Evidence/update proportionality is related to anchoring and adjustment and refers to the idea that we should like to update our beliefs when encountering evidence in proportion to the strength of that evidence. Some examples of failures:
- We strongly believe a particular story and cite that three news outlets support a particular interpretation of the facts, ignoring that all three news outlets were operating off the same scarce data and also are following a very similar set of heuristics for turning said data into a story. Instead, we should like to count each piece of data as some amount of evidence for each model consistent with that data.
- We support a particular position in the face of any and all evidence to the contrary until a critical threshold has been reached, at which point we flip and strongly take up the opposite position. Instead, we should like to remain uncertain about the two positions, and gradually update towards one or the other as new evidence comes in.
- We hold confirming and disconfirming evidence to different standards, making isolated demands of rigor for evidence that disagrees with a current working model.
In general, assuming we are following good search strategies, our updates will generally decrease in size as we converge on consistent interpretations of the available data. Scientific theories and laws are models that have such strong and varied pieces of evidence supporting them that it would be shocking to find evidence strong enough to cause large updates about them.