1. Steps to a statistical analysis
  2. Distributional Thinking
  3. Statistical significance
    1. p-value
  4. generalizability
  5. Cause and Effect
    1. Standard deviation

This article goes into how we can study phenomena using statistical methods. It comes with a glossary at the bottom of the page. Some psychological studies have been included to illustrate each point, some of which I have used to make my notes easier to understand in the future.

There is also a very interesting example on how to dissect a paper and its claims by asking for each of the key components (see Steps to a statistical analysis).


Key components to a statistical investigation

  • Planning the study” (🔗)
    • “For example, how long was the study period of the coffee study? How many people were recruited for the study, how were they recruited, and from where?” (🔗)
  • Examining the data” (🔗)
    • “in the coffee study, did the proportions differ when we compared the smokers to the non-smokers?” (🔗)
  • Inferring from the data” (🔗)
    • “is the 10%–15% reduction in risk of death something that could have happened just by chance?” (🔗)
    • “Were the people in the coffee study older? Healthy? Living in cities?” (🔗)
  • Drawing conclusions
    • “Are scientists now saying that the coffee drinking is the cause of the decreased risk of death?” (🔗)

Distributional Thinking

“The most fundamental principle of statistics is that data vary.” (🔗)

Statistical Significance

“how can we determine whether patterns we see in our small set of data is convincing evidence of a systematic phenomenon in the larger process or population?” (🔗)

“Although this random component cannot be controlled, we can apply a probability model to investigate the pattern of results that would occur in the long run if random chance were the only factor.” (🔗)

“The probability of observing a particular outcome in a sample, or more extreme, under a conjecture about the larger population or process.” (🔗)

Update 2022-11-07

“The p-value tells you how often a random process would give a result at least as extreme as what was found in the actual study, assuming there was nothing other than random chance at play.” (🔗)

“We often compare the p-value to some cut-off value (called the level of significance, typically around 0.05). If the p-value is smaller than that cut-off value, then we reject the hypothesis that only random chance was at play here.” (🔗)


“In its simplest form, random sampling involves numbering every member of the population and then using a computer to randomly select the subset to be surveyed.” (🔗)

“Random assignment should produce groups that are as similar as possible except for the type of motivation, which presumably eliminates all those other variables as possible explanations for the observed tendency for higher scores in the intrinsic group.” (🔗)

“is it possible that an unlucky random assignment is responsible for the observed difference in creativity scores between the groups?” (🔗)

“Only 2 of the 1,000 simulated random assignments produced a difference in group means of 4.41 or larger. In other words, the approximate p-value is 2/1000 = 0.002.” (🔗)

“Statistical thinking involves the careful design of a study to collect meaningful data to answer a focused research question, detailed analysis of patterns in the data, and drawing conclusions that go beyond the observed data. Random sampling is paramount to generalizing results from our sample to a larger population, and random assignment is key to drawing cause-and-effect conclusions.” (🔗)

“A result is statistically significant if it is unlikely to arise by chance alone.” (🔗)