Avoid Misleading Statements: Calculation Results should not be Displayed as a Hint rather than an Example, which is misleading

[TullyMonster]

https:www.biostarhandbook.com/survival-guide.html#statistical-models:~:text=w%20%3D%20wilcox,0.7

The significance of differences in TP53 gene expression levels is discussed here:

x <- c(1887.7, 1298.1, 1799.6)
y <- c(800.2, 785.3, 800.4)
t <- t.test(x, y)
print(t$p.value)  # 0.04199959

The article goes on to say that significance should not be judged lightly on the basis of the above calculations. You should consider how many comparisons you have participated in, and if you add another group, you should p.adjust(t$p.value, n = 2) and prompt that the P value at this point is 0.0839 > 0.05.

BUT, it is true that the above code block itself is only compared once. Should be: p.adjust(t$p.value, n = 1), the result is still 0.04199959.

If the results used for hints are presented, the meaning of n will be misunderstood.

An inexperienced point of view from a beginner.

[ialbert]

The entire context is framed as repeating the experiment.

Thus, it is the second attempt, and it now produces a p-value under the threshold. We need to correct it since we have made two attempts.

It is true that if we did not look at the results of the first attempt, then we could take this p-value at face value with no correction - but if we did look at the previous results, then we cannot anymore.

Statistics is notorious for counterintuitive and surprising behaviors. As an example, many people have argued forcefully that a statistical threshold may not be set "after" looking at the results. Statistical significance is only meaningful if determined before.

So we can't say "hey I originally planned the significance to be 0.05 but I get lots of results even at 0.01, so I could drop the significance to 0.01 and have "more credible" evidence" ... as it happens changing the threshold after seeing what effect it has makes the statistic invalid altogether (all technically speaking of course, in practice, this is done all the time :-) ). Regardless, I found this example quite illuminating.

There is quite a large body of work, going back to 1930s where the creators of modern statistics vehemently argued with one another about what exactly certain terms mean and in what context the interpretation is accurate.

Long story short, I understand the concern that some of the text might be misunderstood - but I think that is unavoidable. Sometimes, trying to be too accurate has unintended consequences.

In this case, my goal is solely to raise awareness that repeated measurements need to be accounted for.

[TullyMonster]

Thanks, I misunderstood myself, but thank you for your clarification and sincerely bless you.