<p>First, I considered posting this on the academia SE, but since this is more a question of statistical validity, I thought it more appropriate to post here. If there are similar questions/answers here or there, please direct me to them, as I've searched for a while and not yet found anything useful.</p>
<p>A colleague with the same background as me (Ecology) is now working in a different research field and has started a project. Both of us have concerns about the validity of tests and results used in similar studies (potential pseudo-replication, wrong choice of hypothesis test, not correcting for multiple comparisons), yet it seems pervasive in the field. Relevant journals have decent impact factors, and I assume that this field has resolved any issues that we perceive as problematic.</p>
<p>I recognize there are often differences in how data or results are presented from one discipline to another, and I may simply be overlooking something that is implied or established knowledge in a field different from mine. However, the 'Statistical Analysis' sub-heading within Methods often does not describe any actual methods; rather, it states (in a single sentence or two) the software used, and relegates the actual hypothesis test used and associated stats/metrics to Figure captions. This baffles me.</p>
<p>Journals typically require detailed statistical analysis methodology explaining the reasoning for choosing a particular test, that assumptions were checked and data transformed if necessary, type of post-hoc analysis, etc. However, I am encountering articles that lack this detail and present figures that (to me) look like results of ANOVA, but state that t-tests were used. Thus, I'd like to ask some questions here in hopes that the community can help me understand if/why these particular analyses are correct.</p>
<p>It is not my goal to find errors or call out publications, just to understand the analyses and address any gaps in my own knowledge. My question is:</p>
<p>Am I allowed to refer to a particular article(s) with the analysis I have a question about, or do I need to create a similar reproducible example with relevant context?</p>
