Checklist: Statistical Problems to Document and Avoid

Update 2010-04-21: I forgot to post the link last time. That would have been helpful. Here you go:

Vanderbilt Biostatistics: Statistical Problems to Document and to Avoid

.....

At the Regression Modeling Strategies course I attended a few weeks ago, Frank Harrell pointed out the checklist on the biostatistics department's website of statistical problems to document and avoid. It was recommended that authors of any paper employing statistical analysis should go through this checklist before writing and submitting a manuscript.  Some of the topics include:

• Design and sample size issues
• Inefficient use of continuous variables (don't categorize!)
• Assumptions of parametric tests
• Inappropriate analysis of repeated measures data
• P-value interpretation
• Filtering results
• Missing data
• Multiple testing concerns
• Model building and specification
• Use of stepwise variable selection (don't do it)
• Overfitting

Be sure to check this out before writing up results, and ideally before you even plan any experiments, especially if you are relatively new to quantitative analysis.