<p>I would add a simple point that questions can be ethical and technical at the same time.</p>
<p>Is it a good or at least defensible idea to omit outliers? is a frequent example, where one simplified answer might start with</p>
<ul>
<li><p>Yes possibly, if it is clear that an outlier is wrong and can't be corrected, or turns out to be irrelevant to your goals, but be clear about what you are doing</p>
</li>
<li><p>No almost certainly, if the motive is that such outliers are awkward or inconvenient for your actual or intended analysis</p>
</li>
</ul>
<p>-- and what underlies the second point is a concern for being honest and realistic about the data.</p>
<p>Other examples could be</p>
<ul>
<li><p>Binning variables where one motive is to reduce the possibility of tracing data points back to individuals (often desirable or even mandatory), but otherwise the usual statistical reaction is that such practice is throwing away information</p>
</li>
<li><p>Blurring variables by adding noise where similar concerns arise</p>
</li>
<li><p>Transforming variables where one concern (usually misplaced, but it exists) is that this amounts to changing the data, which would be wrong</p>
</li>
</ul>
<p>and perhaps above all</p>
<ul>
<li><p>The tension between <em>exploratory data analysis</em> (advised or presented positively as looking at the data carefully to spot problems and possibilities not necessarily evident in advance) and <em>data snooping</em> (advised against on several grounds)</p>
</li>
<li><p>Shady or dubious practices with multiple significance tests, where one dimension of concern is how that is statistically or scientifically honest, where honesty is ethical as well as technical</p>
</li>
</ul>
<p>Leo Breiman has this advice on p.1 of his (excellent, but seemingly little known) text <em>Statistics: With a View Toward Applications</em> (Boston: Houghton Mifflin, 1973)</p>
<blockquote>
<p>The main thing to learn about statistics is what is sensible and
honest and possible.</p>
</blockquote>
<p>Flagging <em>honest</em> is a little unusual in such a text -- not that texts advocate the opposite! -- but it is totally in order.</p>
