<p>I will take this question seriously, which may or may not be helpful.</p>

<p>Perhaps you are familiar with the discipline of <a href="http://en.wikipedia.org/wiki/Psychometrics" rel="nofollow">psychometrics</a> (which is the study of how to accurately measure psychological constructs).  You will need to work out a theory of what it is you want to measure and how it is related to other theoretical constructs (or at least, available measures of those constructs).  Often it is not possible to directly measure the target of your theorizing, but a variety of indirect measures can be found.  <a href="http://www.psych.cornell.edu/darlington/factor.htm" rel="nofollow">Factor analysis</a> is typically used to assess if the indirect measures really do 'hang together', and if they are unitary.  From there, you can work on building a measurement system, and checking its <a href="http://en.wikipedia.org/wiki/Reliability_%28psychometric%29" rel="nofollow">reliability</a>, and convergent and discriminant <a href="http://en.wikipedia.org/wiki/Validity_%28statistics%29" rel="nofollow">validity</a>.  </p>

<p>With some basics mentioned, let me throw out some specific thoughts.  Raw values (e.g., reputation scores and badge counts) are rarely considered very useful.  For instance, consider the total reputation score, an analogy can be made between it and p-values:  Just as p-values measure effect size and $N$ simultaneously, and thus are often not a good thing to focus on (depending on what you want to know), so reputation measures both level of quality and amount of activity.  Even mean upvotes minus downvotes <em>per question</em> is confounded with the amount of attention each question has received as you recognized.  So a decent first pass at the typical quality of someone's (answering) efforts might be a weighted mean vote total, with the weights determined by each question's views.  We can, of course, always push further.  For example, some people may be only inclined to answer a question if they are confident they can give a really good answer, whereas others may answer anything where they suspect their marginal contribution is above zero (think of how some people are more talkative than others).  Is that something that you want to capture in the measurement, or is it a contaminant you want to filter out?  Note that we can run through analogous considerations with badges.  For what it's worth, I agree with MansT (+1).  </p>
