<p>Joe, you are a statistician, and must have taken courses on experiment design. Design an experiment to see how your programs scale on the existing architecture first: take a typical data set, decrease it in size by factors of 3 and 10, and increase it in size by factors of 3 and 10. Likewise, find a way to wary the computational complexity. This may not be so easy, although increasing/decreasing number of variables in MI and number of parameters in MCMC should be a step in that direction. Come up with a factorial design and time it.</p>

<p>One would naturally expect the computational time to grow with the problem size and difficulty. My experience timing <a href="http://stata-journal.com/article.html?article=st0169" rel="nofollow">my own Stata factor analysis code</a> was that $$T \sim n^{0.7} p^{2.3}$$ where $n$ was the sample size and $p$ was the dimensionality of the problem. The exponent for the latter is actually the best existing rate for matrix inversion which is one of the intense tasks in the program. The exponent on the sample size is intriguing, as I was expecting the time to be linear in the sample size; this may have to do with the small sample likelihoods being more weird and difficult to optimize, though.</p>

<p>If you start seeing huge increases in computing time beyond these natural power laws, you must be hitting the memory limits, which would be an indication that you need larger RAM. If you don't see any issues with the memory size, for the problems you will likely be working on, you would be far much better off buying a faster chip with more parallel threads.</p>
