So you sat down at your PC and came up with a super-complex new trading strategy that mints money in everyone of your hypothetical back tests?
Watch out as Andrew Lo warns:
“..the first [rule of thumb] being that no matter how complex and subtle a strategy is and no matter how sophisticated it might be, it has to be possible to describe that strategy in relatively simple and intuitive terms to a sophisticated investor. In other words, regardless of how subtle and impressive and sophisticated the strategy is, I’ve never come across anything that couldn’t be described in relatively straightforward terms as to what the value added of that strategy was, whether it was risk transfer, superior information, better executions, mean reversion, and so on. So, I think that’s the first principle that I think is obvious to many investors; but to some who are not as familiar with quantitative methods, they may feel that they’re just not really smart enough to understand. But, I think that’s just not the case. The second rule of thumb is that you’ll never see a bad back test. Now, again, this may be obvious to the experienced investor, but there’s a very specific set of quantitative models that you can use to be able to gauge the bias that comes about from selection. The fact is that when I’ve talked with investors about doing due diligence, I’ve often said that, you know, whatever back test you’d like to see, I can certainly produce it for you. If you torture the data long enough, it will basically tell you anything you want…”