BackTest, how it works, design, architecture, data flow, brain dump, how I got here…

How I got here;

  • Sometime in early ’06 some friend’s talk about trading futures finally got to me ;^)
  • Sometime in mid ’06 I realized what I wanted to do was automated backtesting of algorithmic trading strategies.
  • I began looking at all the retail software I could find.
  • Read the web sites of many, did trials of several and spent weeks with 2 or 3.
  • Here are the notes I took on various software that I evaluated
  • None of them gave me the dopamine I so craved.
  • Primary problem to be solved: The retail apps suck like a black hole.
  • Goal: Build a backtesting app that doesn’t suck like a black hole.
  • Main problems with retail: (putting personal distaste aside as much as possible) claims to do backtesting but they only scratch the surface – tip of the iceberg. NeoTicker showed hope but very unstable, poorly designed, inconsistent interface and apis. And it still lacked ability for serious backtest analysis. Great 3d visual map though. There were only 2 or 3 apps that I found that even knew what backtesting is. Others claim it but don’t come close.
  • Problem with retail backtesting environments: they’re trying to make it easy(ier) for civilians to do programming. Inherent, often arbitrary restrictions result.
  • Ok let me lob one shot at TradeStation. You know the BackTest Strategy Tab (on BackTest, not TradeStation), where you set the desired parameters. Well with TradeStation, every time you want to modify any of the parameters, you have to set each and every one of them! (Or at least something as obnoxious as that if not exactly that). As annoying as that is, what is more important IMHO (obviously) is that it is telling of the company behind it. I.e. they’re not a software co.
  • NeoTicker was a hopeful since they are in fact a software co, not a broker.
  • SmartQuant ($$$ several thousands) (trading development libraries and IDE) looked very compelling. I almost jumped off the cliff but hesitated due to commitment to C#. Got so close to buying SmartQuant that I went ahead and ordered MS VS2005 for $600 or $700, which I had to do eventually anyway.
  • Came across mention of fmlabs (~ $400) on elitetrader.com. Bingo. Perfect fit. C library. Allowed me to write in C++/MFC which I know/hate so well ;^) mainly reduced learning curve down to learning new MS ide/compiler (VS2005). Plus I could leverage personal libraries previously written for Muzikbrowzer.
  • Along about Oct/Nov ’06 I was burnt out of evaluating retail trading software and I needed to finish my Muzikbrowzer release.
  • My log shows Jan 25 ’07 to be my last commit to my Muzikbrowzer source repository, and I know I was anxious to get started building BackTest. So I must have begun right about the 1st of Feb ’07.

Initial goals as best as I recall

  • With Neo and others there was no easy was to save/retrieve previous results. In some cases the apps didn’t even let you name a profit report so that you could tell how it was generated. They provided no way to organize the results in an orderly fashion. Imagine looking at a profit report or spread sheet and all you see are the NetProfit, Gross Profit etc. No mention of the strategy name, nor parameters used to generate. Further more imagine having to manually click “Save As” each time you generate a report, then typing in a name for the file name. So you basically create your own management system with file names/folders and use windows explorer to manage. And since the report itself contains no information about *how* it was generated, the file name must contain all the meta data!While this may seem like a small or only very annoying limitation, IMHO (obviously) it is so very tedious that serious backtest analysis cannot be done with them.
  • So primary goals were to do backtesting well and excel at results management, at least relative to what I’d seen.