May 7, 2014
From time to time we receive questions about why you can get different results back-testing the same code.
There are five reasons for differences in backtest results:
- Different data (for example if past history is updated/changed due to splits for example or backfill)
- Different settings / parameters (if your formula uses Param() functions that output values that may be changed from the Parameter window)
- Different formula (sometimes even slight change to the formula causes big change in the results, for example if your formula uses #include and included code has changed)
- The formula that self-references its previously generated results. Such code produces some data that is later used to produce next run output (for example your code produces composites that are later used – if those composites change – the input data change so the results change, or if your formula uses previous backtest equity)
- The formula directly or indirectly calls any function that produces random numbers such as Random or mtRandom, or functions that read external data that have changed