May 7, 2014
From time to time we receive questions about why you can get different results back-testing the same code.
There are five reasons for differences in backtest results:
- Different data (for example if past history is updated/changed due to splits for example or backfill)
- Different settings / parameters (if your formula uses Param() functions that output values that may be changed from the Parameter window). Note that Parameters in Analysis window use shared ChartID=0, which means that if two formulas use same parameter names they would share parameter values as well.
- Different formula (sometimes even slight change to the formula causes big change in the results, for example if your formula uses #include and included code has changed)
- The formula that self-references its previously generated results. Such code produces some data that is later used to produce next run output (for example your code produces composites that are later used – if those composites change – the input data change so the results change, or if your formula uses previous backtest equity or if your formula uses STATIC variables without deleting/clearing them at the beginning)
- The formula directly or indirectly calls any function that produces random numbers such as Random or mtRandom, or functions that read external data that have changed