February 12, 2008
Getting started with automatic Walk-Forward optimization
Recently released AmiBroker 5.05 BETA features the automatic Walk-Forward Optimization mode.
The automatic Walk forward optimization is a system design and validation technique in which you optimize the parameter values on a past segment of market data (”in-sample”), then test the system forward in time on data following the optimization segment (”out-of-sample”). You evaluate the system based on how well it performs on the test data (”out-of-sample”), not the data it was optimized on.
To use Walk-Forward optimization please follow these steps:
- Goto Tools->Automatic Analysis
- Click Settings button, then switch to “Walk-Forward tab”
- Here you can see Walk forward settings for In-sample optimization, out-of-sample backtest
“Start” and “End” dates mark initial period begin / end
This period will be moved forward by “Step” until the “End” reaches the “Last” date.
The “Start” date can move forward by “step” too, or can be anchored (constant) if “Anchored” check is on.
If you mark “Use today” then “Last” date entered will be ignored and TODAY (current date) will be used insteadBy default an “EASY MODE” is selected which simplifies the process of setting up WF parameters.
It assumes that:
a) Out-of-sample segment immediatelly follows in-sample segment
b) the length of out-of-sample segment equals to the walk-forward stepBased on these two assumptions the “EASY” mode takes in-sample END date and sets
out-of-sample START date to the following day. Then adds in-sample STEP and this becomes out-of-sample END date.
In-sample and Out-of-sample step values are set to the same values.The “EASY” mode guarantees correctness of WF procedure settings.
In the “ADVANCED” mode, the user has complete control over all values, to the extent that
they may not constitute valid WF procedure.
The interface allows to selectivelly disable in-sample and out-of-sample phases using checkboxes at top
(for special things like runnign sequential backtests without optimization).All settings are immediatelly reflected in the PREVIEW list that shows all generated IS/OOS segments and their dates.
The “Optimization target” field defines the optimization raport COLUMN NAME that
will be used for sorting results and finding the BEST one. Any built-in column can be used
(as appears in the optimization output), or you can use any custom metric that you define
in custom backtester. The default is CAR/MDD, you can however select any other built-in metric from the combo.
You can also TYPE-IN any custom metric that you have added via custom backtester interface. - Once you defined Walk-Forward settings, please go to Automatic Analysis and
- press the dropdown ARROW on the Optimize button and select “Walk Forward Optimization”
This will run sequence of optimizaitons and backtest and the results will be displayed in the “Walk Forward” document that is open in the main application frame.
When optimization is running you can click “MINIMIZE” button on the Progress dialog to minimize it - this allows to see the Walk Forward output during the optimization steps.
IN-SAMPLE and OUT-OF-SAMPLE combined equity
Combined in-sample and out-sample equities are available by
~~~ISEQUITY and ~~~OSEQUITY composite tickers (consecutive periods of IS and OOS are concatenated and scaled to
maintain continuity of equity line - this approach assumes that you generally speaking are compounding profits)
To display IS and OOS equity you may use for example this:
PlotForeign("~~~ISEQUITY","In-Sample Equity", colorRed, styleLine);
PlotForeign("~~~OSEQUITY","Out-Of-Sample Equity", colorGreen, styleLine);
Title = "{{NAME}} - {{INTERVAL}} {{DATE}} {{VALUES}}";
Filed by Tomasz Janeczko at 7:54 am under AFL, Systems
3 Comments