HomeKnowledge Base

Using optimum parameter values in backtesting

After Optimization process has found optimum values for parameters of our trading system, typically we want to use optimum values in subsequent backtesting or explorations. In order to achieve that, we need to manually update default_val (second) argument of Optimize function with the values obtained from the optimization report.

The arguments of Optimize function are shown below (note second parameter marked in dark red color – this is the default value parameter we will be changing after optimization run):

some_var = Optimize( "description", default_val, min_val , max_val, step );

Let us consider the following example formula used for optimization process:

periods Optimize"Periods"2550); // note that default value is 2
Level Optimize"Level"22150); // note that default value is 2

Buy CrossCCIperiods ), -Level );
Sell CrossLevelCCIperiods ) );

If we perform Optimization process and check the results (for this example we use Net Profit as the optimization target), we can see that the best results use Periods = 6 and Level = 126.

Optimization result

Now in order to run backtest and obtain exactly the same results as in the respective line of the above Optimization results, we need to enter the values into default argument, so the modified code will look like this:

periods Optimize"Periods"6550); // we changed default value to 6
Level Optimize"Level"1262150); // we changed default value to 126

Buy CrossCCIperiods ), -Level );
Sell CrossLevelCCIperiods ) );

Now we can use the code with modes other than Optimization and the formula will use optimized values we retrieved from the results.

Getting started with automatic Walk-Forward optimization

Recently released AmiBroker 5.05 BETA features the automatic Walk-Forward Optimization mode.

The automatic Walk forward optimization is a system design and validation technique in which you optimize the parameter values on a past segment of market data (“in-sample”), then test the system forward in time on data following the optimization segment (“out-of-sample”). You evaluate the system based on how well it performs on the test data (“out-of-sample”), not the data it was optimized on.

To use Walk-Forward optimization please follow these steps:

  1. Goto Tools->Automatic Analysis
  2. Click Settings button, then switch to “Walk-Forward tab”
  3. Here you can see Walk forward settings for In-sample optimization, out-of-sample backtest
    “Start” and “End” dates mark initial period begin / end
    This period will be moved forward by “Step” until the “End” reaches the “Last” date.
    The “Start” date can move forward by “step” too, or can be anchored (constant) if “Anchored” check is on.
    If you mark “Use today” then “Last” date entered will be ignored and TODAY (current date) will be used instead

    By default an “EASY MODE” is selected which simplifies the process of setting up WF parameters.
    It assumes that:
    a) Out-of-sample segment immediatelly follows in-sample segment
    b) the length of out-of-sample segment equals to the walk-forward step

    Based on these two assumptions the “EASY” mode takes in-sample END date and sets
    out-of-sample START date to the following day. Then adds in-sample STEP and this becomes out-of-sample END date.
    In-sample and Out-of-sample step values are set to the same values.

    The “EASY” mode guarantees correctness of WF procedure settings.

    In the “ADVANCED” mode, the user has complete control over all values, to the extent that
    they may not constitute valid WF procedure.
    The interface allows to selectivelly disable in-sample and out-of-sample phases using checkboxes at top
    (for special things like runnign sequential backtests without optimization).

    All settings are immediatelly reflected in the PREVIEW list that shows all generated IS/OOS segments and their dates.

    The “Optimization target” field defines the optimization raport COLUMN NAME that
    will be used for sorting results and finding the BEST one. Any built-in column can be used
    (as appears in the optimization output), or you can use any custom metric that you define
    in custom backtester. The default is CAR/MDD, you can however select any other built-in metric from the combo.
    You can also TYPE-IN any custom metric that you have added via custom backtester interface.

  4. Once you defined Walk-Forward settings, please go to Automatic Analysis and
  5. press the dropdown ARROW on the Optimize button and select “Walk Forward Optimization”

This will run sequence of optimizaitons and backtest and the results will be displayed in the “Walk Forward” document that is open in the main application frame.
When optimization is running you can click “MINIMIZE” button on the Progress dialog to minimize it – this allows to see the Walk Forward output during the optimization steps.

IN-SAMPLE and OUT-OF-SAMPLE combined equity

Combined in-sample and out-sample equities are available by
~~~ISEQUITY and ~~~OSEQUITY composite tickers (consecutive periods of IS and OOS are concatenated and scaled to
maintain continuity of equity line – this approach assumes that you generally speaking are compounding profits)
To display IS and OOS equity you may use for example this:

PlotForeign("~~~ISEQUITY","In-Sample Equity"colorRedstyleLine); 
PlotForeign("~~~OSEQUITY","Out-Of-Sample Equity"colorGreenstyleLine); 
Title "{{NAME}} - {{INTERVAL}} {{DATE}} {{VALUES}}"