amibroker

HomeKnowledge Base

Troubleshooting procedure when backtest shows no trades

When we run backtest and get no results at all – there may be several reasons of such behaviour. The main potential causes are the following:

  1. our system does not generate any entry signals within the tested range
  2. our settings do not allow the backtester to take any trades

To verify if we are getting any signals – the first thing to do is to run a Scan. This allows us to check if we are getting any Buy or Short signals at all. If there are none, then we need to check the formula and make sure that data interval we are working on are correct (in Periodicity in Analysis->Settings->General).

If Scan works fine and returns trading signals, but backtester still does not produce any output, it usually means that the settings are wrong, i.e. the constraints set in the settings prevent trades from being opened mainly because requested position size is too big or too small.

To check what is going on, it is best to switch Report mode to Detailed log and re-run backtest.

Report - Detailed log

Once you run backtest in Detailed Log mode you will be able to find out exact reasons why trades can not be opened for each and every bar:

Detailed log output

Using the following settings may be helpful to minimize chances of not entering trades because of various constraints:

In Analysis->Settings, General tab:

  1. check if Initial Equity is high enough
  2. set Periodicity to the appropriate interval
  3. Allow position size shrinking – turn it On
  4. Round Lot Size – set it to 0
  5. in Min. Shares box enter 0.01
  6. in Min. pos. value enter 0
  7. Account Margin – set it to 100

Settings - General

in Portfolio tab, enter 0 in Limit trade size as % of entry bar volume box.

Settings - Portfolio

How to exclude top ranked symbol(s) in rotational backtest

Rotational trading is based on scoring and ranking of multiple symbols based on user-defined criteria. For each symbol a user-definable “score” is assigned on bar by bar basis. Then, each bar, symbols are sorted according to that score and N top ranked symbols are bought, while existing positions that don’t appear in top N rank are closed.

Sometimes however, we may want to exclude the highest ranking symbol (or a couple of them) from trading. The code below shows how to do that using custom backtester.

ExcludeTopN 1// how many top positions to exclude
SetCustomBacktestProc("");

if ( 
Status"action" ) == actionPortfolio )
{
    
bo GetBacktesterObject();
    
bo.PreProcess();

    for ( 
bar 0bar BarCountbar++ )
    {
        
Cnt 0;
        for ( 
sig bo.GetFirstSignalbar ); sigsig bo.GetNextSignalbar ) )
        {
            if ( 
Cnt ExcludeTopN )
                
sig.Price = -1// exclude

            
Cnt++;
        }

        
bo.ProcessTradeSignalsbar );
    }

    
bo.PostProcess();
}

EnableRotationalTradingTrue );

SetOption"MaxOpenPositions");
SetOption"WorstRankHeld"10 );
SetPositionSize20spsPercentOfEquity );
PositionScore RSI14 )

The code is pretty straightforward mid-level custom backtest loop but it uses one trick – setting signal price to -1 tells AmiBroker to exclude given signal from further processing. Note also that signals retrieved by GetFirstSignal / GetNextSignal are already sorted, so the highest ranked signal appears first in the list.

Time compression of data retrieved from another symbol

AmiBroker’s Time-Frame functions (http://www.amibroker.com/guide/h_timeframe.html) allow to use multiple intervals within a single formula and combine them together. Another set of functions in AFL (Foreign and SetForeign) allow us to retrieve data of another symbol from the database, so we can implement strategies where rules are based on multiple symbols.

This article shows how to combine these two features together and properly use Time-Frame functions on data retrieved from another symbol. Let us consider an example of a strategy, which works on daily data, but uses an additional filter based on weekly readings of S&P500 index.

The following sequence is required to code such conditions properly:

  1. switch to the other symbol with SetForeign
  2. compress data into higher interval with TimeFrameSet
  3. store the weekly values / conditions in custom variables
  4. with TimeFrameRestore() or RestorePriceArrays() functions restore the original arrays of the tested symbol (in the original time-frame)
  5. use custom variables assigned in step (3) expanded to original time-frame using TimeFrameExpand()

Here is the AFL formula, which implements the above conditions:

// first switch to ^GSPC symbol
SetForeign"^GSPC" );
//
// compress data to weekly interval
TimeFrameSetinWeekly );
//
// assign weekly values to custom variables
indexWeeklyClose Close;
indexWeeklyMA =  MAClose52 );
indexWeeklyFilter Close MAClose52 );
//
// restore original arrays (back to the primary symbol)
// RestorePriceArrays() function is an equivalent
TimeFrameRestore();
//
// align data back to original interval
indexFilterExpanded TimeFrameExpandindexWeeklyFilterinWeekly );
//
// exploration shows the results, note that all weekly values
// need to be expanded if we haven't done it yet
//
Filter 1;
AddColumnClose"Close AAPL" );
AddColumnTimeFrameExpandindexWeeklyCloseinWeekly ), "Weekly close ^GSPC" );
AddColumnTimeFrameExpandindexWeeklyMAinWeekly ), "Weekly MA ^GSPC" );
AddColumnindexFilterExpanded"Weekly index filter")

Let us compare the readings obtained from the code with a sample chart – both ^GSPC raw reading and 52-week MA values match the chart and the condition is properly aligned to the bars starting on 2011-10-28 and extends until new weekly bar is formed.

TimeFrame + Foreign

There is also an alternative method we can use:

  1. retrieve values from ^GSPC using Foreign() function
  2. compress these readings into weekly interval using TimeFrameCompress
  3. perform calculations on weekly compressed array
  4. expand the compressed data back to the original timeframe using timeFrameExpand
indexClose Foreign("^GSPC","C");
indexWeeklyClose2 TimeFrameCompressindexCloseinWeekly );
indexWeeklyMA2 MAindexWeeklyClose252 );
indexWeeklyFilter2 indexWeeklyClose2 indexWeeklyMA2;
//
Filter 1;
AddColumnClose"Close AAPL" );
AddColumnTimeFrameExpandindexWeeklyClose2inWeekly ), "Weekly close ^GSPC" );
AddColumnTimeFrameExpandindexWeeklyMA2inWeekly ), "Weekly MA ^GSPC" );
AddColumnTimeFrameExpandindexWeeklyFilter2inWeekly ), "Weekly index filter")

How to customize list-view columns

A list-view is a view that displays a list of scrollable items in a table-like format. List-views are used in Real-Time quote window, Analysis window, Symbol list, etc. The columns in any list-view in AmiBroker can be customized in various ways to better match our needs and display the required statistics and readings the way we find it most useful. For the sake of example let us consider Analysis window result list.

Many of the customization actions can be performed directly on the column headers. It is possible to re-order the columns by dragging them with mouse cursor:

Column drag

and their width can be re-sized by dragging the divider lines between columns (double-clicking on that area will auto-resize the columns to match their contents).

Column resize

Hint: You can auto-resize all columns to their content at once by holding down Ctrl key and pressing + (plus sign) key on the numeric keypad.

For more operations it is possible to use Setup Columns… menu available from the context menu, which displays after right-clicking on the headers.

Column resize

Setup Columns dialog allows to re-order, hide/show selected columns.

Column setup

To hide a column uncheck the box, to show it back again, check the box. To re-arrange columns, select a column and click Move Up/Move Down buttons.

It is important to remember that the set of columns will depend on the last run mode, so it will be different for Scan, for the Backtest Trade List, for Summary type of report or Optimization.

Column setup 2

Further customization options are available programmatically. Custom Backtest interface allows to add your own metrics to the backtest report (more info: http://www.amibroker.com/guide/a_custommetrics.html)

We can also define in our code where those additional columns are positioned in the report (this includes both custom metrics added to the report or optimized parameter values in the Optimization). By default they would be listed at the very end, but SetOption() function allows to set different position, for example:

SetOption("ExtraColumnsLocation")

Using price levels with ApplyStop function

ApplyStop function by default requires us to provide stop amount (expressed in either dollar or percentage distance from entry price). Therefore, if we want to place stop at certain price level, then we need to calculate the corresponding stop amount in our code.

This example shows how to place stops at previous bar Low (for long trades) and previous bar High (for short trades).

Stop amount parameter is simply the distance between entry price and desired trigger price (exit point). For long trade it is entry price minus stop level, while for short trade it is trigger (exit) price minus entry price. Additionally we may check if calculated distance is at least 1-tick large. We can distinguish between long and short entry by checking if one of entry signals is present (if a Buy signal is active then it is long entry, otherwise short). We only need to take care about the fact that if we are using trade delays we need to get delayed Buy signal as shown in the code below:

TradeDelay 1// set it to 0 for no delays

SetTradeDelaysTradeDelayTradeDelayTradeDelayTradeDelay );
TickSize 0.01;

// sample entry rules
Buy CrossMACD(), Signal() );
Short CrossSignal(), MACD() );
Sell Cover 0// no other exit conditions, just stops

BuyPrice SellPrice ShortPrice CoverPrice Close;

// define stop level
stopLevelLong RefL, -); // use previous bar low
stopLevelShort RefH, -); // use previous bar high

// calculate stop amount
stopAmountLong BuyPrice stopLevelLong;
stopAmountShort stopLevelShort ShortPrice;

// make sure stop-amount is at least one tick
stopAmountLong MaxTickSizeBuyPrice stopLevelLong );
stopAmountShort MaxTickSizestopLevelShort ShortPrice );

// assign stop amount conditionally by checking if there is a Buy signal on given bar
IsLong RefBuy, -TradeDelay );
stopAmount IIfIsLongstopAmountLongstopAmountShort );
ApplyStopstopTypeLossstopModePointstopAmountTrue )

Per-symbol profit/loss in a portfolio backtest

Backtesting engine in AmiBroker allows to add custom metrics to the report, both in the summary report and in the trade list. This is possible with Custom Backtester Interface, which allows to modify the execution of portfolio-level phase of the test and (among many other features) adjust report generation.

The example presented below shows how to retrieve individual profit/loss figures for each traded symbol in a portfolio test and add the results as custom metrics to the report. The code performs backtest, then iterates through the list of trades and stores each symbol profit in separate variables. Variables are created with VarSet function, which allows to build variable names dynamically, based on the symbol name. There are 2 variables generated per symbol, one holding profit for long trades and one for short trades. In the last part the code reads the created variables and adds input into the backtest report.

function ProcessTradetrade )
{
  global 
tradedSymbols;
  
symbol trade.Symbol;
  
//
  
if( ! StrFindtradedSymbols"," symbol "," ) )
  {
    
tradedSymbols += symbol ",";
  }
  
//
  // HINT: you may replace it with GetPercentProfit if you wish
  
profit trade.GetProfit();
  
//
  
if( trade.IsLong() )
  {
      
varname "long_" symbol;
      
VarSetvarnameNzVarGetvarname ) ) + profit );
  }
  else
  {
      
varname "short_" symbol;
      
VarSetvarnameNzVarGetvarname ) ) + profit );
  }
}
//
SetCustomBacktestProc"" );
//
/* Now custom-backtest procedure follows */
//
if ( Status"action" ) == actionPortfolio )
{
    
bo GetBacktesterObject();
    
//
    
bo.Backtest(); // run default backtest procedure
    //
    
tradedSymbols ",";
    
//
    //iterate through closed trades
    
for ( trade bo.GetFirstTrade( ); tradetrade bo.GetNextTrade( ) )
    {
        
ProcessTradetrade );
    }
    
//
    //iterate through open positions
    
for ( trade bo.GetFirstOpenPos( ); tradetrade bo.GetNextOpenPos( ) )
    {
        
ProcessTradetrade );
    }
    
//
    //iterate through the list of traded symbols and generate custom metrics
    
for ( 1; ( sym StrExtracttradedSymbols) ) != ""i++ )
    {
        
longprofit VarGet"long_" sym );
        
shortprofit VarGet"short_" sym );
        
allprofit Nzlongprofit ) + Nzshortprofit );
        
// metric uses 2 decimal points and
        // 3 (calculate sum) as a "combine method" for walk forward out-of-sample
        
bo.AddCustomMetric"Profit for " symallprofitlongprofitshortprofit2);
    }
}
//
SetOption"MaxOpenPositions"10 );
//
Buy CrossMACD(), Signal() );
Sell CrossSignal(), MACD() );
Short Sell;
Cover Buy;
SetPositionSize10spsPercentOfEquity 

Once we run the Backtest, we will get the following output in the report, showing individual profit/loss figures for each symbol in test.

Per-symbol profit

If you prefer percent profits instead of dollar profits, just replace GetProfit() call with GetPercentProfit().

Position sizing based on risk

One of most popular position sizing techniques is Van Tharp risk-based method. Van Tharp defines risk as the maximum amount that can be lost in a trade. Typically you limit your loses by setting up a maximum loss stop.

The amount risked should not be confused with amount invested. If your stop is 15% away from entry price, in worst case you risk losing 15% of the position size (amount invested), not the entire amount. So risk practically means the amount of maximum loss stop.

Now, imagine that we only allow to lose 1% of entire portfolio equity in single trade. If our stop is placed 15% away, it means that to risk just 1% of entire equity we can put 1/15 part of our available equity into this trade. As we can see desired position size is inversely proportional to stop amount.

Buy CrossCMAC20 ) ); // some trading rules
Sell CrossMAC20 ), );
//
PositionRisk 1// how much (in percent of equity) we risk in single position
TradeRisk 15// trade risk in percent equals to max. loss percentage stop
PctSize 100 PositionRisk TradeRisk;
//
ApplyStopstopTypeLossstopModePercentTradeRiskTrue );
SetPositionSizePctSizespsPercentOfEquity )

Let us see how it works, say we have equity of $60,000, and we only want to risk 1% in single trade ($600). We set our protective stop to 15%. If stock entry price is say $20, we would put our protective stop at $20-15% = $17, so we will risk $3 per share. Given the fact that we want to risk only $600 in that trade, we could buy 200 shares (position risk 200 * $3 = $600). 200 shares @ $20 each gives position value of $4000. $4000 represents 6.667% of $60,000 and this is actual percentage position size we would open. As we can clearly see 6.667% is what we would get if we divide 1 (percent position risk) by 15 (percent loss amount): 1/15 = 6.667%

Instead of setting our stop as fixed percentage, we can use more sophisticated methods. For example we can adjust our maximum loss (so the risk) dynamically, using average true range, so it will get wider if stock is volatile and narrower if stock prices move in a narrow range. Say we want to set our stop to twice the amount of ATR( 20 ) at the entry bar and risk 3% of portfolio equity in a single trade:

Buy CrossCMAC20 ) ); // some trading rules
Sell CrossMAC20 ), );
//
RiskPerShare =  ATR20 );
ApplyStopstopTypeLossstopModePointRiskPerShareTrue );
//
// risk 3% of entire equity on single trade
PositionRisk 3;
//
// position size calculation
PctSize =  PositionRisk BuyPrice RiskPerShare;
SetPositionSizePctSizespsPercentOfEquity )

This time our maximum loss (so the risk per share) is expressed in dollars not in percents. Let us verify the above calculation. Assume that our equity is $90,000, stock price is $18, ATR(20) is $1. Now risk per share (the stop amount) equals 2 * $1 = $2, so our calculated position size (required % of equity) from the above formula would be:

PctSize = 3 * $18 / $2 = 27%

27% of $90,000 means trade size of $24,300, i.e. 1350 shares (@ $18 each). Since we risk $2 on each share, the total risk is $2 * 1350 shares = $2700, which is exactly 3% of our total equity ($90,000 * 3%).

In case of futures, we would need to take into account the fact that our position size depends on Margin Deposit, while the stop size (expressed in dollars) depends on the Point Value, so the position sizing formula would need to be modified.

Buy CrossCMAC20 ) ); // some trading rules
Sell CrossMAC20 ), );
//
RiskPerContract ATR20 );
ApplyStopstopTypeLossstopModePointRiskPerContractTrue );
//
// risk 1% of entire equity on single trade
PositionRisk 1;
PctSize =  PositionRisk MarginDeposit  / ( RiskPerContract PointValue );
SetPositionSizePctSizespsPercentOfEquity )

Let us assume that we are trading a contract with $5000 margin deposit, point value $50, our equity is 1 million and ATR(20) is equal 5 big points. Risk per contract is then 10 big points. Now the above formula would give us:

PctSize = 1% * $5000 / ( 10 * $50 ) = 10%

10% of our 1 million equity is $100K, which allows us to buy 20 contracts (20 * $5000 each). Since our risk is 10 big points and each big point has a value of $50 we are risking 10 * $50 = $500 per contract. We have 20 contracts so entire position represents a risk 20 * $500 = $10,000 which is 1% of our 1 million equity.

How to setup automatic periodic scans & explorations

One of the most powerful features of AmiBroker is the ability of screening even hundreds of symbols in real-time and monitor the occurrence of trading signals, chart patterns and other market conditions we are looking for. This can be done in Analysis module with Scan or Exploration features.

The main difference between Scan and Exploration is that Exploration allows to customize the output shown in Analysis window (this is explained in details in the following tutorial chapter: http://www.amibroker.com/guide/h_exploration.html), while Scan performs search for at least one of Buy, Sell, Short, Cover signals and displays predefined set of columns. Both these features allow for continuous screening of the database in real-time conditions.

The following procedure shows how to configure basic scan formula and generate alerts when conditions coded in the formula are met. We assume that AmiBroker is already configured to receive real-time data from one of realtime data vendors – the list of recommended datasources is available here: http://www.amibroker.com/guide/h_quotes.html

We need to do the following:
– open Formula Editor window with Analysis->Formula Editor command from the menu
– in the editor window enter or paste the code below

// example trading signals defined here
Buy CrossMACD(), Signal() );
Sell CrossSignal(), MACD() );
//
// additional part of the formula which generates audio alerts when condition is detected
AlertIFBuy"SOUND C:\\Windows\\Media\\Ding.wav""Audio alert");
AlertIFSell"SOUND C:\\Windows\\Media\\Ding.wav""Audio alert")

After entering the code use Tools->Send to Analysis as shown below:

Send to Analysis

Then in the Analysis window select Apply To: All Symbols, Range: 1 Recent bar, this defines which symbols are included in the screening and what time-range will be shown in the results list.

Range setting

To enable continuous screening, mark Auto-repeat (AR) Scan/Explore option and enter the repeat interval. The interval can be specified in minutes or seconds (for example entering 10s means 10-seconds, while 5m means 5-minutes). The below example uses 15-second repeat interval:

Auto-repeat setting

NOTE: If that is the very first screening after launching the database and it may require filling the historical quotes, then it is also required to mark Wait for Backfill (applies to data sources, which support this feature, see: http://www.amibroker.com/guide/h_rtsource.html for more details).

Now press Scan button to initiate the screening process:

Scan

The results window will show the hits and generated alerts will also be logged in Alert Output window and the scan will be automatically repeated every 15 seconds in search for new signals.

Scan

More information about generating and configuration formula-based alerts is presented in this tutorial: http://www.amibroker.com/guide/h_alerts.html

How generate backtest statistics from a list of historical trades stored in a file

Apart from testing mechanical rules based on indicator readings, backtester can also be used to generate all statistics based on a list of pre-defined trades, list of our real trades from the past or a list of trades generated from another software.

To achieve that, first we need to create an input information for AmiBroker where it could read the trades from. A convenient way would be to use an input file in text format, which could store information about trades, including the type of transaction (buy or sell), dates and position sizes. A sample input file may look like this:
Symbol,Trade,Date,Price,Shares
AAME,Buy,2000-04-06,2.66,375.94
AAME,Buy,2000-04-10,2.66,378.922
AAPL,Buy,2000-04-27,31.23,32.0862
AAON,Buy,2000-04-06,3.19,313.48
ABAX,Buy,2000-04-26,7.67,132.101
AB,Buy,2000-04-25,20.23,50.0337
A,Buy,2000-04-27,84.66,11.8362
AAME,Buy,2000-05-10,2.6,373.627
ABCB,Buy,2000-05-11,6.08,159.406
A,Buy,2000-05-15,82.27,11.736
AAON,Buy,2000-05-18,3.84,246.242
AB,Buy,2000-05-15,20.84,46.3303
ABAX,Buy,2000-05-18,5.84,161.913
ABCB,Buy,2000-05-15,6.08,158.803
AAME,Buy,2000-05-19,2.6,363.763
AB,Buy,2000-06-05,22.78,43.3501
ABC,Buy,2000-05-18,4.49,210.595

We can read and backtest such input with the formula presented below. It is important to remember that this particular code can work with input files of identical format (columns in identical order, signals specified with exact Buy / Sell words, position sizes specified as shares). Changing the input format would also require to update the formula to match the input.

Path to the file is specified in the very first line (note that double backslashes need to be used).

The formula reads the file line by line, then on a bar with matching date/time it generates a new Buy or Sell signal that is then combined with existing signals (coming from other bars).

file "C:\\TEMP\\trades.csv"// change this to real location of your data file
dt DateTime();
//
// Initialize variables
Buy Sell possize 0;
//
fh fopenfile"r" );
//
if( fh )
 {
     while( ! 
feoffh ) )
     {
         
line fgetsfh );
         
// get the ticker symbol from the file
         
sym StrExtractline);
         
// if ticker matches current symbol
         
if ( Name() == sym )
         {
             
// extract data from line of text
             
trade StrExtractline);
             
trade_datetime StrToDateTimeStrExtractline) );
             
price StrToNumStrExtractline) );
             
shares StrToNumStrExtractline) );
             
//
             
if ( trade == "Buy" )
             {
                 
newbuy dt == trade_datetime;
                 
Buy Buy OR newbuy// combine previous buy signals with new
                 
BuyPrice IIfnewbuypriceBuyPrice );
                 
possize IIfnewbuysharespossize );
             }
             
//
             
if ( trade == "Sell" )
             {
                 
newsell dt == trade_datetime;
                 
Sell Sell OR newsell// combine previous sell signals with new
                 
SellPrice IIfnewsellpriceSellPrice );
             }
         }
     }
     
//
     
fclosefh );
 }
 else
 {
     
Error"ERROR: file can not be open" );
 }
//
SetPositionSizepossizespsShares )

Debugging techniques – Part 1 – Exploration

From time to time people send us their formulas asking what happens in their own code. Or they do not know why given trade is taken or not. These questions are usually caused by the fact that people lack the insight what is happening inside and what values values their variables hold.

The first general-purpose debugging technique is using Exploration.

You need to add several AddColumn statements and run your code as Exploration, so you can actually see the values of all variables. This will reveal whenever you really have values that you expect and would make it easier for you to understand what is happening inside your code.

In simplest form add this code to your system formula:

Filter 1// show all bars
AddColumnBuy"Buy" )

and it will show you if you are getting expected values in Buy array. You can use the same technique to track the content of any variable. Add as many columns as you want. You would be surprised how much insight into your own code you will get.

You can use Exploration to learn how particular function works, for example, if you don’t understand how ValueWhen works, you can display its results this way:

Filter 1// show all bars
//
MAC10 );
cond CrossC);
bi BarIndex();
//
AddColumnC"Close" );
AddColumnm"Mov Avg" );
AddColumncond"Condition");
AddColumnbi"BarIndex" );
AddColumnValueWhencondbi ), "ValueWhen( cond, BarIndex() )" );
AddColumnValueWhencondClose), "ValueWhen( cond, Close )" )

If you run above code you will clearly see how ValueWhen picks the value when condition is true and “holds” it for all other bars (when condition is false).

Debug using Exploration

Once you get this level of insight into your code you will be better equipped to fix any errors.
Exploration is number one choice in getting detailed view on what is happening inside your code.

For more information about Exploration see http://www.amibroker.com/guide/h_exploration.html

« Previous PageNext Page »