amibroker

HomeKnowledge Base

How to write to single shared file in multi-threaded scenario

The problem is as follows: during multiple-symbol Scan (or any other multi-threaded Analysis operation) we want to create a single, shared file and append content generated from multiple symbols to it.

There are two things that we must consider if we are running in multiple treaded scenario.
1. If we want to get just single-run results, before appending content to the file, we need first to delete file generated in previous runs.

2. We have to take care to open the file in share-aware mode so multiple threads do not write at the same time (preventing corruption).

A sample formula is presented below.

// our scanning code
Buy CrossMACD(), Signal() );

filepath "C:\\ScanExport.txt";

if( 
Status("stocknum") == )
{
   
// delete previous file before anything else
   
fdeletefilepath );
}

// open file in "share-aware" append mode
fh fopenfilepath"a"True );

// proceed if file handle is correct
if ( fh )
{
   
lastbuyDT =  LastValueValueWhenBuyDateTime() ) ) ;

   
// write to file
   
fputsName() +", Last Buy: " DateTimeToStrlastBuyDT ) +"\n"fh );

   
// close file handle
   
fclosefh );
}
else
{
  
_TRACE("Failed to open the file");

One important thing to remember is that in multi-threaded environment threads execute independently and there is no guarantee they will all execute sequentially, so the order of items (symbols) in the file may not be alphabetical.

If we want strictly sequential execution, then we must limit ourselves to just running in single-thread. A single-thread execution in New Analysis window can be achieved by placing the following pragma call at the top of the formula.

#pragma maxthreads 

#pragma maxthreads limits the number of parallel threads used by New Analysis window. This command is available in AmiBroker version 6 or higher.

How to run certain piece of code only once

There are situations where we may need to run certain code components just once, e.g. to initialize some static variables before auto-trading execution or perform some tasks (such as ranking) at the very beginning of backtest or exploration. The following techniques may be useful in such cases:

When we want to execute certain part of code just once after starting AmiBroker, we may use a flag written to a static variable that would indicate if our initialization has been triggered or not.

if( NzStaticVarGet("InitializationDone") ) == )
{
   
StaticVarSet("InitializationDone"1);
   
// code for first execution

If we want to run certain part of code at the beginning of the test run in Analysis window, we can use:

if ( Status("stocknum") == )
{
   
// our code here

When Status(“stocknum”) is detected in the code, then execution is performed in a single thread for the very first symbol. Only after processing of this first symbol has finished the other threads will start.

A practical example showing use of this feature is presented in the following tutorial:

http://www.amibroker.com/guide/h_ranking.html

How to move a window to another monitor

By default document windows like Charts, Analysis, Account Manager, Web Research all open inside of main AmiBroker frame window. In multi-monitor setups it may be useful however to move some of them to another screen.

Let us say we want to move Analysis window to second monitor screen. This can be done by switching the window to special “Floating” mode by using Window->Floating option from the menu after opening Analysis.

Window menu

Using floating mode detaches Analysis from the main frame and then the window can be moved outside of it.

Window menu

Exactly the same procedure works for Chart windows, Account Manager or Web Research windows.

The procedure (for chart window) has also been shown in this video:
http://www.amibroker.com/video/FloatAndLink.html

Timestamps explained

When AmiBroker is fed with the data, say 1-minute data, it can create all other time intervals by compressing source data on-the-fly. So if you display say 13-minute chart, AmiBroker takes source 1-minute data and builds 13-minute blocks of data to create 13-minute bars. For this process to work correctly, source data need to have timestamps that point to the START of each bar interval. So with 1-minute data, the bar that has a timestamp 9:30:00 is supposed to cover trades from the period of 9:30:00.000 upto 9:30:59.999. All our data plugins provide data in that format.

Now, provided that we have say 1-minute data, AmiBroker can compress data to any other N-minute interval. When doing so, it can assign timestamps to compressed bars in different ways. This can be controlled through Tools->Preferences->Intraday dialog.

Timestamps

Let us check it on an example of a 5-minute bar based on input 1-minute quotes for e-mini contract.

Timestamps

As explained in the manual (http://www.amibroker.com/guide/w_preferences.html) – there are four choices available:

  1. Time of FIRST tick inside bar – when selected the bar gets the time stamp of the very first trade inside given time slot (bar). With this choice the bar will be stamped with 9:30:00 because this is the first tick (quote) available within that 5-min period
  2. Time of the LAST tick inside bar – when selected the bar gets the time stamp of the very last trade inside given time slot (bar). In this case the bar will be stamped with 9:34:00 because this is the last quote available within that 5-min period
  3. START time of the interval – when selected the bar is time-stamped with start time of the time slot (bar). The bar will be stamped with 9:30:00 because that is a beginning of the selected time period.
    NOTE: This is recommended and the default setting as it provides consistency with how source bars are timestamped. It should not be changed unless you have really good reason to do so.

  4. END time of the interval – when selected the bar is time-stamped with start time of the time slot (bar). The bar will be stamped with 9:34:59 timestamp, because that’s the very end of this 5-min period.

There is also an additional setting available (Override: Weekly/monthly bars use day of last trade), which allow to modify the behaviour in case of Weekly/Monhtly bars, no matter what is the main setting we use. This allows us to e.g. use START time of interval to identify intraday quotes, however – on a weekly chart display e.g. Wednesday date (if that is most recent day in current week) or Friday date for complete weeks.

We need to remember that the timestamps identify the whole bar and all trades within that bar, so if we use START time of interval for time-stamping, in the backtest use Close array for as BuyPrice and 5-minute periodicity, then in our report we will see:

Timestamps

So, we see the time 9:30:00, but this bar refers to trading activity from period 9:30:00-9:34:59 and the actual price is read from the tick being the Close of the whole 5-minute period (at 9:34:00 in the table above).

For the same reason – when we use weekly data for backtesting, we trade at Open, but for time-stamps we use Override box (so weekly bars are stamped with the data of the last day within given week) – then in the report we will see e.g. Friday dates because of the fact that we use such approach to time-stamp bars. This does not really mean that trade happened on Friday, but only that we use Friday date to identify the whole Monday-to-Friday week.

Using optimum parameter values in backtesting

After Optimization process has found optimum values for parameters of our trading system, typically we want to use optimum values in subsequent backtesting or explorations. In order to achieve that, we need to manually update default_val (second) argument of Optimize function with the values obtained from the optimization report.

The arguments of Optimize function are shown below (note second parameter marked in dark red color – this is the default value parameter we will be changing after optimization run):

some_var = Optimize( "description", default_val, min_val , max_val, step );

Let us consider the following example formula used for optimization process:

SetOption("ExtraColumnsLocation"1);
periods Optimize"Periods"2550); // note that default value is 2
Level Optimize"Level"22150); // note that default value is 2

Buy CrossCCIperiods ), -Level );
Sell CrossLevelCCIperiods ) )

If we perform Optimization process and check the results (for this example we use Net Profit as the optimization target), we can see that the best results use Periods = 6 and Level = 126.

Optimization result

Now in order to run backtest and obtain exactly the same results as in the respective line of the above Optimization results, we need to enter the values into default argument, so the modified code will look like this:

SetOption("ExtraColumnsLocation"1);
periods Optimize"Periods"6550); // we changed default value to 6
Level Optimize"Level"1262150); // we changed default value to 126

Buy CrossCCIperiods ), -Level );
Sell CrossLevelCCIperiods ) )

Now we can use the code with modes other than Optimization and the formula will use optimized values we retrieved from the results.

Using multiple watchlists as a filter in the Analysis

The Filter window in the Analysis screen allows us to define a filter for symbols according to category assignments, for example watchlist members (or a result of mutliple criteria search).

The filter allows us to select one watch list for “inclusion” and one for “exclusion”. To include members of more than one watchlist, we can not simply pick them both in the Filter window – we need to combine these symbols together in another dedicated watchlist storing symbols from both lists.

Let us say we want to run a test on members of List 1 and List 2. To combine these watchlists together we need to follow the instructions below.

  1. Click on List 1, then in the bottom part of the Symbols window mark all tickers. A multiple selection is done by clicking on first and last item in the list while holding down the Shift key. We may also select all symbols by clicking on any symbol and pressing Ctrl+A key.

    Select symbols from watch list

  2. Now click on the selection with right mouse button and choose Watch list->Add selected symbol(s)

    Add symbols to watch list

  3. Pick an empty watchlist that we will use to combine our tickers (e.g. List 5 ) and confirm to add multiple symbols:

    Confirm adding multiple symbols

  4. Repeat the above steps 1-3 with List 2 members
  5. Now we can pick List 5 in the Filter window and run the test on all the tickers

    Create new watch list

An alternative solution to this is to filter out unwanted symbols in the code. In this case AmiBroker would need to run analysis for all tickers (so Apply to would need to be set to All symbols) and apply filtering while executing your formula. To do so you may use code like this for backtesting (filtering Buy signals):

Buy /* your regular trading rules here */;

watchlistCheck InWatchList) OR InWatchList);
Buy watchlistCheck  AND Buy// combine watch list filter with your rule

or code like this in exploration (adding extra condition to Filter variable):

Filter /* your regular exploration filter here */;
watchlistCheck InWatchList) OR InWatchList);
Filter watchlistCheck AND Filter// combine watch list filter with your rule

Please keep in mind that filtering in the code is significantly slower. Using this method AmiBroker needs to read the data for all tickers, prepare arrays, then evaluate the formula and verify the condition – so using Filter window and the first approach will be faster, as the filtering is done before the formula execution, saving lots of time required for data retrieval and AFL execution.

Text output in Explorations

Explorations allow to display not only numerical data but also text, there are however certain restrictions what can and can’t be displayed in the exploration result list as a text.

AddTextColumn() function allows to display strings, so we can use it for displaying e.g. full name of the symbol or category assignment information:

Filter 1;
AddTextColumnFullName(), "FullName");
AddTextColumnSectorID(1), "Sector");
AddTextColumnIndustryID(1), "Industry");
AddColumnClose"Close")

Exploration Text output

It is worthwhile to note that these strings displayed above do not vary across historical bars. That is important, because there is no such structure in AFL as an ‘array of strings’, therefore an attempt to generate a text, which varies on each bar will not work. Instead a string representing selected array value (or last value) will be displayed.

Let us check such a formula to illustrate the above statement:

condition Close Open;

Filter 1;
AddColumnOpen"Open" );
AddColumnClose"Close" );
AddColumncondition"Condition"1.0);

// WriteIf returns a SINGLE STRING representing condition at last bar of selected range
text WriteIfcondition"Close above Open""Close below Open" );
// the text variable represents value AT THE LAST BAR of selected range
AddTextColumntext"text" )

If we look at the output over more than one bar, then we can see that the condition from the last bar determines the text output in the column:

Exploration Text output

Therefore, such approach as above can only be used in situations where we run the exploration applied e.g. to 1-Recent bar, because it’s the last bar from the range which determines the text displayed in the column in such situation.

If you want to display the value for other bars than last bar of selected range, you need an extra column, like this:

condition Close Open;

Filter 1;
AddColumnOpen"Open" );
AddColumnClose"Close" );
AddColumncondition"Condition"1.0);

// WriteIf returns a SINGLE STRING representing condition at last bar of selected range
text WriteIfcondition"Close above Open""Close below Open" );
AddTextColumntext"Last bar text" ); 

// Note that we are now using Ref() function to reference previous bar data
text2 WriteIfRefcondition, -), "Close above Open""Close below Open" );
AddTextColumntext2"Previous bar text" )

You can use functions like Ref() or ValueWhen() to refer to other bar’s data, or you can use array subscript operator like this condition[ 1 ] to get value of condition at bar with index 1.

There is an alternative method to display values that change on bar by bar basis as letters though. Instead of displaying full string we can display single characters in a column using formatChar parameter, as shown in the code below:

Version5.90 ); // only works for version 5.90 and above
Buy CrossMACD(), Signal() );
Sell CrossSignal(), MACD() );

Filter Buy OR Sell;
AddColumnIIfBuy'B''S' ), "Signal"formatChar )

Exploration Text output

Note: If you are using version older than 5.90, you need to use Asc function instead of single-character literals, as shown below:

Buy Cross(MACD(),Signal());
Sell Cross(Signal(),MACD());

Filter Buy OR Sell;
AddColumnIIfBuyAsc("B"), Asc("S")), "Signal"formatChar )

More information about explorations can be found in the manual:
http://www.amibroker.com/guide/h_exploration.html

How to copy backtest trade list to a spreadsheet

There are several ways to transfer the backtest results to a spreadsheet.

  1. Immediately after the test we can just click on the results list with right mouse button and choose Copy from the menu. It is also possible to click on the results and use Ctrl+C key shortcut.

    Copy Trade List

    The operation will copy the entire list, so there is no need to select all rows manually.

  2. After the test, we can also use File->Export option from the main program menu to export the results list to a CSV or HTML file, which could be opened from Excel later on.

    Export Trade List

  3. Backtest results are also accessible through the Report Explorer:

    Backtest Report Explorer

    In order to open detailed report for the particular test it is enough to double-click on the selected line. Then, after we navigate to Trade List page, to copy the results, the best option to use is Edit->Copy Table

    Copy Table

    Unlike the regular Copy option, Copy Table transforms HTML tables into CSV format and copies it into clipboard so tables can be pasted easily to Excel. Also it divides Entry/Exit columns into separate Entry/exit date/price columns.

How to backtest symbols individually

By default, when we run backtest over a group or watchlist of symbols – AmiBroker will perform a portfolio test. However, there is also an Individual mode of the backtest available, where every symbol is tested individually and independently.

Once we send the formula to Analysis window and define group of symbols to run code on (Apply To), in order to run an individual backtest, it is necessary to unfold the menu next to Backtest button and choose Individual Backtest from the menu.

Individual Backtest

To get full report generated for each of the tests, it is required to first go to Analysis–>Settings->Report tab and mark Generate detailed reports for each symbol in individual backtests option.

Individual Backtest Report

Then the full reports can be accessed through the Report Explorer.

Report Explorer

The letter I indicates that the report contains results of an individual test. Double-clicking on the particular results line will show full contents of the backtest report.

Report Explorer List

How to display correlation between symbols

For the purpose of calculating the correlation between two data-arrays, there is a Correlation() function in AFL which can be used.

In order to display a correlation chart, please select Analysis–>Formula Editor menu and enter the following code:

SetChartOptions0chartShowArrows|chartShowDates);
Ticker ParamStr"Symbol"Name() );
range Param"Periods"25225001);

corr CorrelationCloseForeignTicker"C"), range );
Plotcorr"Correlation " Name() + "/" ticker,
      
ParamColor"Color"colorRed ), ParamStyle"Style"styleLine ) );

// check if different symbols are used
if( ticker == Name() )
    
Title "Please select different symbol from Parameter dialog"

Now select Tools->Apply Indicator. Initially the code will pick the selected symbol’s Close prices for both arrays, so we either need to change the selected ticker in the chart or the second symbol, which can be defined in Parameters dialog.

We can also use Exploration feature to display a correlation matrix e.g. for the watchlist members. The below example shows the process for Watchlist 0 members.

The formula to display correlation table looks as follows:
// read the list of symbols from Watchlist 0
symlist CategoryGetSymbolscategoryWatchlist);

// display only last bar from the Analysis range
Filter Status"lastbarinrange" );

// iterate through symbols
for ( 0; ( sym StrExtractsymlist) ) != ""i++ )
{
    
// calculate correlation over 252 bars
    
Corr CorrelationCForeignsym"C" ), 252 );

    
// set color dynamically based on correlation values
    // and display the output in exploration column
    
Clr 32 SelectedValueCorr ) * 32;
    
AddColumnCorrsym1.2,
               
ColorHSB128 Clr255255 ),
               
ColorHSBClr255255 ) );
}

SetSortColumns)

To use the formula we need to do the following:

  1. assign some symbols to watchlist 0
  2. select Analysis->Formula Editor menu
  3. in the AFL Editor enter the code listed above
  4. select Tools->Send to Analysis menu
  5. in the Analysis window, select Apply to: Filter (in Include tab hit Clear and pick watchlist 0)

    Filter dialog

  6. select Range: 1 Recent bar (in case of longer range, last bar of the range will be used for output)
  7. press Explore button

Here is a sample output table:

Correlation Matrix

Be careful and try not to put 10000 items in the watch list because it would need to create a table with 10K columns. Windows has some limits on pixel width of the list view and it would truncate display when the display width (scrollable area inside list) exceeds 32767 pixels. That makes it practical only to display matrices of not more than about 1000-2000 columns.

Next Page »