June 27, 2008 14:22

__THIS IS A BETA VERSION OF THE SOFTWARE. EXPECT BUGS !!!__

**Backup your data files and entire AmiBroker folder
first!
**

**INSTALLATION INSTRUCTIONS**

**IMPORTANT: This archive is update-only. You have to install full version
5.10 first.**

Just run the installer and follow the instructions.

Then run AmiBroker. You should see "AmiBroker 5.13.0 beta" written in the About box.

**See CHANGE LOG below for detailed list of changes.**

**CHANGE LOG**

**CHANGES FOR VERSION 5.13.0 (as compared to 5.12.2)**

- Main app window maximize state on 2nd monitor is saved OK now.

- Trendlines drawn in smaller interval (such as 1-minute), was moved one
bar in higher interval (such as 5 minute) if compressed intraday timestamps
ware set to START time of interval or FIRST tick

- The left-hand handle moved to the next bar when it was off-screen and right-hand
handle was adjusted by the user. Now it is fixed. (FC#890)

- When drawing is clicked without moving the mouse, the co-ordinates stay
untouched ( snap is not activated until you move the mouse)

- AFL: AddSummaryRows changed

AddSummaryRows( flags, format = 0, onlycols = 0, ...)AddSummaryRows automatically adds "summary" row(s) to the exploration output.

the flags parameter can be combination of the following

1 - add TOTAL row

2 - add AVERAGE row

4 - add MIN row

8 - add MAX row

16 - add COUNT rowformat - defines the numeric formating in WriteVal style so 1.2 for example means 2 decimal digits.

If default value of zero is used (or parameter not specified) the default formatting of "maximum precision"

is used - upto 15 digits are printedonlycols - defines for which columns you want to display summary row values.

Note that if you do not specify any columns - ALL will be printed.If you are using onlycols, you can define upto 10 columns, columns, like

in SetSortColumns are numbered starting from 1. For example:AddSummaryRows( 1, 1.2, 3, 5, 7, 9 );

Display sum for columns: 3, 5, 7, and 9.

Generally you should call this funciton only once, using combination of flags desired.

But it is possible to call AddSummaryRows multiple times and the result will be "accumulation" (i.e. bitwise OR)

in case of "flag" parameter. format and onlycols are always overwritten by last call.Example:

Filter=1;

AddColumn(V, "Volume" );

AddSummaryRows( 31, 1.2 ); // add Total, Average, Min, Max, and Count rows (1+2+4+8+16)=31 - with two decimal places

summary rows are added at the top of the list

- Handles from selected study line are respected (i.e when study is already
selected, and multiple studies exists under "click" point, the
selection does not change) (FC#726)

- Magnet mode can be asynchronously (temporarily) toggled by holding SHIFT
key while drawing. Toggle means that if you are in magnet mode and hold down
shift it will turn it off and vice versa

- Magnet mode implemented for horizontal price levels and working correctly
now (stay horizontal) (FC#728)

- During optimization Progress bar shows engine ID, opt. target, best value,
step and best combination of parameters found upto "now". These
figures are refreshed every second.

- Optimization (in non-WF mode) results
are sorted by optimization target column (too)

- New optimization engine added:
CMAE (Covariance Matrix Adaptation Evolutionary Strategy) optimizer plug-in

CMA-ES (Covariance Matrix Adaptation Evolutionary Strategy) is state-of-the-art non-exhaustive optimizer.

For scientific background see:

http://www.bionik.tu-berlin.de/user/niko/cmaesintro.html

According to scientific benchmarks outperforms nine other, most popular evolutionary strategies (like PSO, Genetic and Differential evolution).

http://www.bionik.tu-berlin.de/user/niko/cec2005.htmlThe CMAE.DLL plugin implements "Global" variant of search with several restarts with increasing population size

CMAE.DLL comes with FULL SOURCE CODE (inside "ADK" folder)By default number of runs (or restarts) is set to 5.

It is advised to leave the default number of restarts.

You may vary it using OptimizerSetOption("Runs", N ) call, where N should be in range 1..10.

Specifying more than 10 runs is not recommended, although possible.

Note that each run uses TWICE the size of population of previous run so it grows exponentially.

Therefore with 10 runs you end up with population 2^10 greater (1024 times) than the first run.There is another parameter "MaxEval". The default value is ZERO which means that plugin will automatically calculate MaxEval required. It is advised to NOT to define MaxEval by yourself as default works fine.

The algorithm is smart enough to minimize the number of evaluations required and it converges VERY fast to solution point, so usually it finds solutions way faster than other strategies.

It is normal that the plugin will skip some evaluations steps, if it detects that solution was found, therefore you should not be surprised that optimization progress bar may move very fast at some points. The plugin also has ability to increase number of steps over initially estimated value if it is needed to find the solution. Due to its adaptive nature, the "estimated time left" and/or "number of steps" displayed by the progress dialog is only "best guess at the time" and may vary during optimization course.

To use CMA-ES optimizer, you just need to add one line to your code:

OptimizerSetEngine("cmae");

This will run the optimization with default settings which are fine for most cases.

It should be noted, as it is the case with many continouos-space search algorithms, that decreasing "step" parameter in Optimize() funciton calls does not significantly affect optimization times. The only thing that matters is the problem "dimension", i.e. the number of different parameters (number of optimize function calls). The number of "steps" per parameter can be set without affecting the optimization time, so use the finest resolution you want. In theory the algorithm should be able to find solution in at most 900*(N+3)*(N+3) backtests where "N" is the dimension. In practice it converges a LOT faster. For example the solution in 3 (N=3) dimensional parameter space (say 100*100*100 = 1 million exhaustive steps) can be found in as few as 500-900 CMA-ES steps.

- New
optimization engine added: "Tribes" adaptive PSO optimizer
implemented

Tribes is adaptive, parameter-less version of PSO (particle swarm optimization) non-exhaustive optimizer.

For scientific background see:

http://www.particleswarm.info/Tribes_2006_Cooren.pdfIn theory it should perform better than regular PSO, because it can automatically adjust the swarm sizes and algorithm strategy to the problem being solved.

Practice shows that its performance is quite similar to PSO.

To find solutions significantly faster I can recommend CMA-ES (Covariance Matrix Adaptation Evolutionary Strategy) algorithm instead.

The Tribes.DLL plugin implements "Tribes-D" (i.e. dimensionless) variant. Based on http://clerc.maurice.free.fr/pso/Tribes/TRIBES-D.zip by Maurice Clerc. Original source codes used with permission from the author

Tribes.DLL comes with FULL SOURCE CODE (inside "ADK" folder)

Supported parameters:

" MaxEval" - maximum number of evaluations (backtests) per run (default = 1000).

OptimizerSetOption("MaxEval", 1000 );

You should increase the number of evaluations with increasing number of dimensions (number of optimization params).

The default 1000 is good for 2 or maximum 3 dimensions."Runs" - number of runs (restarts). (default = 5 )

You can leave the number of runs at default value of 5.By default number of runs (or restarts) is set to 5.

To use Tribes optimizer, you just need to add one line to your code:OptimizerSetEngine("trib");

OptimizerSetOption("MaxEval", 5000 ); // 5000 evaluations max

**CHANGES FOR VERSION 5.12.2 (as compared to 5.12.0)**

- In 5.12.0 BETA the walk-forward test was stopping after first in-sample cycle. Now it is fixed.
- AddSummaryRows total did not include the very first item. Fixed.

**CHANGES FOR VERSION 5.12.0 (as compared to 5.11.1)**

- ADK: A new interface for external optimization
engines, see optimizer.html

- Added two example non-exhaustive optimizers: Standard Particle Swarm
Optimizer (spso), and Monte Carlo (random pick) optimizer( moca)

Note that BOTH optimizers are provided for demonstration purposes.

Particularly Monte Carlo optimizer is meant as a "the most trivial and simple" or even "dumb" example of optimizer DLL coding.

It works by randomly picking parameter combinations without ANY additional logic. The results are thus random

and most probably sub-optimum.On the other hand Standard Particle Swarm Optimizer is based on SPSO2007 code that is supposed to produce

good results provided that correct parameters (i.e. Runs, MaxEval) are provided for particular problem.

Picking correct options for the PSO optimizer can be tricky therefore results may significantly vary from case to case.The source codes for both optimizers are OPEN and provided as illustration how to implement optimizer engines using

" simple" and "advanced" methods as described in optimizers.html file. You can find full source codes inside "ADK" subfolder.In the future, I will provide more robust non-exhaustive optimizers using various methods.

Example code for Standard Particle Swarm Optimizer:

(finding optimum value in 1000 tests within search space of 10000 combinations)OptimizerSetEngine("spso");

OptimizerSetOption("Runs", 1 );

OptimizerSetOption("MaxEval", 1000 );sl = Optimize("s", 26, 1, 100, 1 );

fa = Optimize("f", 12, 1, 100, 1 );Buy = Cross( MACD( fa, sl ), 0 );

Sell = Cross( 0, MACD( fa, sl ) );Example Code for Monte Carlo optimizer:

(finding sub-optimum value in 1000 test within search space of 10000 combinations)OptimizerSetEngine("moca");

OptimizerSetOption("NumSteps", 1000 );sl = Optimize("s", 26, 1, 100, 1 );

fa = Optimize("f", 12, 1, 100, 1 );Buy = Cross( MACD( fa, sl ), 0 );

Sell = Cross( 0, MACD( fa, sl ) ); - Increased limit of optimization parameters to 100
- AFL: new OptimizerSetOption("name", value ) function

The function set additional parameters for external optimization engine. The parameters are engine-dependent.

For example SPSO optimizer supports "Runs" (number of runs) and "MaxEval" (maximum evaluations (tests)per single run) parameters. Monte Carlo optimizer supports "NumSteps" (number of steps) parameter.

- AFL: new OptimizerSetEngine("name") function

The function selects external optimization engine defined by name. For demonstration two engines are provided: Standard Particle Swarm Optimizer ("spso") and Monte Carlo (random pick) optimizer ("moca")

Example:

OptimizerSetEngine("moca");

- ADK: new example C++ source codes: PSOSample, MOCASample
- AFL: AddSummaryRows( flags )

AddSummaryRows automatically adds "summary" row(s) to the exploration output.

the flag parameter can be combination of the following

1 - add TOTAL row

2 - add AVERAGE row

4 - add MIN row

8 - add MAX row

16 - add COUNT rowYou can call AddSummaryRows multiple times and the result will be "accumulation" (i.e. bitwise OR)

Example:

Filter=1;

AddColumn(V, "Volume" );

AddSummaryRows( 31 ); // add Total, Average, Min, Max, and Count rows (1+2+4+8+16)=31summary rows are added at the top of the list

**CHANGES FOR VERSION 5.11.1 (as compared to 5.11.0)**

- Fixed problem with Walk Forward picking sometimes not the best parameter when
thousand separator was used and metric values were greater than 1000

**CHANGES FOR VERSION 5.11.0 (as compared to 5.10.1)**

**Backtester: Implemented SeparateLongShortRank**

To enable separate long/short ranking use:

SetOption("SeparateLongShortRank", True );When separate long/short ranking is enabled, the backtester maintains TWO separate "top-ranked" signal lists, one

for long signals and one for short signals. This ensures that long and short candidates are independently even if position score

is not symetrical (for example when long candidates have very high positive scores while short candidates have only fractional negative scores).

That contrasts with the default mode where only absolute value of position score matters, therefore one side (long/short) may completely dominate ranking if score values are asymetrical.When SeparateLongShortRank is enabled, in the second phase of backtest, two separate ranking lists are interleaved to form final signal list by

first taking top ranked long, then top ranked short, then 2nd top ranked long, then 2nd top ranked short, then 3rd top ranked long

and 3rd top ranked short, and so on... (as long as signals exist in BOTH long/short lists, if there is no more signals of given kind, then

remaining signals from either long or short lists are appended)For example:

Entry signals(score):ESRX=Buy(60.93), GILD=Short(-47.56), CELG=Buy(57.68), MRVL=Short(-10.75), ADBE=Buy(34.75), VRTX=Buy(15.55), SIRI=Buy(2.79),As you can see Short signals get interleaved between Long signals even though their absolute values of scores are smaller than corresponding scores of long signals. Also there were only 2 short signals for that particular bar so, the rest of the list shows long signals in order of position score

Although this feature can be used independently, it is intended to be used in combination with MaxOpenLong and MaxOpenShort options.

**Backtester: MaxOpenLong/MaxOpenShort implemented**

MaxOpenLong - limits the number of LONG positions that can be open simultaneously

MaxOpenShort - limits the number of SHORT positions that can be open simultaneouslyExample:

SetOption("MaxOpenPositions", 15 );

SetOption("MaxOpenLong", 11 );

SetOption("MaxOpenShort", 7 );The value of ZERO (default) means NO LIMIT. If both MaxOpenLong and MaxOpenShort are set to zero (

or not defined at all) the backtester works old way - there is only global limit active (MaxOpenPositions) regardless of type of trade.Note that these limits are independent from global limit (MaxOpenPositions).

This means that MaxOpenLong + MaxOpenShort may or may not be equal to MaxOpenPositions.If MaxOpenLong + MaxOpenShort is greater than MaxOpenPositions

then total number of positions allowed will not exceed MaxOpenPositions, and individual long/short limits will apply too.

For example if your system MaxOpenLong is set to 7 and maxOpenShort is set to 7 and MaxOpenPositions is set to 10

and your system generated 20 signals: 9 long (highest ranked) and 11 short, it will open 7 long and 3 shorts.If MaxOpenLong + MaxOpenShort is smaller than MaxOpenPositions (but greater than zero), the system won't be able to

open more than (MaxOpenLong+MaxOpenShort).Please also note that MaxOpenLong and MaxOpenShort only cap the number of open positions of given type (long/short).

They do NOT affect the way ranking is made. I.e. by default ranking is performed using ABSOLUTE value of positionscore.If your position score is NOT symetrical, this may mean that you are not getting desired top-ranked signals from one side.

Therefore, to fully utilise MaxOpenLong and MaxOpenShort in rotational balanced ("market neutral") long/short systems

it is desired to perform SEPARATE ranking for long signals and short signals.

To enable separate long/short ranking use:

SetOption("SeparateLongShortRank", True );

**Added ability to running Walk forward test from OLE, using Optimize(3)**

Analysis.Optimize( mode )

when mode == 3 it runs walk forward testAB = new ActiveXObject("Broker.Application");

AA = AB.Analysis;

AA.Optimize(3);

**AFL: DaysSince1900() function**

It returns the number of days that passed since January 1st, 1900, counting from 1. January 1, 1900 is serial number 1, and January 1, 2008 is serial number 39448 because it is 39,448 days after January 1, 1900. Technically is equal to Windows OLEDATE and Excel's DATEVALUE function.

The function can be used for calculations that involve calendar days as opposed to trading days and replaces previously proposed AFL solution

http://www.amibroker.com/kb/2007/03/15/calendar-day-index/Now RefDays can be implemeted as follows:

SetBarsRequired( 365, 0 );

**function**RefDays( Array, Days )

{

td = DaysSince1900();

result =**Null**;

**if**( Days < 0 )

{

**for**( i =**BarCount**-1; i >= -Days; i = i - 1 )

{

backday = td[ i ] + Days; // Days is negative

**for**( j = -Days/2; j < i; j++ )

{

**if**( td[ i - j ] <= backday )

{

result[ i ] = Array[ i - j ];

**break**;

}

}

}

}

**return**result;

}

Plot(**C**, "C",**colorRed**);

Plot( Ref(**C**, -252 ), "Close 252 bars back",**colorBlue**);

Plot( RefDays(**C**, -365 ), "Close 365 days back",**colorGreen**);

**HOW TO REPORT BUGS**

If you experience any problem with this beta version please send detailed description of the problem (especially the steps needed to reproduce it) to support at amibroker.com