Performance tuning tips
AmiBroker is one of the fastest (if not the fastest) technical analysis program,
still the user by applying incorrect settings, poor formula coding and other
sub-optimum choices can significantly slow it down. This short chapter we will
give few hints on how to make program perform as it was originally designed
There are three areas of performance tuning:
- Operating system / machine level
- AmiBroker settings level
- Formula coding
On operating system level you should do:
- Avoid installing unnecessary 3rd party programs
Specifically avoid all kinds of "memory turbo", "windows optimizers", "windows
cleaners" and such tools as they bring just more load to the CPU and system
resources and provide zero or virtually zero benefit. What is worse, they
could affect normal operation of Windows and cause compatibility problems
- Turn OFF virus checking for DATA files
Many antiviruses turn on so called "live" protection on all files on all
disks. This will bring system performance to its knees. Why? Simply because
anti-virus will intercept each and every file access and run its extensive
checking on each data access. This could result in drastic slow down of file
accesses to the extent that Windows boots 3 times slower than normal.
See the following article
shows some real measurements on impact of anti-virus.
While I am not saying to turn off antivirus completely, it is strongly
recommended to turn OFF anti-virus checking on DATA
FILES, specifically on all data files inside AmiBroker folder. Generally
speaking there are two kind of files: executable (.exe, .dll, .ocx) and data
files (.txt, .html, .wav, .jpg, .gif, .chm, etc...) . Checking data files for
viruses makes actually very little or no sense at all because they don't contain
therefore can not really be the source of infection. As long as file does not
include any executable part - it is safe. Note however that some data files
(such as Excel .xls, can contain macros and macros _are_ executable and can
be infected). But as far as AmiBroker goes, anything inside AmiBroker directory
that does not have .exe, .dll, or .js extension is non-executable
- Choose anti-virus wisely
As pointed out above, some anti-virus products can be really resource hogs
and you should check the performance tests (see link above) before deciding
what antivirus to use.
On AmiBroker settings level
- Avoid unnecesarily large "Number of bars" in File->Database Settings
This is the most often mistake people make. For example I have seen 50000
entered into "Number of bars" entered for end-of-day database. This does
not make sense because 50000 daily bars is 192 years!!! There is no data
source that offers such history and entering that amount is waste of memory.
You need to keep in mind that what ever you enter here forces AmiBroker to
bars. Each data bar is 40 bytes. So if you enter 100000 bars, you will force
AmiBroker to actually allocate 4MB of memory *per symbol*. It may not sound
too much but AmiBroker keeps a cache of recently used data and if you have
defined the cache that is 500 symbols large you will be able to hit 2GB
of memory that way. So always look at what File->Database Settings dialog
displays next to "Number of bars" field. It will display the number of days
equivalent to entered number of bars and choosen base time interval. Make
sure that you only enter resonable values and be aware that whatever you
enter here has performance consequences. The consequences do not end with
memory consumption alone, but also in speed. Processing more bars means more
CPU time, especially considering the fact that while small amounts of data
can be kept in on-chip CPU cache, large amounts can not. Now considering
that on-chip cache is usually 10 (ten) times faster than regular memory you
immediately realize that specifying too much bars here result in performance
drop. Again: one bar is 40 bytes. For best performance, make sure that you
don't exceed CPU on-chip cache size, so if your CPU has 4MB cache, for best
performance it is strongly advised not to use more than 100000 bars in File->Database
- Decrease the size of in-memory cache, if you are using very large databases
(>2GB in disk size)
The "in-memory cache" size defined in Tools->Preferences, "Data" tab controls
the number of symbols in the cache and maximum amount of memory consumed
by the cache. While larger cache generally speeds up processing because data
do not need to be read from disk and written back all the time, you may run
out of memory on certain situations when your cache is too large, especially
on 32-bit operating system that limits the available virtual memory per process
to 2GB. To avoid running out of memory, go to Tools->Preferences, "Data"
and decrease the size of in-memory cache. To set it to minimum, enter "11"
(eleven) into "in-memory cache (max. symbols)".
IMPORTANT: It is advised NOT to decrease the cache when your databases are
relatively small (few hundred MB) as larger cache will speed up the access.
Large cache is good thing as long as you do not run out of memory.
On Formula coding level
Poor formula coding is the foremost reason of slow down. People coming from
"other" languages often do not realize the full potential of AFL array processing
and code everything "old" style (i.e. with loops). Loops can be 10..50 times
slower than equivalent array-based code. So for best speed you should avoid
at all replacing them with array processing, or at least make looping code
as short as possible.
Consider the following code:
SetBarsRequired( sbrAll );
for( i = 0;
med[ i ] = (H[ i
] + L[ i ])/2;
"Loop time: "+GetPerformanceCounter(1);
med = ( H + L )/2;
"Array time: "+GetPerformanceCounter(1);
When running this code on 350000 bars the loop version it takes 100ms (0.1s)
to execute loop version and only 2ms (0.002ms) to execute array version (so array
code is 50 times faster than looping). The difference is not so significant with
less bars, but still with even 300 bars, loop requires 0.1ms and array needs
0.01ms so it is 10 times faster.
So there are few guidelines for AFL coding:
- Use array processing instead of loops wherever possible. To learn more
about array processing, please read the "Understanding
the AFL" tutorial
- Inside loops use scalars only or refer to individual array elements using
 subscript operator
- Array operations and all loop invariant code should be moved outside of
loops whenever possible to avoid repeated evaluation of the same function
Consider this code:
i = 0; i < BarCount;
i++ ) The problem with the above code is that it repeats the same
array-base moving average calculation with exactly same input over and over
again (BarCount-times). The code like this is loop-invariant (does not depend
on loop counter and does not have any input that is calculated depending on
loop counter, so it can be calculated ONCE and as such should be moved outside
(before) the loop, as shown below:
x = MA( C, 10 ); //
WRONG ! loop invariant code is called over and over ! It should be OUTSIDE
y[ i ] = C[
i ]/x[ i ];
x = MA( C, 10 ); //
CORRECT ! loop invariant code located OUTSIDE of loop
for( i = 0;
i < BarCount;
y[ i ] = C[ i
]/x[ i ];
- Use the AFL Formula Editor, Tools->Code check & Profile to
find out which functions are called how many times and which ones take the
most time. Start your code
tuning with functions that are called the most often. Check if they are loop
invariants and if so, move them OUTSIDE the loop.
- If you need to store array data in one formula and read it back in another,
and were using AddToComposite/Foreign before, consider using array static
instead because static variables work faster. Note that static variable life
is limited to program run lifetime (they are not persistent as AddToComposite
tickers are), unless you set Persistent parameter to True
In order to help you in real-time monitoring of program performance, AmiBroker
provides two tools. First is Performance monitor window,
second is Performance indicator that is located on the right most
side of the AmiBroker status bar.
The status bar performance indicator shows:
- for real time databases: percentage
load factor indicator
- for off-line
intraday and eod databases: free virtual memory
is a percentage value that shows relative 'snappiness' of the program.
The load factor is calculated as (total chart refresh time in milliseconds)/2
(total data access time in milliseconds)/2 +
(free virtual memory below 20% of total memory available).
So it will reach 100% if any of situations listed below happen:
- total chart
refresh time is higher than 200ms
- total data access time is higher than
- free virtual memory drops below 10% of total memory (or combination of
Total chart refresh time is a sum of times needed to redraw completely all
charts displayed on screen,
it includes AFL execution time for each chart pane and GDI (graphics) output
(It does not include data access)
Total data access time is sum of times required to access fresh data via plugin
for all displayed
symbols plus time required to apply time filtering, and time compression from
interval to displayed interval.
Plug-in time per symbol is time spent in plug-in GetQuotes call per *single*
If you display 10 symbol charts at once AmiBroker will call GetQuotes 10 times
this time gets multiplied by number of symbols displayed (this total plugin
time *is* included in total data access time figure - listed above)
If Plug-in time per symbol exceeds 10ms it means that plugin is slow (or does
not use new ADK 2.0),
if this is the case you should contact plugin vendor to get updated plugin
that uses ADK 2.0.
Recommended is to keep this load factor below 100%. When load factor is
100% AmiBroker is able to keep updating
all charts in real-time (more frequently than 5 times per second) and maintain
responsive and smooth user-interface.
With load factor of 200% AMiBroker is still able to keep updating all charts
as frequenty as 2.5 times per second,
but user interface reaction time may be impaired a bit. Keeping load at 100%
or less is recommended.
200% is maximum value that allows more or less "normal" operation.
When load factor rises above 100% the warning tooltip will pop up once informing
what is the reason of poor performance
When load factor rises above 300% the above tooltip will reappear every minute.
AmiBroker will continue working with loads even above 1000% however the performance
will be bad (one update per 5 seconds or more).
For more information about optimum use of multithreading and other general
remarks regarding performance see Efficient
Use of Multithreading.