Selecting a tick data source

Modified on Wed, 01 Feb 2023 at 03:23 PM

The Tick Data Suite provides several tick data sources so that each user can select the one that is best suited for their purposes.


There is no such thing as "the best tick data source". The source that is selected by default in the Tick Data Suite is Dukascopy because of the variety of symbols available, good date coverage, clean data (no gaps/spikes) and the fact that Dukascopy is very diligent with the upload process. The Dukascopy data doesn't have the best spreads nor the best tick density but it is generally great as a starting point.


If you are wondering which data source is best for you, we can suggest the following attributes for the evaluation criteria:

  • Symbol availability
  • Average spread
  • Average number of ticks per minute
  • Date range available
  • Gaps
  • Price spikes


You can use these to create a personalized formula that should help you determine which source is best suited for you by quantifying their worth and allowing a comparison.

We will provide some brief suggestions on how to deal with each evaluation criterion.



Symbol availability


This would be the most straightforward criterion - if a data source does not have the symbol(s) you need, simply exclude it.



Average spread


It might make sense to select a tick data source that has average spreads that are close to your broker's average spreads. 

Naturally, in order to do that you must measure the average spreads for the selected tick data source as well as the average spreads for your broker for the same time period and compare them.


The average spread is arguably not a critically important criterion because the tick data spread can be adjusted as described in the support article Using variable spread - you will also find an EA attached to this article which can be used to measure spread averages.



Average number of ticks per minute


This is perhaps one of the most defining attributes for a tick data source as it is immutable.


You can find out the average number of ticks per minute for your live account by exporting the M1 history for your target symbol from the history center (in MT4 Tools -> History center or just press F2) then using Excel or some similar software to compute the average Volume for your desired time period (for example, the previous 4 weeks). If you didn't already know, the MT4 concept of volume in the bar data refers to the number of ticks that occurred within that bar.


In order to compute a similar average number of ticks per minute for the tick data, open the Tick Data Manager, select the desired data source, click the ... button next to the target symbol in order to open the Symbol operations dialog, navigate to the Export bars tab, configure the Start and End dates to match the same time period used for the live account (the previous 4 weeks in the example above), set the Format to Metatrader, select only the M1 timeframe and start the export. Once the export is finished, use Excel or some other spreadsheet software to determine the average as before.


Naturally, sources with an average number of ticks per minute that is close to that of your live account should be preferred.


You can employ variations of this technique such as only computing the average number of ticks per minute during the NY session if that's when your EA trades.



Date range available


This is also a criterion that is easy to filter by: if the tick data source doesn't have data for your symbol(s) of interest going as far back as you would like, just exclude it from the potential set of sources.



Gaps


A gap occurs when the source failed to record tick data for some time. This can be as short as a few minutes or as long as a few days.


One way to check for gaps is by using the Remove data dialog that can be opened by clicking the button with the same name located in the Manage tab of the Symbol operations dialog (... next to the symbol in Tick Data Manager). However, this dialog does not provide an overview and it only takes into account gaps that exceed one hour so it may not be the best solution.


A good alternative is the data integrity checker EA by FxBlue which you can backtest in order to get an overview. It is available at FX Blue Data Integrity Checker - forex trading and services (it requires registration which is free).


Naturally, it is ideal for the data to have as few gaps as possible but in the grand scheme of things, when you're backtesting an EA over several years of data a few gaps of some hours typically won't have a big impact on the outcome.



Price spikes


A price spike occurs when the price jumps up or down to a value that is clearly incorrect and that should not have been tradable. For example, the EURUSD price jumping from 1.23456 to 1.56789 from one tick to the next and then back to the 1.2 range would be a price spike upwards.


You can check for spikes by configuring the Spike filter in the Settings dialog in Tick Data Manager and inspecting the download logs for messages related to filtered spikes. Note that the setting does not apply retroactively but rather to new downloads only.


You could also check for spikes by running a visual backtest using the tick data from your target source - this way you can visually inspect the data for spikes by looking at the chart bars.


However, neither of the above is very efficient. The same data integrity checker EA mentioned above (under Gaps) can also detect such spikes in the data and report them.


As opposed to gaps, price spikes can have a huge impact on your backtests. It can happen that an EA opens or closes a position within a spike which can result in abnormally large wins or losses.


Price spikes can be filtered using the Tick Data Suite, but this requires tweaking the price spike filter and downloading the affected data once more.