RoboMiner Pro Robot Review - LIVE ACCOUNT

[Hyperdimension]

Originally Posted by PipCop

HyperDimension, if you are backtesting with the default settings, then you have FIFO = true. That means all trades will close at exactly the same time. This is to allow customers to use RM on NFA compliant brokerages.

Ok that's what is causing it. I wonder how the results would be with FIFO = false.

Here's an excerpt from the Strategy Tester results that show 30 positions built up over a 4 month period and then finally close at once in a flash:

615    2008.04.15 11:12    sell    308    0.10    1.17740    0.00000    0.00000    0.00    35571.50
616    2008.04.16 08:36    sell    309    0.10    1.18142    0.00000    0.00000    0.00    35571.50
617    2008.04.16 12:19    sell    310    0.10    1.18540    0.00000    0.00000    0.00    35571.50
618    2008.04.16 17:36    sell    311    0.10    1.18942    0.00000    0.00000    0.00    35571.50
619    2008.04.24 00:15    sell    312    0.10    1.19340    0.00000    0.00000    0.00    35571.50
620    2008.04.25 10:02    sell    313    0.10    1.19744    0.00000    0.00000    0.00    35571.50
621    2008.04.29 11:05    sell    314    0.10    1.20140    0.00000    0.00000    0.00    35571.50
622    2008.04.29 18:17    sell    315    0.10    1.20541    0.00000    0.00000    0.00    35571.50
623    2008.05.08 01:45    sell    316    0.10    1.20953    0.00000    0.00000    0.00    35571.50
624    2008.05.08 02:28    sell    317    0.10    1.21341    0.00000    0.00000    0.00    35571.50
625    2008.05.08 04:42    sell    318    0.10    1.21745    0.00000    0.00000    0.00    35571.50
626    2008.05.08 15:13    sell    319    0.10    1.22143    0.00000    0.00000    0.00    35571.50
627    2008.05.09 15:10    sell    320    0.10    1.22542    0.00000    0.00000    0.00    35571.50
628    2008.05.13 02:12    sell    321    0.10    1.22942    0.00000    0.00000    0.00    35571.50
629    2008.05.14 02:27    sell    322    0.10    1.23340    0.00000    0.00000    0.00    35571.50
630    2008.05.15 10:58    sell    323    0.10    1.23740    0.00000    0.00000    0.00    35571.50
631    2008.06.05 00:11    sell    324    0.10    1.24140    0.00000    0.00000    0.00    35571.50
632    2008.06.05 04:04    sell    325    0.10    1.24545    0.00000    0.00000    0.00    35571.50
633    2008.06.05 15:41    sell    326    0.10    1.24940    0.00000    0.00000    0.00    35571.50
634    2008.06.06 04:19    sell    327    0.10    1.25340    0.00000    0.00000    0.00    35571.50
635    2008.06.10 12:43    sell    328    0.10    1.25740    0.00000    0.00000    0.00    35571.50
636    2008.06.10 17:05    sell    329    0.10    1.26140    0.00000    0.00000    0.00    35571.50
637    2008.06.26 01:47    sell    330    0.10    1.26546    0.00000    0.00000    0.00    35571.50
638    2008.07.03 16:20    sell    331    0.10    1.26940    0.00000    0.00000    0.00    35571.50
639    2008.07.14 00:55    sell    332    0.10    1.27340    0.00000    0.00000    0.00    35571.50
640    2008.07.18 20:00    sell    333    0.10    1.27787    0.00000    0.00000    0.00    35571.50
641    2008.07.21 06:32    sell    334    0.10    1.28140    0.00000    0.00000    0.00    35571.50
642    2008.07.23 11:42    sell    335    0.10    1.28540    0.00000    0.00000    0.00    35571.50
643    2008.07.24 00:00    sell    336    0.10    1.28967    0.00000    0.00000    0.00    35571.50
644    2008.07.24 00:00    sell    337    0.10    1.29357    0.00000    0.00000    0.00    35571.50
645    2008.08.15 17:30    close    337    0.10    1.23273    0.00000    0.00000    400.21    35971.72
646    2008.08.15 17:30    close    336    0.10    1.23273    0.00000    0.00000    374.46    36346.18
647    2008.08.15 17:30    close    335    0.10    1.23273    0.00000    0.00000    346.05    36692.23
648    2008.08.15 17:30    close    334    0.10    1.23273    0.00000    0.00000    319.48    37011.71
649    2008.08.15 17:30    close    333    0.10    1.23273    0.00000    0.00000    296.09    37307.79
650    2008.08.15 17:30    close    332    0.10    1.23273    0.00000    0.00000    266.13    37573.92
651    2008.08.15 17:30    close    331    0.10    1.23273    0.00000    0.00000    239.07    37812.99
652    2008.08.15 17:30    close    330    0.10    1.23273    0.00000    0.00000    212.53    38025.52
653    2008.08.15 17:30    close    329    0.10    1.23273    0.00000    0.00000    184.41    38209.94
654    2008.08.15 17:30    close    328    0.10    1.23273    0.00000    0.00000    157.99    38367.93
655    2008.08.15 17:30    close    327    0.10    1.23273    0.00000    0.00000    131.43    38499.36
656    2008.08.15 17:30    close    326    0.10    1.23273    0.00000    0.00000    104.95    38604.30
657    2008.08.15 17:30    close    325    0.10    1.23273    0.00000    0.00000    78.86    38683.16
658    2008.08.15 17:30    close    324    0.10    1.23273    0.00000    0.00000    52.11    38735.26
659    2008.08.15 17:30    close    323    0.10    1.23273    0.00000    0.00000    24.16    38759.42
660    2008.08.15 17:30    close    322    0.10    1.23273    0.00000    0.00000    -2.47    38756.96
661    2008.08.15 17:30    close    321    0.10    1.23273    0.00000    0.00000    -28.83    38728.13
662    2008.08.15 17:30    close    320    0.10    1.23273    0.00000    0.00000    -55.39    38672.73
663    2008.08.15 17:30    close    319    0.10    1.23273    0.00000    0.00000    -81.82    38590.91
664    2008.08.15 17:30    close    318    0.10    1.23273    0.00000    0.00000    -108.10    38482.82
665    2008.08.15 17:30    close    317    0.10    1.23273    0.00000    0.00000    -134.78    38348.04
666    2008.08.15 17:30    close    316    0.10    1.23273    0.00000    0.00000    -160.41    38187.63
667    2008.08.15 17:30    close    315    0.10    1.23273    0.00000    0.00000    -188.42    37999.22
668    2008.08.15 17:30    close    314    0.10    1.23273    0.00000    0.00000    -214.90    37784.32
669    2008.08.15 17:30    close    313    0.10    1.23273    0.00000    0.00000    -241.20    37543.12
670    2008.08.15 17:30    close    312    0.10    1.23273    0.00000    0.00000    -267.95    37275.17
671    2008.08.15 17:30    close    311    0.10    1.23273    0.00000    0.00000    -294.96    36980.21
672    2008.08.15 17:30    close    310    0.10    1.23273    0.00000    0.00000    -321.51    36658.70
673    2008.08.15 17:30    close    309    0.10    1.23273    0.00000    0.00000    -347.80    36310.90
674    2008.08.15 17:30    close    308    0.10    1.23273    0.00000    0.00000    -374.42    35936.47

I think I'd simply use the EA on a non NFA regulated broker .

Originally Posted by PipCop

IMO, backtests are useless.

But how then do EA developers develop and optimize their EAs? They need historical data, so in that sense, historical data is essential, and backtesting over that data is an essential part of the development process. Would you buy an EA that was never backtested, not even once? Maybe some people would if it was forward tested for some time. But that forward testing time may be too short , i.e. not enough to produce statistically significant results if they are all you have to go by.

Originally Posted by PipCop

I have never seen a backtest that comes anywhere close to the results of a forward test. This is due to the fact that the data which MetaTrader uses is not tick -by-tick , but at best minute by minute. So the quality of your data is impacting your results, which is probably why you are seeing a loss on some trades.

You are preaching to the converted. I have been thinking about this all the time for the past month or so, and have been searching far and wide for good quality data to test over for scalping strategies in which a few pip differences can make a huge impact on profitability. On other forums I am the one who is the preacher regarding backtesting and historical data. I fully understand that MetaTrader creates fake tick data from 1 minute data that is stored on the computer, and I have literally spent days analyzing and working with it, together with gigabytes of Gain Capital tick data that I've downloaded. The Robominer strategy is nowhere near as sensitive to ticks as scalping strategies though, so I think it's ok to use M1 data and not worry that the ticks being tested over are fake.

One of my significant findings is that the data that is easily downloadable from the MetaTrader History Center is from MetaQuotes (if the warning message box appears) and such data is "indicative data", i.e. data that is from multiple sources and averaged together, as opposed to real data from a trade server. The averaging process would make the data a lot smoother than real data, which can then make EA performance results look much better.

So this is the reason why I used FXDD data from their web site (and not from the MetaTrader History Center download) because I think the data on their web site comes from their trade server.

Those who have searched for good historical data may have come across many references to Alpari data. Alpari UK used to provide their trade server data on their web site (http://www.alpari.co.uk/en/dc/databank.html), but some months ago they stopped and replaced the page with instructions to download the data through History Center. I have analyzed the new data, and it is just about exactly the same as the data from other brokers' MetaTrader terminals, i.e. it's just MetaQuotes data. I would have still been using the Alpari data from their web site if they were still available. Though the good thing about testing over FXDD data for the Robominer EA is that it is a popular broker to run the EA on.

[PipCop]

Originally Posted by Hyperdimension

I think I'd simply use the EA on a non NFA regulated broker .

Agreed!

Originally Posted by Hyperdimension

But how then do EA developers develop and optimize their EAs? They need historical data, so in that sense, historical data is essential, and backtesting over that data is an essential part of the development process. Would you buy an EA that was never backtested, not even once? Maybe some people would if it was forward tested for some time. But that forward testing time may be too short , i.e. not enough to produce statistically significant results if they are all you have to go by.

They can start by backtesting ... like I used to do ... but then they need to forward test on a live account, and on multiple brokerages, and for a long time. That is the only legitimate way to get real results. So I would really only trust a robot which i have seen forward tested for 6+ months.

So yes, it seems like we are on the same page. I also searched around for historical data and found pretty much what you are finding now. All in all, however, I realized that I was not a good enough programmer, so I stopped trying to make my own robots, and I stopped trusting backtests, and only wanted to see forward tests. That's why I started this site really, to share my findings with others.

[veeps]

Robominer EA uses no custom indicator .No past data is used to calculate the buy or sell signals like MACD,RSI etc.Then what is the use of historical data supplied by FX servers for this EA?.

[PipCop]

RoboMiner does use past data to determine the high and low ranges for the currency . That is how it adjusts it's grid.

[Hyperdimension]

Originally Posted by PipCop

So yes, it seems like we are on the same page.

Not exactly the same page, because you said "backtests are useless", whereas I think backtests are valuable (and certainly better than nothing).

The amount of value of a backtest depends on a number of things like:

• who does it (backtest results by the vendor may not be reliable due to some bias);

• Where the historical data is from (MetaQuotes data downloaded through the History Center? Actual trade server data from broker 's web site? The data source MUST be specified when presenting backtest results... if I don't see any source specified I assume they used the junk data from MetaQuotes because that's the one that most easily obtainable)

• how it's done (i.e. test methodology... if you studied science you'd know good and bad ways to conduct tests/experiments).

There are many variables and issues to address, like the ones we already covered. But if/when it's done in a scientific/"best practice" way, the results would be highly valuable.

That doesn't mean that it would be wise to then blindly go and trade with big capital; forward testing should still be done, in which there also are many variables and issues, and I'm sure you are deeply aware of that through the operation of this web site.

I think it would actually be a good idea to provide backtest results for each of the robots you have under test, so that people can see whether the forward test results bear any similarity to the backtest results. We may then be able to see which robots were over-optimized by the developers just to show a smooth backtest equity curve.

If you also do updates to the backtest as time goes by, we can then see how well live forward testing matches backtest results over the recent data.

Originally Posted by PipCop

I was not a good enough programmer

It just takes practice, like most other skills. My first exposure to programming was when I had a Commodore 64 computer... I think I was maybe 8 years old! Now it's my primary career, and lately I've been applying my programming skills to trading.

I stopped trusting backtests, and only wanted to see forward tests.

You don't have to completely trust backtests... (I sure don't), but as I wrote in this message, you can use backtests as part of an entire due-diligence analysis of a robot, which include forward tests. I think anyone who is thinking of using a robot with real money should conduct their own backtests of it (and compare with others' published results, including the vendor's), as well as forward, instead of only forward as you seem to suggest. That is why I have been doing a backtest of Robominer... which is still going; I've never had a backtest go for this long. Usually I also run optimizations of particular parameters to see how the performance varies with different parameter values, but if a single optimization pass is going to take days, I think I may simply accept the vendor's optimal settings.

That's why I started this site really, to share my findings with others.

I commend you for doing a good job. There are other web sites that also conduct forward tests, and that doesn't mean that there is competition; it means people can cross-check test results, which is additionally valuable in the context of doing an entire performance examination of a robot. Robominer in particular does well on other web sites that are forward testing it.

[pipdetector]

Originally Posted by Hyperdimension

(...)That is why I have been doing a backtest of Robominer (which is still going... I've never had a backtest go for this long).

I have noticed that GT-Shadow --the bigger brother of Robominer also takes quite long to backtest, but it never took more than several hours. GT-Shadow has an additional parameter EnableFastBackTests, maybe you should use GT-Shadow (I assume that both have the same logic, but Robominer has limited access to parameters that are available to the user).

[PipCop]

I really need to take more time and be more clear when I post!

HyperDimension, you make very good points. I should point out that I don't think backtests are useless, but I am just frustrated by the companies who use them as marketing material when they bear no resemblance to the live results. So, in that regard, that is when I think they are useless.

I agree that people should do backtests and compare them to the live results. However, I will tell you that in the several months that I spent learning to program, I made several robots which had GREAT backtest results. I am talking about "Wow! I'm going to be rich!" backtests. Then when I put them on a live account, the account tanked.

As far as not being a good programmer, that's not exactly what I meant. I actually am a programmer for a living, just not a MQL4 programmer. However, it was trivial to learn the language. What was missing was the background in understanding the Forex markets and the quality of data that I needed to test. MetaTrader is a good platform, but not good enough to backtest if the data is simulated. In the Forex market, where data is tick -by-tick , I cannot use something which only uses 1 minute levels, or worse, generates a random number to determine the levels in the minute (which is what MT does.) That would be like testing a digital watch down to the minute, and making up how many seconds were in that minute. 40 seconds this minute. 22 the next. That is NOT data quality, so I grew frustrated and gave up programming.

I did search around for real tick -by-tick data, and did find it, but it either cost too much or was incomplete (not enough data). And the other problem was that when I did find it, there was so MUCH data it was hard to download it all. Besides, it was not compatible with MT4, so then I had to find a different program, and then I could not use MT4, etc. Since so many people use MT4 as beginners, I really wanted to stay within this arena so I could serve the community of MT4 users better.

As far as doing backtests, I've thought about it but I am trying to keep things simple. The site is already so complex that I don't want to make it too confusing or too hard to use for new visitors/traders. So I figured I would focus on the meat of the issue - how does a robot do going forward. The back tests to me are really just marketing material anyway. It is very rare when a robot actually matches it's back test results. The other issue is that a back test from 12 months ago won't reflect the current market. 12 months ago the market was in flux. Now things are more stable. So if I put up a back test, it would only be reflective of that period of time. Obviously I could keep re-doing them, but that take a lot of effort for me to do, especially with so many robots, so I don't have the resources to do so.

I am glad other sites are also doing comparisons. I am always amazed when I see a loser on my site being a winner elsewhere. It just goes to show that some times when you start a robot has a lot to do with how it works, as well as what settings are used. Since each site typically seems to use different settings it is hard to really compare.

[Hyperdimension]

Here are the backtest results of Robominer II 2.17.

Notice the peculiar round bumps in the equity curve. If the horizontal scale was calendar or time based, the bumps would not actually be so round; they'd actually be perfectly vertical drops because they are when multiple open positions are closed at once, half of which are losing positions.

[Hyperdimension]

Originally Posted by PipCop

I don't think backtests are useless, but I am just frustrated by the companies who use them as marketing material when they bear no resemblance to the live results. So, in that regard, that is when I think they are useless.

Ok so what you mean is that the backtests provided by the vendors are often useless. That is the reason why why need independent third parties to do their own backtests to compare with the vendors' backtests, otherwise there is no knowing how true/accurate/reliable the vendors' backtest results are. If independent backtests show that results are nowhere close to the vendors' backtest results, then we can add that to the list of false claims about the robot. The more false claims there are, the more reason there is to stop wasting any further work with this robot. This leads me into my next point :

Backtesting can give some indication of whether it would be actually be worth doing forward testing. Forward testing can take a lot of time and effort, and you'd only want to do it on robots that would be worth looking further into. Instead of blindly forward testing based on the vendor's claims and backtest results, some due diligence should be done, which include your own backtests. A robot that performs poorly in a backtest is highly unlikely to do very well in forward testing. Whereas a robot that performs very well in a backtest may continue to do well in future, and we can only know for sure by progressing further through forward testing. So this is another way in which backtesting is very valuable - it can save you time that you would have otherwise wasted in forward testing a bad robot.

I agree that people should do backtests and compare them to the live results. However, I will tell you that in the several months that I spent learning to program, I made several robots which had GREAT backtest results. I am talking about "Wow! I'm going to be rich!" backtests. Then when I put them on a live account, the account tanked.

If you only spent a few months in the endeavour before giving up, then I'd say that it was not enough time to gain enough proficiency to produce accurate and robust EAs suitable for live trading with real money. There could have been numerous flaws in your programming as well as development and testing methodology. Were you using MetaQuotes data (downloaded through the MetaTrader terminal)? That data is just indicative data. As I said previously, the averaging process smoothens the data fluctuations and can cause your EA performance to look great, whereas in reality the data you get from a single trade server would not be so smooth.

MetaTrader also has some quirks that you need to work around. You have to write code to handle them e.g. when trade context is busy.

It appears to me that you gave up too soon, but I understand also that you probably have other commitments (including running this site and babysitting the forward tests).

What was missing was the background in understanding the Forex markets and the quality of data that I needed to test. MetaTrader is a good platform, but not good enough to backtest if the data is simulated. In the Forex market, where data is tick -by-tick , I cannot use something which only uses 1 minute levels, or worse, generates a random number to determine the levels in the minute (which is what MT does.)

For strategies that use the 1 hour timeframe or higher, the MetaQuotes data is accurate enough and the tick data being fake is not a problem. A pip difference here or there would be insignificant, because the take profit and stop loss levels would be large, as opposed to scalping strategies whose take profit levels are usually under 10 pips. Scalping strategies would need good tick data, and I agree that it is lacking. I have worked with Gain Capital data but the quality is bad for the early years, though it has been improving.

I did search around for real tick -by-tick data, and did find it, but it either cost too much or was incomplete (not enough data). And the other problem was that when I did find it, there was so MUCH data it was hard to download it all. Besides, it was not compatible with MT4, so then I had to find a different program, and then I could not use MT4, etc.

You must be referring to the Gain Capital data. Yes it a huge amount of data, and I did managed to convert their data to MetaTrader format, though it took a lot of time and effort. But there are week-long holes in the data, in some places, so it's not that great, but it's still interesting to see how strategies perform over the data compared with other data.

I think practically, the best we can do at the moment is use FXDD data that is available from their web site: MetaTrader 1-Minute (M1) Data. I believe this data is from their trade server as opposed to MetaQuotes.

The back tests to me are really just marketing material anyway.

Backtests that are done by independent third parties (including yourself) are not marketing material. Backtesting is one step in an analysis programme of a robot. Forward testing is a later step, and as I said before, it may be worth doing backtests to weed out, at an early stage, the obviously bad robots.

The other issue is that a back test from 12 months ago won't reflect the current market. 12 months ago the market was in flux. Now things are more stable. So if I put up a back test, it would only be reflective of that period of time.

You could actually say a similar thing with forward tests. If you do a 12 month forward test, all the results are from the past, and the market behaviour may be a lot different at the end of the 12 month period. Markets are always changing.

I am glad other sites are also doing comparisons. I am always amazed when I see a loser on my site being a winner elsewhere. It just goes to show that some times when you start a robot has a lot to do with how it works, as well as what settings are used. Since each site typically seems to use different settings it is hard to really compare.

I think most sites would use the default vendor recommended settings. Though the forexpeacearmy one is using 0.1 lot position size for AUDNZD: Forex Peace Army Robominer test

Also check out the 4xProject Test.

[traderx]

will this ea work with oanda? www.fxtrade.oanda.com