A bot is only its rules
A trading bot removes hesitation and missed signals. It does not change whether its rules make money after costs. If the rules lose, the bot loses faster and more consistently than a person would. Studies cited by ESMA found that 74% to 89% of retail CFD accounts lost money; automation does not exempt a strategy from that.
When TradeLab tests a bot independently, we run its own logic on years of broker data with that broker's costs, split time, and run luck tests. The same three findings come up most often: no edge after costs; trades the code takes that the written rules never intended (duplicate signals, missing timeouts); and results that depend on one market or one period.
This tutorial turns our ten-point due-diligence checklist into steps you can follow before paying for, renting or switching on any bot, expert advisor or signal service. You do not need to code to use it; you need to ask for evidence and know what good evidence looks like.
The ten checks, as steps
- Get the exact rules or the code. Entry, exit, stop, target, sizing, filters. Refusal, or rules that change when you ask twice, is a red flag.
- Ask for results in R per trade after all costs, with the number of trades. Percent-of-account curves can be made to look like anything by changing size.
- Check the costs match your broker and your hours. Which spreads were used: recorded at each trade, or one typed-in number?
- Ask which years built the bot and which tested it. One continuous curve with no untouched period proves only that the tuning worked.
- Ask how many versions were tried. At a 1-in-20 bar, 806 tries would produce about 40 lucky winners.
- Ask for a luck test: the bot should beat 95% of 200 coin-flip copies of its own trades, and buy-only bots should also beat random long entries.
- Ask where the profit came from: longs or shorts, which market, which year.
- Stress the costs: still positive at 1.5 x the spread, with swap.
- Check that the code does what the rules say: look at individual trades for duplicates, trades that never time out, stops inside the spread, limit fills better than the market offered, exits at the daily break.
- Know how it loses: worst losing streak and largest drawdown in R, and whether it ever adds to losing positions (grid, averaging down, martingale).
Worked example: a scalper's pips into R
Before the spread: 2 / 8 = +0.25R per trade
At a 0.6-pip median spread: (2 - 0.6) / 8 = +0.175R
At a 0.8-pip 90th-percentile spread: (2 - 0.8) / 8 = +0.15R
At the 1.5 x stress spread (0.9 pips): (2 - 0.9) / 8 = +0.1375R
The spread takes 30% to 45% of the reported edge before slippage, latency or swap
The spreads are the EURUSD median and 90th percentile we measured since 2024. If the bot's real edge before costs is smaller than reported, or your broker is wider, the same arithmetic can turn it negative. In our 39-strategy assessment, 16 strategies were positive before the spread and 3 after it.
Before you switch it on
- Freeze the version and settings; note a fingerprint of the code if you can.
- Write down in advance what result after how many trades counts as success, and what makes you stop.
- Run it on a demo account or at the smallest size your broker allows, at the hours you will really trade.
- Compare live trades with the backtest's trades for the same period, one by one.
- Change nothing during the test; a bot tuned during its forward test is a new bot.
Signal services: the same questions
- How many signals, over how long? A few months is rarely enough to tell skill from a good run.
- Is every signal published at the time, with its stop? Without the stop you cannot compute R; edited or 'cancelled' signals make a record meaningless.
- How many channels or accounts does the provider run? If only the best is promoted, its record was selected by luck.
- Can you get the same fills? Copying a short-term signal minutes later, at another broker, gives a different entry and often most of the edge back.
Limits
Strategy-tester reports depend on settings chosen by whoever ran them: how prices inside each bar were modelled, which spread, which dates, and whether the settings were optimised on the same dates. Ask for all of these before reading any statistic.
Key takeaways
- A bot does not change the odds; only rules with an edge after costs do.
- Ask for R per trade after costs, an untouched test period, a count of tries and a luck test.
- Look at individual trades: duplicates and missing timeouts are the bugs we find most.
- Forward-test the frozen bot before real money.
Sources
- European Securities and Markets Authority (2018). ESMA agrees to prohibit binary options and restrict CFDs to protect retail investors. Link
Education only, not investment advice. Results quoted from TradeLab research are historical simulations; trading can lose more than you expect.
