Articles /
Backtest on 60 Race Files: How Accurate Is the Engine
AeroPacer's finish-time prediction runs on a physics engine, not a statistical regression. The upside of a physics engine is that it can compute a new course on day one. The downside is that you have to prove it computes correctly. This article lays out how I validated it.
How I tested
The method is called replay: take the power record the athlete actually produced on race day, feed it to the engine segment by segment, add that day's hourly wind direction and speed plus the course's cornering slowdowns, and compute how long the engine thinks that ride "should" have taken. Then compare that with the actual time recorded on the head unit.
The most important detail is CdA (drag area). If you back out CdA from a race's own file and then use it to predict that same race, that is cheating. So for every race, the CdA comes only from that athlete's other races: the estimates are weighted by recency and the median is taken. The race's own data is left out.
Sample
- Nine athletes, 2019 to 2026, 74 race files, on courses in more than 16 countries.
- All body weights are measured values. Bikes are grouped by type (tri bikes and road bikes get separate CdA baselines).
- Files with documented problems were excluded: two with no power, four with a faulty barometer, one duplicate upload, three with GPS distortion in tunnel sections. Their CdA estimates stay in the baseline pool; only their overall times are kept out of the statistics.
That left 64 races that could be evaluated. Four extreme outliers were then removed, giving a final sample of 60.
Results
| Metric | Value |
|---|---|
| Median absolute error | 2.2% |
| Within ±5% | 92% |
| Within ±3% | 73% |
| Median for 70.3 (113 km) | 1.7% |
| Median for full distance (226 km) | 2.3% |
| Signed median | +0.7% (slightly conservative) |
The website says "median error about 2%, nine out of ten races within ±5%". Both numbers are rounded in the conservative direction.
How the error is distributed
Predicted vs actual
Where the error comes from
- Body weight. With assumed weights the median was actually lower, because when the weight is wrong, the CdA estimate compensates in the other direction and hides the error. Once measured weights went in, the mismatches in equipment and era were exposed. This is also why the Console asks for your real weight.
- Equipment changes. One athlete's implied CdA dropped from 0.22 to between 0.16 and 0.18 starting in 2024. A big equipment upgrade, but the median over the full history cannot keep up. This is the unavoidable cost of estimating CdA from other races.
- Distance to the weather station. There is no weather station near Port Macquarie and the wind was strong; two athletes in the same race came out at −7.4% and −4.1%.
- Barometers overstating elevation gain on long flat courses. This one affects course building, not the engine; the course library handles it with official routes and cross-checks against multiple sources.
Races used for validation
The final 60 races come from 45 race-years, listed below. Multiple files for the same race are different athletes. Error is each file's prediction relative to actual (positive = predicted slower). The three "training rides" are my own training rides at race-course length, added to fill out the 2026 sample.
| Year | Race | Bike leg | Files | Error per file |
|---|---|---|---|---|
| 2026 | Tielifang · Guangde 113 | 113 · 88 km | 1 | +3.3% |
| 2026 | Training ride (not a race) | 116 km | 3 | -1.5% -0.5% +1.1% |
| 2026 | IRONMAN Vietnam · Da Nang | 226 · 180 km | 1 | +4.1% |
| 2026 | IRONMAN Taiwan · Penghu | 226 · 178 km | 1 | +2.2% |
| 2026 | IRONMAN Hamburg | 226 · 177 km | 2 | -0.7% +3.7% |
| 2026 | IRONMAN 70.3 Shanghai Chongming | 113 · 90 km | 3 | +1.0% +3.5% +4.5% |
| 2026 | Challenge Taiwan 226 | 226 · 180 km | 1 | +6.3% |
| 2025 | Tielifang · Guangde 113 | 113 · 89 km | 1 | +0.1% |
| 2025 | Beauty of Taitung 113 | 113 · 89 km | 1 | +3.8% |
| 2025 | IRONMAN World Championship · Nice | 226 · 178 km | 1 | -1.5% |
| 2025 | IRONMAN World Championship · Kona | 226 · 181 km | 1 | +2.7% |
| 2025 | IRONMAN Taiwan · Penghu | 226 · 184 km | 1 | +5.2% |
| 2025 | IRONMAN Malaysia · Langkawi | 226 · 179 km | 1 | -0.9% |
| 2025 | IRONMAN Gurye | 226 · 176 km | 3 | -1.5% +0.9% +1.1% |
| 2025 | IRONMAN Frankfurt | 226 · 175 km | 2 | +0.9% +2.4% |
| 2025 | IRONMAN California · Sacramento | 226 · 180 km | 1 | +2.3% |
| 2025 | IRONMAN 70.3 Kenting | 113 · 90 km | 1 | -0.9% |
| 2025 | IRONMAN 70.3 Shanghai Chongming | 113 · 90 km | 4 | -0.1% +1.3% +2.7% +2.7% |
| 2025 | IRONMAN 70.3 Puerto Princesa | 113 · 90 km | 2 | -1.7% -1.7% |
| 2024 | Tielifang · Guangde 113 | 113 · 89 km | 1 | -2.7% |
| 2024 | IRONMAN World Championship · Kona (partial file) | 73 km (partial) | 1 | +3.5% |
| 2024 | IRONMAN World Championship · Kona | 226 · 180 km | 1 | +2.2% |
| 2024 | IRONMAN Texas | 226 · 178 km | 1 | +4.1% |
| 2024 | IRONMAN Cairns | 226 · 179 km | 1 | -0.9% |
| 2024 | IRONMAN 70.3 World Championship · Taupō | 113 · 90 km | 2 | -2.4% -0.7% |
| 2024 | IRONMAN 70.3 Subic Bay | 113 · 91 km | 1 | +2.3% |
| 2024 | IRONMAN 70.3 Desaru Coast | 113 · 89 km | 1 | -2.7% |
| 2024 | IRONMAN 70.3 Bangsaen | 113 · 90 km | 1 | +2.7% |
| 2024 | IRONMAN 70.3 Bahrain | 113 · 90 km | 2 | +0.5% +0.9% |
| 2024 | Challenge Xiamen 113 | 113 · 89 km | 1 | +3.9% |
| 2024 | Challenge Taiwan 226 | 226 · 179 km | 1 | +4.2% |
| 2024 | Challenge Roth | 226 · 178 km | 1 | +2.3% |
| 2023 | Tielifang · Guangde 113 | 113 · 86 km | 2 | -7.0% +0.6% |
| 2023 | Puyuma Triathlon 226 | 226 · 178 km | 1 | -2.0% |
| 2023 | IRONMAN France · Nice | 226 · 179 km | 1 | -5.9% |
| 2023 | IRONMAN Austria · Klagenfurt | 226 · 179 km | 1 | -2.4% |
| 2023 | IRONMAN Australia · Port Macquarie | 226 · 180 km | 1 | -4.1% |
| 2023 | IRONMAN 70.3 Kenting | 113 · 90 km | 1 | -2.3% |
| 2023 | IRONMAN 70.3 Puerto Princesa | 113 · 90 km | 1 | -0.8% |
| 2022 | IRONMAN Taiwan · Penghu | 226 · 179 km | 1 | -1.9% |
| 2022 | IRONMAN 70.3 Taitung | 113 · 88 km | 1 | -0.3% |
| 2019 | IRONMAN World Championship · Kona | 226 · 180 km | 1 | +5.5% |
| 2019 | IRONMAN 70.3 Xi'an | 113 · 90 km | 1 | -0.8% |
| 2019 | IRONMAN 70.3 Liuzhou | 113 · 90 km | 1 | -2.1% |
| 2019 | IRONMAN 70.3 Xiamen | 113 · 90 km | 1 | -0.3% |
What this means
For an age-group athlete, here is what the number means: ride to the plan and your finish time will land within about ±5% of the prediction. Upload your ride file after race day for the comparison, and your own CdA converges tighter with each race, so the error on your next one shrinks again.
A note on credibility
- Every number in this article is computed directly from the backtest results table, and the charts are drawn from the same data. No races were picked out separately.
- The excluded files and the reasons are all listed above. Exclusions were decided on data quality before seeing the results, not on the size of the error.
- The sample comes from nine real athletes plus myself as coach, and covers courses in Asia, Europe, North America and Oceania. Not a single race used its own file to estimate its own CdA.
- The backtest script and results table are kept in the project and re-run with every engine upgrade. If the numbers change, they will be updated here along with the reason.
The engine will keep being backtested against new ride files.
