1. Know that all your tools are reliable: your data, your trading platform, etc.
2. Always be aware of where the theoretical world diverges from the real one.
Know with each step that you’re not “cheating”—knowing something you couldn’t access in the real world. Trade Station won’t let you pollute today’s data with something from tomorrow—but a spreadsheet might, if you inadvertently referenced a “future” cell down the page. A Trade Station (probably all platforms) limit order will be shown as filled if the price is merely touched but not penetrated.–untrue more often than not in the real world. An exit-on-close order will be shown as filled at the posted closing price, but that price is calibrated after the official close. An automated strategy won’t get you out in that case.
3. Have some idea of what you’re testing for before you start. (Momentum? Mean reversion?) Haphazardly churning numbers is a good way to produce something that doesn’t have much promise for future robustness.
4. Make sure you have a large enough time frame in which you’re testing. Five years is generally good.
5. Don’t include the previous year in your study. You want to hold some time off so that you can test your finished system in data that had nothing to do with testing procedures. It’s considered “virgin data”—a virtual time frame not unlike the unknowable future.
6. Initial testing should be “wide”—use a simple concept unadorned by stops, filters etc. They can be added in the refining stages, after you’ve proven your simple technique has a bias. Don’t include slippage/commission—you don’t want small biases to get by you at the early testing stage.
7. Narrow in with additional rules, stops and/or filters.
Adding conditions is a balancing act—you need results to improve dramatically enough to offset the decrease in statistical reliability that is inevitable as you make qualifiers more stringent. The less additional parameters it takes to hit the desired threshold, the better.
8. You need enough trades for statistical significance—thousands for intraday, probably a few hundred for overnights. Revise or discard if you come up short.
9. Optimize. Use variable letters in the command section of your code—“n” rather than “5”. Assign “n” a value in the input section (5). When you optimize, you can see results of all possible numbers between [low number] and [high number] in increments of [your choice]. This is how to compare and contrast performance summaries with alternate values for n. You can test for one variable, or for more than one combined in the same study.
10. Study your optimization results. They will contain, by definition, a worst and best performer and an array in between. Settle on a best number that resides in a neighborhood of similarly good numbers—not a diamond in the rough. Don’t use the best one (25) if the adjacent results (22,23,26) are only fair to poor. Use the second best (7). Or the third best if 7 is also isolated.
11. Add slippage commission to your settled-on strategy version. At least $5 round trip for a mini contract, $25 for a full-sized one—more would be even more desirable. This greatly lessens the possibility that your results were fluky.
12. Test the system in your withheld virgin data. If results aren’t reasonably positive, your system is suspect.
13. Plan on trading a market that has both good liquidity and volatility. (The S&Ps and Russell are the most ideal futures. The 10 and 30 year bonds are decent). If you decide to trade one with less liquidity, increase your per-trade slippage/commission.
14. All aspects of your settled-on performance summary should range from reasonable to exceptional. Can you live with the biggest drawdown? (More specifically, you should be projecting 150% that amount under the theory that your worst drawdown is always ahead of you). Can you tolerate the worst string of losses? Beware if your average trade doesn’t rise far enough above slippage/commission.
15. Examine your trades list and periodical returns to make sure your gains are dispersed more or less evenly throughout the data field. Distrust any system that gets most of its gains from a very small time period.
16. Look for similarly positive results in other markets, particularly closely related ones in the same sector such as the DOW and S&P. Don’t trust a system that becomes an anti-system money loser in a related market even if the results are exceptionally good in your chosen market.
17. Ideal goal for trading multiple systems should be lack of correlation between them; in other words, they should be doing different things and producing different results.
18. If you settle on multiple systems that have many overlapping trades, just realize you’re in effect increasing your position size even if there is some divergence.
19. Have Plan B’s firmly in place. Computers go down—have your trade desk phone number at hand. Have a plan for instantly dealing with missed trades, trades that shouldn’t have been made and all other conceivable errors. Despite meticulous planning, glitches will crop up sooner or later.
20. Go back and double check everything. Don’t breeze over anything and don’t assume you’re engaging in a pointless exercise because you nailed everything the first time. You’ll probably catch more lapses than you imagined. And we’re only talking your money after all.