← Back to research
Working Paper · SSRN

Overcoming the Transaction Cost Trap

Why machine-learning equity signals die in execution — and how cross-sectional conviction ranking keeps the alpha alive under realistic frictions.

ProblemML classifiers reach statistically significant accuracy yet lose money live: naive threshold execution churns the portfolio and transaction costs quietly consume the gross alpha — the transaction cost trap.
ApproachTreat calibrated ensemble probabilities as ordinal conviction. Each day, hold only the Top-K highest-ranked assets — preserving the signal while structurally constraining turnover.
ResultA genuinely significant cross-sectional signal (IC 0.0197, p = 0.034). Top-1 returns 45.8% a year at Sharpe 1.18 — 7.3× the Sharpe of a random one-stock baseline.
SignificanceThe edge survives to a 24.2 bp break-even cost (4.8× institutional friction); a 1,000-run permutation test places it at the 99.9th percentile (p = 0.001).
0.0197Cross-sectional IC · p = 0.034
45.8%Top-1 annualised return
1.18Top-1 Sharpe (net 5 bps)
24.2 bpsBreak-even cost · 4.8× friction

The Transaction Cost Trap

Gradient-boosting and neural models routinely beat linear factor models in cross-sectional return prediction — and then fail to make money in live execution. The failure is not in the signal; it is in the translation of probabilities into trades. A model with 55% directional accuracy that fires ~400 round-trip trades a year at 10 bps forfeits roughly 4% annually to friction alone — enough to consume, and often invert, a competitive 6–8% gross alpha. Worse, the standard remedy of raising the decision threshold (P > 0.60) collapses to a Sharpe of 0.071 at a 6.8% win rate: it trades a cost problem for a frequency problem and discards the signal's information content.

Cross-Sectional Conviction Ranking

Rather than thresholding absolute probabilities, the framework ranks every asset by its calibrated ensemble probability each day and allocates capital only to the Top-K. The ensemble is an equal-weight blend of CatBoost, a Random Forest, and a deep MLP (47 → 256 → 128 → 64 → 1) over 47 strictly causal technical indicators, with isotonic calibration fit per fold on a held-out slice — essential, because uncalibrated tree outputs compress toward the class mean and would rank variance artefacts rather than conviction. K = 1 is the recommended configuration: highest Sharpe (1.183) and return (45.8%) on the fewest trades.

Honest Out-of-Sample Validation

All estimation runs on an expanding walk-forward protocol — minimum 756-day train, 126-day test, a two-day embargo (the label looks two days ahead, so the gap prevents contamination) — across 12 folds spanning Oct 2018 – Oct 2024 (1,512 out-of-sample days). The headline test is a permutation test: shuffling the daily cross-sectional rankings 1,000 times yields a null Sharpe distribution (mean 0.233, 95th percentile 0.666); the observed Top-1 Sharpe of 1.183 sits at the 99.9th percentile (p = 0.001). The Lo (2002) HAC-adjusted Sharpe confidence interval is [0.382, 1.984] (t = 2.894, p = 0.0019) — positive after autocorrelation correction.

Where the Edge Is Thin

The result is reported with its limits. The universe is seven ex-post dominant technology equities, so the findings are conditional on a concentrated, survivor-selected cross-section — not a generalisable sector claim. The signal is also regime-dependent: it shines in the volatile COVID/Growth period (Sharpe 2.83, +117% annualised), holds slightly positive through the 2022–24 rate shock (Sharpe 0.10), and is negative in the early ZIRP bull (Sharpe −0.95) — though every strategy, including buy-and-hold, was negative there. Technical signals extract the most relative information during structural transitions, least in calm trending markets.

Methodology Pipeline

7 STOCKS 47 INDICATORS ENSEMBLE + ISOTONIC CAL. TOP-K CONVICTION RANK WALK-FWD + PERMUTATION

Key Result — Same Concentration, Different Signal

Top-1 · ML conviction rankingSharpe 1.18 · 45.8%
Random Top-1 · one stock, no signalSharpe 0.16 · 6.9%
Threshold P > 0.60 · the trapSharpe 0.07 · 6.8% win

Top-1 and Random Top-1 each hold exactly one stock per day — the only difference is the ML ranking. Net of 5 bps one-way costs, 1,512 out-of-sample days.