Understanding Moving Averages

Go: open a terminal in a folder you can write to — cd ~/Desktop on macOS or Linux, cd %USERPROFILE%\Desktop on Windows.

Do: save this as prices.py and run python3 prices.py. The random seed is fixed, so every number on this page will match yours exactly.

"""Build a price series once, so every later step works on the same numbers."""
import random

random.seed(11)                     # fixed, so your figures match this page exactly

# Eight regimes of 50 bars: up, down, flat, repeating. The turns are at
# bar 50, 100, 150, 200, 250, 300, 350 -- known, because we put them there.
REGIMES = [0.35, -0.30, 0.0, 0.35, -0.30, 0.0, 0.35, -0.30]

price, series = 100.0, []
for bar in range(400):
    drift = REGIMES[bar // 50]
    price += drift + random.uniform(-1.2, 1.2)
    series.append(round(price, 2))

with open("closes.txt", "w") as f:
    for p in series:
        f.write("%.2f\n" % p)

print("bars written :", len(series))
print("first 5      :", series[:5])
print("last 5       :", series[-5:])
print("high / low   : %.2f / %.2f" % (max(series), min(series)))
print("regime turns : bars 50, 100, 150, 200, 250, 300, 350")
print()
print("Three kinds of market on purpose -- rising, falling and flat, each")
print("appearing more than once. Most indicator advice is written as if only")
print("the first two exist.")

You should see: four hundred bars written to a file, with the turns stated:

bars written : 400
first 5      : [100.24, 100.73, 102.1, 102.36, 102.73]
last 5       : [116.86, 116.63, 115.51, 114.5, 114.51]
high / low   : 134.48 / 92.25
regime turns : bars 50, 100, 150, 200, 250, 300, 350

Three kinds of market on purpose -- rising, falling and flat, each
appearing more than once. Most indicator advice is written as if only
the first two exist.

Using generated prices rather than a real download is deliberate and is what makes the rest of the page possible: we know where the trend actually turns, because we put the turns there. With real market data you can measure what an indicator did, but never how late it was, because nobody can say when the market “really” turned.

If not: PermissionError means the folder is not writable — cd somewhere you own, since this script writes closes.txt for the later steps to read. If your first five values differ, the random.seed(11) line is missing or was moved after the loop began.

Go: the same folder.

Do: save this as sma.py and run python3 sma.py.

"""The simple moving average, and a hand-check that it is right."""

closes = [float(line) for line in open("closes.txt")]

def sma(values, period, index):
    """Average of the `period` values ending at `index`. None before there are enough."""
    if index < period - 1:
        return None
    window = values[index - period + 1: index + 1]
    return sum(window) / period

print("bar  close    SMA(5)     the five values it averaged")
for i in range(3, 9):
    v = sma(closes, 5, i)
    window = closes[max(0, i - 4): i + 1]
    print("%3d  %6.2f  %8s   %s" % (
        i, closes[i], "%.4f" % v if v else "  --  ", window))

# Check one value by hand, the way you would on paper.
i = 8
window = closes[4:9]
by_hand = (window[0] + window[1] + window[2] + window[3] + window[4]) / 5
print()
print("hand arithmetic at bar 8:")
print("   (%s) / 5" % " + ".join("%.2f" % v for v in window))
print("   = %.4f" % by_hand)
print("   function says %.4f  ->  agree: %s" % (sma(closes, 5, 8), abs(by_hand - sma(closes, 5, 8)) < 1e-12))

You should see: the first four bars empty, then the arithmetic agreeing with the function:

bar  close    SMA(5)     the five values it averaged
  3  102.36      --     [100.24, 100.73, 102.1, 102.36]
  4  102.73  101.6320   [100.24, 100.73, 102.1, 102.36, 102.73]
  5  103.29  102.2420   [100.73, 102.1, 102.36, 102.73, 103.29]
  6  102.89  102.6740   [102.1, 102.36, 102.73, 103.29, 102.89]
  7  103.27  102.9080   [102.36, 102.73, 103.29, 102.89, 103.27]
  8  103.93  103.2220   [102.73, 103.29, 102.89, 103.27, 103.93]

hand arithmetic at bar 8:
   (102.73 + 103.29 + 102.89 + 103.27 + 103.93) / 5
   = 103.2220
   function says 103.2220  ->  agree: True

Two things are worth taking from this. First, an average of five values does not exist until five values exist — that is why the first four bars are blank, and why every indicator has a “warm-up” period at the left edge of a chart where it simply is not defined.

Second, the hand-check at the bottom matters more than it looks. It is the habit of proving a function does what its name claims on one case you worked out yourself, before trusting it on four hundred. Every later step depends on this one being right.

If not: if the last line prints agree: False, the slice in the hand-check and the window inside sma cover different bars — both must be the five values ending at bar 8, i.e. closes[4:9]. A ZeroDivisionError means period arrived as 0.

Go: the same folder.

Do: save this as ema.py and run python3 ema.py.

"""The exponential moving average: the same idea, weighted toward the present."""

closes = [float(line) for line in open("closes.txt")]

def ema_series(values, period):
    k = 2.0 / (period + 1)              # the smoothing factor
    out = [None] * (period - 1)
    seed = sum(values[:period]) / period    # start from an SMA, as MT5 does
    out.append(seed)
    for v in values[period:]:
        out.append(v * k + out[-1] * (1 - k))
    return out

def sma_series(values, period):
    return [None if i < period - 1 else sum(values[i - period + 1:i + 1]) / period
            for i in range(len(values))]

e = ema_series(closes, 10)
s = sma_series(closes, 10)

print("smoothing factor k for period 10 = 2/(10+1) = %.6f" % (2.0 / 11))
print()
print("bar  close    SMA(10)   EMA(10)   EMA-SMA")
for i in (9, 10, 11, 40, 41, 42):
    print("%3d  %6.2f  %8.4f  %8.4f  %+7.4f" % (i, closes[i], s[i], e[i], e[i] - s[i]))

print()
print("The EMA has no window. Every past price is still in it, with a weight")
print("that halves roughly every %.1f bars -- so nothing is ever dropped, it" % (0.693 / (2.0/11)))
print("just fades. That is why an EMA turns sooner than an SMA of the same period.")

You should see: the two averages starting identical and then separating:

smoothing factor k for period 10 = 2/(10+1) = 0.181818

bar  close    SMA(10)   EMA(10)   EMA-SMA
  9  104.98  102.6520  102.6520  +0.0000
 10  104.36  103.0640  102.9625  -0.1015
 11  104.23  103.4140  103.1930  -0.2210
 40  114.95  111.6580  112.1301  +0.4721
 41  114.65  112.2440  112.5883  +0.3443
 42  114.49  112.7790  112.9340  +0.1550

The EMA has no window. Every past price is still in it, with a weight
that halves roughly every 3.8 bars -- so nothing is ever dropped, it
just fades. That is why an EMA turns sooner than an SMA of the same period.

The exponential average has no window at all. Each new value is mixed in with weight k, and everything already there is kept with weight 1−k — so a price from two hundred bars ago is still in the number, with a weight so small it no longer matters. Nothing is ever dropped; it fades.

That is the whole reason an EMA turns sooner than an SMA of the same period, and it is also why an EMA has no exact “lookback”. When somebody says an EMA(10) “looks at the last ten bars”, they are describing the SMA. Note the two averages are identical at bar 9, because MetaTrader seeds the EMA with an SMA — the difference only begins once there is a previous EMA to carry forward.

If not: if the two columns never converge at bar 9, the seed line was changed — the first EMA value must be the simple average of the first ten closes, which is what makes the EMA-SMA column start at exactly +0.0000.

Go: the same folder. This is the number nobody quotes.

Do: save this as lag.py and run python3 lag.py.

"""How many bars behind the price is the average? Measure it."""

closes = [float(line) for line in open("closes.txt")]

def sma_series(values, period):
    return [None if i < period - 1 else sum(values[i - period + 1:i + 1]) / period
            for i in range(len(values))]

def turned_down_at(series, start):
    """First bar at or after `start` where the series stops rising."""
    for i in range(max(start, 2), len(series)):
        if series[i] is None or series[i - 1] is None:
            continue
        if series[i] < series[i - 1] and series[i - 1] < series[i - 2]:
            return i
    return None

# The series was built to turn down at bar 50. That is the ground truth.
TRUE_TURN = 50
print("the trend actually reverses at bar :", TRUE_TURN)
print()
print("period   SMA notices the turn at   bars late")
for period in (5, 10, 20, 50):
    s = sma_series(closes, period)
    t = turned_down_at(s, TRUE_TURN)
    print("%6d %26s %11s" % (period, t, t - TRUE_TURN if t else "never"))

print()
print("A moving average cannot lead. It is an average of prices that already")
print("happened, so a longer period is smoother AND later, always, and no")
print("setting escapes the trade-off. Choosing a period IS choosing how much")
print("lag you will accept in exchange for how few false signals.")

You should see: the lag growing with the period, in bars:

the trend actually reverses at bar : 50

period   SMA notices the turn at   bars late
     5                         54           4
    10                         57           7
    20                         61          11
    50                         75          25

A moving average cannot lead. It is an average of prices that already
happened, so a longer period is smoother AND later, always, and no
setting escapes the trade-off. Choosing a period IS choosing how much
lag you will accept in exchange for how few false signals.

The market turned at bar 50. A five-period average noticed four bars later; a fifty-period average noticed twenty-five bars later. On an hourly chart that is twenty-five hours after the fact.

This is not a defect to be tuned away, and no setting escapes it. An average of past prices cannot know about a price that has not happened, so smoother always means later. Choosing a period is not choosing a “good” setting — it is choosing your position on that trade-off, and the right answer depends on whether false signals or late signals cost you more.

If not: if a row prints never, that period's average never produced two consecutive falling values after the turn, which happens if the period is longer than the regime — try adding a shorter one. The exact bar numbers depend on the seeded series, so they will match this page but not a different dataset.

Go: the same folder.

Do: save this as whipsaw.py and run python3 whipsaw.py.

"""Count the crossovers in each regime. This is where the money goes."""

closes = [float(line) for line in open("closes.txt")]

def sma_series(values, period):
    return [None if i < period - 1 else sum(values[i - period + 1:i + 1]) / period
            for i in range(len(values))]

fast, slow = sma_series(closes, 5), sma_series(closes, 20)

def crossovers(a, b, lo, hi):
    n = 0
    for i in range(max(lo, 1), hi):
        if None in (a[i], b[i], a[i-1], b[i-1]):
            continue
        was_above = a[i-1] > b[i-1]
        is_above  = a[i]   > b[i]
        if was_above != is_above:
            n += 1
    return n

# Eight 50-bar regimes: up, down, flat, up, down, flat, up, down.
RISING  = [(0, 50), (150, 200), (300, 350)]
FALLING = [(50, 100), (200, 250), (350, 400)]
FLAT    = [(100, 150), (250, 300)]
REGIMES = [("rising  ", RISING), ("falling ", FALLING), ("flat    ", FLAT)]

print("MA(5) crossing MA(20), over 400 bars:")
total, counts = 0, {}
for label, spans in REGIMES:
    n = sum(crossovers(fast, slow, lo, hi) for lo, hi in spans)
    bars = sum(hi - lo for lo, hi in spans)
    counts[label] = (n, bars)
    total += n
    print("   %-10s %3d crossings in %3d bars   (%.2f per 100 bars)"
          % (label, n, bars, 100.0 * n / bars))

flat, flat_bars = counts["flat    "]
print()
print("   total      %3d crossings in 400 bars" % total)
print("   in the flat %3d  (%.0f%% of all signals, from %.0f%% of the bars)"
      % (flat, 100.0 * flat / total, 100.0 * flat_bars / 400))
print()
print("Most of a crossover system's signals arrive in the market where it")
print("cannot work. The indicator is not broken in the flat section -- it")
print("is answering 'which way is price leaning' in a market that is not")
print("leaning, and every answer costs a spread.")

You should see: the flat market producing signals seven times as fast as the rising one:

MA(5) crossing MA(20), over 400 bars:
   rising       1 crossings in 150 bars   (0.67 per 100 bars)
   falling      3 crossings in 150 bars   (2.00 per 100 bars)
   flat         5 crossings in 100 bars   (5.00 per 100 bars)

   total        9 crossings in 400 bars
   in the flat   5  (56% of all signals, from 25% of the bars)

Most of a crossover system's signals arrive in the market where it
cannot work. The indicator is not broken in the flat section -- it
is answering 'which way is price leaning' in a market that is not
leaning, and every answer costs a spread.

Read the per-hundred-bars column rather than the raw counts, since the regimes are different lengths. The flat market generates 5.00 crossings per hundred bars against 0.67 in the rising one — roughly seven times the rate — and produces 56% of all signals from 25% of the bars.

Those flat-market crossings are the ones that lose money, and not because the indicator malfunctions. It is answering the question it was asked — which way is price leaning? — in a market that is not leaning either way, so the answer flips back and forth, and every flip costs a spread. The failure mode of a trend indicator is not being wrong; it is being used where there is no trend.

If not: if a regime shows zero crossings, check the span lists against the comment: the series alternates up, down, flat every fifty bars, so RISING, FALLING and FLAT between them must cover all four hundred bars.

Go: the same folder. This step is about arithmetic, not about trading.

Do: save this as filter.py and run python3 filter.py.

"""The same average used as a filter -- and how much a small sample can tell you."""
from math import comb

closes = [float(line) for line in open("closes.txt")]

def sma_series(values, period):
    return [None if i < period - 1 else sum(values[i - period + 1:i + 1]) / period
            for i in range(len(values))]

fast, slow, trend = sma_series(closes, 5), sma_series(closes, 20), sma_series(closes, 50)

def run(use_filter):
    taken, wins, pnl = [], 0, 0.0
    for i in range(51, len(closes) - 5):
        if None in (fast[i], slow[i], fast[i-1], slow[i-1], trend[i]):
            continue
        if not (fast[i-1] <= slow[i-1] and fast[i] > slow[i]):
            continue                        # not an upward crossover
        if use_filter and closes[i] < trend[i]:
            continue                        # only buy above the long-term average
        move = closes[i + 5] - closes[i]    # hold five bars, no stop, no costs
        taken.append((i, closes[i], trend[i], move))
        pnl += move
        wins += move > 0
    return taken, wins, pnl

taken, wins, pnl = run(False)
print("every upward crossover:")
print("  bar   close    SMA(50)  above?   move over 5 bars")
for i, c, t, m in taken:
    print("  %4d %7.2f %8.2f   %-6s  %+.2f" % (i, c, t, c > t, m))
print("  signals %d, wins %d, total move %+.2f" % (len(taken), wins, pnl))

taken_f, wins_f, pnl_f = run(True)
print()
print("with the SMA(50) filter:")
print("  signals %d  -- every one was rejected, because the 'above?' column" % len(taken_f))
print("  above is False on all of them.")

print()
print("That is the filter WORKING, not failing: all four upward crossings")
print("happened while price was still under its own long-term average, which")
print("is the condition the filter exists to refuse.")
print()
n, k = len(taken), wins
print("But now the honest question. The unfiltered rule won %d of %d." % (k, n))
print("How much does that tell you? If the rule were a pure coin flip, the")
p = sum(comb(n, j) for j in range(k, n + 1)) / 2 ** n
print("chance of getting %d or more wins out of %d by luck alone is %.1f%%." % (k, n, 100 * p))
print()
print("Nothing with a %.0f%% chance of happening by accident is evidence." % (100 * p))
print("Four trades cannot separate a good rule from a lucky one -- and this")
print("is exactly the arithmetic behind every screenshot of a 'proven'")
print("strategy with a handful of trades on it.")

You should see: the filter rejecting every signal, and then the reason that is not the interesting part:

every upward crossover:
  bar   close    SMA(50)  above?   move over 5 bars
   112   96.96   102.27   False   +2.58
   152   95.57    96.64   False   +2.05
   266  105.12   110.07   False   -2.27
   281  103.77   106.29   False   +0.88
  signals 4, wins 3, total move +3.24

with the SMA(50) filter:
  signals 0  -- every one was rejected, because the 'above?' column
  above is False on all of them.

That is the filter WORKING, not failing: all four upward crossings
happened while price was still under its own long-term average, which
is the condition the filter exists to refuse.

But now the honest question. The unfiltered rule won 3 of 4.
How much does that tell you? If the rule were a pure coin flip, the
chance of getting 3 or more wins out of 4 by luck alone is 31.2%.

Nothing with a 31% chance of happening by accident is evidence.
Four trades cannot separate a good rule from a lucky one -- and this
is exactly the arithmetic behind every screenshot of a 'proven'
strategy with a handful of trades on it.

The filter removing all four signals is the filter working: every upward crossing in this series happened while price was still below its own long-term average, which is precisely the condition it exists to refuse.

But the last paragraph is the one worth keeping. The unfiltered rule won three of four, a 75% win rate — and a coin flip produces three or more heads in four tosses 31.2% of the time. A result that arises by accident in nearly a third of attempts is not evidence of anything at all.

This is the arithmetic behind every screenshot of a “proven” strategy with a handful of trades on it, and behind a great deal of what is sold. Before believing any backtest, including your own, ask how many trades it contains — and treat anything under a few hundred as a story rather than a measurement.

If not: if the filtered run reports more than 0 signals, the comparison was written as > rather than < — the filter must skip when the close is below the long average. The comb import needs Python 3.8 or newer; on anything older, from math import comb raises ImportError.