Anomaly detection: process-behaviour charts and change-point analysis

Not every bad period is an anomaly, and not every anomaly looks like a bad period. A corridor can record its worst punctuality figure in two years and still be inside the range its process normally produces. Another corridor can post an unremarkable headline number while sitting in a sustained downward shift that began six periods ago.

Two distinct questions are at stake. The first is whether a single observation falls outside what the process normally produces — the domain of statistical process control. The second is whether the process itself has shifted to a new operating level — the domain of change-point detection.

Answering the wrong question generates editorial noise. Gauge Intelligence uses formal methods for both, and publishes anomaly annotations only when those methods trigger.

Process-behaviour charts (XmR)

The process-behaviour chart, developed by Donald Wheeler, plots individual observations on an X chart and the two-point moving range on an mR chart. Together they describe the natural variation a process produces when nothing unusual is happening.

The natural process limits on the X chart are X̄ ± 2.66·R̄, where R̄ is the mean of the moving ranges. The constant 2.66 is the inverse of d₂ for n=2 (1.128) multiplied by 3, giving three-sigma equivalents estimated from short-range variation rather than long-run standard deviation.

Three signal rules trigger an anomaly annotation: one point beyond the natural process limits; eight consecutive points on the same side of the centreline; or three of four consecutive points in the outer third of the limits. When none of these triggers, the observation is common-cause variation, the process behaving as it always behaves, and is not published as a news event.

The reference treatments are Wheeler, Understanding Variation (1993), and Montgomery, Introduction to Statistical Quality Control, Ch 6.

The chart is applied to Corridor::DailyResidual, the residual after STL seasonal decomposition, so that ordinary weekly and period-of-year patterns do not generate false signals. A Friday-evening peak that recurs every Friday is not an anomaly, and the residual series strips it out before the limits are computed.

The closest live counterpart to a corridor signal in the public archive is the median-polish residual table on the Great Eastern Main Line corridor page. There, NR Infrastructure (49) sits 12.2 pp below its predicted 81% baseline on the 90-day window. The flagged cell is computed by the same Tukey decomposition described under Resistant summary statistics below.

Service-level z-scores

XmR and PELT operate on a corridor’s daily residual series. A complementary question runs at the service level: does this individual freight service’s punctuality this week sit far enough from its own recent baseline to merit flagging?

Gauge Intelligence applies a trailing z-score to each service-week. The trailing window is the twelve weeks preceding the observation; the current week is excluded from its own baseline. The flag fires when the absolute z-score exceeds 2 — the week’s on-time rate is more than two standard deviations from the same service’s own twelve-week mean.

The z-score answers a different question from the process-behaviour chart. SPC asks whether the process is in control as it accumulates points and is therefore a property of the time series; the z-score is point-in-time, computed against a moving baseline that walks forward with each new week.

SPC can fire on a run of eight consecutive points on one side of the centreline even when no single point is extreme; the z-score will be silent on that pattern. Conversely, a single severe service-week will trip the z-score but may sit inside the XmR limits that have been widened by prior variability.

The figure is read as standard deviations from baseline, not as a percentage. A z of −2.4 on the 4M61 Felixstowe to Ditton service in a given week means that week’s on-time rate sat 2.4 within-service SDs below its trailing twelve-week mean. That is a flag-worthy departure from this service’s own recent behaviour, irrespective of whether 4M61’s overall level is good or bad.

Each operator reference page surfaces the trailing twelve-week mean and SD alongside the current period figure, so every published A2F point ships paired with its own trailing context.

Change-point detection

Process-behaviour charts detect individual signal points. Change-point detection answers a different question: has the process itself shifted to a new level, and if so, when?

The PELT algorithm (Killick & Eckley, Journal of Statistical Software, 2014) performs an exact, computationally efficient search for the optimal segmentation of a time series under a penalised cost function. It returns a set of change points, the dates at which the underlying process is best modelled as having shifted, together with the segment means.

When a change point is confirmed, the process-behaviour limits and the forecasting baselines are recomputed from the new regime only. Mixing pre-change and post-change data into a single baseline produces wide, uninformative limits and forecasts that lag the current state of the network.

This matters for corridor reports. “This corridor has been performing below its historical average for eight periods” is the wrong framing if the corridor shifted regimes in Period 3. The correct framing is “this corridor has operated at a lower level since Period 3, and is performing to that new baseline”.

Resistant summary statistics (Tukey)

The arithmetic mean and standard deviation are non-resistant: a single large incident can shift both substantially. In freight performance data, where a single locomotive failure or a single major possession can produce one extreme observation in a period of forty, that sensitivity is a problem.

Gauge Intelligence uses the median and median absolute deviation (MAD) as resistant alternatives where the underlying distribution is likely to contain outliers. The MAD scaled by 1.4826 is a consistent estimator of the standard deviation under normality, and degrades gracefully in its absence.

For two-way operator × corridor tables, median polish decomposes the table into an overall effect, row (operator) effects, column (corridor) effects, and residuals; it is resistant to outlying cells. The residuals answer a sharp question: is this operator performing above or below what their corridor baseline would predict, once the operator’s general level and the corridor’s general level have been removed?

Worked example

Consider three operators measured on two corridors over a four-week period. Punctuality (percentage of services arriving within fifteen minutes of schedule) is as follows:

Computed via the operator lens — operator effects estimated after corridor structure is removed. See league-table methodology.

	North–South	East–West
Operator A	88.1	83.4
Operator B	76.2	71.8
Operator C	84.6	82.9

Raw ranking by row mean: A (85.8 %), C (83.8 %), B (74.0 %).

After median polish, the decomposition absorbs corridor structure into column effects (North–South baseline: 86.0 %; East–West baseline: 81.6 %, a 4.4 pp structural gap). The row effects measure each operator’s level relative to Operator C, the median operator:

Operator	Row effect	Residual rank
A	+2.0 pp	1
C	0.0 pp	2
B	−9.8 pp	3

The ranking order is unchanged, but the corridor structure is now explicit. B’s raw 74.0 % includes the East–West structural discount (4.4 pp below the North–South baseline). An operator posting 72 % exclusively on the East–West corridor is doing structurally harder work than an operator posting 72 % exclusively on the North–South corridor. The residuals make that visible where raw averages do not.

The reference is Tukey, Exploratory Data Analysis (1977), Ch 11–12.

Publication thresholds

A period is annotated as anomalous in published output only when it triggers a formal signal rule. “Below the trailing average” without a signal-rule trigger is not published as a finding; it is presented as context and is not given headline framing.

Change-point confirmations are published with the estimated date of the shift and the regime means before and after. Readers can therefore see both that a change occurred and what it changed from.

The complete signal history for a corridor (every rule trigger, date, moving range, and regime transition) is available in licensed analytical content. Licensed access enables dispute analysis and trend attribution against a documented signal record rather than a narrative summary.

Provisional signals, Wheeler’s rules 2 (run of eight) and 3 (three-of-four in the outer third), carry a disclosure note that the signal is provisional and may resolve. Rule 1 (a single point beyond the natural process limits) is the strongest signal and is published without that qualifier.

Version history

Version 1.0 — April 2026. Initial publication. XmR process-behaviour charts on STL residuals, PELT change-point detection, and Tukey- resistant summary statistics. Applies to all anomaly annotations in the public archive from April 2026 onward.

Where this is implemented

Periodic::DailyA2fSeriesAssessment (at app/models/periodic/daily_a2f_series_assessment.rb) is the entry point for period-report daily A2F panels. Given an operator and a date range it computes the daily series, applies the Western Electric SPC rules described above, and returns the run-rule signals with the rendered sparkline. Every “SPC run-rule signal fired” annotation in a period report resolves through this class.

The underlying daily-series computation sits in Periodic::DailyA2fSeries (app/models/periodic/daily_a2f_series.rb) — the assessment wrapper adds the SPC overlay, severity classification, and prose summary. The 3σ control limits use the same Tukey-resistant standard-deviation estimate documented in the section above.

Sources

Killick, R. & Eckley, I. A. (2014). “changepoint: An R Package for Changepoint Analysis”. Journal of Statistical Software, 58(3).

Montgomery, D. (2019). Introduction to Statistical Quality Control (8th ed). Wiley. Ch 6.

Tukey, J. W. (1977). Exploratory Data Analysis. Addison-Wesley. Ch 11–12.

Wheeler, D. J. (1993). Understanding Variation: The Key to Managing Chaos. SPC Press.