Chemometric Outlier Detection: Mahalanobis Distance vs. Hotelling's T²

The Role of Outlier Detection in Spectroscopy

When deploying quantitative calibration models (built via Partial Least Squares or PLS regression) on an active production line, the system relies on the assumption that the scanned samples are similar to the calibration database. If a sample is scanned that contains an unexpected contaminant, exhibits an extreme temperature variation, or has an completely different raw material matrix, the model will output a prediction. However, that prediction will be mathematically invalid.

To prevent bad predictions from writing corrupt data into your SCADA system, process spectrometers must evaluate whether each scanned spectrum is a "multivariate outlier" before outputting predictions. This is accomplished by projecting the sample spectrum into the multidimensional space of the calibration model and calculating statistical distance limits.

"Outlier detection acts as the immune system of a process analyzer. It guarantees that predictions are only outputted when the sample matches the calibration space."

Mahalanobis Distance: Measuring Covariance

Simple Euclidean distance measures the straight-line distance between a sample spectrum and the calibration mean. However, Euclidean distance fails in chemometrics because it ignores the strong correlation (covariance) between adjacent wavelengths in NIR spectra.

**Mahalanobis Distance (MD)** resolves this by scaling the distance by the covariance matrix of the calibration dataset. Effectively, MD accounts for the direction of variance, assigning less weight to variations in directions of high normal calibration variance, and high weight to deviations in directions where the calibration data is tightly bound.

Hotelling's T—: Confidence Thresholds

While Mahalanobis Distance provides a raw distance value, we need a statistically sound threshold to flag outliers. **Hotelling's T—** normalizes the Mahalanobis Distance in a Principal Component (PC) projection space, calculating a multivariate score. Because T² follows an F-distribution, we can establish formal statistical confidence limits (typically 95% or 99%):

Parameter	Mahalanobis Distance (MD)	Hotelling's T²
Mathematical Focus	Measures generalized distance in raw or covariance-scaled wavelength space.	Measures score distance within the Principal Component space (PC scores).
Covariance scaling	Scales by the full wavelength covariance matrix.	Scales by the variance of individual PC scores.
Threshold Limit	Heuristic or Chi-Squared ($\chi^2$) distribution.	Strict F-distribution confidence intervals ($1-\alpha$, where $\alpha = 0.05$ or $0.01$).
Best Use Case	Flagging bad raw materials, instrumentation drift, or hardware faults.	Validating class membership and flagging samples outside the model space.

Setting Outlier Limits in caliX Suite

To implement outlier limits in your calibrations, follow these steps in **caliX Suite**:

1. Compile PCA/PLS scores: Open your calibration file and calculate the principal component loading vectors.
2. Set Confidence Interval: Navigate to the Outliers tab and select `Hotelling T—`. Choose a 99% confidence threshold ($\alpha = 0.01$) for critical control points, or 95% ($\alpha = 0.05$) for routine monitoring.
3. Enable ProChem Alarms: Sync the model to the process spectrometer. In **ProChem**, map the outlier flag output to your PLC register to hold the last valid prediction when an outlier is triggered.

Conclusion

Deploying robust outlier detection using Mahalanobis Distance and Hotelling's T² ensures process analyzer integrity. By identifying and isolating anomalous spectra, manufacturers can prevent calibration failures and secure automated process loops.