1/25/2017 - 12:30 AM

outliers median methodology

Outliers detection using Median Filtering


Filtering technique based on the median absolute deviation (MAD) which allow detecting the variability of an univariate sample of quantitative data.

The median absolute deviation (MAD) is defined as the median of deviations from the median

and you can find out more about it here

def get_median_filtered(signal, threshold, isreplace=False):
  signal: is numpy array-like
  returns: signal, numpy array 
  import numpy as np
  difference = np.abs(signal - np.nanmedian(signal))
  median_difference = np.nanmedian(difference)
  s = 0 if median_difference == 0 else difference / float(median_difference)
  mask = s > threshold
  if isreplace: signal[mask] = np.nanmedian(signal) # replace
  else: signal[mask] = np.nan                       # delete
  return signal