jmquintana79
1/25/2017 - 12:30 AM

outliers median methodology

Outliers detection using Median Filtering

"""
MEDIAN ABSOLUTGE DEVIATION:

Filtering technique based on the median absolute deviation (MAD) which allow detecting the variability of an univariate sample of quantitative data.

The median absolute deviation (MAD) is defined as the median of deviations from the median

MAD=mediani(|Xi−medianj(Xj)|)
and you can find out more about it here http://en.wikipedia.org/wiki/Median_absolute_deviation
"""

def get_median_filtered(signal, threshold, isreplace=False):
  """
  signal: is numpy array-like
  returns: signal, numpy array 
  """
  import numpy as np
  difference = np.abs(signal - np.nanmedian(signal))
  median_difference = np.nanmedian(difference)
  s = 0 if median_difference == 0 else difference / float(median_difference)
  mask = s > threshold
  if isreplace: signal[mask] = np.nanmedian(signal) # replace
  else: signal[mask] = np.nan                       # delete
  return signal