Anomaly detection

Outlier / Anomaly detection

Anomaly detection of numeric data

• Median
• Range Checks
• All values is event or odd
• The values are the same even the column is totally different

Anomaly detection of categorical data (qualitative variable)

• Normal distribution e.g. The interest of audiences should be very different NOT coherent

Anomaly detection for time series data

• Trend
• Dramatically Increase or decrease of rows count for each time period
  • Example: Regularly scheduled web scraping that collects 9k records per week suddenly drops to 3k records
• Future data

Anomaly detection for consumer data

For consumer data

• Season issue: consumption data of coat should increase in cold weather
• Holiday issue: consumption data of some gift e.g. moon cake should increase in special holiday e.g. Mid-Autumn Festival

Anomaly detection for string data

• Created time of the text message
• Time frequency of the text message
• Length of the text message
• NULL or empty value
• Minor differences of text content<ref>How to Measure Text Similarity: A Comprehensive Guide | by Ahmet Münir Kocaman | Medium</ref>
• Character encoding e.g. Fix garbled message text

More on: Outlier - Wikipedia

References

<references />