Tune Your Microservices by Learning from Traces

## Overview This talk presents an approach to monitoring and optimizing microservices by analyzing distributed traces using machine learning. The speakers describe their architecture for processing large-scale trace data (approximately 700,000 traces daily) and using predictive models to detect anomalies and inform auto-scaling decisions.

## Architecture and Data Pipeline The system uses Kubeflow and TensorFlow for machine learning workflow deployment. Traces are collected from microservices and stored in Elasticsearch, with Kibana for visualization. The pipeline extracts key information from traces including service names, origins, destinations, and request durations. Data is prepared and cleaned before being fed into machine learning models, which are trained via Kubeflow and stored in object storage.

## Technical Approach The team employs time series analysis for anomaly detection, specifically using LSTM (Long Short-Term Memory) networks to capture temporal patterns in service duration data. They observe service duration across multiple time frames (30 minutes, 12 hours, 7 days) to understand behavior at different scales. By predicting future time sequences based on historical data and comparing predictions to actual measurements, they identify deviations that indicate anomalies.

## Key Challenges and Solutions A major challenge addressed is distinguishing actual anomalies from normal fluctuations across different time scales. The team uses multiple models with different time windows to avoid false positives—what appears anomalous in a short timeframe might be normal behavior in a longer context. They mention using clustering and specific algorithms for threshold detection rather than relying solely on traditional feature engineering.

## Applications The detected anomalies trigger notifications to operations teams and inform auto-scaling decisions for microservices. Models are versioned and deployed as APIs for real-time use. The approach enables proactive service optimization by predicting performance issues before they impact users.

## Limitations and Future Work The team notes challenges with distributed model training and acknowledges that models are service-specific rather than universal. Discussions in Q&A reveal ongoing work on optimizing training performance and determining appropriate scaling thresholds.