GCP BigQuery as Security Detection Platform

The speakers, from the game studio King, open with the problem of storing and querying high-volume telemetry. The on-premise platforms they used were costly, hard to maintain, and slow or congested when analysts needed them most. They argue that Google Cloud's BigQuery, a managed warehouse that runs SQL over very large datasets, can also serve as a detection source, starting with Cloud Audit Logs such as admin activity and data access records.

Norberto García Marín then walks through the framework design. Each detection is a template holding an identifier, MITRE technique and tactic, category, severity, a description and a SQL query. Scheduler templates select groups of detections by metadata and set how often they run. Cloud Storage, Cloud Functions, Cloud Scheduler and Pub/Sub handle ingestion, orchestration and result publishing, while Terraform and Atlantis deploy changes from GitHub pull requests. Cloud Monitoring raises alerts when a function fails, and results are emitted as JSON messages for downstream tools. A sample SQL injection detection illustrates the output format.

The talk closes with planned work: an anomaly detection layer built from log data, using BigQuery ML clustering or Vertex AI, with Dataflow handling streaming feature extraction. Diana Kramer shares lessons on the learning curve of using a general data platform for security, and advises teams to start from tools they already use and to consult internal BigQuery experts. The main takeaway is that managed cloud services can be assembled into a detection pipeline that scales without heavy operational work.