Advancing Network Threat Detection Thru Standardized Feature Extraction & Dynamic Ensemble Learning

The talk starts from a common problem with machine learning for intrusion detection: many earlier studies trained a single classifier on small, narrow datasets that looked strong in the lab but generalized poorly to real traffic, with efficacy often falling below 40 percent once models met unseen traffic. The research asks whether detection accuracy can improve by working directly from packet-level data.

The approach has three parts. First, a standardized feature extraction framework built on packet metadata and flow-level statistics, avoiding reliance on IP addresses and port numbers that do not scale to new traffic. Second, a set of classifiers that includes random forest, isolation forest, GMM, QDA, boosting methods and neural networks. Third, a custom ensemble that weights each model's output by confidence, giving more influence to less certain predictions. Training combined public labeled captures with benign traffic gathered from home and academic networks. The ensemble reached roughly 98 percent accuracy and the most balanced precision and recall among the models tested.

The speaker was clear about limits: the classifiers were static, testing used captured traffic rather than live capture, and some GMM and neural models underperformed. Future work includes tuning hyperparameters, exploring CNN and RNN approaches, and testing on live traffic, with a small Raspberry Pi prototype already being tried on open Wi-Fi. The framework is released under a GPL license. The main takeaway is that feature choice can matter as much as model choice, and that ensembles can offset individual model weaknesses.