Who Ate My Resources? Trace LLM Performance in Real Time With OTel
Explores who Ate practitioner Resources? Trace LLM Performance in Real Time With OTel and evaluates key implications for security practitioners.
Summary
No summary yet — watch the video above, or open it on YouTube.
Related talks
The Linux Foundation·52 min
HW, SW, Performance and Costs for Llama-2 70b and Mixtral 8x7b LLM Inference with Low...- Ivan Baldo
Low...- Ivan Baldo
Details HW, SW, Performance and Costs for Llama-2 70b and Mixtral 8x7b LLM Inference with Low...- Ivan Baldo and evaluates key implications for security practitioners.
The Linux Foundation·32 min
Seokhwan Kong
Outlines beyond Round-Robin: GPU-Aware Load Balancing for LLM Inference in Kubernetes and evaluates key implications for security practitioners.
The Linux Foundation·43 min
Kevin Hannon
Discusses being a Maintainer in the Age of LLM Mania and evaluates key implications for security practitioners.