Episode 10 · Planned
Slurm and Kubernetes orchestration
Compare representative scheduled jobs and long-running services as platform choices, separate from engines.
Use Slurm for representative queued batch training, scheduled experiments and tightly coordinated multi-node jobs. Use Kubernetes for representative long-running services, platform APIs, deployment controllers, autoscaling and serving orchestration. Validate GPU discovery and allocation with the NVIDIA GPU Operator/device plugin, then study GitOps-managed KServe and Ray Serve deployment, with operators such as KubeRay managing the relevant cluster lifecycle. Run a metrics-driven autoscaling load test and a controlled scaling incident; use measured SLO capacity and recorded prices to build a capacity plan. Compare these platform choices separately from runtime engines such as vLLM, SGLang and TensorRT-LLM. Treat these as common strengths rather than exclusive boundaries: Kubernetes can schedule training, and Slurm environments can host inference services.
Evidence: comparable job and service definitions plus an explicit decision based on queue semantics, gang scheduling, topology awareness, failure recovery, service discovery, rollout control and operator burden. Keep platform health evidence separate from engine benchmarks.