Episode 04 · Planned
Batching, scheduling and mixed traffic
Study chunked prefill and scheduler behavior under controlled long/short traffic mixes.
Hold the workload fixed while changing chunked-prefill support or configuration, chunk size and token budget one variable at a time. Then vary the long/short traffic mix. Continuous batching is part of the scheduler; do not assume every engine exposes a valid off mode. A static-batch reference is a separate workload if the runtime supports it.
Evidence: verified effective flags, running/waiting requests, scheduled tokens per iteration when exposed, queue time, TTFT and decode gaps by request class, maximum wait, starvation indicators and KV time series.