Episode 08 · Planned
Parallelism across nodes
Test placement and communication costs on a documented multi-node network topology.
Compare replica placement and TP/PP/EP layouts on a documented network topology. Do not carry within-node assumptions across Ethernet or InfiniBand boundaries. Scale only when model fit or measured capacity justifies the communication and operational cost.
Evidence: node and accelerator placement, NIC/link speeds, selected transport, collective latency, throughput, transfer volume, failures and recovery behavior.