GPU Scheduling Is a Business Policy Problem: Designing NVIDIA Run:ai Quotas, Fairness, and Preemption

This article explains that GPU scheduling in shared clusters is fundamentally a business policy problem, not just a technical configuration. It focuses on NVIDIA Run:ai as an orchestration platform, detailing how to design quotas, fairness, and preemption to align with business priorities. Key concepts include guaranteed quota, over-quota access, maximum consumption limits, fair-share scheduling, and priority/preemptibility. The article emphasizes treating GPU allocation as enterprise service policy, using whole GPUs by default, and measuring queue time, completion time, useful utilization, guarantee attainment, and preemption cost. It provides architectural guidance for managing shared AI infrastructure across departments and workloads.