Fair-Share GPU Scheduler for University Lab Workstations and Student Projects
Build a lightweight job scheduler for a university department's handful of GPU workstations, where students currently book GPUs on a spreadsheet or log in and fight over them. Students submit containerised training jobs; the scheduler queues them, shares GPUs fairly across groups, checkpoints and pauses long jobs to let short ones through, and reports who used what. It is far simpler to run than Slurm.
Intermediate
A small team, or one strong student willing to learn something new.1 to 2 semesters · 2 students
7 to ship
4 optional extrasSuggested stack
What you should ship
- A job submission portal and command-line tool: container image, GPU count, time limit, project
- A node agent on each workstation that starts containers, pins GPUs and reports usage
- A fair-share queue across groups with priorities for deadlines such as final year demos
- Checkpoint-friendly preemption: signal a job to save and stop, then resume later
- Detection of idle GPUs held by jobs that stopped using them
- Usage reports per student, group and supervisor
- A trial in a real lab or a simulation from logged usage, comparing waiting times with first come, first served
If you have time left
- Sharing one GPU between small jobs with memory limits or MIG where supported
- Jupyter sessions that are scheduled like jobs and stopped when idle
- Energy and temperature reporting per job
- Email or WhatsApp notifications when jobs start and finish
The problem: three GPUs, sixty students, one spreadsheet
Many computer science departments in Pakistan have a small number of GPU workstations bought for research and final year projects. Access is managed by a shared spreadsheet, a WhatsApp group or whoever logs in first. A student starts a training run, it finishes at 3 a.m., and the GPU sits idle until the morning. Another student's job crashes because someone else's process took all the memory. Before demo week, everyone needs the GPUs at once, and the loudest group wins.
Supervisors have no data on usage, so they cannot argue for more hardware or set fair rules.
What already exists and the gap
- Slurm is the standard scheduler for clusters and supports GPUs, but setup and administration are a real burden for a department without a dedicated system administrator.
- Kubernetes with GPU operators can schedule GPU jobs, with considerable operational overhead.
- Commercial GPU orchestration platforms such as NVIDIA Run:ai target enterprises.
- Cloud notebooks offer limited free GPU time, unreliable for long training runs.
The gap is a small, friendly scheduler for a few workstations that a lab technician or student volunteers can run, with fairness across groups, preemption that respects checkpoints and usage data supervisors can use.
Inspiration
"Gandiva: Introspective Cluster Scheduling for Deep Learning" by Wencong Xiao and colleagues at Microsoft Research (OSDI 2018) exploits the repetitive mini-batch structure of deep learning jobs to time-slice GPUs across jobs and migrate them, improving latency and cluster efficiency. This project takes two of its ideas, suspend and resume at iteration boundaries and introspection of job progress, and applies them at the scale of a department lab.
What you build
- Students submit a job with an image, a command and a time limit, and see their place in the queue and their job's logs.
- Supervisors set group shares and approve priority for deadlines.
- Lab admins see utilisation per GPU, idle allocations and monthly reports.
The technically interesting part
- Fair share. Combine group shares, recent usage and waiting time into a priority, so a group that used little recently gets ahead of one that has used a lot. Evaluate fairness with standard measures.
- Preemption that does not waste work. Send a signal to a running job, give it time to write a checkpoint, then stop it; resume it later from the checkpoint. Provide a tiny helper library for PyTorch training loops that makes this easy.
- Placement. Jobs that need two GPUs need two on the same machine. Avoid fragmentation where every machine has one free GPU but no two are together.
- Idle detection. Watch GPU utilisation and memory per job, and warn or reclaim when a job holds a GPU but has done nothing for a set time.
- Isolation. Each job sees only its assigned GPUs and its own files, with limits on CPU and memory too.
- Honest time limits. Students underestimate run times. Use each job's own progress reports, or its loss logging, to estimate time left, and warn before the limit rather than killing a nearly finished run.
How to evaluate it
- Simulation: collect or reconstruct a semester of usage requests, replay them through first come, first served and through your scheduler, and compare waiting times, utilisation and fairness.
- Lab trial: with the department's permission, run for several weeks and compare utilisation and student satisfaction with the previous approach.
- Preemption cost: time lost per preemption with and without checkpoints.
Suggested plan
- Weeks 1 to 3: Docker with GPU access on one workstation; collect current usage habits from students.
- Weeks 4 to 7: node agent, job submission and a simple queue.
- Weeks 8 to 11: fair share, placement and idle detection.
- Weeks 12 to 15: preemption with checkpoints, reports, simulation and report.
- Optional second semester: lab trial, GPU sharing and notebooks.
Who it is for, concretely
A department with three or four workstations of one or two GPUs each, a lab technician who is not a Linux expert, and dozens of students who need the GPUs most in the last six weeks of each semester.
Path to a product
Universities, research groups and small AI companies with a few on-premises GPUs all share this problem. An open source scheduler with paid setup and support, or a small hosted control plane that manages on-premises workstations, are realistic. It also makes a strong platform engineering portfolio.
Scope warning
Do not replace Slurm for large clusters; target three to ten machines. Rely on containers for isolation and do not try to virtualise GPUs yourself. Fairness and preemption matter more than a beautiful portal.
Learn what you need on Sefism
- MLOps and LLMOps roadmap: training infrastructure and checkpoints.
- DevOps with Docker: containers, images and resource limits.
- Operating Systems: Three Easy Pieces: scheduling and fairness from first principles.
- Platform engineering and SRE roadmap: building internal platforms people actually use.
- Prometheus Documentation: collecting GPU and job metrics.
Get help with this idea
The risk is building a portal and running out of time for fairness and preemption. Tauseef can help you plan the scheduler logic first, design the simulation and approach your department for a trial. If you are new here, create a free account first.
Ideas and guidance, not finished projects
These are project ideas and scoping guidance, published free for students to use as a starting point. I do not build, write, or sell final-year projects, and I do not complete coursework for anyone. Take an idea, make it yours, and build it.
Not sure this one fits you?
I help students pick an idea that matches their skills and their deadline, then scope it down until it is finishable. Guidance only, never done for you.
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.
Follow along
New writing, resources and project ideas land here first.