I looked into this a bit more and here is the summary: This is meant to show off a candidate kernel feature that allows for running different schedulers in userland.
Task scheduling has become much more complex as CPUs have grown in size and have had new developments in architecture, so the need to develop more complex and robust schedulers is steadily rising.
The kernel feature is meant to lower the barrier of entry for anyone who wants to try getting into schedulers, as well as enable quicker development iteration, by removing the need to completely recompile the linux kernel every time you want to test your code.
Read more at the main project’s github: https://github.com/sched-ext/scx
Not quite running in userspace. To the best of my own understanding:
The new kernel feature is to allow writing schedulers in eBPF, a “language” the kernel runs in kernelspace that is heavily restricted.
For example, all eBPF programs must complete in bounded time, and the kernel’s static checker must be able to verify that before the program can even begin executing. eBPF is a rare language that is not touring complete.
“For scx_simple, suspending the scheduler process doesn’t affect scheduling behavior because all that the userspace component does is print statistics. This doesn’t hold for all schedulers.”
So, it may be that eBPF also makes it easier to write a truly userspace scheduler, but that’s not the primary purpose, and it’s not what is being done with scx_simple.
https://lwn.net/Articles/909095/ for more about (e)BPF.
Thank you for the correction! Reading up on eBPF is fascinating.
Additional resource that adds to your secondary point that this is more than just allowing schedulers to be run in userland: https://github.com/sched-ext/scx/blob/main/scheds/c/README.md
Is eBPF dependently typed?
Wow, that’s a huge difference. I wonder if it’s just a niche case that fixes Terraria specifically, it would be nice to see more benchmarks on other games.
I didn’t watch it all. (OK, didn’t realize it was only a minute. I basically saw everything).
But I wonder if it’s simpler than that, and it’s just doing a better job prioritizing the game. Presumably there’s a reason they’re doing it while running a heavy background process.
This obviously still has value if it’s the case, and is a big part of the point of a scheduler, but the headline implied (to me) that it’s a general performance improvement, and the video doesn’t demonstrate that.
If this is in user space, does this mean we can switch schedulers on the fly? Put it in game mode when gaming, power saving mode when on battery, etc?
Now if only gccrs would mature soon!
LLVM-based is fine for most case, but I bet a lot of people would want to stick with gcc for compiling the kernel.
For that usecase
rustc_codegen_gcc
works too and is much more likely to be mature soon.It seems to still require LLVM, tho
What makes you think that? The whole point of it is to create a
rustc
backend that useslibgccjit
instead of LLVM.