Lael Labs

Kernel Bench

Lael Labs · public benchmark

tasks87subsystems14languages3models17

Measuring frontier coding agents on original, long-horizon kernel engineering tasks

Read the writeup →Run Kernel Bench

Leaderboard

87 tasks · updated July 21, 2026Models (17/17)
Kernel Bench scorehover · click to pin
0%20%40%60%80%$25$20$15$10$5$0Kernel Bench scoreAvg cost per taskmost efficient ↗gpt-5.6-solMAXclaude-fable-5MAXgpt-5.6-terraMAXkimi-k3MAXgpt-5.6-lunaMAXgpt-5.5XHIGHclaude-opus-4.8MAXclaude-sonnet-5MAXgrok-4.5HIGHmuse-spark-1.1XHIGHgpt-5.4XHIGHgemini-3.6-flashHIGHglm-5.2MAXgemini-3.5-flashMEDIUMkimi-k2.7-codeMAXclaude-sonnet-4.6HIGHgemini-3.1-proHIGH
Models (17/17)
ModelPass@1Avg costOut tokSteps
gpt-5.6-sol [max]
73% ±3%$8.3960k61
claude-fable-5 [max]
70% ±4%$21.63119k88
gpt-5.6-terra [max]
70% ±3%$4.9572k76
kimi-k3 [max]
69% ±5%$4.6581k98
gpt-5.6-luna [max]
67% ±4%$3.0373k102
gpt-5.5 [xhigh]
67% ±6%$7.2346k82
claude-opus-4.8 [max]
59% ±2%$13.22135k120
claude-sonnet-5 [max]
54% ±4%$26.40214k268
grok-4.5 [high]
54% ±2%$2.4236k61
muse-spark-1.1 [xhigh]
53% ±3%$2.3674k96
gpt-5.4 [xhigh]
52% ±2%$5.6571k70
gemini-3.6-flash [high]
49% ±5%$3.5397k108
glm-5.2 [max]
44% ±2%$3.9278k129
gemini-3.5-flash [medium]
37% ±2%$7.34276k86
kimi-k2.7-code [max]
31% ±1%$2.8259k149
claude-sonnet-4.6 [high]
30% ±4%$5.5276k134
gemini-3.1-pro [high]
12% ±2%$9.48196k81

All models run on a fixed mini-agent harness for consistency. Read why.

Today's leading public coding benchmarks are starting to saturate at the frontier: top models cluster within a narrow score band where adjacent configurations often overlap on confidence intervals. Kernel Bench is a long-horizon systems benchmark built to separate them. It delivers four advances over existing public suites:

  • Contamination free: Tasks are authored against private kernel branches and synthetic regressions, not scraped from public commits, so solutions are not sitting in pretraining data.
  • High diversity: Tasks span 14 subsystems across memory, scheduling, filesystems, networking, BPF, KVM, and drivers — in C, Rust, and assembly.
  • Real-world complexity: Prompts stay short; solutions require multi-file edits, lock discipline, and hardware-facing edge cases that pure algorithmic benches never touch.
  • Reliable verification: Verifiers boot test kernels, run kselftests / custom harnesses, and score behavior — not string match on patches.

The result is a benchmark that reflects how frontier coding agents actually perform on kernel engineering work.

Task Examples

Fix race in page fault handler under concurrent munmap

Ensure concurrent unmaps cannot free a page still held by a faulting core; preserve TLB shootdown ordering.

linux/mmC

Restore fair CFS load after cgroup freeze/thaw

Thawed tasks must re-enter the runqueue with correct vruntime; no priority inversion across cgroups.

linux/schedC

Add io_uring fixed-buffer cancel on process exit

Drain and unmap fixed buffers when the last file ref drops; no use-after-free on completion CQEs.

linux/io_uringC

Close TOCTOU in BPF map update with RCU

Map updates must be RCU-safe under concurrent lookups; reject stale gen when relocating elements.

linux/bpfC

Harden KVM nested VMX MSR bitmap merge

Nested guests must inherit filtered MSRs only; intercept bits compose correctly across L1/L2.

linux/kvmC

Fix Rust binder death recipient teardown

Death notifications must not fire after the recipient is dropped; align with binder_alloc lifetime.

linux/androidRUST
All 87 tasks →

Read the
full writeup

Open →
  1. 01IntroductionWhy a kernel benchmark
  2. 02OverviewWhat separates Kernel Bench
  3. 03MethodologyHow tasks and verifiers are built
  4. 04ResultsWhere frontier models diverge
  5. 05Qualitative analysisHow agents fail in the kernel
  6. 06Limitations & future workWhat we will do differently