SWE-bench ProPublic set (731 tasks) · 731 tasks; harness not stated
50.4%Checked A person checked this number against its source on 7 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.
9.4%Checked A person checked this number against its source on 7 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.
39.7%Checked A person checked this number against its source on 7 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.