OSWorld 2.1
Created by XLANG Lab, 16 Sep 2026 · Higher is better · Website
- Other labs
Settings and run setups
| Task set | Scoring | Harness | Providers |
|---|---|---|---|
| Full set (108 tasks) | Partial score | Not stated | |
| Full set (108 tasks) | Partial score | Common internal framework | |
| Full set (108 tasks) | Strict pass rate | Not stated | |
| Full set (108 tasks) | Strict pass rate | Common internal framework | |
| Offline set (82 tasks) | Partial score | Not stated | |
| Offline set (82 tasks) | Partial score | Gemini CUA harness | |
| Offline set (82 tasks) | Strict pass rate | Not stated | |
| Offline set (82 tasks) | Not stated | Not stated | |
| OpenAI compares GPT-6.1 Terra with Claude Sonnet 5.5’s 83.9%, which Anthropic lists under “8.9.3 OSWorld 2.1 offline subset” | |||
| Not stated | Not stated | Kimi Agent harness | |
| Tencent lists Kimi K3.1 Pro’s 79.6 under “OSWorld 2.1 (Full, partial score)” | |||
Results
| Model | Score | Source |
|---|---|---|
| Kimi K3.1 Pro24 Oct 2026 | 79.6% | Kimi K3.1 Pro Tech Blog: Advancing Open IntelligenceChecked A person checked this number against its source on 11 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Kimi Agent harness · max reasoning effortOur capture |
| Tencent lists it under “OSWorld 2.1 (Full, partial score)” | ||