SWE-BENCH VERIFIED · ALL 136 PUBLIC SUBMISSIONS · OFFICIAL 500-TASK RUN
CAIN-42: #136 of 136
0/500 resolved · 0.0% · Wilson 95% 0.0–0.8% · leader 396/500 (79.2%) · to pass #135: +3 · to #1: +397
Open the live interactive board · raw feed · evidence · refreshed 2026-11-20T18:55:01Z · checksum f90f34d321e5fd7d
| # | submission | resolved | rate | 50% horizon | gap from CAIN |
|---|---|---|---|---|---|
| 1 | 20251215_livesweagent_claude-opus-4-5 | 396/500 | 79.2% | 4.3 h | CAIN needs +397 |
| 2 | 20251205_sonar-foundation-agent_claude-opus-4-5 | 396/500 | 79.2% | 4.2 h | CAIN needs +397 |
| 3 | 20250928_trae_doubao_seed_code | 394/500 | 78.8% | 4.1 h | CAIN needs +395 |
| 4 | 20251127_openhands_claude-opus-4-5 | 388/500 | 77.6% | 3.3 h | CAIN needs +389 |
| 5 | 20251120_livesweagent_gemini-3-pro-preview | 387/500 | 77.4% | 2.5 h | CAIN needs +388 |
| 6 | 20250902_atlassian-rovo-dev | 384/500 | 76.8% | 2.1 h | CAIN needs +385 |
| 7 | 20250804_epam-ai-run-claude-4-sonnet | 384/500 | 76.8% | 1.7 h | CAIN needs +385 |
| 8 | 20250819_ACoder | 382/500 | 76.4% | 1.9 h | CAIN needs +383 |
| 9 | 20250901_warp | 378/500 | 75.6% | 2.7 h | CAIN needs +379 |
| 10 | 20250612_trae | 376/500 | 75.2% | 1.7 h | CAIN needs +377 |
| 11 | 20251103_sonar-foundation-agent_claude-sonnet-4-5 | 374/500 | 74.8% | 2.6 h | CAIN needs +375 |
| 12 | 20250731_harness_ai | 374/500 | 74.8% | 1.4 h | CAIN needs +375 |
| 13 | 20250915_JoyCode | 373/500 | 74.6% | 2.3 h | CAIN needs +374 |
| 14 | 20250720_Lingxi-v1.5_claude-4-sonnet-20250514 | 373/500 | 74.6% | 1.7 h | CAIN needs +374 |
| 15 | 20251015_Prometheus_v1.2.1_gpt5 | 372/500 | 74.4% | 2.5 h | CAIN needs +373 |
| 16 | 20250603_Refact_Agent_claude-4-sonnet | 372/500 | 74.4% | 1.8 h | CAIN needs +373 |
| 17 | 20251103_SalesforceAIResearch_SAGE_OpenHands | 369/500 | 73.8% | 1.8 h | CAIN needs +370 |
| 18 | 20250522_tools_claude-4-opus | 366/500 | 73.2% | 2.3 h | CAIN needs +367 |
| 19 | 20251021_SalesforceAIResearch_SAGE_bash_only | 365/500 | 73.0% | 2.1 h | CAIN needs +366 |
| 20 | 20250522_tools_claude-4-sonnet | 362/500 | 72.4% | 2.0 h | CAIN needs +363 |
| 21 | 20260901_mini-v2.4.2_gemini-3-5-flash | 359/500 | 71.8% | 3.0 h | CAIN needs +360 |
| 22 | 20250807_openhands_gpt5 | 359/500 | 71.8% | 1.5 h | CAIN needs +360 |
| 23 | 20250929_Prometheus_v1.2_gpt5 | 356/500 | 71.2% | 1.7 h | CAIN needs +357 |
| 24 | 20250710_bloop | 356/500 | 71.2% | 1.4 h | CAIN needs +357 |
| 25 | 20251014_Lingxi_kimi_k2 | 356/500 | 71.2% | 1.2 h | CAIN needs +357 |
| 26 | 20250715_qodo_command | 356/500 | 71.2% | 1.2 h | CAIN needs +357 |
| 27 | 20250623_warp | 355/500 | 71.0% | 1.7 h | CAIN needs +356 |
| 28 | 20250611_moatless_claude-4-sonnet-20250514 | 354/500 | 70.8% | 1.3 h | CAIN needs +355 |
| 29 | 20250519_trae | 353/500 | 70.6% | 1.2 h | CAIN needs +354 |
| 30 | 20250610_augment_agent_v1 | 352/500 | 70.4% | 1.4 h | CAIN needs +353 |
| 31 | 20250524_openhands_claude_4_sonnet | 352/500 | 70.4% | 1.3 h | CAIN needs +353 |
| 32 | 20250515_Refact_Agent | 352/500 | 70.4% | 1.2 h | CAIN needs +353 |
| 33 | 20250519_devlo | 351/500 | 70.2% | 1.1 h | CAIN needs +352 |
| 34 | 20250430_zencoder_ai | 350/500 | 70.0% | 1.1 h | CAIN needs +351 |
| 35 | 20250805_openhands-Qwen3-Coder-480B-A35B-Instruct | 348/500 | 69.6% | 1.1 h | CAIN needs +349 |
| 36 | 20250516_cortexa_o3 | 341/500 | 68.2% | 1.1 h | CAIN needs +342 |
| 37 | 20250930_zai_glm4-6 | 341/500 | 68.2% | 1.1 h | CAIN needs +342 |
| 38 | 20250522_sweagent_claude-4-sonnet-20250514 | 333/500 | 66.6% | 1.0 h | CAIN needs +334 |
| 39 | 20250514_aime_coder | 332/500 | 66.4% | 1.1 h | CAIN needs +333 |
| 40 | 20250415_openhands | 329/500 | 65.8% | 0.9 h | CAIN needs +330 |
| 41 | 20250716_openhands_kimi_k2 | 327/500 | 65.4% | 0.9 h | CAIN needs +328 |
| 42 | 20250316_augment_agent_v0 | 327/500 | 65.4% | 0.8 h | CAIN needs +328 |
| 43 | 20250405_amazon-q-developer-agent-20250405-dev | 327/500 | 65.4% | 0.8 h | CAIN needs +328 |
| 44 | 20250117_wandb_programmer_o1_crosscheck5 | 323/500 | 64.6% | 0.9 h | CAIN needs +324 |
| 45 | 20250503_patchpilot-v1.1-o4-mini | 323/500 | 64.6% | 0.9 h | CAIN needs +324 |
| 46 | 20250728_zai_glm4-5 | 321/500 | 64.2% | 0.9 h | CAIN needs +322 |
| 47 | 20250206_agentscope | 317/500 | 63.4% | 0.7 h | CAIN needs +318 |
| 48 | 20250224_tools_claude-3-7-sonnet | 316/500 | 63.2% | 0.7 h | CAIN needs +317 |
| 49 | 20250110_blackboxai_agent_v1.1 | 314/500 | 62.8% | 0.7 h | CAIN needs +315 |
| 50 | 20250228_epam-ai-run-claude-3-5-sonnet | 314/500 | 62.8% | 0.6 h | CAIN needs +315 |
| 51 | 20250225_sweagent_claude-3-7-sonnet | 312/500 | 62.4% | 0.6 h | CAIN needs +313 |
| 52 | 20241221_codestory_midwit_claude-3-5-sonnet_swe-search | 311/500 | 62.2% | 0.6 h | CAIN needs +312 |
| 53 | 20250203_openhands_4x_scaled | 304/500 | 60.8% | 0.6 h | CAIN needs +305 |
| 54 | 20250901_entroPO_R2E_QwenCoder30BA3B_tts | 302/500 | 60.4% | 0.6 h | CAIN needs +303 |
| 55 | 20250110_learn_by_interact_claude3.5 | 301/500 | 60.2% | 0.5 h | CAIN needs +302 |
| 56 | 20250629_deepswerl_r2eagent_tts | 294/500 | 58.8% | 0.5 h | CAIN needs +295 |
| 57 | 20241213_devlo | 291/500 | 58.2% | 0.5 h | CAIN needs +292 |
| 58 | 20250410_cortexa | 291/500 | 58.2% | 0.5 h | CAIN needs +292 |
| 59 | 20241223_emergent | 286/500 | 57.2% | 0.4 h | CAIN needs +287 |
| 60 | 20250924_artemis_agent_v2 | 285/500 | 57.0% | 0.5 h | CAIN needs +286 |
| 61 | 20241208_gru | 285/500 | 57.0% | 0.4 h | CAIN needs +286 |
| 62 | 20250405_swe-rizzo_claude37 | 283/500 | 56.6% | 0.4 h | CAIN needs +284 |
| 63 | 20241212_epam-ai-run-claude-3-5-sonnet | 277/500 | 55.4% | 0.4 h | CAIN needs +278 |
| 64 | 20241202_amazon-q-developer-agent-20241202-dev | 275/500 | 55.0% | 0.4 h | CAIN needs +276 |
| 65 | 20241108_devlo | 271/500 | 54.2% | 0.4 h | CAIN needs +272 |
| 66 | 20251110_frogboss-32b | 268/500 | 53.6% | 0.3 h | CAIN needs +269 |
| 67 | 20250804_codesweep_sweagent_kimi_k2_instruct | 267/500 | 53.4% | 0.3 h | CAIN needs +268 |
| 68 | 20250120_Bracket | 266/500 | 53.2% | 0.4 h | CAIN needs +267 |
| 69 | 20241029_OpenHands-CodeAct-2.1-sonnet-20241022 | 265/500 | 53.0% | 0.3 h | CAIN needs +266 |
| 70 | 20241212_google_jules_gemini_2.0_flash_experimental | 261/500 | 52.2% | 0.3 h | CAIN needs +262 |
| 71 | 20250901_entroPO_R2E_QwenCoder30BA3B | 261/500 | 52.2% | 0.3 h | CAIN needs +262 |
| 72 | 20241125_enginelabs | 259/500 | 51.8% | 0.3 h | CAIN needs +260 |
| 73 | 20250122_autocoderover-v2.1-claude-3-5-sonnet-20241022 | 258/500 | 51.6% | 0.3 h | CAIN needs +259 |
| 74 | 20250805_openhands-Qwen3-Coder-30B-A3B-Instruct | 258/500 | 51.6% | 0.3 h | CAIN needs +259 |
| 75 | 20241202_agentless-1.5_claude-3.5-sonnet-20241022 | 254/500 | 50.8% | 0.3 h | CAIN needs +255 |
| 76 | 20241028_solver | 250/500 | 50.0% | 0.3 h | CAIN needs +251 |
| 77 | 20241125_marscode-agent-dev | 250/500 | 50.0% | 0.3 h | CAIN needs +251 |
| 78 | 20241105_nfactorial | 246/500 | 49.2% | 0.2 h | CAIN needs +247 |
| 79 | 20241022_tools_claude-3-5-sonnet-updated | 245/500 | 49.0% | 0.2 h | CAIN needs +246 |
| 80 | 20241025_composio_swekit | 243/500 | 48.6% | 0.2 h | CAIN needs +244 |
| 81 | 20241106_navie-2-gpt4o-sonnet | 236/500 | 47.2% | 0.2 h | CAIN needs +237 |
| 82 | 20250616_Skywork-SWE-32B+TTS_Bo8 | 235/500 | 47.0% | 0.2 h | CAIN needs +236 |
| 83 | 20250520_openhands_devstral_small | 234/500 | 46.8% | 0.2 h | CAIN needs +235 |
| 84 | 20241023_emergent | 233/500 | 46.6% | 0.2 h | CAIN needs +234 |
| 85 | 20241108_autocoderover-v2.0-claude-3-5-sonnet-20241022 | 231/500 | 46.2% | 0.2 h | CAIN needs +232 |
| 86 | 20250528_patchpilot_Co-PatcheR | 230/500 | 46.0% | 0.2 h | CAIN needs +231 |
| 87 | 20240924_solver | 227/500 | 45.4% | 0.2 h | CAIN needs +228 |
| 88 | 20240824_gru | 226/500 | 45.2% | 0.2 h | CAIN needs +227 |
| 89 | 20251110_frogmini-14b | 225/500 | 45.0% | 0.2 h | CAIN needs +226 |
| 90 | 20250118_codeshellagent_gemini_2.0_flash_experimental | 221/500 | 44.2% | 0.2 h | CAIN needs +222 |
| 91 | 20240920_solver | 218/500 | 43.6% | 0.2 h | CAIN needs +219 |
| 92 | 20250214_agentless_lite_o3_mini | 212/500 | 42.4% | 0.1 h | CAIN needs +213 |
| 93 | 20250527_amazon.nova-premier-v1.0 | 212/500 | 42.4% | 0.1 h | CAIN needs +213 |
| 94 | 20250629_deepswerl_r2eagent | 211/500 | 42.2% | 0.1 h | CAIN needs +212 |
| 95 | 20250806_SWE-Exp_DeepSeek-V3 | 210/500 | 42.0% | 0.1 h | CAIN needs +211 |
| 96 | 20250112_ugaiforge | 208/500 | 41.6% | 0.1 h | CAIN needs +209 |
| 97 | 20241030_nfactorial | 208/500 | 41.6% | 0.1 h | CAIN needs +209 |
| 98 | 20250226_swerl_llama3_70b | 206/500 | 41.2% | 0.1 h | CAIN needs +207 |
| 99 | 20241113_nebius-search-open-weight-models-11-24 | 203/500 | 40.6% | 0.1 h | CAIN needs +204 |
| 100 | 20241016_composio_swekit | 203/500 | 40.6% | 0.1 h | CAIN needs +204 |
| 101 | 20240820_honeycomb | 203/500 | 40.6% | 0.1 h | CAIN needs +204 |
| 102 | 20241022_tools_claude-3-5-haiku | 203/500 | 40.6% | 0.1 h | CAIN needs +204 |
| 103 | 20250511_sweagent_lm_32b | 201/500 | 40.2% | 0.1 h | CAIN needs +202 |
| 104 | 20241029_epam-ai-run-claude-3-5-sonnet | 198/500 | 39.6% | 0.1 h | CAIN needs +199 |
| 105 | 20240721_amazon-q-developer-agent-20240719-dev | 194/500 | 38.8% | 0.1 h | CAIN needs +195 |
| 106 | 20241028_agentless-1.5_gpt4o | 194/500 | 38.8% | 0.1 h | CAIN needs +195 |
| 107 | 20240628_autocoderover-v20240620 | 192/500 | 38.4% | 0.1 h | CAIN needs +193 |
| 108 | 20250725_sweagent_devstral_small_2507 | 190/500 | 38.0% | 0.1 h | CAIN needs +191 |
| 109 | 20250616_Skywork-SWE-32B | 190/500 | 38.0% | 0.1 h | CAIN needs +191 |
| 110 | 20240617_factory_code_droid | 185/500 | 37.0% | 0.1 h | CAIN needs +186 |
| 111 | 20240620_sweagent_claude3.5sonnet | 168/500 | 33.6% | 0.1 h | CAIN needs +169 |
| 112 | 20250306_SWE-Fixer_Qwen2.5-7b-retriever_Qwen2.5-72b-editor | 164/500 | 32.8% | 0.1 h | CAIN needs +165 |
| 113 | 20240612_MASAI_gpt4o | 163/500 | 32.6% | 0.1 h | CAIN needs +164 |
| 114 | 20241120_artemis_agent | 160/500 | 32.0% | 0.0 h | CAIN needs +161 |
| 115 | 20241007_nfactorial | 158/500 | 31.6% | 0.1 h | CAIN needs +159 |
| 116 | 20241128_SWE-Fixer_Qwen2.5-7b-retriever_Qwen2.5-72b-editor_20241128 | 151/500 | 30.2% | 0.1 h | CAIN needs +152 |
| 117 | 20241002_lingma-agent_lingma-swe-gpt-72b | 144/500 | 28.8% | 0.0 h | CAIN needs +145 |
| 118 | 20241016_epam-ai-run-gpt-4o | 135/500 | 27.0% | 0.1 h | CAIN needs +136 |
| 119 | 20240615_appmap-navie_gpt4o | 131/500 | 26.2% | 0.0 h | CAIN needs +132 |
| 120 | 20241001_nfactorial | 129/500 | 25.8% | 0.0 h | CAIN needs +130 |
| 121 | 20240509_amazon-q-developer-agent-20240430-dev | 128/500 | 25.6% | 0.0 h | CAIN needs +129 |
| 122 | 20240918_lingma-agent_lingma-swe-gpt-72b | 125/500 | 25.0% | 0.0 h | CAIN needs +126 |
| 123 | 20240820_epam-ai-run-gpt-4o | 120/500 | 24.0% | 0.0 h | CAIN needs +121 |
| 124 | 20250627_agentless_MCTS-Refine-7B | 116/500 | 23.2% | 0.0 h | CAIN needs +117 |
| 125 | 20240728_sweagent_gpt4o | 116/500 | 23.2% | 0.0 h | CAIN needs +117 |
| 126 | 20240402_sweagent_gpt4 | 112/500 | 22.4% | 0.0 h | CAIN needs +113 |
| 127 | 20241002_lingma-agent_lingma-swe-gpt-7b | 91/500 | 18.2% | 0.0 h | CAIN needs +92 |
| 128 | 20240402_sweagent_claude3opus | 79/500 | 15.8% | 0.0 h | CAIN needs +80 |
| 129 | 20240918_lingma-agent_lingma-swe-gpt-7b | 51/500 | 10.2% | 0.0 h | CAIN needs +52 |
| 130 | 20240402_rag_claude3opus | 35/500 | 7.0% | 0.0 h | CAIN needs +36 |
| 131 | 20231010_rag_claude2 | 22/500 | 4.4% | 0.0 h | CAIN needs +23 |
| 132 | 20240402_rag_gpt4 | 14/500 | 2.8% | 0.0 h | CAIN needs +15 |
| 133 | 20231010_rag_swellama7b | 7/500 | 1.4% | 0.0 h | CAIN needs +8 |
| 134 | 20231010_rag_swellama13b | 6/500 | 1.2% | 0.0 h | CAIN needs +7 |
| 135 | 20231010_rag_gpt35 | 2/500 | 0.4% | 0.0 h | CAIN needs +3 |
| 136 | CAIN-42 (70B tool agent, E124) — LAST PLACE | 0/500 | 0.0% | — |
Every row is a published SWE-bench Verified submission. Scaffolds and model versions differ, so this is a SWE-bench rank, not METR HCAST. CAIN's 0/500 is reported as measured. Generated by scripts/e124/rank_agents.py (7 agents, hash-chained) — no row is typed by hand. AGI status: NOT ACHIEVED.