Beyond Pass@k: Developing Human-Centric Evaluation Metrics for AI Pair Programmers
ICML 2026 Workshop on Deep Learning for Code
A human-centric score for AI-written code; cut developer review time by 28%.
A written machine learning competition, open worldwide. Do a project, write it up, and the strongest report wins $10,000 in Anthropic credits.
ICML 2026 Workshop on Deep Learning for Code
A human-centric score for AI-written code; cut developer review time by 28%.
ACL 2026 Workshop on Advances in Language and Vision Research (ALVR)
Fixes relational blind spots in vision-language models: +8.6 on Winoground, no fine-tuning.
NeurIPS 2025 Workshop on Imageomics: Discovering Biological Knowledge from Images Using AI
Hyperbolic embeddings that follow the plant family tree, on PlantCLEF.
| September 1, 2026 | Submissions open |
| December 6, 2026 | Deadline, 11:59 pm AoE |
| January 18, 2027 | Finalists announced |
| February 8, 2027 | Winner announced |
Gold
$10,000
in Anthropic credits
Silver
$6,000
in Anthropic credits
Bronze
$4,000
in Anthropic credits
Anthropic credits pay for the Claude API. Use them for development: build, test and ship your next machine learning project.
High school and undergraduate students worldwide.
Yes. The work should be done, or substantially extended, during the season.
Yes. Failed research directions are valuable contributions to science and will be treated as such.
They are usage credits for Anthropic's API, which runs the Claude models.
No. Judges grade the report. Your code, a Python .zip of up to 150 MB, is used to verify the work.
The $25 registration fee pays our review committee of ML professionals and researchers.
Yes, as long as you disclose how you used them. Read the AI policy.