Cognition AI

Devin

Under evaluation
Maker
Cognition AI
Origin
United States
Released
Mar 12, 2024
Category
Coding & Agentic

Strengths & weaknesses

  • Runs autonomously in the cloud, including overnight and unattended
  • Strong on large, well-defined jobs like migrations and refactors
  • Usage-based pricing gets expensive fast for heavy workloads
  • Needs clear upfront specs; weak at open-ended, exploratory work

Evaluation

Awaiting evaluation
  • Code qualityw45—

    No independent measurement yet

  • Web developmentw40—

    No independent measurement yet

  • Codebase contextw15—

    No independent measurement yet

Price & availability

Measured the same way, deliberately kept out of the score: what a model costs doesn’t change what it can do.

  • Cost efficiencynot scored—

    No independent measurement yet

w = weight, each criterion’s share of the overall score. Missing marks don’t count for or against. How scoring works

More in Coding & Agentic

View all →
StepFun

Step 5 Preview

Under evaluation
  • 600B mixture-of-experts with 27B active and a 1M-token context, built for long-horizon agentic work
  • $1.00 input and $2.70 output per million tokens, with open weights promised for October 2026
  • Generated roughly 1.7× the median output tokens on Artificial Analysis's index, so real cost runs well above the headline rate
  • A preview with no independent coding-benchmark result yet, so it carries no overall score here
OpenAI

GPT-5.3 Codex

Under evaluation
  • Excellent terminal automation, git operations, and CI/CD debugging
  • Far more token-efficient than reasoning-heavy rivals on routine tasks
  • Needs detailed, unambiguous instructions; struggles with vague requests
  • Smaller context window than some rivals, a constraint on huge monorepos
Anthropic

Claude Opus 4.6 (Claude Code)

7.5/10
  • Strong at inferring intent from vague prompts and architectural context
  • 1M-token context supports coherent multi-file, cross-repo refactors
  • Can use many more tokens than leaner coding models on routine work
  • Narrates its reasoning at length, which slows down quick tasks
Alibaba

Qwen3 Coder 480B

4.7/10
  • Open-weight performance within striking distance of proprietary leaders
  • Free to self-host, appealing for cost-sensitive or air-gapped teams
  • Requires serious infrastructure to run at full size
  • Tooling and IDE integrations are less mature than Copilot, Cursor, or Codex