Why could the same AI rebuild a 16,000-line program without seeing its code, yet fail to customize a library for METR? The answer shapes what you hand it.

The assumption that extraordinary AI performance in one area implies broader competence needs re-examination, especially with AI models demonstrating exceptional abilities in specific domains like mathematics. This uneven progress, where AI excels dramatically in certain tasks while lagging in others, could lead to a future of astonishing successes coupled with difficult-to-reconcile failures. The way AI training feedback concentrates gains in areas with plentiful data might exacerbate this unevenness, making overall ‘smartness’ a less useful descriptor.

• We tend to assume AI progress in one difficult area signifies deeper, general intelligence that should help in others.
• OpenAI’s release of mathematical research manuscripts from an internal model highlights narrow superintelligence – extraordinary performance in specific tasks.
• It’s possible AI can improve dramatically in many things, but its most exceptional abilities may advance much faster, creating unevenness.
• This uneven progress could result in astonishing AI successes alongside failures that are hard to explain, making generalized ‘smartness’ less predictive.
• Training feedback concentration in areas with cheap and plentiful data may lead to disproportionately faster improvements in those specific abilities.
  https://bender.layer3.press/articles/962f603b-eb99-4b4f-a580-d5a6522ed2cb