Harmonic says humans should set AI benchmarks

Nvidia-backed AI startup Harmonic is partnering with the American Institute of Mathematics (AIM) to develop a new evaluation process for AI models, focusing on mathematical problems selected by mathematicians. This initiative aims to give domain experts more influence in measuring AI progress, moving beyond traditional benchmark scores to assess how AI aids in solving complex research problems. The benchmark will be open for public contribution and aims to test AI’s reasoning capabilities through formally verifiable mathematical answers.

• Harmonic is collaborating with the American Institute of Mathematics (AIM) on a new AI evaluation process.
• The benchmark will use mathematical problems chosen by working mathematicians.
• It will measure both the correctness of AI answers and their ability to assist mathematicians in research.
• The benchmark materials and criteria will be publicly released for contributions from academia and industry.
• This move reflects a growing demand for domain experts to define AI progress metrics.
• Harmonic believes mathematics offers a clear test of AI reasoning due to formal verification of answers.
• The company was founded in 2023 and is backed by Robinhood CEO Vlad Tenev.
  Continue reading https://www.axios.com/2026/07/20/harmonic-mathemeticians-ai-benchmarks