Did you know AI Benchmark Group Under Fire for Delayed Disclosure of OpenAI Funding

 

FrontierMath is being scrutinized for failing to disclose details about funding received from OpenAI until after it went public.

For those still wondering, FrontierMath is known for creating AI benchmarks in the world of mathematics. It purposefully chose not to disclose more on the matter. This includes how a lot of investments were made at the start, yet so many people in its own organization were unaware of this.

The matter has only recently been raised which has drawn serious scrutiny linked to impropriety from those in the AI world. FrontierMath comprises expert-level sums used to check AI’s math skills. It was also one of the benchmarks used by OpenAI when debuting its new AI o3.

Thanks to a contractor from Epoch AI who goes by the name Meemi, the matter was brought to light of how clueless contributors were about the matter and how they were not informed of the AI giant’s involvement until it went public.

Meemi shared how most communication on this front was non-transparent instead of being out there in the open if funds from OpenAI were received. Contractors should also provide more transparent data about the potential of work used for capabilities and if they would like to work on such a benchmark or not.

Other users found on social media shared how many concerns they had about the secrecy and how it could tarnish FrontierMath’s reputation in the end. Furthermore, OpenAI had insights about all the sums and solutions present inside the benchmark. Still, the launch of o3 was done without anyone knowing details about the background story.

Another post published by Carina Hong spoke about OpenAI having privileged access to FrontierMath thanks to its bond with EpochAI. That matter is not sitting down well with a lot of contributors.

Several math experts contributed to the benchmark and the fact that they were unaware that the AI giant OpenAI would have special access to it while others would not is just wrong. Many others expressed how they aren’t sure if they would have signed up in the first place had they known about this funding.

Epoch AI did admit to being wrong in this situation in a new post shared after the news went out. They did confirm they made a huge error in terms of transparency but also reaffirmed that no integrity related to FrontierMath was compromised. There were added talks about OpenAI having access but also a verbal agreement in place that bars them from using the problem set at FrontierMath to train AI.

The matter is another major example of the many challenges related to designing new empirical benchmarks to gauge AI. This includes security the important resources for developing benchmarks without the need of attacking conflicts of interest.

Other experts in the world of math spoke about launching their individual tests to evaluate the situation and then reporting if the above is true.