According to BlockBeats, OpenAI has released a new AI agent evaluation benchmark called PaperBench. This benchmark, unveiled at 1 a.m. UTC+8, focuses on assessing the capabilities of AI agents in areas such as search, integration, and execution. It requires the replication of top papers from the 2024 International Conference on Machine Learning, testing the agents' understanding of the content, code writing, and experiment execution.
OpenAI's test data reveals that while renowned large models have not yet surpassed top machine learning Ph.D. experts, they are proving beneficial in assisting with learning and understanding research content.