AgentBench (2024年)（エージェントベンチ）

概要

AgentBench は2024年成熟・Industry-leading LLM-as-Agent benchmark 2024年Tsinghua + UC Berkeley AgentBench LLM-as-Agent benchmark position確立。Industry-leading 8 distinct environments + API + Sim + OS + DB + KG + Card + CodeKG + Lateral + GPT-4 leads + open-source gap exposed + 2023年8月 paper → 2024 mature。

主な特徴・仕組み

• Industry-leading Xiao Liu (Tsinghua) + Hao Yu + Hanchen Zhang + Yifan Xu + Xuanyu Lei + Hanyu Lai + Yu Gu + Hangliang Ding + Kaiwen Men + Kejuan Yang + Shudan Zhang + Xiang Deng + Aohan Zeng + Zhengxiao Du + Chenhui Zhang + Sheng Shen + Tianjun Zhang + Yu Su + Huan Sun + Minlie Huang + Yuxiao Dong + Jie Tang 2023年8月 paper → 2024 mature
• Industry-leading AgentBench LLM-as-Agent benchmark signature
• Industry-leading 8 distinct environments signature: OS + DB + KG + DCG + LTP + ALFWorld + WebShop + WebArena
• Industry-leading API + Sim + OS + DB + KG + Card + CodeKG + Lateral coverage signature
• Industry-leading GPT-4 leads (4.8 avg score) signature 2023→2024
• Industry-leading open-source gap exposed signature (Llama-2-70B 2.7 avg)
• Industry-leading multi-turn open-ended task evaluation signature
• Industry-leading agent capability comprehensive assessment signature
• Industry-leading 2023年8月 ICLR 2024 paper "AgentBench Evaluating LLMs as Agents"
• Industry-leading Apache 2.0 license signature
• Industry-leading Tsinghua + UC Berkeley + collaborator institutions

まとめ

AgentBench = 2024年成熟Tsinghua + UC Berkeley発表のpremium LLM-as-Agent benchmark。Industry-leading 8 distinct environments OS + DB + KG + DCG + LTP + ALFWorld + WebShop + WebArena + API + Sim + OS + DB + KG + Card + CodeKG + Lateral coverage + GPT-4 leads 4.8 avg score 2023→2024 + open-source gap exposed Llama-2-70B 2.7 avg + multi-turn open-ended task evaluation + agent capability comprehensive assessment + 2023年8月 ICLR 2024 paper "AgentBench Evaluating LLMs as Agents" + Apache 2.0 + Tsinghua + UC Berkeley + collaborator institutions position確立。