发现信息

  • 作者:Yixu Wang, Xin Eric Wang, Yang Yao, Xinyuan Li, Xibang Yang, Yan Teng, Xingjun Ma, Yingchun Wang
  • 年份:2026
  • 发表源:Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
  • OpenAlex ID:https://openalex.org/W7166865524
  • DOI:https://doi.org/10.18653/v1/2026.findings-acl.727
  • arXiv:待核验
  • 检索主题:self-evolving
  • OpenAlex 引用数快照:2(2026-11-12)

摘要线索

The rapid integration of Large Language Models (LLMs) into high-stakes domains necessitates reliable safety and compliance evaluation.However, existing static benchmarks are ill-equipped to address the dynamic nature of AI risks and evolving regulations, creating a critical safety gap.This paper introduces a new paradigm of agentic safety evaluation, reframing evaluation as a continuous and selfevolving process rather than a one-time audit.We then propose a novel multi-agent framework AgenticEval, which autonomously ingests unstructured policy documents to generate and perpetually evolve a comprehensive safety benchmark.AgenticEval leverages a synergistic pipeline of specialized agents and incorporates a Self-evolving Evaluation loop, where the system learns from evaluation results to craft progressively more sophisticated and targeted test cases.Our experiments demonstrate the effectiveness of AgenticEval, showing a consistent decline in model safety as the evaluation hardens.For instance, GPT-5's safety rate on the EU AI Act drops from 72.50% to 36.36% over successive iterations.These findings reveal the limitations of static assessments and highlight our framework's ability to uncover deep vulnerabilities missed by traditional methods, underscoring the urgent need for dynamic evaluation ecosystems to ensure the safe and responsible deployment of advanced AI.

核验与筛选

  • 当前状态:待筛选、去重和一手来源核验。
  • OpenAlex 元数据和引用数只用于发现,不作为正式引文或质量判断。
  • 记录正式版本、venue、方法细节、实验协议和局限后,才迁入稳定论文卡片。