发现信息
- 作者:Kevin M. Chen, Marco Cusumano-Towner, Brody Huval, Petrenko, Aleksei, Jackson Hamburger, Vladlen Koltun, Philipp Krähenbühl
- 年份:2025
- 发表源:arXiv (Cornell University)
- OpenAlex ID:https://openalex.org/W4407183653
- DOI:https://doi.org/10.48550/arxiv.2502.01600
- arXiv:2502.01600
- 检索主题:memory-learning
- OpenAlex 引用数快照:123(2026-11-12)
摘要线索
Interactive digital agents (IDAs) leverage APIs of stateful digital environments to perform tasks in response to user requests. While IDAs powered by instruction-tuned large language models (LLMs) can react to feedback from interface invocations in multi-step exchanges, they have not been trained in their respective digital environments. Prior methods accomplish less than half of tasks in sophisticated benchmarks such as AppWorld. We present a reinforcement learning (RL) approach that trains IDAs directly in their target environments. We formalize this training as a partially observable Markov decision process and derive LOOP, a data- and memory-efficient variant of proximal policy optimization. LOOP uses no value network and maintains exactly one copy of the underlying LLM in memory, making its implementation straightforward and as memory-efficient as fine-tuning a single LLM. A 32-billion-parameter agent trained with LOOP in the AppWorld environment outperforms the much larger OpenAI o1 agent by 9 percentage points (15% relative). To our knowledge, this is the first reported application of RL to IDAs that interact with a stateful, multi-domain, multi-app environment via direct API calls. Our analysis sheds light on the effectiveness of RL in this area, showing that the agent learns to consult the API documentation, avoid unwarranted assumptions, minimize confabulation, and recover from setbacks.
核验与筛选
- 当前状态:待筛选、去重和一手来源核验。
- OpenAlex 元数据和引用数只用于发现,不作为正式引文或质量判断。
- 记录正式版本、venue、方法细节、实验协议和局限后,才迁入稳定论文卡片。