[논문리뷰] AdaExploreBench: Benchmarking LLM Agents for Proactive Exploration and Test-time Adaptation in Evolving Environments

링크: 논문 PDF로 바로 열기

The paper is about "AdaExploreBench: Benchmarking LLM Agents for Proactive Exploration and Test-time Adaptation in Evolving Environments".

I need to extract the following information:

• Authors: Rui Wang, Zishen Lai, Jiayu Liu, Cheng Qian, Jeonghwan Kim, Heng Wang, Xuejun Zhang, Jiateng Liu, Bingxuan Li, Shujin Wu, Qing Zong, Tianshi Zheng, Chunyang Li, Qihan Lin, Chen Li, Dadi Guo, Xiusi Chen, Yangqiu Song
• Keywords: Need to identify 5-8 from the abstract and introduction.
  • Proactive Exploration
  • Test-time Adaptation
  • LLM Agents
  • Evolving Environments
  • Environmental Dynamics
  • Latent Rules
  • Rule Drift
  • Benchmarking
• Key Terms & Definitions:
  • Proactive Exploration: Agents actively interact with an unknown environment to collect evidence and understand its dynamics.
  • Adaptive Exploration: The ability to detect, re-explore, and revise in response to unannounced environmental dynamics changes.
  • Rule Drift: Unannounced changes in the latent rules governing environmental dynamics, requiring agents to detect and adapt.
  • Calibrated Accuracy (CA): A metric that grants partial credit for near-optimal choices, calibrated so a random policy gets 0 and an optimal policy gets 1.
  • Factor Recognition: Measures how accurately an agent identifies the relevant factors of a rule through auxiliary assessment.
  • Change Recognition: Measures whether an agent correctly identifies what changed in the rule dynamics.
• Motivation & Problem Statement:
  • LLM agents are deployed in unfamiliar and evolving environments.
  • They need to proactively explore and adapt to changing environmental dynamics.
  • Existing benchmarks rarely test proactive discovery and adaptation in evolving environments.
  • Limitations of existing benchmarks (Table 1): often provide evidence or use static environments.
  • Central question: Can LLM agents proactively explore to understand and adapt to environmental dynamics, especially with unannounced changes?
• Method & Key Results:
  • AdaExploreBench: A benchmark for proactive exploration and test-time adaptation in evolving environments.
  • Instantiates environmental dynamics as hidden latent rules that map actions to outcomes.
  • Agents must actively acquire evidence to infer these rules.
  • Features unannounced rule drift where latent rules change without notification, requiring agents to detect performance degradation and re-explore.
  • Scalable Construction Pipeline: Uses an LLM-based generation pipeline to synthesize complex and diverse evaluation instances.
  • Evolving Environment Design: Reflects real-world challenges with latent rules and rule changes.
  • Runtime Protocol: Agents interact by querying cases (Probe Batch) or proceeding (Request Next Batch), and submitting actions (Submit Action Batch), receiving utility feedback. Evaluation History Masking isolates proactive exploration. Auxiliary Assessment directly evaluates factor and change recognition.
  • Experiment Results (Table 2):
    • Eight leading LLMs evaluated.
    • All models fall below 70% Calibrated Accuracy (CA) in the default setting.
    • GPT-6 Sol achieves 61.9% CA, while Gemini-3.8-Flash achieves 67.25% CA (best).
    • Providing sufficient evidence or removing rule drift significantly improves average performance by 9.4% and 6.2% respectively (Figure 2). This implies agents struggle with evidence acquisition and adaptation.
    • DeepSeek-V4.1-Flash probes the most but has lower CA than Gemini-3.8-Flash and GPT-6-Sol, suggesting exploration quality is more important than quantity.
    • Change Recognition ability strongly correlates with end-to-end performance.
  • Analysis (Figure 2, 3, Table 3, 4):
    • Removing rule drift significantly recovers performance for most models (average 6.2% CA gain).
    • Providing sufficient evidence externally significantly improves performance (average 10.3% CA gain).
    • Forcing agents to explore more (Forced-Probe) barely helps, sometimes even degrades performance (mean -2.0% CA change). This suggests probe informativeness is a bottleneck.
    • Accuracy drops immediately at rule drift and never fully recovers for any model (Figure 3).
    • Re-probing helps within-model change recognition but doesn't fully explain cross-model differences, implying reasoning ability over evidence is crucial.
    • Agents tend to settle on incomplete environmental models, with precision generally exceeding recall in Factor and Change Recognition (Table 4).
    • Agents struggle more with direction reversals of existing factors than with factor replacements (Table 10).
• Conclusion & Impact:
  • AdaExploreBench is a new benchmark for proactive exploration and test-time adaptation of LLM agents in evolving environments.
  • Current LLM agents are limited in proactive exploration and test-time adaptation.
  • They struggle to autonomously acquire evidence and keep their environmental understanding updated, especially when rule drift occurs.
  • The research highlights the need for agents to actively challenge established assumptions and revise knowledge.
  • AdaExploreBench provides a foundation for developing agents capable of reliable exploration and adaptation in dynamic environments.

Figures to Select:

1. Figure 1: Overview of AdaExploreBench. This is a good candidate for the overall architecture.
   • Image URL: https://arxiv.org/html/2610.25950v1/x1.png
   • Caption: "AdaExploreBench의 전체 개요"
2. Figure 2: Calibrated Accuracy under default and the three other settings. This shows key results for different experimental settings.
   • Image URL: https://arxiv.org/html/2610.25950v1/x2.png
   • Caption: "다양한 설정에서의 Calibrated Accuracy"
3. Figure 3: Calibrated Accuracy per evaluation batch under default, with batch 0 marking the last pre-change batch. This figure shows the performance drop after rule drift.
   • Image URL: https://arxiv.org/html/2610.25950v1/x3.png
   • Caption: "평가 배치별 Calibrated Accuracy 변화"

Let's double-check the image URLs by looking for <img> tags in the browsed content for 2610.25950v1/x1.png, 2610.25950v1/x2.png, 2610.25950v1/x3.png. The HTML shows these images.

Now, construct the output.저자: Rui Wang, Zishen Lai, Jiayu Liu, et al.

1. Key Terms & Definitions (핵심 용어 및 정의)

• Proactive Exploration: LLM agents가 알려지지 않은 환경에서 스스로 상호작용하여 증거를 수집하고 환경의 Dynamics를 이해하려는 능력.
• Adaptive Exploration: 환경 Dynamics의 Unannounced 변화를 감지하고, 기존 지식이 Old해졌음을 인식하며, Re-explore하여 적응하는 능력.
• Rule Drift: 환경 Dynamics를 제어하는 Latent Rule이 예고 없이 변경되는 현상으로, Agent가 이를 감지하고 재학습해야 하는 도전 과제이다.
• Calibrated Accuracy (CA): Agent의 End-to-End Performance를 측정하는 Metric으로, Optimal하지 않은 선택에도 부분 점수를 부여하며, Random Policy는 0, Optimal Policy는 1을 얻도록 보정된다.
• Change Recognition: Agent가 Rule 내에서 발생한 변화를 얼마나 정확하게 식별하는지를 측정하는 Metric.

2. Motivation & Problem Statement (연구 배경 및 문제 정의)

본 연구는 LLM agents가 Unfamiliar하고 Evolving하는 환경에서 Proactive Exploration 및 Test-time Adaptation 능력을 벤치마킹하는 AdaExploreBench를 제안한다. 기존 연구들은 LLM agents의 환경 이해 및 적응 능력을 충분히 테스트하지 못했다는 한계점을 가지고 있다. 많은 기존 벤치마크는 Agent에게 증거를 제공하거나(Evidence Given), 환경이 Stationary한 상태로 유지된다(Static Environments) [cite: 1, Table 1]. 이러한 한계점들로 인해, Agent가 능동적으로 환경 규칙을 Discover하고, 특히 Latent Rule이 예고 없이 변경되는 Rule Drift 상황에서 지식을 Updated할 수 있는지 평가하는 것이 불가능했다. 따라서 본 논문은 Agents가 자율적으로 환경을 Explore하고, Rule Drift를 인식하여 Adaptation하는 능력을 평가하기 위한 새로운 벤치마크의 필요성을 강조한다.

3. Method & Key Results (제안 방법론 및 핵심 결과)

저자들은 LLM agents의 Proactive Exploration 및 Test-time Adaptation 능력을 평가하기 위해 AdaExploreBench를 제안한다. 이 벤치마크는 환경 Dynamics를 Latent Rules로 정의하고, Agent는 Probe Batch를 통해 증거를 수집하여 규칙을 추론하며, Unannounced Rule Drift 발생 시 이를 감지하고 Re-explore하도록 요구한다. AdaExploreBench는 Scalable Construction Pipeline을 통해 복잡하고 다양한 평가 Instance를 생성하며, Runtime Protocol을 통해 Agent의 상호작용 및 평가를 관리한다 [cite: 1, Figure 1].

실험에서는 8개의 최신 LLM이 AdaExploreBench에서 평가되었으며, 그 결과 Proactive Exploration 및 Test-time Adaptation이 여전히 매우 도전적인 과제임을 보여준다 [cite: 1, Table 2]. 모든 모델은 Default Setting에서 70% 미만의 Calibrated Accuracy (CA)를 기록했으며, 최고 성능을 보인 Gemini-3.8-Flash조차 67.25% CA에 그쳤다 [cite: 1, Table 2]. 특히, Rule Drift가 발생하면 모든 모델의 Calibrated Accuracy가 즉시 하락하며, 완전히 회복되지 못하는 경향을 보인다 [cite: 1, Figure 3]. 반면, Sufficient Evidence가 외부에서 제공되거나 Rule Drift가 제거될 경우, 모델의 평균 성능이 각각 약 9.4%와 6.2% 향상되었다 [cite: 1, Figure 2]. 이는 현재 LLM agents가 Informative Evidence를 활용하는 능력은 우수하지만, 스스로 올바른 Evidence를 능동적으로 획득하고 기존 지식이 더 이상 유효하지 않음을 인식하는 데 어려움을 겪는다는 것을 시사한다. 또한, Exploration 노력의 양(Mean Unique Probes)이 반드시 End-to-End Performance 향상으로 이어지지 않았으며, Change Recognition 능력은 End-to-End Performance와 더 강한 상관관계를 보였다 [cite: 1, Table 2]. Agents는 Factor Replacement보다 Direction Reversal 변화를 인식하는 데 더 큰 어려움을 겪는 것으로 나타났다.

4. Conclusion & Impact (결론 및 시사점)

본 연구는 LLM agents의 Proactive Exploration 및 Test-time Adaptation 능력을 평가하는 새로운 벤치마크인 AdaExploreBench를 성공적으로 제시했다. 실험 결과는 현재 LLM agents가 Evolving Environment에서 자율적으로 Evidence를 획득하고, Rule Drift에 따라 지식을 Updated하는 데 상당한 한계가 있음을 명확히 보여준다. 이 연구는 Agent가 Established Assumption에 적극적으로 도전하고, 환경 변화에 따라 지식을 수정할 필요성을 강조한다. AdaExploreBench는 향후 동적 환경에서 신뢰할 수 있는 Exploration 및 Adaptation이 가능한 Agent를 개발하기 위한 중요한 기반을 제공하며, 해당 분야의 학계 및 산업계 연구 발전에 크게 기여할 것으로 기대된다.

⚠️ 알림: 이 리뷰는 AI로 작성되었습니다.