#Benchmarking

91개의 포스트

[논문리뷰] How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings

arXiv에 게시된 'How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Agents #Agentic Skills #Skill Retrieval #Skill Refinement #Benchmarking #Test-time Adaptation

2026년 4월 7일

[논문리뷰] SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks

arXiv에 게시된 'SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks' 논문에 대한 자세한 리뷰입니다.

#Review #SlopCodeBench #Coding Agents #Iterative Development #Code Quality #Structural Erosion #Verbosity #Benchmarking #Long-Horizon Tasks

2026년 3월 26일

[논문리뷰] FinMCP-Bench: Benchmarking LLM Agents for Real-World Financial Tool Use under the Model Context Protocol

arXiv에 게시된 'FinMCP-Bench: Benchmarking LLM Agents for Real-World Financial Tool Use under the Model Context Protocol' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Agents #Financial Tool Use #Benchmarking #Model Context Protocol #Multi-tool Reasoning #Multi-turn Conversation #Evaluation Metrics

2026년 3월 26일

[논문리뷰] Can Large Language Models Keep Up? Benchmarking Online Adaptation to Continual Knowledge Streams

arXiv에 게시된 'Can Large Language Models Keep Up? Benchmarking Online Adaptation to Continual Knowledge Streams' 논문에 대한 자세한 리뷰입니다.

#Review #Online Adaptation #Continual Learning #Knowledge Streams #Large Language Models #Benchmarking #State Tracking #Retrieval Augmented Generation #Agentic Memory

2026년 3월 11일

[논문리뷰] $OneMillion-Bench: How Far are Language Agents from Human Experts?

arXiv에 게시된 '$OneMillion-Bench: How Far are Language Agents from Human Experts?' 논문에 대한 자세한 리뷰입니다.

#Review #Language Agents #Benchmarking #Expert Evaluation #Economic Value #Professional Tasks #Rubric-based Evaluation #Multi-step Reasoning #Reliability #Domain Adaptation

2026년 3월 9일

[논문리뷰] T2S-Bench & Structure-of-Thought: Benchmarking and Prompting Comprehensive Text-to-Structure Reasoning

linyueqian이 arXiv에 게시한 'T2S-Bench & Structure-of-Thought: Benchmarking and Prompting Comprehensive Text-to-Structure Reasoning' 논문에 대한 자세한 리뷰입니다.

#Review #Benchmarking #Text-to-Structure #LLM Prompting #Structure-of-Thought #Multihop Reasoning #Graph Extraction #Scientific Documents #Text Processing

2026년 3월 4일

[논문리뷰] MobilityBench: A Benchmark for Evaluating Route-Planning Agents in Real-World Mobility Scenarios

arXiv에 게시된 'MobilityBench: A Benchmark for Evaluating Route-Planning Agents in Real-World Mobility Scenarios' 논문에 대한 자세한 리뷰입니다.

#Review #Large Language Models #Route Planning Agents #Benchmarking #Real-World Mobility #API Replay Sandbox #Multi-dimensional Evaluation #Tool-augmented Agents

2026년 2월 26일

[논문리뷰] NanoKnow: How to Know What Your Language Model Knows

arXiv에 게시된 'NanoKnow: How to Know What Your Language Model Knows' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Knowledge #Pre-training Data #Retrieval-Augmented Generation (RAG)#FineWeb-Edu #nanochat #Benchmarking #Question Answering #Data Attribution

2026년 2월 25일

[논문리뷰] DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories

arXiv에 게시된 'DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal Agents #Image Retrieval #Context-Aware #Visual Histories #Benchmarking #Vision-Language Models #Agentic AI

2026년 2월 16일

[논문리뷰] Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation

Ryan Rossi이 arXiv에 게시한 'Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation' 논문에 대한 자세한 리뷰입니다.

#Review #RAG Security #Knowledge Extraction Attack #Benchmarking #Privacy Leakage #Defense Mechanisms #Large Language Models #Retrieval Augmented Generation

2026년 2월 16일

[논문리뷰] SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents

Huayu Sha이 arXiv에 게시한 'SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Agents #Tool-use #Scientific Reasoning #Benchmarking #Interactive Environment #Data Synthesis #Error Recovery #Multi-step Tasks

2026년 2월 15일

[논문리뷰] Adapting Vision-Language Models for E-commerce Understanding at Scale

arXiv에 게시된 'Adapting Vision-Language Models for E-commerce Understanding at Scale' 논문에 대한 자세한 리뷰입니다.

#Review #E-commerce #Vision-Language Models #Multimodal Understanding #Instruction Tuning #Attribute Extraction #Fine-tuning #Benchmarking #LLMs

2026년 2월 12일

[논문리뷰] FeatureBench: Benchmarking Agentic Coding for Complex Feature Development

Jiahe Wang이 arXiv에 게시한 'FeatureBench: Benchmarking Agentic Coding for Complex Feature Development' 논문에 대한 자세한 리뷰입니다.

#Review #Agentic Coding #Benchmarking #LLMs #Feature Development #Software Engineering #Test-Driven Development #Scalability

2026년 2월 11일

[논문리뷰] LOCA-bench: Benchmarking Language Agents Under Controllable and Extreme Context Growth

arXiv에 게시된 'LOCA-bench: Benchmarking Language Agents Under Controllable and Extreme Context Growth' 논문에 대한 자세한 리뷰입니다.

#Review #Large Language Models #Language Agents #Long Context #Context Rot #Benchmarking #Context Management #Tool Use #Agent Evaluation #Dynamic Environments

2026년 2월 9일

[논문리뷰] OdysseyArena: Benchmarking Large Language Models For Long-Horizon, Active and Inductive Interactions

heroding77이 arXiv에 게시한 'OdysseyArena: Benchmarking Large Language Models For Long-Horizon, Active and Inductive Interactions' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Agents #Benchmarking #Inductive Reasoning #Long-Horizon Tasks #Active Exploration #World Models #Autonomous Discovery

2026년 2월 8일

[Loki] LogQL 벤치마크에 오브젝트 스토리지 지연 시뮬레이션 추가

Loki LogQL 벤치마크에 S3/GCS 같은 오브젝트 스토리지 지연을 시뮬레이션하는 플래그를 추가하여 프로덕션 환경에 가까운 성능 측정을 가능하게 한 PR 분석.

#Grafana Loki #Go #Benchmarking #Object Storage #Latency Simulation #LogQL

2026년 2월 7일

[논문리뷰] SAGE: Benchmarking and Improving Retrieval for Deep Research Agents

Chen Zhao이 arXiv에 게시한 'SAGE: Benchmarking and Improving Retrieval for Deep Research Agents' 논문에 대한 자세한 리뷰입니다.

#Review #Deep Research Agents #Scientific Literature Retrieval #LLM-based Retrievers #Benchmarking #Test-time Scaling #Information Retrieval #Query Decomposition #RAG

2026년 2월 5일

[논문리뷰] WideSeek: Advancing Wide Research via Multi-Agent Scaling

Zhongtao Jiang이 arXiv에 게시한 'WideSeek: Advancing Wide Research via Multi-Agent Scaling' 논문에 대한 자세한 리뷰입니다.

#Review #Wide Research #Multi-Agent Systems #Reinforcement Learning #Information Seeking #Benchmarking #LLM Agents #Knowledge Graphs

2026년 2월 3일

[논문리뷰] AdaptMMBench: Benchmarking Adaptive Multimodal Reasoning for Mode Selection and Reasoning Process

Shilin Yan이 arXiv에 게시한 'AdaptMMBench: Benchmarking Adaptive Multimodal Reasoning for Mode Selection and Reasoning Process' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal Reasoning #Adaptive Learning #Vision-Language Models (VLMs)#Benchmarking #Mode Selection #Tool Learning #Reasoning Process Evaluation #Matthews Correlation Coefficient (MCC)

2026년 2월 3일

[논문리뷰] WildGraphBench: Benchmarking GraphRAG with Wild-Source Corpora

arXiv에 게시된 'WildGraphBench: Benchmarking GraphRAG with Wild-Source Corpora' 논문에 대한 자세한 리뷰입니다.

#Review #GraphRAG #Benchmarking #Retrieval-Augmented Generation #Wild-Source Corpora #Multi-document Aggregation #Heterogeneous Data #Wikipedia #Long-Context Reasoning

2026년 2월 2일

[논문리뷰] OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Execution

Yusai Zhao이 arXiv에 게시한 'OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Execution' 논문에 대한 자세한 리뷰입니다.

#Review #GUI Agent #Multimodal AI #MoE #Data Synthesis #Reinforcement Learning #Cross-Platform #Benchmarking

2026년 1월 28일

[논문리뷰] AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security

arXiv에 게시된 'AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security' 논문에 대한 자세한 리뷰입니다.

#Review #AI Agents #Safety Guardrails #Explainable AI (XAI)#Risk Taxonomy #Benchmarking #LLM Safety #Tool Use #Agent Alignment

2026년 1월 27일

[논문리뷰] Scientific Image Synthesis: Benchmarking, Methodologies, and Downstream Utility

arXiv에 게시된 'Scientific Image Synthesis: Benchmarking, Methodologies, and Downstream Utility' 논문에 대한 자세한 리뷰입니다.

#Review #Scientific Image Synthesis #Multimodal Reasoning #Text-to-Image #Benchmarking #Programmatic Synthesis #Large Multimodal Models #Synthetic Data

2026년 1월 26일

[논문리뷰] DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints

arXiv에 게시된 'DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Agents #Long-Horizon Planning #Benchmarking #Verifiable Constraints #Tool Use #Constraint Optimization #Information Acquisition #Travel Planning #Shopping Planning

2026년 1월 26일

[논문리뷰] Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces

Harsh Raj이 arXiv에 게시한 'Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces' 논문에 대한 자세한 리뷰입니다.

#Review #AI Agents #LLM Evaluation #Benchmarking #Command Line Interface #Software Engineering #Realistic Tasks #Error Analysis

2026년 1월 22일

[논문리뷰] PRiSM: Benchmarking Phone Realization in Speech Models

arXiv에 게시된 'PRiSM: Benchmarking Phone Realization in Speech Models' 논문에 대한 자세한 리뷰입니다.

#Review #Phone Recognition #Speech Models #Benchmarking #Phonetic Analysis #Cross-lingual Speech #LALMs #Intrinsic Evaluation #Extrinsic Evaluation

2026년 1월 20일

[논문리뷰] MemoryRewardBench: Benchmarking Reward Models for Long-Term Memory Management in Large Language Models

arXiv에 게시된 'MemoryRewardBench: Benchmarking Reward Models for Long-Term Memory Management in Large Language Models' 논문에 대한 자세한 리뷰입니다.

#Review #Reward Models #LLM Memory Management #Benchmarking #Long Context #Evaluation Metrics #Generative RMs #Memory Management Patterns

2026년 1월 20일

[논문리뷰] KAGE-Bench: Fast Known-Axis Visual Generalization Evaluation for Reinforcement Learning

Aleksandr I. Panov이 arXiv에 게시한 'KAGE-Bench: Fast Known-Axis Visual Generalization Evaluation for Reinforcement Learning' 논문에 대한 자세한 리뷰입니다.

#Review #Reinforcement Learning #Visual Generalization #Distribution Shift #Benchmarking #JAX #Controlled Environments #PPO

2026년 1월 20일

[논문리뷰] Advances and Frontiers of LLM-based Issue Resolution in Software Engineering: A Comprehensive Survey

arXiv에 게시된 'Advances and Frontiers of LLM-based Issue Resolution in Software Engineering: A Comprehensive Survey' 논문에 대한 자세한 리뷰입니다.

#Review #LLM-based Issue Resolution #Software Engineering #Autonomous Agents #Code Generation #Benchmarking #Reinforcement Learning #Supervised Fine-tuning #Multimodal LLMs

2026년 1월 20일

[논문리뷰] SIN-Bench: Tracing Native Evidence Chains in Long-Context Multimodal Scientific Interleaved Literature

arXiv에 게시된 'SIN-Bench: Tracing Native Evidence Chains in Long-Context Multimodal Scientific Interleaved Literature' 논문에 대한 자세한 리뷰입니다.

#Review #Long-Context Understanding #Multimodal AI #Scientific Literature #Evidence-based Reasoning #MLLM Evaluation #Benchmarking #Cross-modal Reasoning #Information Synthesis

2026년 1월 19일

[논문리뷰] ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development

arXiv에 게시된 'ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development' 논문에 대한 자세한 리뷰입니다.

#Review #Backend Development #LLM Agents #Code Generation #Benchmarking #DevOps #Containerization #End-to-End Testing #Environment Configuration

2026년 1월 19일

[논문리뷰] A^3-Bench: Benchmarking Memory-Driven Scientific Reasoning via Anchor and Attractor Activation

Kai He이 arXiv에 게시한 'A^3-Bench: Benchmarking Memory-Driven Scientific Reasoning via Anchor and Attractor Activation' 논문에 대한 자세한 리뷰입니다.

#Review #Scientific Reasoning #Memory-Driven AI #Benchmarking #Large Language Models (LLMs)#Anchor-Attractor Activation #Episodic Memory #Knowledge Retrieval

2026년 1월 14일

[Loki] Plain 디코더 벤치마크 추가 및 코드 개선

Loki dataobj의 plain bytes 디코더에 체계적인 벤치마크를 추가하고, 불필요한 조건 체크를 제거하여 디코딩 성능을 개선한 PR 분석.

#Grafana Loki #Go #Benchmarking #Decoder #Data Object #Performance

2026년 1월 14일

[논문리뷰] Towards Comprehensive Stage-wise Benchmarking of Large Language Models in Fact-Checking

Zhen Ye이 arXiv에 게시한 'Towards Comprehensive Stage-wise Benchmarking of Large Language Models in Fact-Checking' 논문에 대한 자세한 리뷰입니다.

#Review #Fact-Checking #Large Language Models (LLMs)#Benchmarking #Multi-agent System #Stage-wise Evaluation #Claim Evolution #Trustworthy AI

2026년 1월 13일

[논문리뷰] InSight-o3: Empowering Multimodal Foundation Models with Generalized Visual Search

Jierun Chen이 arXiv에 게시한 'InSight-o3: Empowering Multimodal Foundation Models with Generalized Visual Search' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal AI #Visual Search #Foundation Models #Multi-agent Systems #Reinforcement Learning #Benchmarking #Visual Reasoning

2025년 12월 28일

[논문리뷰] SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios

Nghi D. Q. Bui이 arXiv에 게시한 'SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios' 논문에 대한 자세한 리뷰입니다.

#Review #Coding Agents #Software Evolution #Benchmarking #Long-Horizon Tasks #Large Language Models (LLMs)#Software Engineering #Code Generation

2025년 12월 24일

[논문리뷰] Step-DeepResearch Technical Report

arXiv에 게시된 'Step-DeepResearch Technical Report' 논문에 대한 자세한 리뷰입니다.

#Review #Deep Research Agents #LLMs #Reinforcement Learning #Supervised Fine-tuning #Agentic AI #Multi-hop Reasoning #Benchmarking #Cost-effectiveness

2025년 12월 23일

[논문리뷰] Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows

Yuhao Zhou이 arXiv에 게시한 'Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows' 논문에 대한 자세한 리뷰입니다.

#Review #Scientific General Intelligence (SGI)#LLMs #Benchmarking #Scientist-Aligned Workflows #Practical Inquiry Model #Multi-modal Reasoning #Code Generation #Test-Time Reinforcement Learning (TTRL)

2025년 12월 21일

[논문리뷰] Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMs

Carlos Escolano이 arXiv에 게시한 'Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMs' 논문에 대한 자세한 리뷰입니다.

#Review #Speech-to-Text Translation #Multimodal LLMs #Speech Foundation Models #Cascaded Systems #Benchmarking #Speech Modality Integration #Robustness #Evaluation Metrics

2025년 12월 18일

[논문리뷰] Step-GUI Technical Report

arXiv에 게시된 'Step-GUI Technical Report' 논문에 대한 자세한 리뷰입니다.

#Review #GUI Automation #Self-Evolving Pipeline #Reinforcement Learning #Multimodal LLMs #Privacy-Preserving AI #Human-Computer Interaction #Model Context Protocol #Benchmarking

2025년 12월 17일

[논문리뷰] V-REX: Benchmarking Exploratory Visual Reasoning via Chain-of-Questions

Kwesi Cobbina이 arXiv에 게시한 'V-REX: Benchmarking Exploratory Visual Reasoning via Chain-of-Questions' 논문에 대한 자세한 리뷰입니다.

#Review #Visual Reasoning #Multi-step Exploration #Chain-of-Questions (CoQ)#Vision-Language Models (VLMs)#Benchmarking #Planning #Following

2025년 12월 15일

[논문리뷰] From Macro to Micro: Benchmarking Microscopic Spatial Intelligence on Molecules via Vision-Language Models

arXiv에 게시된 'From Macro to Micro: Benchmarking Microscopic Spatial Intelligence on Molecules via Vision-Language Models' 논문에 대한 자세한 리뷰입니다.

#Review #Vision-Language Models #Microscopic Spatial Intelligence #Molecular Structures #Benchmarking #PDBbind Dataset #Spatial Reasoning #Drug Discovery

2025년 12월 11일

[논문리뷰] Same Content, Different Answers: Cross-Modal Inconsistency in MLLMs

arXiv에 게시된 'Same Content, Different Answers: Cross-Modal Inconsistency in MLLMs' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal Large Language Models (MLLMs)#Cross-Modal Consistency #Reasoning Inconsistency #OCR Performance #Modality Gap #Benchmarking #Render Equivalence

2025년 12월 9일

[논문리뷰] DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecycle

arXiv에 게시된 'DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecycle' 논문에 대한 자세한 리뷰입니다.

#Review #Data Agents #Benchmarking #Data Engineering #Data Analysis #LLM-as-Judge #Full Data Intelligence Lifecycle #Repository-Level #Open-Ended Tasks

2025년 12월 4일

[논문리뷰] WiseEdit: Benchmarking Cognition- and Creativity-Informed Image Editing

Wendong Bu이 arXiv에 게시한 'WiseEdit: Benchmarking Cognition- and Creativity-Informed Image Editing' 논문에 대한 자세한 리뷰입니다.

#Review #Image Editing #Benchmarking #Cognitive AI #Creativity #Multimodal AI #Knowledge-based Reasoning #Diffusion Models #MLLMs

2025년 12월 1일

[논문리뷰] Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insights

arXiv에 게시된 'Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insights' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal AI #Text-to-Multi-Image #Causal Reasoning #World Knowledge #Benchmarking #Spatiotemporal Consistency #Generative Models #Evaluation Metrics

2025년 12월 1일

[논문리뷰] From Pixels to Feelings: Aligning MLLMs with Human Cognitive Perception of Images

Filippos Kokkinos이 arXiv에 게시한 'From Pixels to Feelings: Aligning MLLMs with Human Cognitive Perception of Images' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal LLM #Human Cognition #Image Perception #Benchmarking #Supervised Fine-tuning #Image Generation #Aesthetics #Memorability

2025년 11월 30일

[논문리뷰] TurkColBERT: A Benchmark of Dense and Late-Interaction Models for Turkish Information Retrieval

arXiv에 게시된 'TurkColBERT: A Benchmark of Dense and Late-Interaction Models for Turkish Information Retrieval' 논문에 대한 자세한 리뷰입니다.

#Review #Information Retrieval #Turkish Language #Late-Interaction Models #ColBERT #Dense Retrieval #MUVERA #Benchmarking #Low-Resource NLP #Fine-tuning

2025년 11월 20일

[논문리뷰] TiViBench: Benchmarking Think-in-Video Reasoning for Video Generative Models

Qingyang Liu이 arXiv에 게시한 'TiViBench: Benchmarking Think-in-Video Reasoning for Video Generative Models' 논문에 대한 자세한 리뷰입니다.

#Review #Video Generative Models #Visual Reasoning #Benchmarking #Image-to-Video #TiViBench #VideoTPO #Prompt Optimization

2025년 11월 17일

[논문리뷰] DiscoX: Benchmarking Discourse-Level Translation task in Expert Domains

arXiv에 게시된 'DiscoX: Benchmarking Discourse-Level Translation task in Expert Domains' 논문에 대한 자세한 리뷰입니다.

#Review #Discourse-Level Translation #Expert Domains #Benchmarking #LLM Evaluation #Reference-Free Metric #Chinese-English Translation #Contextual Coherence #Domain-Specific Terminology

2025년 11월 16일

[논문리뷰] Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following

Karishma Mandyam이 arXiv에 게시한 'Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following' 논문에 대한 자세한 리뷰입니다.

#Review #LLM #Instruction Following #Reinforcement Learning #Rubric-based Evaluation #Benchmarking #Reward Shaping #Rubric Verifier #AdvancedIF

2025년 11월 13일

[논문리뷰] Benchmarking Diversity in Image Generation via Attribute-Conditional Human Evaluation

arXiv에 게시된 'Benchmarking Diversity in Image Generation via Attribute-Conditional Human Evaluation' 논문에 대한 자세한 리뷰입니다.

#Review #Text-to-Image Models #Diversity Evaluation #Human Evaluation #Attribute-Conditional #Vendi Score #Generative AI #Benchmarking

2025년 11월 13일

[논문리뷰] DigiData: Training and Evaluating General-Purpose Mobile Control Agents

arXiv에 게시된 'DigiData: Training and Evaluating General-Purpose Mobile Control Agents' 논문에 대한 자세한 리뷰입니다.

#Review #Mobile Control Agents #User Interface Automation #Large-Scale Dataset #Benchmarking #LLM Judges #Data Diversity #Task Success Rate

2025년 11월 10일

[논문리뷰] TabTune: A Unified Library for Inference and Fine-Tuning Tabular Foundation Models

arXiv에 게시된 'TabTune: A Unified Library for Inference and Fine-Tuning Tabular Foundation Models' 논문에 대한 자세한 리뷰입니다.

#Review #Tabular Foundation Models #Fine-Tuning #PEFT #Meta-Learning #Calibration #Fairness #Unified Library #Benchmarking

2025년 11월 9일

[논문리뷰] CostBench: Evaluating Multi-Turn Cost-Optimal Planning and Adaptation in Dynamic Environments for LLM Tool-Use Agents

Shijue Huang이 arXiv에 게시한 'CostBench: Evaluating Multi-Turn Cost-Optimal Planning and Adaptation in Dynamic Environments for LLM Tool-Use Agents' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Agents #Tool Use #Cost-Optimal Planning #Dynamic Environments #Benchmarking #Multi-Turn Interaction #Economic Reasoning

2025년 11월 9일

[논문리뷰] RoboChallenge: Large-scale Real-robot Evaluation of Embodied Policies

arXiv에 게시된 'RoboChallenge: Large-scale Real-robot Evaluation of Embodied Policies' 논문에 대한 자세한 리뷰입니다.

#Review #Robotics #Real-robot Evaluation #Embodied AI #Vision-Language-Action Models #Benchmarking #Online Testing System #Robotics Control #Large-scale Evaluation

2025년 11월 9일

[논문리뷰] ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation

Feng Li이 arXiv에 게시한 'ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal AI #Benchmarking #Cross-Modal Reasoning #Omnimodal Generation #Visual Generation #Verbal Generation #Unified Multimodal Models

2025년 11월 9일

[논문리뷰] Do Vision-Language Models Measure Up? Benchmarking Visual Measurement Reading with MeasureBench

arXiv에 게시된 'Do Vision-Language Models Measure Up? Benchmarking Visual Measurement Reading with MeasureBench' 논문에 대한 자세한 리뷰입니다.

#Review #Vision-Language Models #Benchmarking #Visual Measurement Reading #Synthetic Data Generation #Fine-grained Perception #Spatial Grounding #Reinforcement Learning

2025년 11월 9일

[Ray] 단일 노드 LLM 배치 추론 성능 기준선 벤치마크 및 회귀 가드 추가

Ray Data LLM의 단일 노드 vLLM 배치 추론 벤치마크를 추가하고 환경 변수 기반 회귀 감지 임계값을 설정한 분석.

#Ray #Python #Performance #Benchmarking #LLM

2025년 10월 30일

[논문리뷰] The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution

Haoze Wu이 arXiv에 게시한 'The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution' 논문에 대한 자세한 리뷰입니다.

#Review #Language Agents #Tool Use #Benchmarking #Long-Horizon Tasks #Realistic Environments #Multi-Application #Execution-Based Evaluation #Model Context Protocol (MCP)

2025년 10월 30일

[논문리뷰] InteractComp: Evaluating Search Agents With Ambiguous Queries

Yani Fan이 arXiv에 게시한 'InteractComp: Evaluating Search Agents With Ambiguous Queries' 논문에 대한 자세한 리뷰입니다.

#Review #Search Agents #Interactive AI #Ambiguous Queries #Benchmarking #Language Agents #Information Retrieval #Overconfidence #Reinforcement Learning

2025년 10월 29일

[논문리뷰] AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite

Bhavana Dalvi이 arXiv에 게시한 'AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite' 논문에 대한 자세한 리뷰입니다.

#Review #AI Agents #Benchmarking #Scientific Research #LLM Evaluation #Agentic AI #Tool Use #Reproducibility #Cost-Aware Evaluation

2025년 10월 27일

[논문리뷰] From Charts to Code: A Hierarchical Benchmark for Multimodal Models

Dongxing Mao이 arXiv에 게시한 'From Charts to Code: A Hierarchical Benchmark for Multimodal Models' 논문에 대한 자세한 리뷰입니다.

#Review #Chart-to-Code #Multimodal Models #Hierarchical Benchmark #Chart Understanding #Code Generation #Evaluation Metrics #Benchmarking

2025년 10월 23일

[논문리뷰] RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems

arXiv에 게시된 'RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems' 논문에 대한 자세한 리뷰입니다.

#Review #Large Language Models #Retrieval Augmented Generation #Agentic Systems #Benchmarking #Intermediate Tasks #Error Analysis #LLM Evaluation

2025년 10월 17일

[논문리뷰] DialectGen: Benchmarking and Improving Dialect Robustness in Multimodal Generation

arXiv에 게시된 'DialectGen: Benchmarking and Improving Dialect Robustness in Multimodal Generation' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal Generation #Dialect Robustness #Text-to-Image #Text-to-Video #Benchmarking #Diffusion Models #Text Encoder Tuning #Low-Resource Dialects

2025년 10월 17일

[논문리뷰] MLLM as a UI Judge: Benchmarking Multimodal LLMs for Predicting Human Perception of User Interfaces

Sungchul Kim이 arXiv에 게시한 'MLLM as a UI Judge: Benchmarking Multimodal LLMs for Predicting Human Perception of User Interfaces' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal LLMs #UI Evaluation #Human Perception #Benchmarking #UX Research #MLLM-as-a-Judge #Cognitive Factors #Pairwise Comparison

2025년 10월 15일

[논문리뷰] PhysToolBench: Benchmarking Physical Tool Understanding for MLLMs

Xu Zheng이 arXiv에 게시한 'PhysToolBench: Benchmarking Physical Tool Understanding for MLLMs' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal Large Language Models (MLLMs)#Physical Tool Understanding #Benchmarking #Embodied AI #Visual Question Answering (VQA)#Tool Affordances #Reasoning

2025년 10월 13일

[논문리뷰] BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution

Hange Liu이 arXiv에 게시한 'BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution' 논문에 대한 자세한 리뷰입니다.

#Review #Code Generation #Human Preference #LLM Evaluation #Execution Feedback #Benchmarking #Crowdsourcing #Software Engineering #Large Language Models

2025년 10월 13일

[논문리뷰] NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agents

Baixuan Xu이 arXiv에 게시한 'NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agents' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Agents #Scientific Law Discovery #Benchmarking #Metaphysical Shifts #Interactive Environments #Exploration-Exploitation #Tool Use

2025년 10월 10일

[논문리뷰] U-Bench: A Comprehensive Understanding of U-Net through 100-Variant Benchmarking

Heqin Zhu이 arXiv에 게시한 'U-Bench: A Comprehensive Understanding of U-Net through 100-Variant Benchmarking' 논문에 대한 자세한 리뷰입니다.

#Review #U-Net #Medical Image Segmentation #Benchmarking #Performance Evaluation #Efficiency Metrics #Zero-shot Generalization #U-Score

2025년 10월 9일

[논문리뷰] Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methods

Yiyu Wang이 arXiv에 게시한 'Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methods' 논문에 대한 자세한 리뷰입니다.

#Review #Visual Token Compression #MLLMs #Evaluation Framework #Benchmarking #Downsampling #Data Filtering #Model Efficiency

2025년 10월 9일

[논문리뷰] GEM: A Gym for Agentic LLMs

arXiv에 게시된 'GEM: A Gym for Agentic LLMs' 논문에 대한 자세한 리뷰입니다.

#Review #Agentic LLMs #Reinforcement Learning #Environment Simulator #Multi-turn Interactions #Return Batch Normalization #Tool Integration #Benchmarking

2025년 10월 2일

[논문리뷰] VitaBench: Benchmarking LLM Agents with Versatile Interactive Tasks in Real-world Applications

arXiv에 게시된 'VitaBench: Benchmarking LLM Agents with Versatile Interactive Tasks in Real-world Applications' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Agents #Benchmarking #Interactive Tasks #Real-world Applications #Tool Use #Multi-turn Conversation #Task Complexity

2025년 10월 1일

[논문리뷰] BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software

arXiv에 게시된 'BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Agents #Open-Source Software #Compilation #Benchmarking #Software Engineering #Error Resolution #Retrieval-Augmented Generation

2025년 10월 1일

[논문리뷰] UltraHorizon: Benchmarking Agent Capabilities in Ultra Long-Horizon Scenarios

Zeyu Qin이 arXiv에 게시한 'UltraHorizon: Benchmarking Agent Capabilities in Ultra Long-Horizon Scenarios' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Agents #Long-Horizon Reasoning #Benchmarking #Partially Observable #Tool Use #Memory Management #Exploration

2025년 9월 29일

[논문리뷰] Instruction-Following Evaluation in Function Calling for Large Language Models

NikolaiSkripko이 arXiv에 게시한 'Instruction-Following Evaluation in Function Calling for Large Language Models' 논문에 대한 자세한 리뷰입니다.

#Review #Function Calling #LLMs #Instruction Following #Benchmarking #JSON Schema #AI Agents #Evaluation Metrics

2025년 9월 29일

[논문리뷰] Logics-Parsing Technical Report

Fan Yang이 arXiv에 게시한 'Logics-Parsing Technical Report' 논문에 대한 자세한 리뷰입니다.

#Review #Document Parsing #Large Vision-Language Models (LVLM)#Reinforcement Learning (RL)#Layout Analysis #Reading Order #Supervised Fine-Tuning (SFT)#HTML Annotation #Benchmarking

2025년 9월 25일

[논문리뷰] WildScore: Benchmarking MLLMs in-the-Wild Symbolic Music Reasoning

Amit Namburi이 arXiv에 게시한 'WildScore: Benchmarking MLLMs in-the-Wild Symbolic Music Reasoning' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal Large Language Models #Symbolic Music Reasoning #Music Score Analysis #Benchmarking #Visual Question Answering #In-the-Wild Data #Music Theory

2025년 9월 8일

[논문리뷰] MobiAgent: A Systematic Framework for Customizable Mobile Agents

Wangbo Gong이 arXiv에 게시한 'MobiAgent: A Systematic Framework for Customizable Mobile Agents' 논문에 대한 자세한 리뷰입니다.

#Review #Mobile Agents #GUI Agents #Vision-Language Models #Agent Acceleration #Benchmarking #Reinforcement Learning #Data Collection

2025년 9월 3일

[논문리뷰] Benchmarking Optimizers for Large Language Model Pretraining

mjaggi이 arXiv에 게시한 'Benchmarking Optimizers for Large Language Model Pretraining' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Optimizers #Benchmarking #Hyperparameter Tuning #AdamW #AdEMAMix #MARS #Mixture of Experts (MoE)#Weight Decay

2025년 9월 3일

[논문리뷰] AHELM: A Holistic Evaluation of Audio-Language Models

Siwei Yang이 arXiv에 게시한 'AHELM: A Holistic Evaluation of Audio-Language Models' 논문에 대한 자세한 리뷰입니다.

#Review #Audio-Language Models #Holistic Evaluation #Benchmarking #Multimodality #Fairness #Robustness #Reasoning #Bias Detection

2025년 9월 1일

[논문리뷰] MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Servers

Shashank Biju이 arXiv에 게시한 'MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Servers' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Agents #Tool Use #Benchmarking #Model Context Protocol (MCP)#Cross-Domain Orchestration #Fuzzy Instructions #Multi-Step Tasks #Real-World Scenarios

2025년 8월 29일

[논문리뷰] Mind the Third Eye! Benchmarking Privacy Awareness in MLLM-powered Smartphone Agents

Yue Yao이 arXiv에 게시한 'Mind the Third Eye! Benchmarking Privacy Awareness in MLLM-powered Smartphone Agents' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal LLMs (MLLMs)#Smartphone Agents #Privacy Awareness #Benchmarking #Sensitive Data Detection #Risk Assessment #UI Automation

2025년 8월 28일

[논문리뷰] LiveMCP-101: Stress Testing and Diagnosing MCP-enabled Agents on Challenging Queries

huuuyeah이 arXiv에 게시한 'LiveMCP-101: Stress Testing and Diagnosing MCP-enabled Agents on Challenging Queries' 논문에 대한 자세한 리뷰입니다.

#Review #AI Agents #Tool Use #Model Context Protocol (MCP)#Benchmarking #Large Language Models (LLMs)#Real-world Tasks #Evaluation #Error Analysis

2025년 8월 22일

[논문리뷰] MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers

Prathyusha Jwalapuram이 arXiv에 게시한 'MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers' 논문에 대한 자세한 리뷰입니다.

#Review #Large Language Models #Benchmarking #Model Context Protocol #Tool Use #Real-World Applications #Agent Evaluation #Long Context #Unknown Tools

2025년 8월 21일

[논문리뷰] Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved Image Generation

Zhenghao Hu이 arXiv에 게시한 'Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved Image Generation' 논문에 대한 자세한 리뷰입니다.

#Review #Synthetic Data #Image Generation #GPT-4o #Multimodal Models #Instruction Following #Surreal Image Generation #Dataset #Benchmarking

2025년 8월 14일

[논문리뷰] OmniEAR: Benchmarking Agent Reasoning in Embodied Tasks

Hongxing Li이 arXiv에 게시한 'OmniEAR: Benchmarking Agent Reasoning in Embodied Tasks' 논문에 대한 자세한 리뷰입니다.

#Review #Embodied AI #Agent Reasoning #LLM #Benchmarking #Tool Use #Multi-Agent Systems #Physical Interaction #Constraint Reasoning

2025년 8월 12일

[논문리뷰] BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent

Kai Zou이 arXiv에 게시한 'BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent' 논문에 대한 자세한 리뷰입니다.

#Review #Benchmarking #Deep-Research Agents #LLMs #Retrieval #Curated Corpus #Evaluation #Fairness #Transparency #Reproducibility

2025년 8월 12일

[논문리뷰] I Think, Therefore I Am Under-Qualified? A Benchmark for Evaluating Linguistic Shibboleth Detection in LLM Hiring Evaluations

Chirag Shah이 arXiv에 게시한 'I Think, Therefore I Am Under-Qualified? A Benchmark for Evaluating Linguistic Shibboleth Detection in LLM Hiring Evaluations' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Bias #Hiring Evaluation #Linguistic Shibboleth #Hedging Language #Fairness #Benchmarking #Sociolinguistics

2025년 8월 8일

[논문리뷰] CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward

Songyang Gao이 arXiv에 게시한 'CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Evaluation #Answer Verification #Reward Model #Benchmarking #Data Augmentation #Reinforcement Learning #Formula Verification #Hallucination Detection

2025년 8월 6일

[논문리뷰] AgroBench: Vision-Language Model Benchmark in Agriculture

Yoshitaka Ushiku이 arXiv에 게시한 'AgroBench: Vision-Language Model Benchmark in Agriculture' 논문에 대한 자세한 리뷰입니다.

#Review #Vision-Language Models #Agriculture #Benchmarking #Disease Identification #Pest Management #Crop Management #Agronomy

2025년 8월 2일