#Evaluation Metrics

34개의 포스트

[논문리뷰] FinMCP-Bench: Benchmarking LLM Agents for Real-World Financial Tool Use under the Model Context Protocol

arXiv에 게시된 'FinMCP-Bench: Benchmarking LLM Agents for Real-World Financial Tool Use under the Model Context Protocol' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Agents #Financial Tool Use #Benchmarking #Model Context Protocol #Multi-tool Reasoning #Multi-turn Conversation #Evaluation Metrics

2026년 3월 26일

[논문리뷰] Making Reconstruction FID Predictive of Diffusion Generation FID

Haotian Zhang이 arXiv에 게시한 'Making Reconstruction FID Predictive of Diffusion Generation FID' 논문에 대한 자세한 리뷰입니다.

#Review #Latent Diffusion Models #VAE #FID #Generative Models #Evaluation Metrics #Image Generation #Reconstruction-Generation Dilemma #Interpolation

2026년 3월 8일

[논문리뷰] DLEBench: Evaluating Small-scale Object Editing Ability for Instruction-based Image Editing Model

FengJiao Chen이 arXiv에 게시한 'DLEBench: Evaluating Small-scale Object Editing Ability for Instruction-based Image Editing Model' 논문에 대한 자세한 리뷰입니다.

#Review #Image Editing #Instruction-based Models #Small Object Editing #Benchmark #Evaluation Metrics #Large Multimodal Models (LMMs)#Visual Consistency

2026년 3월 1일

[논문리뷰] Towards a Science of AI Agent Reliability

arXiv에 게시된 'Towards a Science of AI Agent Reliability' 논문에 대한 자세한 리뷰입니다.

#Review #AI Agents #Reliability #Evaluation Metrics #Consistency #Robustness #Predictability #Safety #Benchmarks

2026년 2월 18일

[논문리뷰] AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents

arXiv에 게시된 'AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents' 논문에 대한 자세한 리뷰입니다.

#Review #AI Research Agents #LLM Agents #Machine Learning Benchmarks #Scientific Discovery #Code Generation #Evaluation Metrics #Scaffolds #Reproducibility

2026년 2월 9일

[논문리뷰] PlanViz: Evaluating Planning-Oriented Image Generation and Editing for Computer-Use Tasks

Zhixin Wang이 arXiv에 게시한 'PlanViz: Evaluating Planning-Oriented Image Generation and Editing for Computer-Use Tasks' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal Models #Image Generation #Image Editing #Benchmark #Computer-Use Tasks #Planning #Evaluation Metrics

2026년 2월 8일

[논문리뷰] MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic Environments

arXiv에 게시된 'MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic Environments' 논문에 대한 자세한 리뷰입니다.

#Review #Mobile GUI Agents #Memory Benchmarking #Short-Term Memory #Long-Term Memory #LLM-as-Judge #Dynamic Environments #Evaluation Metrics #Task Automation

2026년 2월 8일

[논문리뷰] MemoryRewardBench: Benchmarking Reward Models for Long-Term Memory Management in Large Language Models

arXiv에 게시된 'MemoryRewardBench: Benchmarking Reward Models for Long-Term Memory Management in Large Language Models' 논문에 대한 자세한 리뷰입니다.

#Review #Reward Models #LLM Memory Management #Benchmarking #Long Context #Evaluation Metrics #Generative RMs #Memory Management Patterns

2026년 1월 20일

[논문리뷰] On the Fallacy of Global Token Perplexity in Spoken Language Model Evaluation

Ju-Chieh Chou이 arXiv에 게시한 'On the Fallacy of Global Token Perplexity in Spoken Language Model Evaluation' 논문에 대한 자세한 리뷰입니다.

#Review #Spoken Language Models #Evaluation Metrics #Perplexity #Mean Opinion Score #Likelihood-based Evaluation #Model-as-a-Judge #Acoustic Consistency #Speech Generation

2026년 1월 12일

[논문리뷰] DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving

arXiv에 게시된 'DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving' 논문에 대한 자세한 리뷰입니다.

#Review #Generative World Models #Autonomous Driving #Video Generation #Benchmark #Evaluation Metrics #Trajectory Prediction #Temporal Consistency #Data Diversity

2026년 1월 12일

[논문리뷰] Are We on the Right Way to Assessing LLM-as-a-Judge?

arXiv에 게시된 'Are We on the Right Way to Assessing LLM-as-a-Judge?' 논문에 대한 자세한 리뷰입니다.

#Review #LLM-as-a-Judge #Evaluation Metrics #Consistency #Robustness #Positional Bias #Transitivity #Situational Preference #Multi-agent Systems

2025년 12월 21일

[논문리뷰] Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMs

Carlos Escolano이 arXiv에 게시한 'Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMs' 논문에 대한 자세한 리뷰입니다.

#Review #Speech-to-Text Translation #Multimodal LLMs #Speech Foundation Models #Cascaded Systems #Benchmarking #Speech Modality Integration #Robustness #Evaluation Metrics

2025년 12월 18일

[논문리뷰] OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluation

Simeng Qin이 arXiv에 게시한 'OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluation' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal LLMs #Jailbreak Attack #Attack-Defense Evaluation #Benchmark #Safety Alignment #Vulnerability Analysis #Risk Taxonomy #Evaluation Metrics

2025년 12월 8일

[논문리뷰] Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insights

arXiv에 게시된 'Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insights' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal AI #Text-to-Multi-Image #Causal Reasoning #World Knowledge #Benchmarking #Spatiotemporal Consistency #Generative Models #Evaluation Metrics

2025년 12월 1일

[논문리뷰] M3-Bench: Multi-Modal, Multi-Hop, Multi-Threaded Tool-Using MLLM Agent Benchmark

Bangwei Guo이 arXiv에 게시한 'M3-Bench: Multi-Modal, Multi-Hop, Multi-Threaded Tool-Using MLLM Agent Benchmark' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal LLM #Tool Use #Agent Benchmark #Model Context Protocol #Multi-Hop Reasoning #Multi-Threaded Execution #Evaluation Metrics #Similarity Alignment

2025년 11월 24일

[논문리뷰] Large Language Models for Scientific Idea Generation: A Creativity-Centered Survey

Mohammad Hossein Rohban이 arXiv에 게시한 'Large Language Models for Scientific Idea Generation: A Creativity-Centered Survey' 논문에 대한 자세한 리뷰입니다.

#Review #Large Language Models #Scientific Discovery #Idea Generation #Creativity #Survey #AI in Science #Prompt Engineering #Multi-agent Systems #Evaluation Metrics

2025년 11월 16일

[논문리뷰] OSWorld-MCP: Benchmarking MCP Tool Invocation In Computer-Use Agents

arXiv에 게시된 'OSWorld-MCP: Benchmarking MCP Tool Invocation In Computer-Use Agents' 논문에 대한 자세한 리뷰입니다.

#Review #Multimodal Agents #Tool Invocation #Benchmark #Model Context Protocol (MCP)#GUI Automation #Computer-Use Agents #Evaluation Metrics

2025년 10월 29일

[논문리뷰] From Charts to Code: A Hierarchical Benchmark for Multimodal Models

Dongxing Mao이 arXiv에 게시한 'From Charts to Code: A Hierarchical Benchmark for Multimodal Models' 논문에 대한 자세한 리뷰입니다.

#Review #Chart-to-Code #Multimodal Models #Hierarchical Benchmark #Chart Understanding #Code Generation #Evaluation Metrics #Benchmarking

2025년 10월 23일

[논문리뷰] PICABench: How Far Are We from Physically Realistic Image Editing?

Kaiwen Zhu이 arXiv에 게시한 'PICABench: How Far Are We from Physically Realistic Image Editing?' 논문에 대한 자세한 리뷰입니다.

#Review #Image Editing #Physical Realism #Benchmark #VLM-as-a-Judge #Synthetic Data #Physics-Aware AI #Diffusion Models #Evaluation Metrics

2025년 10월 21일

[논문리뷰] ReviewerToo: Should AI Join The Program Committee? A Look At The Future of Peer Review

Christopher Pal이 arXiv에 게시한 'ReviewerToo: Should AI Join The Program Committee? A Look At The Future of Peer Review' 논문에 대한 자세한 리뷰입니다.

#Review #Peer Review #AI-Assisted Review #Large Language Models #LLM Agents #Meta-Review #Conference Submissions #Reviewer Personas #Evaluation Metrics

2025년 10월 13일

[논문리뷰] When Benchmarks Age: Temporal Misalignment through Large Language Model Factuality Evaluation

arXiv에 게시된 'When Benchmarks Age: Temporal Misalignment through Large Language Model Factuality Evaluation' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Factuality Evaluation #Benchmark Aging #Temporal Misalignment #Information Retrieval #Question Answering #Evaluation Metrics #GPT-4o-mini #Qwen2.5

2025년 10월 9일

[논문리뷰] Bridging Text and Video Generation: A Survey

G. Maragatham이 arXiv에 게시한 'Bridging Text and Video Generation: A Survey' 논문에 대한 자세한 리뷰입니다.

#Review #Text-to-Video Generation #Generative Models #Diffusion Models #GANs #VAEs #Video Synthesis #Survey #Evaluation Metrics

2025년 10월 9일

[논문리뷰] Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models

arXiv에 게시된 'Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models' 논문에 대한 자세한 리뷰입니다.

#Review #Code-switching #Multilingual NLP #Large Language Models #NLP Survey #Data Augmentation #Evaluation Metrics #Low-Resource Languages

2025년 10월 9일

[논문리뷰] Revisiting Modeling and Evaluation Approaches in Speech Emotion Recognition: Considering Subjectivity of Annotators and Ambiguity of Emotions

arXiv에 게시된 'Revisiting Modeling and Evaluation Approaches in Speech Emotion Recognition: Considering Subjectivity of Annotators and Ambiguity of Emotions' 논문에 대한 자세한 리뷰입니다.

#Review #Speech Emotion Recognition #Annotator Subjectivity #Emotion Ambiguity #Soft Labels #Multi-label Classification #Evaluation Metrics #Loss Functions

2025년 10월 8일

[논문리뷰] BiasFreeBench: a Benchmark for Mitigating Bias in Large Language Model Responses

Julian McAuley이 arXiv에 게시한 'BiasFreeBench: a Benchmark for Mitigating Bias in Large Language Model Responses' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Bias Mitigation #Benchmark #Evaluation Metrics #Prompt Engineering #Fine-tuning #Bias-Free Score #Fairness

2025년 10월 2일

[논문리뷰] Instruction-Following Evaluation in Function Calling for Large Language Models

NikolaiSkripko이 arXiv에 게시한 'Instruction-Following Evaluation in Function Calling for Large Language Models' 논문에 대한 자세한 리뷰입니다.

#Review #Function Calling #LLMs #Instruction Following #Benchmarking #JSON Schema #AI Agents #Evaluation Metrics

2025년 9월 29일

[논문리뷰] Do You Hear What I Mean? Quantifying the Instruction-Perception Gap in Instruction-Guided Expressive Text-To-Speech Systems

Hung-yi Lee이 arXiv에 게시한 'Do You Hear What I Mean? Quantifying the Instruction-Perception Gap in Instruction-Guided Expressive Text-To-Speech Systems' 논문에 대한 자세한 리뷰입니다.

#Review #Instruction-Guided TTS #Expressive Speech Synthesis #Human Perception #Subjective Evaluation #Controllability #Instruction Following #Evaluation Metrics

2025년 9월 22일

[논문리뷰] SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledge

Dipanjan Das이 arXiv에 게시한 'SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledge' 논문에 대한 자세한 리뷰입니다.

#Review #LLM Factuality #Parametric Knowledge #Benchmark #Question Answering #Data Curation #Evaluation Metrics #Hallucination Mitigation #Large Language Models

2025년 9월 10일

[논문리뷰] Why Language Models Hallucinate

Edwin Zhang이 arXiv에 게시한 'Why Language Models Hallucinate' 논문에 대한 자세한 리뷰입니다.

#Review #Language Models #Hallucination #Pretraining #Post-training #Evaluation Metrics #Binary Classification #Uncertainty Quantification #Calibration

2025년 9월 8일

[논문리뷰] T2R-bench: A Benchmark for Generating Article-Level Reports from Real World Industrial Tables

Yu Zhao이 arXiv에 게시한 'T2R-bench: A Benchmark for Generating Article-Level Reports from Real World Industrial Tables' 논문에 대한 자세한 리뷰입니다.

#Review #Table-to-Report Generation #Large Language Models (LLMs)#Benchmark Dataset #Industrial Applications #Table Reasoning #Evaluation Metrics #Real-world Data

2025년 9월 2일

[논문리뷰] SpotEdit: Evaluating Visually-Guided Image Editing Methods

Ersin Yumer이 arXiv에 게시한 'SpotEdit: Evaluating Visually-Guided Image Editing Methods' 논문에 대한 자세한 리뷰입니다.

#Review #Visually-Guided Image Editing #Multimodal Models #Benchmark #Hallucination #Diffusion Models #Autoregressive Models #Evaluation Metrics

2025년 8월 26일

[논문리뷰] INTIMA: A Benchmark for Human-AI Companionship Behavior

Yacine Jernite이 arXiv에 게시한 'INTIMA: A Benchmark for Human-AI Companionship Behavior' 논문에 대한 자세한 리뷰입니다.

#Review #AI Companionship #Benchmark #Language Models (LLMs)#Human-AI Interaction #Emotional AI #Boundary Setting #Psychological Frameworks #Evaluation Metrics

2025년 8월 22일

[논문리뷰] Advances in Speech Separation: Techniques, Challenges, and Future Trends

Zhuo Chen이 arXiv에 게시한 'Advances in Speech Separation: Techniques, Challenges, and Future Trends' 논문에 대한 자세한 리뷰입니다.

#Review #Speech Separation #Deep Neural Networks #Cocktail Party Problem #Transformer Architecture #Unsupervised Learning #Supervised Learning #Evaluation Metrics #Datasets

2025년 8월 20일

[논문리뷰] WideSearch: Benchmarking Agentic Broad Info-Seeking

Yan Gao이 arXiv에 게시한 'WideSearch: Benchmarking Agentic Broad Info-Seeking' 논문에 대한 자세한 리뷰입니다.

#Review #Agentic Search #LLM #Benchmark #Information Seeking #Structured Output #Evaluation Metrics #Multi-agent Systems

2025년 8월 12일