[논문리뷰] Calibri: Enhancing Diffusion Transformers via Parameter-Efficient Calibration

링크: 논문 PDF로 바로 열기

The paper "Calibri: Enhancing Diffusion Transformers via Parameter-Efficient Calibration" by Danil Tokhchukov, Aysel Mirzoeva, Andrey Kuznetsov, and Konstantin Sobolev from MSU and FusionBrain Lab, AXXX, discusses a new method called Calibri.

I have extracted the content of the paper. Now I will structure the summary according to the user's requirements.

Part 1: Markdown Summary

• Authors : Identify from the paper.
• Keywords : Extract 5-8 technical terms.
• Key Terms & Definitions : Define 3-5 core concepts.
• Motivation & Problem Statement : Explain why the research was done.
• Method & Key Results : Describe Calibri and its performance.
• Conclusion & Impact : Summarize the findings and implications.
• Figure Citations : Add [Figure N] where appropriate.

Part 2: JSON Figure Information

• Select up to 3 important figures.
• Get image_url for each (full path).
• Provide a caption_kr (short Korean description).

Let's break down the content and fill these sections.

Authors: Danil Tokhchukov, Aysel Mirzoeva, Andrey Kuznetsov, Konstantin Sobolev.

Keywords brainstorm: Diffusion Transformers, Parameter-Efficient Calibration, Generative Models, Black-Box Optimization, CMA-ES, Text-to-Image Synthesis, Inference Steps, Reward Model.

Key Terms & Definitions:

• Diffusion Transformers (DiTs) : The backbone architecture for advanced generative models like Stable Diffusion 3 and FLUX, replacing traditional UNet.
• Parameter-Efficient Calibration : A method to enhance model performance by adjusting only a small subset of parameters (around 10^2) rather than fine-tuning the entire model.
• Black-Box Reward Optimization : Framing the calibration parameter search as an optimization problem where the internal workings of the reward function are not directly accessible, relying solely on its output.
• CMA-ES (Covariance Matrix Adaptation Evolution Strategy) : A gradient-free evolutionary algorithm used to efficiently solve the black-box optimization problem for finding optimal calibration coefficients.
• Inference Steps : The number of iterative steps required for a diffusion model to generate an image; reducing this number improves efficiency.

Motivation & Problem Statement: The paper highlights that despite the uniform architecture of Diffusion Transformers (DiTs), their constituent blocks contribute unevenly to the overall generative quality. Previous work identified "vital layers," and the authors' analysis further reveals that selectively disabling certain DiT blocks can sometimes improve image quality, while a simple re-weighting of each block's output with a learned scalar consistently enhances performance. This indicates that the standard DiT architecture is sub-optimally weighted. The core problem is to efficiently calibrate these DiT blocks to significantly improve generation quality and reduce inference steps without extensive fine-tuning.

Method & Key Results: Calibri proposes a parameter-efficient approach to calibrate DiT components. It formulates the calibration as a black-box reward optimization problem, where the goal is to maximize the quality of model outputs as measured by a reward model (e.g., HPSv3). This optimization is efficiently solved using the gradient-free evolutionary algorithm, CMA-ES , modifying only approximately 10^2 parameters. The method introduces three granularity levels for internal-layer calibration: Block Scaling , Layer Scaling , and Gate Scaling , with Layer Scaling showing consistent improvements across reward functions. Furthermore, the paper introduces Calibri Ensemble , which integrates multiple calibrated models to further boost generative performance.

Experimental results demonstrate Calibri's effectiveness across various state-of-the-art text-to-image models, including FLUX.1-dev , Stable Diffusion 3.5 Medium (SD-3.5M) , and Qwen-Image . For instance, Calibri significantly improves HPSv3 scores (e.g., from 11.41 to 13.48 for FLUX) and ImageReward scores, while drastically reducing the Number of Function Evaluations (NFE) , often by 2x to 3.3x (e.g., from 30 to 15 for FLUX, 80 to 30 for SD-3.5M, and 100 to 30 for Qwen-Image). A user study confirmed human preference for Calibri-generated images in both Overall Preference and Text Alignment . Calibri also integrates efficiently with existing alignment methods like Flow-GRPO , achieving comparable performance improvements with significantly fewer updated parameters (e.g., 216 parameters for Calibri vs. 18.78M for Flow-GRPO to optimize PickScore on SD-3.5M). The calibration cost ranges from 32 to 356 H100 GPU-hours , which is a one-time offline cost providing a permanent speed-up at inference.

Conclusion & Impact: Calibri successfully demonstrates that enhancing DiT generative capabilities can be achieved through a parameter-efficient calibration approach, focusing on a single learned scaling parameter. By leveraging black-box optimization with CMA-ES , the method significantly improves generation quality and remarkably reduces the number of inference steps required, thereby boosting computational efficiency without compromising output quality. This research provides a practical and effective solution for enhancing state-of-the-art diffusion models, offering substantial benefits for real-world applications where both high-quality generation and efficiency are critical. Its compatibility with existing alignment methods further broadens its applicability and impact on the field of visual content generation.

Figures Selection:

1. Figure 2: Illustration of DiT architectural components. This shows the basic building blocks.
2. Figure 3: Motivational Experiment: DiT block ablation and scaling. This visually explains the core insight for Calibri.
3. Figure 4: Illustration of calibration parameter search procedure. This shows the methodology.

Let's verify the image_url for these figures.

• Figure 2: 2603.24800v1/x1.png for (a) DiT block scheme and 2603.24800v1/x2.png for (b) MM-DiT block scheme. The prompt says "Figure 2: Illustration of DiT architectural components." but it shows two subfigures. I should pick one or combine it if possible, but the instruction is for a single image_url per figure_id. Let's use x1.png for Figure 2 and denote it as "DiT block scheme" or perhaps Figure 2(a) if the JSON structure allows. The example JSON {"figure_id": "Figure 1" suggests using the main figure number. I will choose the first one (x1.png) and make a general caption for Figure 2.
• Figure 3: 2603.24800v1/x3.png for (a) DiT block ablation and 2603.24800v1/x4.png for (b) DiT block scaling. Similar issue. I will choose x3.png for Figure 3 and a general caption for the motivational experiment.
• Figure 4: 2603.24800v1/x5.png. This looks like the main methodology diagram.

I will try to get the URLs from the browsed content precisely. From the HTML:

• Figure 2(a) DiT block scheme is 2603.24800v1/x1.png
• Figure 2(b) MM-DiT block scheme is 2603.24800v1/x2.png
• Figure 3(a) DiT block ablation is 2603.24800v1/x3.png
• Figure 3(b) DiT block scaling is 2603.24800v1/x4.png
• Figure 4: Illustration of calibration parameter search procedure. is 2603.24800v1/x5.png

The instructions say "Figure/Table을 최대 3개 선별합니다." (select up to 3 figures/tables). And then {"figure_id": "Figure 1", ...}. It seems they want the main Figure number, not sub-figures. I will select:

1. Figure 2: As it illustrates the architecture. I'll use x1.png which represents the Standard DiT block scheme.
2. Figure 3: As it illustrates the motivation. I'll use x3.png which represents DiT block ablation.
3. Figure 4: As it illustrates the method. I'll use x5.png.

Rechecking constraints:

1. Technical Terminology in English: Done.
2. Natural Korean Phrasing: Done, mixed with English terms.
3. Mandatory Metadata: Authors and Keywords at the top.
4. Output format: Markdown for Part 1, JSON for Part 2 with `

⚠️ 알림: 이 리뷰는 AI로 작성되었습니다.