Multimodal Diffusion Language Models for Thinking-Aware Editing and Generation

While thinking-aware generation aims to improve performance on complex tasks, we identify a critical failure mode where existing sequential, autoregressive approaches can paradoxically degrade performance due to error propagation. To systematically analyze this issue, we propose ParaBench, a new benchmark designed to evaluate both text and image output modalities. Our analysis using ParaBench reveals that this performance degradation is strongly correlated with poor alignment between the generated reasoning and the final image. To resolve this, we propose a parallel multimodal diffusion framework that enables continuous, bidirectional interaction between text and images throughout the entire denoising trajectory. This model, MMaDA-Parallel, is trained with supervised finetuning and then further optimized by Parallel Reinforcement Learning (ParaRL), a novel strategy that applies semantic rewards along the trajectory to enforce cross-modal consistency. Experiments validate that our approach significantly improves cross-modal alignment and semantic consistency, achieving a 6.9% improvement in Output Alignment on ParaBench compared to the state-of-the-art model, Bagel, establishing a more robust paradigm for thinking-aware image synthesis.

• [2025-11-11] We release our codes and models for MMaDA-Parallel, with two released 8B models MMaDA-Parallel-A and MMaDA-Parallel-M.
• [2025-11-10] We release our research paper for Parallel Multimodal Large Diffusion Language Models for Thinking-Aware Editing and Generation.

Note: Our model has been successfully validated on synthetic datasets focusing on environments, still life, architecture, and natural landscapes. Its performance on out-of-distribution inputs—such as human faces or real-world photographic imagery—has not yet been fully explored. We are actively expanding our training corpus to include more diverse datasets.

First, start with a torch environment with torch 2.3.1 or higher version, then install the following dependencies:

pip install -r requirements.txt

We provide two varients of MMaDA-Parallel with different tokenizers. MMaDA-Parallel-A is trained with tokenizer Amused-VQ, and MMaDA-Parallel-M is trained with tokenizer Magvitv2.

You can directly use the local gradio app to experience the parallel generation with MMaDA-Parallel-A:

Or you can use the inference script to generate the parallel generation results:

cd MMaDA-Parallel-A
python inference.py \
    --checkpoint tyfeld/MMaDA-Parallel-A \
    --vae_ckpt tyfeld/MMaDA-Parallel-A \
    --prompt "Replace the laptops with futuristic transparent tablets displaying holographic screens, and change the drink to a cup of glowing blue energy drink." \
    --image_path examples/image.png \
    --height 512 \
    --width 512 \
    --timesteps 64 \
    --text_steps 128 \
    --text_gen_length 256 \
    --text_block_length 32 \
    --cfg_scale 0 \
    --cfg_img 4.0 \
    --temperature 1.0 \
    --text_temperature 0 \
    --seed 42 \
    --output_dir output/results_interleave

cd MMaDA-Parallel-M
python inference.py interleave_root=./interleave_validation  

• Release the MMaDA-Parallel code and paper.
• Evaluation on ParaBench code.
• Refine MMaDA-Parallel-M and update the corresponding checkpoint.
• Training code for SFT and ParaRL.

@article{tian2025mmadaparallel,
  title={MMaDA-Parallel: Multimodal Large Diffusion Language Models for Thinking-Aware Editing and Generation},
  author={Tian, Ye and Yang, Ling and Yang, Jiongfan and Wang, Anran and Tian, Yu and Zheng, Jiani and Wang, Haochen and Teng, Zhiyang and Wang, Zhuochen and Wang, Yinjie and Tong, Yunhai and Wang, Mengdi and Li, Xiangtai},
  journal={arXiv preprint arXiv:2511.09611},
  year={2025}
}

This work is heavily based on MMaDA and Lumina-DiMOO. Thanks to all the authors for their great work.