TY - JOUR
T1 - TSCM
T2 - Efficient image synthesis using latent diffusion
AU - Lin, Zhiwei
AU - Wang, Xiaolong
AU - Zhang, Songchuan
AU - Wang, Tao
AU - Guo, Zhonghua
AU - Liu, Tai
N1 - Publisher Copyright:
© 2026 Elsevier B.V.
PY - 2026/4
Y1 - 2026/4
N2 - This paper introduces a novel method for distilling diffusion models, called the Two Step Consistency Model (TSCM). TSCM significantly enhances the generation speed of diffusion models, particularly in generating guiding images, achieving real-time, high-resolution output. Unlike traditional consistency models, TSCM avoids marginal effects associated with distillation by first resolving the diffusion model in one step and then executing consistency model calculations. Furthermore, TSCM incorporates Low-Rank Adaptation (LORA) technology to prevent a significant increase in spatial complexity. Notably, TSCM requires only a minor increase in storage space to significantly improve model inference speed and supports training while keeping the original model weights frozen. To validate TSCM’s effectiveness, experiments were conducted on the Deepfashion dataset and the LAION-5B dataset, evaluating the distillation effects. In the Deepfashion dataset, TSCM improved the Frechet Inception Distance (FID) from 11.238 to 10.352 and the Learned Perceptual Image Patch Similarity (LPIPS) from 0.1921 to 0.1824. In the LAION-5B dataset, TSCM effectively avoided image blurring when reducing the number of generation steps, and its CLIP score increased by 4.21 compared to previous sampling methods. Additionally, TSCM can compress the generation time by an average of 8.8 times while maintaining the same level of image generation quality.
AB - This paper introduces a novel method for distilling diffusion models, called the Two Step Consistency Model (TSCM). TSCM significantly enhances the generation speed of diffusion models, particularly in generating guiding images, achieving real-time, high-resolution output. Unlike traditional consistency models, TSCM avoids marginal effects associated with distillation by first resolving the diffusion model in one step and then executing consistency model calculations. Furthermore, TSCM incorporates Low-Rank Adaptation (LORA) technology to prevent a significant increase in spatial complexity. Notably, TSCM requires only a minor increase in storage space to significantly improve model inference speed and supports training while keeping the original model weights frozen. To validate TSCM’s effectiveness, experiments were conducted on the Deepfashion dataset and the LAION-5B dataset, evaluating the distillation effects. In the Deepfashion dataset, TSCM improved the Frechet Inception Distance (FID) from 11.238 to 10.352 and the Learned Perceptual Image Patch Similarity (LPIPS) from 0.1921 to 0.1824. In the LAION-5B dataset, TSCM effectively avoided image blurring when reducing the number of generation steps, and its CLIP score increased by 4.21 compared to previous sampling methods. Additionally, TSCM can compress the generation time by an average of 8.8 times while maintaining the same level of image generation quality.
KW - Consistency models
KW - Diffusion models
KW - Human image generation
UR - https://www.scopus.com/pages/publications/105029677348
U2 - 10.1016/j.patrec.2026.02.005
DO - 10.1016/j.patrec.2026.02.005
M3 - Article
AN - SCOPUS:105029677348
SN - 0167-8655
VL - 202
SP - 57
EP - 62
JO - Pattern Recognition Letters
JF - Pattern Recognition Letters
ER -