[Pipelines] Add DreamLite text-to-image and image-edit pipelines

[Carlofkl]

Context

This PR integrates DreamLite — ByteDance's text-to-image / image-edit diffusion model — into diffusers, following an invitation from @NielsRogge to release the model on the Hub in diffusers format.

Related issue: ByteVisionLab/DreamLite#3 (comment)

Model cards (public, ungated):

• Base (3-branch dual CFG): https://huggingface.co/carlofkl/DreamLite-base
• Mobile (distilled, single forward): https://huggingface.co/carlofkl/DreamLite-mobile

Both repos use a diffusers branch (loaded via revision="diffusers") to keep the original ByteDance-internal main branch intact for backward compatibility with existing users.

What's added

src/diffusers/
├── models/unets/unet_dreamlite.py            # DreamLiteUNetModel
├── pipelines/dreamlite/
│   ├── __init__.py
│   ├── pipeline_dreamlite.py                  # DreamLitePipeline (3-branch dual CFG)
│   ├── pipeline_dreamlite_mobile.py           # DreamLiteMobilePipeline (distilled)
│   └── pipeline_output.py
└── (registered in src/diffusers/__init__.py, models/__init__.py,
    pipelines/__init__.py, utils/dummy_*.py)

docs/source/en/api/pipelines/dreamlite.md
tests/pipelines/dreamlite/
├── test_pipeline_dreamlite.py
└── test_pipeline_dreamlite_mobile.py

Architecture highlights

• DreamLiteUNetModel — UNet-based denoiser conditioned on Qwen3-VL text/vision embeddings.
• DreamLitePipeline — runs 3 forward passes per step (text-cond / image-cond / uncond) and combines them with a dual-CFG schedule for high-fidelity text-to-image and image edit.
• DreamLiteMobilePipeline — distilled single-pass variant; no CFG; designed for on-device inference. Pairs with AutoencoderTiny.
• Both pipelines use FlowMatchEulerDiscreteScheduler.

Testing

• Loading smoke test against carlofkl/DreamLite-base with revision="diffusers" — all 6 sub-modules resolve to the correct diffusers.* namespace.
• Inference smoke test — generates a 1024×1024 image in ~0.6s/step on a single A800; output stats sane (std≈93, no NaN/Inf).
• Standard pipeline tests in tests/pipelines/dreamlite/.

Before submitting

• Did you read the contributor guideline?
• Did you read our philosophy doc?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case. → Release DreamLite on Hugging Face ByteVisionLab/DreamLite#3
• Did you make sure to update the documentation with your changes? Here are the documentation guidelines.
• Did you write any new necessary tests?

Who can review?

cc @sayakpaul @yiyixuxu @DN6 — thanks in advance for the review!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Carlofkl]

Hi @sayakpaul @yiyixuxu — pushed a small follow-up commit (032412c) that fixes the two check_repository_consistency failures from the previous run:

1. Registered DreamLiteAttnProcessor2_0 in docs/source/en/api/attnprocessor.md (was missing — check_support_list.py).
2. Split combined height / width and guidance_scale / image_guidance_scale docstring entries in both pipelines into separate lines, and added a complete Args block to DreamLiteTransformer2DModel.forward (was tripping check_forward_call_docstrings.py).

No behavioral change — docs/docstrings only. Verified both lints pass locally.

Whenever convenient, could you re-approve the workflows? Thanks!

[Carlofkl]

Hi @yiyixuxu @DN6 @sayakpaul — quick update: CI is now fully green
(all 15 checks passing) and I've just marked the PR as ready for review.
Whenever you have a moment to take a look, I'd really appreciate it. Thanks!

[dg845]

Thanks for the PR! Left an initial design review :).

[dg845]

Also, if I test out the example using the following script:

import torch
from diffusers import DreamLitePipeline
from diffusers.utils import load_image

model_id = "carlofkl/DreamLite-base"
device = "cuda"
dtype = torch.float16

pipe = DreamLitePipeline.from_pretrained(model_id, revision="diffusers", torch_dtype=dtype)
pipe.to(device=device)

# Text-to-image
image = pipe(
    prompt="A serene mountain lake at sunrise",
    generator=torch.Generator(device=device).manual_seed(42),
).images[0]

image.save("dreamlite_t2i.png")

# Image-to-image (instruction-based edit)
image_url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/astronaut.jpg"
init_image = load_image(image_url)
edited = pipe(
    prompt="make it snowy",
    image=init_image,
    generator=torch.Generator(device=device).manual_seed(42),
).images[0]

edited.save("dreamlite_i2i.png")

I get the following T2I sample:

and the following I2I sample:

Is the sample quality expected? The T2I image in particular has a weird block pattern.

[Carlofkl]

@dg845 Thanks for testing!

Confirmed: the artifact only appears with transformers >= 5.0, which changed the underlying tokenization/processor logic for Qwen3-VL in a way that breaks DreamLite. With transformers==4.57.3 it reproduces cleanly on my end (A800, fp16, carlofkl/DreamLite-base) — see attached samples. I've pinned transformers==4.57.3 in the DreamLite README accordingly.

[Carlofkl]

@dg845 thanks for the initial review — pushed updates in 62a5db6 addressing all the inline comments, please take another look when you have time.

@yiyixuxu friendly ping — would love your eyes on this whenever you get a chance, especially the single-file unet decision in src/diffusers/models/unets/unet_dreamlite.py (per dg845's comment).

[dg845]

Thanks for iterating! Left some follow up comments.

[Carlofkl]

@dg845 follow-up review fully addressed across two commits (9fd711a2a + 63304257e):

-1. resnet (DepthwiseSeparableConv + ResnetBlock2DDreamLite) inlined into unet_dreamlite.py; resnet_dreamlite.py deleted.
-2.DreamLiteAttnProcessor2_0 moved into unet_dreamlite.py; attention_processor.py cleaned up; the autodoc entry in attnprocessor.md repointed to the new path.
-3. Dropped the hasattr(F, "scaled_dot_product_attention") ImportError — diffusers already requires torch>=2.0.
-4. Dropped the deprecated scale handling (*args, **kwargs + deprecate(...)); the new signature matches your suggestion exactly.
-5. Switched SDPA to dispatch_attention_fn with _attention_backend / _parallel_config class attrs (mirrors Flux2AttnProcessor). Follow-up fix in 63304257e: kept Q/K/V in (batch, seq, heads, head_dim) layout — that's what dispatch_attention_fn expects, the previous (batch, heads, seq, head_dim) layout collided with the dispatch's internal transpose and broke inference.
-6+7. Renamed block dispatch keys to match the Python class names: DreamLiteCrossAttn{Down,Up}Block2D, DreamLiteCrossAttnNoSelfAttn{Down,Up}Block2D, DreamLite{Down,Up}Block2D, DreamLiteUNetMidBlock2DCrossAttn. The DreamLiteUNetModel.__init__ defaults, the test fixtures, and the unet/config.json on the diffusers branch of carlofkl/DreamLite-{base,mobile} are all updated in lock-step.
-8. retrieve_latents inlined locally in pipeline_dreamlite.py with a # Copied from marker; the cross-pipeline import is gone — same shape as pipeline_dreamlite_mobile.py.
-9. Added check_inputs(prompt, image, height, width) to DreamLitePipeline and to DreamLiteMobilePipeline (mobile uses # Copied from diffusers.pipelines.dreamlite.pipeline_dreamlite.DreamLitePipeline.check_inputs); both pipelines call it from __call__. The image-type validation moved out of prepare_image_latents into check_inputs, plus we now also check that prompt is a str and warn if height/width aren't multiples of vae_scale_factor.

Re-tested end-to-end T2I + I2I locally on A800 / bf16 against carlofkl/DreamLite-base (diffusers branch) with transformers==4.57.3 — both modes still produce clean outputs.

Note on CI: the 5 failing PR test jobs (Fast PyTorch Pipeline / Models CPU tests, PyTorch Example CPU tests, Hub tests, LoRA tests with PEFT main) appear to be failing on main itself as well — looks like an environment / dependency issue upstream, not introduced by this PR. Happy to investigate further if it persists after main goes green.

[dg845]

Would it be possible to use a small Qwen3VLForConditionalGeneration text encoder model for testing here instead of a MagicMock instance (and similarly for the tokenizer and processor)? For example, NucleusMoE-Image does the following:

diffusers/tests/pipelines/nucleusmoe_image/test_nucleusmoe_image.py

Lines 116 to 117 in 1c7f759

	text_encoder = Qwen3VLForConditionalGeneration(config).eval()
	processor = Qwen3VLProcessor.from_pretrained("hf-internal-testing/tiny-random-Qwen2VLForConditionalGeneration")

This should let us run the skipped tests below normally. (I'm still reviewing the rest of the PR but wanted to leave this comment first since the change would make reviewing the tests easier.)