Skip to content

feat(diffusers): add AudioLDM2 generation - #11513

Open
localai-org-maint-bot wants to merge 1 commit into
masterfrom
cron/issue-1699-audioldm2
Open

feat(diffusers): add AudioLDM2 generation#11513
localai-org-maint-bot wants to merge 1 commit into
masterfrom
cron/issue-1699-audioldm2

Conversation

@localai-org-maint-bot

Copy link
Copy Markdown
Collaborator

Description

This PR adds AudioLDM2 text-to-audio generation through the existing diffusers backend and /v1/sound-generation API.

  • implements the diffusers SoundGeneration RPC and writes pipeline output as mono PCM WAV
  • maps duration_seconds to audio_length_in_s and prompt_influence to guidance_scale
  • advertises sound generation in backend capability discovery
  • adds the audioldm2 gallery model and user documentation
  • adds focused request-mapping, WAV serialization, capability, and gallery coverage

Closes #1699

Notes for Reviewers

Verification:

  • python3 -m py_compile backend/python/diffusers/audio_utils.py backend/python/diffusers/backend.py backend/python/diffusers/test.py
  • focused PCM WAV serialization check
  • go test ./core/config -ginkgo.focus="advertises diffusers sound generation"
  • go test ./core/gallery -ginkgo.focus="gallery/index.yaml variant invariants|backend gallery entries are valid"
  • git diff --check

The full go test ./core/config run passes 500 of 501 specs here. Its existing remote-config validation fails because this environment receives HTTP 403 from raw.githubusercontent.com.

The full diffusers runtime test needs the backend environment and model dependencies; this cron did not run a long backend build.

Signed commits

  • Yes, I signed my commits.
  • Documentation updated (docs/content/) for user-facing changes, or not applicable

Expose diffusers audio pipelines through the existing sound-generation RPC. AudioLDM2 can now return PCM WAV output from the model gallery without a separate backend.

Assisted-by: Codex:gpt-5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AudioLDM2Pipeline

1 participant