Skip to content

Reduce peak VRAM by releasing large attention tensors (as soon as they're unnecessary)#3463

Merged
patrickvonplaten merged 1 commit into
huggingface:mainfrom
cmdr2:attn_tensor_free
May 17, 2023
Merged

Reduce peak VRAM by releasing large attention tensors (as soon as they're unnecessary)#3463
patrickvonplaten merged 1 commit into
huggingface:mainfrom
cmdr2:attn_tensor_free

Release large tensors in attention (as soon as they're no longer requ…

5b68fa1
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs