Add AutoPipeline For Text2Video

[naomili0924]

What does this PR do?

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[IlyasMoutawwakil]

does this test pass with "hf-internal-testing/tiny-stable-diffusion-pipe" ?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[sayakpaul]

@yiyixuxu could you give this a look?

[yiyixuxu]

hey @naomili0924

thanks for the PR! could you explain what would be the motivation for this PR? these pipeline groups do not support multiple tasks

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[IlyasMoutawwakil]

@yiyixuxu the idea is to have a single entrypoint for the text2video models / task

[yiyixuxu]

hi @IlyasMoutawwakil

The purpose of AutoPipeline is to enable users to switch between tasks using the same checkpoint. For video models, though, different tasks typically require different checkpoints (see Wan 2.1 for example - their checkpoints all have the task encoded in their names: "T2V" for text-to-video, "I2V" for image-to-video, plus other tasks like audio-to-video, first-last-frame-to-video, etc.). We're also seeing some model makers (e.g. LTX) release checkpoints that handle multiple tasks, which we support with a single pipeline (LTXConditionPipeline).

The only exception where checkpoint sharing happens is text-to-video vs video-to-video, but IMO it's not worth adding and maintaining two auto pipelines for such a limited use case.

I'd like to understand the motivation a bit more - is there a specific project you're working on that requires this? If the goal is just to have a single entry point to load different video pipelines without knowing the class name, DiffusionPipeline.from_pretrained() should already handle that.

[IlyasMoutawwakil]

Thanks @yiyixuxu i think i misunderstood the rationale behind AutoPipelines. The reason why we prefer having a task-specific AutoPipeline is because it makes task-specific testing in optimum-onnx and optimum-intel easy.
DiffusionPipeline does not guarantee which pipeline you will get when you load a checkpoint, while AutoPipelineForTask will always return the pipeline for the task it's related to.
I wasn't aware that video checkpoints only support one task for now, thanks for the clarification, i think we can use DiffusionPipeline with video models in our testing.