SD feature channel

[mldemox]

RuntimeError: Given groups=1, weight of size [320, 4, 3, 3], expected input[2, 9, 64, 64] to have 4 channels, but got 9 channels instead

[SmolBoi96]

which stable diffusion model are you using? I got the same error using:
"stabilityai/stable-diffusion-2-1-base"
"stabilityai/stable-diffusion-2-base"
"runwayml/stable-diffusion-v1-5"

Using the following gave me cuda out of memory instead:
"runwayml/stable-diffusion-inpainting"
"stabilityai/stable-diffusion-2-inpainting"

[chenhonghua]

@SmolBoi96 Basically, we use diffusion inpainting models, and four 32G V100 to train a NeRF scene.

[chenhonghua]

@mldemox Could you please provide more information about where this error occurs?

[SmolBoi96]

I reproduced the error:

Here is are the following changes i did to the current latest commit to reproduce the error:

1. Implement the fix here to fix issue IndexError when running the code. #14
2. remove the line to do data parallelization for Stable Diffusion (this line does not work when I offload to CPU)
3. set vram_o to true to safe on vram memory (cause I always CUDA OOM)
4. use "stabilityai/stable-diffusion-2-1-base" instead of "runwayml/stable-diffusion-inpainting"
5. use fp16 instead of fp32

My Git diff is shown here:

I suspect using SD base instead of SD inpainting is the issue?

[mldemox]

After checking the documentation, I found that the input channel of stable-diffusion-xxx series is 4, so I chose stable-diffusion-inpaint series to test it, and found that the following error is reported:

GPU: A6000 48GB