[quantization] Introduce Qwen3VLVisionMLP wrapper

[stamalakhov]

This commit adds Qwen3VLVisionMLP wrapper and tests for it.

┌───────────── Quantization Error Summary ─────────────
│ Mean |diff|: 0.016181
│ PEIR       : 1.979650 %
└──────────────────────────────────────────────────────
    ┌────────────────────────────────────────────┐
 7.6┤                                            │
    │                                        ••  │
    │                                      ••    │
 5.1┤                                    •••     │
    │                                 ••••       │
    │                               ••••         │
    │                             ••••           │
 2.7┤                           ••••             │
    │                         ••••               │
    │                       ••••                 │
 0.3┤                     ••••                   │
    │                   ••••                     │
    │                 ••••                       │
    │               ••••                         │
-2.1┤             •••                            │
    │           ••••                             │
    │         •••                                │
-4.6┤       •••                                  │
    │     •••                                    │
    │   •••                                      │
    │  ••                                        │
-7.0┤                                            │
    └┬──────────┬──────────┬─────────┬──────────┬┘
   -7.0       -3.3        0.3       3.9       7.6 

Quantized Circle model saved to /mnt/storage/slow_repos/VLM_TICO/TICO/qwen3vl_vision_mlp.q.circle

./ccex test -k quantization.wrapq.wrappers.qwen_vl.test_quant_vision_mlp

RUN unit tests with -k quantization.wrapq.wrappers.qwen_vl.test_quant_vision_mlp ...
test_mode_and_forward (quantization.wrapq.wrappers.qwen_vl.test_quant_vision_mlp.TestQuantQwenVisionMLP.test_mode_and_forward) ... ok
test_calib_quant_export (quantization.wrapq.wrappers.qwen_vl.test_quant_vision_mlp.TestSubgraphExport.test_calib_quant_export) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.893s

OK

Draft: #484

[dayo09]

I believe AutoModelForVision2Seq is included in recent transformer versions.
Could you elaborate your torch and transformers versions on any document?

We need to convert examples into CI someday. It will be helpful to write down the versions.

[stamalakhov]

@dayo09
Ahhh. It occured that AutoModelForVision2Seq is removed in v5.0. I believe i should change it to AutoModelForImageTextToText, which is used in transformers since 4.5 version.

[stamalakhov]

@dayo09
Fixed. Am i supposed to support AutoModelForVision2Seq for versions below 4.5?

[dayo09]

@stamalakhov Thanks so much!

Fixed. Am i supposed to support AutoModelForVision2Seq for versions below 4.5?

No, I don't think so. I believe just mentioning the version is enough for now. It's just for later test. We are preparing for Qwen3-VL model structure's quantization/frontend-compilation possibility. It's not about deployment level yet. :-D

[dayo09]

BTW, could you share your torch and transformers verisons? I encounter this error by running the example. 😓

Loading weights: 100%|████████████████████████████████████████████████████████████████████████████| 625/625 [00:00<00:00, 1913.24it/s, Materializing param=model.visual.pos_embed.weight]
Traceback (most recent call last):
  File "/home/dayo/git/TICO/tico/quantization/wrapq/examples/quantize_qwen_vision_mlp.py", line 74, in <module>
    int8_out = mlp_q(hidden)
  File "/home/dayo/miniconda3/envs/py310-tvm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/dayo/miniconda3/envs/py310-tvm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/dayo/git/TICO/tico/quantization/wrapq/wrappers/ptq_wrapper.py", line 46, in forward
    return self.wrapped(*args, **kwargs)
  File "/home/dayo/miniconda3/envs/py310-tvm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/dayo/miniconda3/envs/py310-tvm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/dayo/git/TICO/tico/quantization/wrapq/wrappers/qwen_vl/quant_vision_mlp.py", line 76, in forward
    fc1 = self.linear_fc1(x_q)
  File "/home/dayo/miniconda3/envs/py310-tvm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/dayo/miniconda3/envs/py310-tvm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/dayo/git/TICO/tico/quantization/wrapq/wrappers/ptq_wrapper.py", line 46, in forward
    return self.wrapped(*args, **kwargs)
  File "/home/dayo/miniconda3/envs/py310-tvm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/dayo/miniconda3/envs/py310-tvm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/dayo/git/TICO/tico/quantization/wrapq/wrappers/nn/quant_linear.py", line 57, in forward
    w = self.obs_weight.fake_quant(w)
  File "/home/dayo/git/TICO/tico/quantization/wrapq/observers/affine_base.py", line 152, in fake_quant
    return torch.fake_quantize_per_channel_affine(
RuntimeError: !needs_dynamic_casting<func_t>::check(iter) INTERNAL ASSERT FAILED at "/pytorch/aten/src/ATen/native/cpu/Loops.h":311, please report a bug to PyTorch. 

[stamalakhov]

No, I don't think so. I believe just mentioning the version is enough for now. It's just for later test. We are preparing for Qwen3-VL model structure's quantization/frontend-compilation possibility. It's not about deployment level yet. :-D

@dayo09
Got it. Thank you.

[dayo09]

@stamalakhov Ah, it was about the module's dtype==torch.bfloat16

[stamalakhov]

BTW, could you share your torch and transformers verisons? I encounter this error by running the example. 😓

Transformers ~ 4.57.6, torch ~ 2.10.0

[dayo09]

LGTM

[mhs4670go]

LGTM