Skip to content

[wrapq] Operation Decomposition After Quantization #489

@dayo09

Description

@dayo09

What?

Problem Statement
When TICO converts (lower/decompose) quantized operator into several operators, some intermediate compute operations remain as float32.
For example,

Conv3d can be lowered into Conv2d + Add + Reshape... or Conv2d + Reshape.
NOTE that our Qwen3-VL patch embedding aims to be lowered down to Conv2d + Reshape without Add when patch size == stride size. This pass is scheduled to be implemented soon.
However, our current pass (26-02-13, main branch) converts Conv3d into Conv2d + Add + Reshape...

Thus, after quantization, Add is additionally generated. This extra Add remains in float32.

How to resolve?

  1. Operator conversion should be done before quantization-calibration, if it produces any computing operations.
  2. If required, we could manipulate qparams for specific operators.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions