Skip to content

chakra_converter PyTorch - conversion failed to due to Cyclic dependecny in DAG #189

@alokprasad

Description

@alokprasad

Describe the Bug

While converting the ET+ pytroch trace to Protobuf format facing conversion error due Cyclic dependency detected.

Steps to Reproduce

Run pytorch resnet18 model with execution trace enable to get both host and device traces.
Use chakra trace link to merge the traces.
use chakra converter to convert to PB.

chakra_converter PyTorch --input chakra_host_device_trace_0.json --output chakra_
host_device_trace_0.pb

chakra repo
commit bf222b2 (HEAD -> main, origin/main, origin/HEAD)

Expected Behavior

file in Protobuf formation.

ERROR [03/23/2025 09:28:44 AM] Cyclic dependency detected: aten::lift_fresh -> aten::result_type -> aten::randint -> aten::sub_ -> aten::sub_ -> aten::view -> aten::view -> aten::item -> aten::is_nonzero -> aten::as_strided -> aten::any -> aten::to -> aten::eq -> aten::lift_fresh -> aten::lift_fresh -> aten::to -> aten::to -> aten::empty -> aten::to -> aten::to -> aten::div -> aten::to_copy -> aten::to -> aten::contiguous -> aten::lift_fresh -> aten::local_scalar_dense -> aten::view -> aten::item -> aten::is_nonzero -> aten::to_copy -> aten::to -> aten::lt -> aten::to -> aten::lt -> aten::uniform -> aten::rand -> aten::item -> aten::random -> aten::empty -> aten::randint -> aten::item -> aten::random -> aten::empty -> aten::randint -> aten::sub_ -> aten::is_nonzero -> aten::view -> aten::fill_ -> aten::as_strided -> aten::any -> aten::lift_fresh. The conversion failed because a cyclic dependency was detected. Cyclic dependencies should not exist. The input and output traces must form a Directed Acyclic Graph (DAG). This is essential for simulation; otherwise, simulators cannot resolve the next dependency-free node and will hang. This indicates a bug in the conversion process. Please investigate or report this issue on GitHub.
ERROR [03/23/2025 09:28:44 AM] Cyclic dependency detected. The conversion failed because a cyclic dependency was detected. Cyclic dependencies should not exist. The input and output traces must form a Directed Acyclic Graph (DAG). This is essential for simulation; otherwise, simulators cannot resolve the next dependency-free node and will hang. This indicates a bug in the conversion process. Please investigate or report this issue on GitHub.
Traceback (most recent call last):
File "/home/marvell/alok/chakra/chakra_env/bin/chakra_converter", line 8, in
sys.exit(main())
File "/home/marvell/alok/chakra/chakra_env/lib/python3.10/site-packages/chakra/src/converter/converter.py", line 113, in main
args.func(args)
File "/home/marvell/alok/chakra/chakra_env/lib/python3.10/site-packages/chakra/src/converter/converter.py", line 36, in convert_pytorch
converter.convert(args.input, args.output, args.simulate)
File "/home/marvell/alok/chakra/chakra_env/lib/python3.10/site-packages/chakra/src/converter/pytorch_converter.py", line 57, in convert
self.identify_cyclic_dependencies(protobuf_node_map)
File "/home/marvell/alok/chakra/chakra_env/lib/python3.10/site-packages/chakra/src/converter/pytorch_converter.py", line 600, in identify_cyclic_dependencies
raise Exception(err_msg)
Exception: Cyclic dependency detected. The conversion failed because a cyclic dependency was detected. Cyclic dependencies should not exist. The input and output traces must form a Directed Acyclic Graph (DAG). This is essential for simulation; otherwise, simulators cannot resolve the next dependency-free node and will hang. This indicates a bug in the conversion process. Please investigate or report this issue on GitHub.

Note trace file is about 13GB.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions