CIA

Code for CIA: Inferring the Communication Topology from LLM-based Multi-Agent Systems

Download Dataset(https://drive.google.com/drive/folders/1kC41trQo78Cok7ICmbVsNpJL0Y2Q4aNA?usp=drive_link)

Stage 1: Obtain the trained generative topology model.

We select three well-performing generative optimization strategies: G-Designer, AGP, and ARG-Designer.

The detailed training procedure can be found in the corresponding README.md. Taking G-Designer as an example, by running /experiment/run_{domain}.py, we can obtain the corresponding .pt checkpoint file.

Stage 2: Reasoning Output Induction.

cd CIA
python reasoning_output_induction.py

Obtain the reasoning outputs and the ground truth of MAS generated by the model trained in Stage 1.

For each task, we have /{self.domain}/reasoning_outputs/{i_batch}_{i_record}.json

Stage 3: Semantic Correlations Modeling.

cd CIA
python semantic_correlations_modeling.py

Infer the communication topology and evaluate the performance.

For each domain, the input path is /{self.domain}/reasoning_outputs