FunGraph

Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interaction

Setup Environment

conda create -n fg python=3.10 --yes
conda activate fg
# install pytorch with the correct cuda version https://pytorch.org/get-started/locally/, 
# for example: conda install pytorch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 pytorch-cuda=11.8 -c pytorch -c nvidia --yes
# or pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu129
pip install -r requirements.txt
pip install -e .

Create a .env file in the root of this project and add your OpenAI API key:

# .env
OPENAI_API_KEY=<YOUR_KEY>

Download SceneFun3D by following the the official documentation.

Download the Functional Element Detector and place it into the models/checkpoints directory.

Edit the paths.yaml and dataset.yaml files to specify your paths, and set the desired scene and video IDs you want to run.

Generate the 3D Scene Graph

python src/generate_scenegraph.py

You can control the hyperparameters of this script in config.yaml.

Interactively Search the Generated 3D Scene Graph

python src/search_scenegraph.py

Citation

If you found this work useful, please consider citing:

@article{rotondi2025fungraph,
  author    = {Dennis Rotondi and Fabio Scaparro and Hermann Blum and Kai O. Arras},
  title     = {FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interaction},
  journal   = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  year      = {2025},
}