[BUG] Memory Overflow and Thread Error in `MWF4WLIKTIFZIAABAAAAADY8_usd` Scene

[zhanglixuan0720]

Description:
I encountered an issue when testing the NavDP model with the MWF4WLIKTIFZIAABAAAAADY8_usd scene in scenes_commercial. The memory usage spikes to 80GB during the test, whereas other scenes only use around 10GB. After this, the following errors appear:

2025-11-20 08:11:29 [1,286,639ms] [Warning] [omni.ujitso] UJITSO : Build storage validation failed for Storage status: 1
http://localhost:8002/navigator_reset
No trajectory available, using zero action
No trajectory available, using zero action
No trajectory available, using zero action   [ERROR:0@1278.827] global parallel_impl.cpp:244 WorkerThread 54: Can't spawn new thread: res[0/1840]
LLVM ERROR: out of memory
[ERROR:0@1278.827] global parallel_impl.cpp:244 WorkerThread 55: Can't spawn new thread: res = 11
[ERROR:0@1278.829] global parallel_impl.cpp:244 WorkerThread 56: Can't spawn new thread: res = 11
[ERROR:0@1278.829] global parallel_impl.cpp:244 WorkerThread 57: Can't spawn new thread: res = 11
[ERROR:0@1278.829] global parallel_impl.cpp:244 WorkerThread 58: Can't spawn new thread: res = 11
[ERROR:0@1278.829] global parallel_impl.cpp:244 WorkerThread 59: Can't spawn new thread: res = 11
LLVM ERROR: out of memory
Aborted (core dumped)

Steps to Reproduce:

1. Clone the NavDP repository and set up the environment.

2. Download and set up the scenes_commercial/MWF4WLIKTIFZIAABAAAAADY8_usd scene

3. Run the model with the scenes_commercial/MWF4WLIKTIFZIAABAAAAADY8_usd scene using the following command:

   python eval_pointgoal_wheeled.py --port 8000 --scene_dir assets/scenes/internscenes_commercial/scenes_commercial --scene_index 17 --scene_scale 0.01 
   

The modefied part of eval_pointgoal_wheeled.py is as follows:

all_scenes = sorted(os.listdir(args_cli.scene_dir))
scene_path = os.path.join(args_cli.scene_dir,all_scenes[args_cli.scene_index]) + "/"

4. Observe the error and the program aborts.

Additional Information:

• This issue only occurs in the MWF4WLIKTIFZIAABAAAAADY8_usd scene. Other scenes, like those in the cluttered_easy or internscenes_commercial directories, do not show this behavior.
• It appears to be related to the memory management or threading during the model's operation.

Environment:

• Operating System: Ubuntu 22.04.5 LTS
• GPU: NVIDIA GeForce RTX 4090
• Memory: 1 TB

Request:
Could you please advise on possible solutions for this issue? Any suggestions for addressing this issue would be appreciated.

[wzcai99]

Thank you for your feedback, we will try to figure this out in the next few days.

[zhanglixuan0720]

Thanks!