Hyper-realistic "Digital Twins" for robotics training generated via World Labs LWMs
Powered by World Labs
WorldLab Robotics Examples bridges the gap between Generative AI and Embodied Intelligence. Traditional simulations are too sterile; real-world data is too expensive. We use World Labs' Large World Models (LWMs) to generate "Physically Grounded Digital Twins"—chaotic, photorealistic, and 3D-consistent environments engineering-grade spatial reasoning.
Whether you are testing a quadruped on a construction site, a manipulator in a messy bedroom, or a social robot in a hotel lobby, this repository provides the "Digital Twin" blueprints you need. Physical intelligence requires physical data. This repository leverages the World Labs API to generate unlimited, photorealistic, and 3D-consistent environments ("Digital Twins") for robotics training—ranging from domestic chores to complex industrial assembly.
Below are the "Digital Twin" blueprints available in this repository. Each prompt is engineered for specific robotics vision and interaction tasks.
Prompt: "A hyper-realistic, robot-centric digital twin of a modern domestic kitchen. Captured from a 1.2m height (robot eye level). Key interaction zones include a granite island with specific grasping targets..."
Prompt: "A physically simulated messy bedroom for high-complexity manipulation training... A tangled pile of laundry (red t-shirt, blue jeans, white socks) rests on an unmade bed..."
Prompt: "A 'lived-in' living room state for housekeeping robots. An open suitcase sits on a rug, spilling patterned clothing... The scene tests the robot's ability to distinguish between 'trash' and 'personal items'..."
Prompt: "A precision-mapped industrial warehouse aisle. High-contrast yellow safety lines painted on polished concrete floors. Metal racking units... Fiducial markers (QR codes) visible on shelf edges..."
Prompt: "Macro-photography view of an electronics assembly workbench. A green PCB (Printed Circuit Board) is clamped in a holder... Nearby components: a tray of tiny 0402 resistors..."
Prompt: "System Role: Senior Simulation Engineer specializing in Deformable Linear Objects (DLOs)... Objective: Generate a high-fidelity 3D simulation environment for 'Data Center Cable Remediation'..."
Prompt: "A heavy-industry automotive welding cell. A raw steel car chassis acts as the central workspace. Orange industrial robotic arms (KUKA-style)... Sparks from a spot welder are frozen in mid-air..."
Prompt: "A 5-star hotel lobby designed for social navigation. Polished marble floors reflecting the environment... The space is populated with 'ghost' proxies of people... Luggage carts with brass railings..."
Prompt: "A dense restaurant dining floor layout. Round tables covered in white cloth... Chairs are pulled out at irregular angles, narrowing the navigation path to <60cm. Ambient lighting is dim/moody..."
Prompt: "A sterile, bright hospital corridor. High-gloss vinyl flooring. Walls are white with color-coded wayfinding strips. Obstacles include a crash cart... Lighting is clinical fluorescent..."
Prompt: "A planogram-compliant supermarket aisle. Shelves are fully stocked with colorful cereal boxes... Several items are intentionally missing (out-of-stock) to test gap detection. Price tags on the shelf edge are legible..."
Prompt: "First-person view from a delivery rover on a city sidewalk. Paved concrete slabs with cracks and chewing gum stains. A fire hydrant (red) and a parking meter (gray) act as static obstacles..."
Prompt: "An active chaotic construction site. Ground is uneven, covered in loose gravel and sand. Piles of red bricks and wooden pallets create traversability barriers. Exposed steel rebar acts as a snagging hazard..."
Prompt: "A simulated post-earthquake rubble pile. Broken slabs of concrete, twisted rebar, and shattered glass... Atmospheric dust/haze reduces visibility to 80%. Lighting is low-angle, simulating dusk..."
Prompt: "A robotic agriculture perception dataset. Parallel rows of green corn crops, approximately 1.5m high. The inter-row path is dirt with random weeds. Sunlight is directly overhead (noon)..."
scripts/:generate_with_api.py: (Core Tool) A powerful script pre-loaded with 15+ robotics scenarios to generate 3D worlds on demand.
prompts/:core_collection.json: Structured JSON tasks linked to specific environments (e.g., "Plug in ethernet cable" -> "Data Center").
worlds/:api_renders/: The output directory where your generated world thumbnails and assets are saved.
You need a World Labs Platform account and API key.
- Create a
.envfile:cp .env.example .env
- Add your key:
WLT_API_KEY=wl_...
We have automated the creation of reference environments. Run the script to generate verified scenarios:
# Generates all configured scenarios using the World Labs API
python3 scripts/generation/generate_with_api.pyNote: This utilizes the Marble 0.1-mini model for speed and efficiency.
Load our structured prompts to test your robot's reasoning against these worlds.
import json
with open("prompts/core_collection.json", "r") as f:
tasks = json.load(f)
# Example: Get a task for the Data Center environment
cable_task = [t for t in tasks[1]["prompts"] if t["id"] == "manip_cable_01"][0]
print(f"Task: {cable_task['task']}")Want to take these "Digital Twins" into a physics-based simulator? Check out our complete guide on exporting World Lab models: From Imagination to Simulation: A Complete Guide to Importing World Labs Marble Models into NVIDIA Isaac Sim
Have a new robotics use case?
- Fork the repo.
- Add your scenario to
generate_with_api.py. - Add a corresponding task to
prompts/core_collection.json. - Submit a Pull Request!
MIT License.