From Words to Bricks: Carnegie Mellon’s LegoGPT AI Turns Text Into Functional LEGO Builds
A team of AI researchers at Carnegie Mellon University has introduced LegoGPT, an innovative system that converts simple text prompts into stable, buildable LEGO models.
This groundbreaking project combines machine learning with robotics, creating a platform where AI aids in design while robotic arms bring the models to life.
LegoGPT is the product of a collaboration between machine learning and robotics experts, aiming to provide both hobbyists and professionals with a tool to translate text-based instructions into structurally sound LEGO creations.
LegoGPT generates a LEGO structure from a user-provided text prompt in an end-to-end manner. Notably, our generated LEGO structure is physically stable and buildable.
Paper: LegoGPT: Generating Physically Stable and Buildable LEGO Designs from Text
Project:… pic.twitter.com/qISNMMATkl
— AI Bites | YouTube Channel (@ai_bites) May 10, 2025
Unlike typical AI generators, which might produce designs that lack physical feasibility, LegoGPT ensures its designs comply with the laws of physics.
The system is trained on a comprehensive dataset of over 47,000 LEGO structures, paired with detailed descriptions, enabling it to predict the optimal placement of each brick in sequence.
This approach ensures that the final design remains stable at every stage of construction.
According to the team, LegoGPT achieves a remarkable 98% success rate in producing physically stable designs.
We've released the code for LegoGPT. This autoregressive model generates physically stable and buildable designs from text prompts, by integrating physics laws and assembly constraints into LLM training and inference.
This work is led by PhD students @AvaLovelace0, @kangle_deng,… pic.twitter.com/wdhkF5NBgC
— Jun-Yan Zhu (@junyanz89) May 9, 2025
LegoGPT Aims to Provide a Host of Purposes
LegoGPT aims to provide a versatile tool that caters to both educational and design needs, allowing users to generate detailed LEGO projects from simple text descriptions.
While the system is still in its early stages, the researchers have made their code and database publicly accessible, inviting individuals and organisations to explore and leverage the tool.
All relevant resources can be found on the project’s official website.
LegoGPT is part of a broader movement toward AI-driven solutions for LEGO design, joining other advanced platforms like BrickCenter, which generates custom LEGO models from text prompts, and Brickit, a mobile app that scans available bricks to recommend buildable structures.