5. RhinoAI: An LLM-Based 3D Modeling Assistant for Procedural Instructions and Demonstrations
24 Fall, Harvard University. With Viola Tan, Kida Huang.
-
Ilya Sutskever
"AI is not about replacing humans. It's about augmenting human potential, freeing us to explore what we thought impossible."
RhinoAI redefines the landscape of 3D modeling by seamlessly integrating large language models (LLMs) with Rhinoceros 3D, offering an intuitive, dynamic, and accessible framework for novice and professional users alike. Through its unique multi-agent architecture, the system bridges the gap between natural language inputs and complex 3D modeling workflows, allowing users to interact via text or images to create detailed procedural instructions and functional Python scripts. This groundbreaking approach democratizes 3D modeling, transforming it from a technical challenge into a creative and accessible experience.
Designed primarily for novice users, RhinoAI aims to guide learners in understanding and mastering the intricacies of 3D modeling. Simultaneously, the system appeals to professionals seeking efficiency in automating repetitive tasks or exploring new workflows. By providing step-by-step guidance, real-time Q&A support, and procedurally generated results, the platform caters to a diverse user base, from architects and designers to engineers and students venturing into digital design.
RhinoAI leverages a multi-agent pipeline consisting of an Image Description Agent, Task Decomposition Agent, and Script Generation Agent. This architecture translates user inputs—textual or visual—into actionable modeling steps. Enhancing this workflow are features such as metadata-driven memory management, automated debugging, and structured object naming systems, ensuring accuracy and adaptability. By iteratively refining scripts and dynamically supporting user queries, RhinoAI provides a robust framework for both education and efficiency.
At its core, RhinoAI is more than a tool; it is a reflection of how AI can augment human creativity. It empowers users to focus on conceptual design rather than technical barriers, fostering an environment where imagination drives innovation. By integrating AI into 3D modeling, RhinoAI invites us to reconsider the boundaries of human-computer collaboration in the creative process.
Full Documentation: