Project Genie | How world sketching works

2026-01-29 · Source: Google DeepMind · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Gaming & Interactive Media · Depth: Fundamental Awareness, quick

Summary

Project Genie is a new tool that allows users to create and fine-tune virtual worlds through a combination of text prompts and sketching. Users begin by describing their desired environment, including a diverse set of objects, and then define a character, specifying its movement and point of view (first or third person). The platform, which utilizes "Nano Banana Pro" for previews, enables users to sketch modifications directly onto a visual representation of their world, such as adding specific elements like a house or mountains. This iterative process of text description and visual sketching helps users bring their imagined worlds to life for exploration. Project Genie is available at labs.google/project genie.

Key takeaway

For creative technologists and product designers building virtual experiences, Project Genie offers a novel approach to world creation. Your teams can leverage its text-to-sketch workflow to rapidly prototype and iterate on immersive environments, potentially reducing development time and enhancing creative control. Consider integrating similar multimodal input methods into your own design pipelines to empower users with more intuitive creation tools.

Key insights

Project Genie combines text prompting and visual sketching to create and customize virtual worlds.

Principles

• Iterative design refines virtual environments.
• Text and sketch inputs enhance world creation.

Method

Describe environment and character via text, then use a "create sketch" feature to visually modify the world with additional text prompts, leading to world generation.

In practice

• Use detailed prompts for rich environments.
• Add diverse objects to world descriptions.

Topics

• Virtual World Generation
• Text-to-World
• Interactive Sketching
• Prompt Engineering
• Generative AI

Best for: Creative Technologist, Product Designer, General Interest

Related on AIssential

• Project Genie | How world remixing works — Project Genie allows users to remix virtual worlds and characters using natural language prompts.
• Project Genie | Skydiving — Project Genie generates interactive 2D worlds from text or image prompts in real time.
• A Solution Looking for a Problem... — Google Genie transforms text prompts into interactive, navigable 3D virtual worlds.
• The Sequence Knowledge #838: Project GENIE: Building Playable Worlds from Pixels — Project GENIE enables AI to simulate and interact within playable pixel-based worlds, fostering agency beyond text-based models.
• Create new worlds in Project Genie with these 4 tips — Project Genie enables interactive world creation and exploration through detailed text and image prompting.
• A collaborative system that teaches AI models to sketch more like humans do — AI models can learn human-like sketching by focusing on stroke order and direction, not just final image.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Google DeepMind.