Generate 3D Gaussian Splats from One Image with AI Video Models

What this is, in plain English

This workflow explains an advanced technique to create a 3D representation of a scene, called “Gaussian splats,” from just a single 2D image. Gaussian splats are a new way to represent 3D objects and environments using many tiny, transparent, colored points (like a cloud of paint splatters) that look realistic from any angle.

The core idea is to use an AI video model to generate a short video that shows different views of the subject from your single input image. Once you have this video, you can then use its frames, along with your original image, to “train” the Gaussian splats, effectively turning a 2D picture into an interactive 3D scene.

This is an advanced workflow because it requires specific software like ComfyUI, which is a technical tool for running AI models. It also demands powerful computer hardware, specifically a graphics card (GPU) with a lot of dedicated memory (VRAM). Because of these requirements, it cannot be reduced to simple copy-paste steps for a beginner and is not suitable for mobile devices.

What you can use it for

• Create 3D assets from photos: Turn a single picture into a 3D object or scene that can be used in games, virtual reality (VR) experiences, or digital art projects.
• Virtual reality experiences: Generate immersive 3D environments that can be explored and interacted with in VR headsets, offering a new level of realism.
• Digital product showcases: Make interactive 3D models of products from just one image, allowing customers to view them from all angles on online stores.
• Artistic expression: Experiment with new forms of digital art by transforming static 2D images into dynamic, explorable 3D scenes.

Tools you need

• Qwen Image 2.1 (paid): An AI model used for advanced image editing, such as changing backgrounds.
• ComfyUI (free): A visual, node-based interface for running Stable Diffusion and other AI models, used here for video generation.

How it actually works

This process involves several technical steps, primarily focused on generating a video from your single image, which then serves as input for creating Gaussian splats. The excerpt focuses on the video generation part, not the final splat training.

1. Prepare your input image: Choose a high-resolution image, ideally around 1440 x 1920 pixels, with a common aspect ratio. This higher resolution is important for the final quality of the splats.

2. Edit the background (optional): If the subject in your image is too close to a wall or other geometry that would interfere with a 360-degree view, you can use an image editing AI like Qwen Image 2.1 to change the background. The author used a prompt like this:

   Change the background to a scene on a red carpet with palm trees in the background.

   This step helps ensure the generated video can show a full orbit around the subject without collisions.

3. Set up ComfyUI: Install and configure ComfyUI on a computer equipped with a powerful graphics card (GPU). The author notes using a card with 12GB of VRAM (Video RAM, the memory on your graphics card) for this process. This setup involves downloading the necessary AI models and workflows.

4. Generate a video from the image: Within ComfyUI, use a specific workflow designed for video generation from a single image (the author used the “Minimax H3 default FL2VA workflow,” which stands for first-last frame to video). You will set your original input image as both the first and the last frame of the video.

   The author rendered their video at 768 x 1024 resolution with 20 steps, which took about 35 minutes on their 12GB VRAM card. You can use a lower resolution or fewer steps (e.g., 8 steps with a “lightning LoRA” (Low-Rank Adaptation) if available) to speed up the process, but this will likely reduce the quality of the output video.

5. Avoid video upscaling: The author advises against using video upscaling tools (like SeedVR 2.5) for this workflow. The quality of the final Gaussian splats benefits more from the original high-resolution input image than from an upscaled video.

6. Train the Gaussian Splats: After generating the video, the next step (not detailed in the excerpt) would be to use the video frames and the original high-resolution image to train the Gaussian splats. The author notes that the single center image can be of higher resolution than the video frames for this training process.

Words you’ll see, explained

• Gaussian Splats: A modern way to represent 3D scenes using many tiny, transparent, colored points (called “splats”) that can be viewed from any angle, creating a realistic 3D effect.
• VRAM (Video RAM): Special memory located on a graphics card (GPU) that is essential for processing complex visual data and running large AI models quickly.
• ComfyUI: A visual, node-based interface that allows users to build and run complex AI workflows, especially for image and video generation, without needing to write code directly.
• LoRA (Low-Rank Adaptation): A technique used to efficiently fine-tune large AI models, allowing them to learn new styles or specific details without requiring extensive retraining of the entire model.
• Aspect Ratio: The proportional relationship between an image’s width and its height (e.g., 16:9 for widescreen videos, 4:3 for older screens).

Original source

This concept was shared by /u/akatash23 on Reddit, building on an earlier tutorial by /u/Many-Ad-6225. The author investigated how AI video models could be used to create Gaussian splats, particularly for use in virtual reality.

Notes & variations

• Do you even need this?: For simpler 3D effects, basic image manipulation, or creating 2D animations, more accessible tools like Canva or mobile photo/video editors might be sufficient. This workflow is specifically for advanced 3D scene reconstruction.
• Free-tier limits: While ComfyUI itself is free and open-source, running it locally requires a powerful computer with a dedicated GPU, which can be a significant upfront cost. Cloud GPU services (like Google Colab Pro, RunPod, or vast.ai) offer access to the necessary hardware but are paid services. Qwen Image 2.1 is likely a paid API or service.
• Common pitfall: Underestimating the hardware requirements. This workflow explicitly mentions needing a GPU with at least 12GB of VRAM. Attempting to run it on less powerful hardware will result in extremely slow processing times or outright failure, leading to frustration.