Improve AI Image Quality with PiD and ComfyUI (Advanced)

What this is, in plain English

PiD (PixelDiT) is a method to create high-quality images from AI models without using a VAE (Variational Autoencoder). A VAE is a component in many AI image generators that helps turn the AI’s internal “idea” of an image into actual pixels. The author notes that Krea 2 uses a VAE that often produces overly smooth images with poor detail and color. PiD bypasses this, directly generating pixels for better results.

This is an advanced workflow because it requires setting up and running ComfyUI, a complex visual programming interface for AI image generation. It also involves downloading large model files and needs a powerful computer with a lot of video memory (VRAM), around 15GB, which is more than most personal computers have. The exact steps are contained within a ComfyUI workflow file that you need to load and understand.

What you can use it for

• Generate sharper, more detailed AI images: Improve the clarity and fine details in pictures created by AI.
• Achieve better color and contrast: Produce images with richer colors and a wider range between light and dark areas.
• Bypass VAE limitations: Overcome issues like overly smooth or blurry outputs from AI models that use less effective VAEs.
• Experiment with advanced image generation techniques: Learn how to use a powerful, node-based tool like ComfyUI for more control over AI art.

Tools you need

• ComfyUI (free): A powerful, node-based interface for creating and running AI image generation workflows on your computer. https://github.com/comfyanonymous/ComfyUI
• PiD (PixelDiT) (free): A method for generating high-quality images directly in pixel space, avoiding the need for a VAE. https://github.com/nv-tlabs/PiD
• Hugging Face (freemium): A platform where you can find and download many AI models, including the specific PiD models needed for this workflow. https://huggingface.co
• Krea.ai (freemium): An online tool for real-time AI image generation, mentioned as using a VAE that this workflow aims to improve upon. https://krea.ai

How it actually works

The author provides a ComfyUI workflow file that outlines the exact steps. To reproduce this, you would generally follow these advanced steps:

1. Set up ComfyUI: You need to install ComfyUI on your computer. This is a technical process that usually involves downloading the software and setting up its required components (like Python). Refer to the official ComfyUI GitHub page (https://github.com/comfyanonymous/ComfyUI) for detailed installation instructions for your operating system.
2. Download PiD model files: Download the specific PiD Text encoder and Decoder files from Hugging Face.
   • PiD Text encoder: https://huggingface.co/Comfy-Org/PixelDiT/blob/main/text_encoders/gemma_2_2b_it_elm_bf16.safetensors
   • PiD Decoder: https://huggingface.co/Comfy-Org/PixelDiT/blob/main/diffusion_models/pid_qwenimage_1024_to_4096_4step_bf16.safetensors You will need to place these files into the correct folders within your ComfyUI installation (e.g., ComfyUI/models/text_encoders and ComfyUI/models/diffusion_models).
3. Download and load the workflow: Download the ComfyUI workflow JSON file provided by the author: https://files.catbox.moe/927hoo.json. Once ComfyUI is running (usually accessible in your web browser at http://127.0.0.1:8188), you can typically drag and drop the downloaded JSON file onto the ComfyUI interface to load the workflow.
4. Adjust settings and disable SageAttention: Within the loaded ComfyUI workflow, you will need to adjust settings like your text prompt (what image you want to create) and potentially other parameters. The author specifically warns that you must disable any “SageAttention” nodes if they are present in the workflow, as using them will result in black images.
5. Run the workflow: Once configured, click the “Queue Prompt” button in ComfyUI to start generating your image.

Words you’ll see, explained

• VAE (Variational Autoencoder): A part of many AI image generators that helps convert the AI’s internal representation of an image into actual pixels you can see.
• PiD (PixelDiT): A method that generates images directly in pixel space, bypassing the VAE to achieve higher detail and better color.
• ComfyUI: A visual interface that lets you connect different AI components (like models and processors) using “nodes” to build custom image generation workflows.
• VRAM (Video Random Access Memory): Special memory on your computer’s graphics card (GPU) that is crucial for running AI models, especially for image generation.
• Hugging Face: A popular online platform where developers and researchers share and host AI models and datasets.
• Workflow (ComfyUI): A set of connected “nodes” in ComfyUI that define a sequence of operations to generate an image, saved as a JSON file.
• SageAttention: A specific component or technique mentioned by the author that, if used with this PiD workflow, causes images to turn out black.

Original source

This advanced technique was shared by Reddit user /u/Total-Resort-3120 on the r/StableDiffusion subreddit. They provided the workflow file and model links to demonstrate how to improve image quality using the PiD method.

Notes & variations

• Do you even need this?: This workflow is for users who are already comfortable with ComfyUI or are willing to learn a complex tool. If you just want to generate images without deep technical setup, online AI image generators like Krea.ai (which has a free tier) or other freemium chat-based AI tools might be a simpler starting point, though they might not offer the same level of quality control.
• Free-tier limits: Running ComfyUI with these models locally requires a powerful computer with at least 15GB of VRAM, which is not common on most personal devices. Cloud GPU services offer this power but are typically paid. Downloading the model files (which are large) will consume significant mobile data if you’re not on Wi-Fi.
• Common pitfall: Forgetting to disable “SageAttention” will result in completely black images, as warned by the author. Always double-check your ComfyUI workflow to ensure this component is not active.
• Tip for better results: Experiment with different text prompts and negative prompts within the ComfyUI workflow. Even with a powerful decoding method like PiD, the quality of your input prompt significantly influences the final image.