ComfyUI Update: What’s new from the last few weeks, SD Turbo, Stable Zero123, Group Nodes, FP8, and more.

2023-12-19 · Source: ComfyUI blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, short

Summary

ComfyUI has released several new features and updates, including support for SDXL Turbo and SD2.1 Turbo for live prompting, with workflows available on the examples page. Frontend improvements now allow users to convert multiple nodes into a single Group Node, utilize CTRL-Z/CTRL-Y for undo/redo, and integrate Reroute nodes with Primitive Nodes. The platform also introduces Stable Zero123, an SD1.x model for generating multi-angle views of objects, primarily for 3D model creation. For memory optimization, FP8 support (e4m3fn and e5m2 formats) is available for UNet and CLIP weights, reducing memory usage at a slight performance and quality cost. Performance enhancements include Python 3.12 and PyTorch Nightly 2.3 updates for the standalone Windows package, along with new features like Self Attention Guidance for sharper images and PerpNeg for more precise negative prompt effects. Additionally, ComfyUI now supports the Segmind Vega Model and its LCM Lora, a SaveAnimatedPNG node, `--deterministic` and `--gpu-only` arguments, and GLora files.

Key takeaway

For Computer Vision Engineers developing generative AI applications, these ComfyUI updates offer critical performance and feature enhancements. You should evaluate integrating SDXL Turbo for real-time generation or Stable Zero123 for 3D asset creation workflows. If you are experiencing memory constraints, consider implementing FP8 support to reduce GPU memory footprint, carefully balancing the trade-off with image quality. Explore the new guidance methods like Self Attention Guidance and PerpNeg to refine output consistency and sharpness.

Key insights

ComfyUI updates enhance performance, memory efficiency, and creative capabilities through new models and UI features.

Principles

• Memory optimization can trade quality for resource efficiency.
• Specific model architectures enable specialized generative tasks.

Method

To optimize memory, launch ComfyUI with `--fp8_e4m3fn-text-enc` and `--fp8_e4m3fn-unet` arguments to store CLIP and UNet weights in FP8 e4m3fn format.

In practice

• Use SDXL Turbo for live prompting workflows.
• Employ Stable Zero123 to generate multi-angle object views.
• Apply Self Attention Guidance for sharper, more consistent images.

Topics

• ComfyUI
• SDXL Turbo
• Stable Zero123
• FP8 Quantization
• Diffusion Models

Code references

• comfyanonymous/ComfyUI
• AIGODLIKE/ComfyUI-BlenderAI-node
• city96/ComfyUI_ExtraModels
• florestefano1975/comfyui-portrait-master
• kijai/ComfyUI-Marigold

Best for: Computer Vision Engineer, AI Engineer, Machine Learning Engineer, Deep Learning Engineer

Related on AIssential

• ComfyUI Update: Stable Video Diffusion on 8GB vram with 25 frames and more. — ComfyUI's latest update enhances video generation, sampling efficiency, and high-resolution image consistency.
• The next chapter for ComfyUI — ComfyUI's creator launched Comfy Org to advance open-source AI inference, focusing on usability and performance.
• INSANE NEW FREE NSFW IMAGE AI KING IS HERE! LESS THAN 8GB VRAM! — Z Image Turbo is a 6B parameter, uncensored text-to-image model offering high quality and speed on low VRAM.
• NVIDIA and ComfyUI Streamline Local AI Video Generation for Game Developers and Creators at GDC — NVIDIA's GDC updates enhance local AI video generation with simplified interfaces and significant performance gains on RTX GPUs.
• Getting Started with ComfyUI on AMD Radeon™ RX 9000 Series GPUs — ComfyUI is now stable and performant on AMD Radeon RX 9000 series GPUs with ROCm 7.1.
• TextureFlow part II: full ComfyUI walkthrough - powerful AI animation tool — TextureFlow is a ComfyUI workflow for abstract animations, integrating IP adapters, ControlNets, and AnimeDiff for detailed control.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by ComfyUI blog.