ComfyUI-BlenderAI-node

ComfyUI-BlenderAI-node

Used for AI model generation, next-generation Blender rendering engine, texture enhancement&generation (based on ComfyUI)

Stars: 561

Visit
 screenshot

ComfyUI-BlenderAI-node is an addon for Blender that allows users to convert ComfyUI nodes into Blender nodes seamlessly. It offers features such as converting nodes, editing launch arguments, drawing masks with Grease pencil, and more. Users can queue batch processing, use node tree presets, and model preview images. The addon enables users to input or replace 3D models in Blender and output controlnet images using composite. It provides a workflow showcase with presets for camera input, AI-generated mesh import, composite depth channel, character bone editing, and more.

README:

Introduction

This is an addon for using ComfyUI in Blender. It will convert ComfyUI nodes into Blender nodes, letting you use ComfyUI inside Blender without having to switch between programs.

Features

  • Converts ComfyUI nodes to Blender nodes
  • Edit launch arguments in addon preferences, or just connect to a running ComfyUI process
  • Add some special Blender nodes like camera input or compositing data
  • Draw masks with Grease pencil
  • Blender-like node groups
  • Queue batch processing with mission excel
  • Node tree, workflow presets and node groups
  • Model previews in Loader node
  • Directly import or replace 3D models
  • Composition output perfect controlnet image
  • Live preview when sampling

Here are some workflow showcases: You can find all these workflow presets in ComfyUI-BlenderAI-node/presets/

Camera input

image

Camera input with roop

https://github.com/AIGODLIKE/ComfyUI-BlenderAI-node/assets/116185401/cb96dd60-b93a-4f09-9ab6-043b66617313

Import or replace AI generated meshes in the 3D Viewport

image

Composite depth channel

image

Pose characters using Blender's bones

image

Automatic AI gap animation(ToonCrafter)

https://github.com/AIGODLIKE/ComfyUI-BlenderAI-node/assets/116185401/2c21173b-84e7-433c-bf2f-e61bcca7162c

Installation

WINDOWS 10\11

  1. Install Blender

First, you need to install Blender(Recommend Blender 3.5, 3.6.X, or previous 4.0).

image

  1. Install this add-on(ComfyUI BlenderAI node)
  • Install from Blender's preferences menu

In Blender's preferences menu, under addons, you can install an addon by selecting the addon's zip file. Blender will automatically show you the addon after it's installed; if you missed it, it's in the Node category, search for "ComfyUI". Don't forget to enable the addon by clicking on the tickbox to the left of the addon's name!

Note: The zip file might not have a preview image. This is normal.

Pasted image 20240319183259

  • Install manually (recommended)

This is a standard Blender add-on. You can git clone the addon to Blender's addon directory:

cd %USERPROFILE%\AppData\Roaming\Blender Foundation\blender\%version%\scripts\addons
git clone https://github.com/AIGODLIKE/ComfyUI-BlenderAI-node.git --recursive

Then you can see the addon after refreshing the addons menu or restarting Blender. It is in the Node category, search for "ComfyUI". Don't forget to enable the addon by clicking on the tickbox to the left of the addon's name!

Linux

If you're using Linux, assuming you have some experience:

  1. Install Blender
  2. Create and activate a Python venv
  3. Install ComfyUI
  4. cd /home/**YOU**/.config/blender/**BLENDER.VERSION**/scripts/addons
  5. git clone https://github.com/AIGODLIKE/ComfyUI-BlenderAI-node.git --recursive
  6. Set your ComfyUI path and your venv /bin/ path in the addon's preferences

Some things will not work on Linux, or might break!

Usage

  1. Prepare ComfyUI

You can download ComfyUI from here: ComfyUI Releases

Or you can build one yourself as long as you follow this path structure:

├── ComfyUI
│   ├── main.py
│   ...
├── python_embeded
│   ├── python.exe
│   ...
  1. Set the "ComfyUI Path" to your ComfyUI directory

image

  1. Set the "Python Path" if you're not using the standard ComfyUI file directory

The default (empty) path is:

├── ComfyUI
├── python_embeded
│   ├── python.exe  <-- Here

If you're using a virtual environment named venv, the executable is in venv/Scripts/python.exe.

  1. Open the ComfyUI Node Editor

Switch to the ComfyUI Node Editor, press N to open the sidebar/n-menu, and click the Launch/Connect to ComfyUI button to launch ComfyUI or connect to it. Or, switch the "Server Type" in the addon's preferences to remote server so that you can link your Blender to a running ComfyUI process. Pasted image 20240319185542

  1. Add nodes/presets

Like in the other Node Editors, you can use the shortcut Shift+A to bring up the Add menu to add nodes. You can also click on the "Replace Node Tree" or "Append Node Tree" buttons in the sidebar to add/append a node tree.

For image previews and input, you must use the Blender-specific nodes this addon adds, otherwise the results may not be displayed properly! Using the Blender-specific nodes won't affect generation, results will still be saved as ComfyUI standard data.

image

  1. Generate

By clicking Excute Node Tree in n-panel, or the little red button on the right in header in node editor, current node tree will add to queue list. You can cancel current running task by click Cancel, clear all task list by click ClearTask.

Loop execution is in advanced executing option at the side of Excute Node Tree button.

image

Special Nodes for Blender

Input Image

image

  • Input image from directory
  • Input image list from directory
  • Input image from render(Supports current and selected frames)
  • Input image from viewport(Supports real-time refresh)

Mask

image

  • Create a mask from Grease Pencil
  • Create a mask by projecting an object on the camera
  • Create a mask by projecting a collection on the camera

Mat Image

image

  • Input texture from object
  • Input textures from collection objects

Save Image

image

  • Normally saves to a folder
  • Can save to an image in Blender to replace it

Multiline Textbox

image

To improve writing long prompts, we made a button that can show all prompts in a separate textbox since Blender doesn't support multiline textboxes in nodes. When you click the button on the side of the textbox, a window will open to write prompts in. The first time you do this, you might need to wait. Keep your cursor over the window while typing.

Shortcut Keys

Link

Select a node, then hold D and drag the cursor to another node's center, you can link all available widgets between them

image

Search Widgets

By pressing R when the cursor is near a widget, a pie menu will display all nodes that have this widget

image

Mask Link

Hold F and drag the cursor to a mask node, it will automatically create a camera to genarate mask from the scene

image

Notes

  • Not every node can work perfectly in Blender, for example nodes regarding videos
  • You can enable the console under Window>Toggle System Console at the top left
  • Model preview images need to have the same name as the model, including the extension, for example - model.ckpt.jpg

Tested Nodes

Here are some interesting nodes we've tested in Blender

√ = works as in ComfyUI web

? = not all functions work

× = only few or no functions work

Custom Node Name Status
3D-Pack
Advanced Encode
Advanced ControlNet
AGL-ComfyUI-Translation
AlekPet Nodes
AnimateAnyone
AnimateDiff ?
AnimateDiff-Evolved
BiRefNet
CLIP Seg
ComfyRoll
ControlNet LLLite
ControlNet Preprocessors
ControlNet Preprocessors AUX
Crystools
Cutoff
Custom-Scripts ×
cg-use-everywhere ?
cg-image-picker
Davemane42 Nodes ×
Dagthomas Nodes
Dynamic Thresholding
Easy Tools
Easy Use
Efficiency Nodes
EllangoK Postprocessing
Essentials
ExLlama nodes
experiments
Fast Decode
FlowtyTripoSR
FreeU Advanced
IPAdapter
IPAdapter_plus
Image Grid
Impact Pack ?
Impact Subpack
Inspire Pack
InstantID (cubiq)
InstantID (ZHO)
KJ Nodes
LaMa Preprocessor
Latent2RGB
LayerDiffuse
LayerStyle
LCM
Manager ×
Masquerade Nodes ×
Math
Mixlab Nodes ?
MoonDream
MotionCtrl
MotionCtrl-SVD
Noise
Portrait Master
Power Noise Suite
Prompt Reader
QR
OneButtonPrompt
ReActor
Restart-Sampling
Roop
rgthree
SD-Latent-Interposer
SDXL_prompt_styler
SeargeSDXL
Segment Anything ?
StabilityNodes
TiledDiffusion
TiledKSampler
TinyTerra
UltimateSDUpscale ?
Vextra Nodes
VLM Nodes
WAS Suite ?
WD14-Tagger
zfkun

Change Log

See Change Log.

Links

Tutorial

[EN]BSLIVE ComfyUI Blender AI Node Addon for Generative AI(By Jimmy Gunawan)

[EN]Generate AI Rendering with Blender ComfyUI AddOn(By Gioxyer)

[CN]无限圣杯完全使用指南

(Please feel free to contact me for recommendations)

Our AI website

AIGODLIKE Community

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for ComfyUI-BlenderAI-node

Similar Open Source Tools

For similar tasks

For similar jobs