ComfyUI_Yvann-Nodes

ComfyUI_Yvann-Nodes

Audio Reactivity Nodes for ComfyUI πŸ”Š Create AI generated audio-driven animations. Compatible with IPAdapter, ControlNets, AnimateDiff...

Stars: 340

Visit
 screenshot

ComfyUI_Yvann-Nodes is a pack of custom nodes that enable audio reactivity within ComfyUI, allowing users to create AI-driven animations that sync with music. Users can generate audio reactive AI videos, control AI generation styles, content, and composition with any audio input. The tool is simple to use by dropping workflows in ComfyUI and specifying audio and visual inputs. It is flexible and works with existing ComfyUI AI tech and nodes like IPAdapter, AnimateDiff, and ControlNet. Users can pick workflows for Images β†’ Video or Video β†’ Video, download the corresponding .json file, drop it into ComfyUI, install missing custom nodes, set inputs, and generate audio-reactive animations.

README:

πŸ”Š ComfyUI_Yvann-Nodes YouTube Tutorials

Made with the help of Lilien unnamed

A pack of custom nodes that enable audio reactivity within ComfyUI, allowing you to generate AI-driven animations that sync with music


What Does This Do?

  • Create Audio Reactive AI videos, enable controls over AI generations styles, content and composition with any audio
  • Simple: Just Drop one of our Workflows in ComfyUI and specify your audio and visuals input
  • Flexible: Works with existing ComfyUI AI tech and nodes (eg: IPAdapter, AnimateDiff, ControlNet, etc.)

Quick Setup

Pick a Workflow (Images β†’ Video or Video β†’ Video)

  1. Images β†’ Video
  2. Video β†’ Video

Load Your Chosen Workflow in ComfyUI

  1. Download the .json file for the workflow you picked:

  2. Drop the .json file into the ComfyUI window.

  3. Open the β€œπŸ§© Manager” β†’ β€œInstall Missing Custom Nodes”

    • Install each pack of nodes that appears.
    • Restart ComfyUI if prompted.
  4. Set Your Inputs & Generate

    • Provide the inputs needed (everything explained here
    • Click Queue button to produce your audio-reactive animation!

That’s it! Have fun playing with the differents settings now !! (if you have any questions or problems, check my Youtube Tutorials


Advanced/Optional Node Details

Click to Expand: Node-by-Node Reference

Audio Analysis πŸ”

Analyzes audio to generate reactive weights for each frame.

Node Parameters
  • audio_sep_model: Model from "Load Audio Separation Model"
  • audio: Input audio file
  • batch_size: Frames to associate with audio weights
  • fps: Frame rate for the analysis

Parameters:

  • analysis_mode: e.g., Drums Only, Vocals, Full Audio
  • threshold: Minimum weight pass-through
  • multiply: Amplification factor

Outputs:

  • graph_audio (image preview),
  • processed_audio, original_audio,
  • audio_weights (list of values).

Load Audio Separation Model 🎧

Loads or downloads an audio separation model (e.g., HybridDemucs, OpenUnmix).

Node Parameters
  • model: Choose between HybridDemucs / OpenUnmix.
  • Outputs: audio_sep_model (connect to Audio Analysis or Remixer).

Audio Peaks Detection πŸ“ˆ

Identifies peaks in the audio weights to trigger transitions or events.

Node Parameters
  • peaks_threshold: Sensitivity.
  • min_peaks_distance: Minimum gap in frames between peaks.
  • Outputs: Binary peak list, alternate list, peak indices/count, graph.

Audio IP Adapter Transitions πŸ”„

Manages transitions between images based on peaks. Great for stable or style transitions.

Node Parameters
  • images: Batch of images.
  • peaks_weights: From β€œAudio Peaks Detection”.
  • blend_mode, transitions_length, min_IPA_weight, etc.

Audio Prompt Schedule πŸ“

Links text prompts to peak indices.

Node Parameters
  • peaks_index: Indices from peaks detection.
  • prompts: multiline string.
  • Outputs: mapped schedule string.

Audio Remixer πŸŽ›οΈ

Adjusts volume levels (drums, vocals, bass, others) in a track.

Node Parameters
  • drums_volume, vocals_volume, bass_volume, others_volume
  • Outputs: single merged audio track.

Repeat Image To Count πŸ”

Repeats a set of images N times.

Node Parameters
  • mask: Mask input.
  • Outputs: Repeated images.

Invert Floats πŸ”„

Flips sign of float values.

Node Parameters
  • floats: list of floats.
  • Outputs: inverted list.

Floats Visualizer πŸ“ˆ

Plots float values as a graph.

Node Parameters
  • floats (and optional second/third).
  • Outputs: visual graph image.

Mask To Float 🎭

Converts a mask into a single float value.

Node Parameters
  • mask: input.
  • Outputs: float.

Floats To Weights Strategy πŸ‹οΈ

Transforms float lists into an IPAdapter β€œweight strategy.”

Node Parameters
  • floats: list of floats.
  • Outputs: dictionary with strategy info.

6. Thank You!

Please give a ⭐ on GitHub it helps us enhance our Tool and it's Free for you !! (:

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for ComfyUI_Yvann-Nodes

Similar Open Source Tools

For similar tasks

For similar jobs