Real-time FLUX.2-Klein model running locally on a single RTX5090 GPU. Conditioned with prompt and RGB camera input. Custom KV cache bank implementation for temporal consistency + a ton of optimizations 😵💫😈
•
#flux2 #realtimerendering #nvidia #touchdesigner #blackforestlabs
Real-time diffusion with FLUX.2-Klein4B model + streamV2V in Touchdesigner 😳
Very impressed with the results!
Running smoothly on Nvidia RTX 5090 GPU locally.
Swipe to view the input ⏩️
•
#flux2 #torch #nvidia #blackforestgermany #touchdesigner
Times like this remind you how much work depends on the people around you.
What begins as a structured set can slowly open into a long improvisation. And every now and then the stage and the visuals briefly meet in the same moment — something you could never fully plan.
Nomad Path Live Performance by @firouzfarmanfarmaian and @interactive.items
Made with @touchdesigner and #streamdiffusion by @dotsimulate
#generativeart #audiovisual #realtime #newmedia
Why does sharing GPU memory between two processes still require copying data through the CPU? At 60fps, that round-trip costs milliseconds you can’t afford. I built CUDA-Link to fix that — an open-source Python library that shares GPU textures and tensors directly between processes. Zero-copy, no CPU detour. Started because I needed live TouchDesigner frames in AI models without the latency hit. The GPU → CPU → GPU path was eating my frame budget, so I wrapped CUDA’s IPC API in pure Python and cut it out entirely. What it does: → Zero-copy GPU sharing between any two processes → Ring buffer — producer and consumer never block each other → PyTorch, CuPy, NumPy output modes → Pure Python — pip install and go
60fps in production. Easy.
Open-source: github.com/forkni/cuda-link (link in bio)
•
#touchdesigner #python #cuda #realtimeai
Live depth feed from Azure Kinect camera → ControlNet + StreamV2V (temporal consistency) → diffused output at 30 FPS with SDXL-Turbo at 2 Timesteps on a single RTX 5090.
Most StreamDiffusion implementations use TensorRT (incompatible with StreamV2V) — torch.compile gives both speed and consistency. Currently working on further pipeline acceleration 🚀🚀🚀
#streamdiffusion #torch #realtimerendering #generativeart #ai
60FPS real-time video generation with StreamDiffusion from Orbbec camera. •
- ControlNet depth for figure shape coherence •
- StreamV2V with smart caching & Feature injection for temporal stability •
•
Everything running real-time in Touchdesigner on RTX5090 GPU •
•
Big shoutout to @ekarasyk for the GPU and trust 🖤
#streamdiffusion #nvidia #torch #realtimerendering #touchdesigner
So, this is it. •
After months of tries and errors, I could manage to adapt StreamV2V code to be compilable!
What you see here - is real-time render with StreamDiffusion + StreamV2V + Depth ControlNet.
3 diffusion steps; 30FPS; SDTurbo diffusion model.
Produced and recorded in real time-time in TouchDesigner.
•
#streamdiffusion #torch #nvidia #realtimerendering #touchdesigner
OK, now we talking…
Real-time video generation with StreamDiffusion from 60FPS input! .
Left part of the video - unprocessed input,
Right part of the video - generated output.
Just made it work, so it’s ruff sketch, but looks very promising for me.
Also it’s running on single NVIDIA 4090 GPU, but with CUDA 12.8, so it will work on 50xx GPU’s as well!
Hell yeah! 🔘
••
#streamdiffusion #realtimerendering #nvidia #pytorch #touchdesigner
