TouchDesigner Performance Rig
Hands, a body and a Kinect playing Ableton Live from a Mac
This is the setup I play live sets on. Echo Garden runs on top of it.
It started from a Windows tutorial project where a Kinect plays Ableton through TouchDesigner. That needs TouchDesigner’s Kinect operators, which only ship on Windows. On my Mac the Kinect arrives over Syphon instead, through my Kinect calibration work.
Hands come from MediaPipe, through torinmb’s TouchDesigner plugin. It runs in a browser and streams hand landmarks back. I mostly use pinch distance, the gap between thumb and index finger, on both hands.
Raw landmarks jitter. Each value goes through a One Euro filter, then a Lag CHOP, then a Math CHOP before it reaches Ableton. TDAbleton maps them onto 7 device racks. I have an 808 set and a dub techno set built this way.
Wekinator handles poses. You show it a few examples and it learns to tell them apart. Its classes crossfade Ableton CCs.
The hard part was ports. A pile of separate scripts fought over the same UDP ports. Now one Python process, the gateway, owns an event bus. Every input publishes to it, every output reads from it, and each port has one owner. Wekinator always sends to 12020.
Note timing skips TouchDesigner. TouchDesigner runs on a 60 fps frame clock, so it would snap every note to a 16.7 ms grid. Notes go from Python straight to Ableton over the IAC MIDI bus. TouchDesigner keeps the slow layer: macros, visuals and the Wekinator training screen.
Some hardware can only have one owner. The Kinect’s USB connection is one, and the projector is another. A start script checks first. Is the Syphon app running? Is a projector attached? Who holds the robot arm’s port? It also never projects onto the laptop screen.