Multi audio channel visuals

[Spudmagic]

Video output canvas size: 3080 x 480, 5 stitched projectors through Resolume Arena, tons of space =D

I am designing an AV system that will incorporate multiple audio channels from the band/performers(at least 5). The goal is to apply each sound channel to a video layer, each musician gets their individualized sound reactive video backdrop. I have approaches for hardware and software combinations already mapped out. My endeavors are to know, has anyone ever done multiple audio signals to a multi-input sound card/MIDIbox etc., analyzed by FLStudio, AbletonLive, or QC and those audio channels are then Rewired as audio reactive signals to GVJ?
that setup is then bussed as 1 video signal via BlackMagic intensity shuttle to the mainbrainDisplay where Arena is controlling the main 3080x480 seamless projection wall.

right now I am deciding the better of my two approaches, based on my thoughts and concerns about system bottlenecking, how smooth the VJ experience is, and exactly how much audio analyzation (5 signals to 5 layers) can the fastest consumer computers handle with the lowest of latency to the audio reactive paramaters. the above method requires less hardware, and time consumed programming it all, than the one i will describe next

my second method would be applying each Audio channel to its own video source computer, multiple VJ interfaces analyzing only 1 or 2 channels of audio, which will hub into the mainPC running Arena, i feel this method will flow smoother because every AV reactive signal layer from each video source computer will be accessible as individual video layers/sources on resolume, which has a greater customizability aspect because specific layers need to be repositioned to where each performer is standing. this method will greatly lessen the feared AV analyzing sync bottleneck, (If there is any) which would enable focus to more video layers and effects which would be applied to each individual signal. the fallbacks of this requires a bit more energy to VJ, programming and acquiring more systems, but it can become much more explosive and intricate due to more computer interfaces incorporating their full potential.

any suggestions, advice, from anyone whose done anything similar please give a shout.

-Spudmagic

[mowgli]

I'm not a gvj user but I have a feeling that you can't input more than 1 sound source to it so the second option will be the only way.

[subpixel]

If you can display all the video layers you want from one computer, that may be the easiest thing to do.

I have an ASUS N61J laptop (2+ years old) running Windows 7. I made my own audio to MIDI application using Processing (you can download the source and change it as you like -- spxlAudioToMidi) and discovered that I can run two copies, each "listening" to a different audio source - the "stereo mix" and the microphone input.

To do this I change the system Sound > Recording setting after starting the first copy of the app (I haven't looked into how to set up the Minim library to choose the input audio device programmatically).

I imagine that if I were to plug in a USB audio device that I'd be able to listen to that separately (at the same time) too.

Another experiment I've done is running my app on another computer (which, of course, has its own microphone input / audio in) and sending the MIDI data over the network (WiFi). There's a program called rtpMIDI which uses Apple Bonjour to be able to talk to Macs (OSX comes with the built-in ability to send MIDI over the network) so I actually had my app running on a MacbookPro, sending MIDI over WiFi to my main Windows laptop, which was running visuals that are driven by the incoming MIDI data.

I also use MIDI-Yoke to route virtual MIDI ports (and allow multiconnection to/from programs on one port)

I made the app so you can choose what MIDI channel the messages are sent on, so you could use that along with some MIDI remapping (using Midi-Ox), but as you have the source you could change it to output different ranges of midi commands instead; if your visuals program can use multiple channels (mine doesn't), that's probably easiest.

The latency of the MIDI traffic over the network (even WiFi) could/should be pretty low; you'll have to try it for yourself to see if it is good enough, and also reliable (WiFi can be flaky!) - running ethernet using a hub wouldn't be a terrible idea.

-spxl