Alternative midi controller/instrument development

[elbows]

Last try for a while as there dont seem to be very many hardware developers here.

Is anyone interested in developing new input methods or instruments that would be of interest to VJs and musicians? This is something I have been thinking of in my brain for several years and now things are finally sorted to proceed with development.

I am starting with the I-CubeX midi interface to save time with that side of things, and will be starting off with some fairly basic additonal sensors and eventually combining many of these into several different areas:

1) A device for VJs to use similar to a midi controler but with a wider range of sensors and control methods

2) Alternative musical instruments

3) Devices for crowd participation at clubs etc.

Preliminary demonstration of the underlying technology should be happening at AVIT, then I expect some custom prototype sensors to follow before end of 2002. Hoping to explore commercial posibilities by early 2003 though this depends on numerous business and financial factors.

So does anyone have a passion for this stuff that would like to get involved?

[MoRpH]

As I said in the other thread, there are plent of COOL dedicated hardware controllers out there @ the momment, its a matter of finding a way to hook them into VJ softwares... I think the DirectInput/HID setup is the easiest way to go.

[robotfunk]

yes i love doing stuff like this. unfortunately i dont have much time on my hands but i'd like to help here.
i've been toying with the idea of making a modular i/o midi based system based on the excellent (open) midibox project www.ucapps.de

[elbows]

Out of interest have you seen any of them in the flesh? Its just im a bit cynical that so many of them are either impossible to actually find or else the picture looks rendered and doesnt reflect true build quality.

Anyway yeah I agree theres no point re-inventing the wheel, but I was thinking of new and unique input methods that feel high quality in terms of their movement/damping etc. eg the midi hardware I have has sliders which dont feel solid enough to me.

[elbows]

Originally posted by robotfunk
yes i love doing stuff like this. unfortunately i dont have much time on my hands but i'd like to help here.
i've been toying with the idea of making a modular i/o midi based system based on the excellent (open) midibox project www.ucapps.de

Snap :) This is great news, I was also planning on starting with that project and using it as the basis for alternative sensors. But I ran out of time to build one before AVIT so hence my mad decision to buy an I-CubeX to skip a stage and speed up prototyping efforts.

Anyway I'll keep you up-to-date with any progress, sure know what you mean about lack of time, where does it all go? :D

[MoRpH]

Originally posted by elbows
Out of interest have you seen any of them in the flesh? Its just im a bit cynical that so many of them are either impossible to actually find or else the picture looks rendered and doesnt reflect true build quality.

Well I have a midi keyboard, toaster and a commandpost and they are all pretty solid, just wish some VJing app would support the commandpost I mean its a pretty wikkid video controller (perfect actually). Also I'm definately getting a mixman controller and will report on it as soon as I play with it abit, it would also make a wikkid video controller for the more Skratch VJ ppl out there :)

[robotfunk]

elbows:
Out of interest have you seen any of them in the flesh? Its just im a bit cynical that so many of them are either impossible to actually find or else the picture looks rendered and doesnt reflect true build quality.
This is exactly why I'm suggesting we use the midibox platform as a starting point. It is an open, modular system that you can fill in as you want with knobs/(motor)faders of any quality/size you'd like. You can make a rugged concrete or flimsy plastic one if you'd like, and the choice is yours how you make yours. Any pot input can be replaced with a sensor of any kind and I'm sure we can get Thorsten interested in adding outputs, and if not we could use some readily available MIDI/CV converters for this. I think noone here including you and me has the knowledge to program the microcontrollers and design the circuitry for a project like this (if there IS someone here who can plz speak up!),while Thorsten has proved a bit of a guru in this field as well as being totally dedicated and helpful to implement any requests for enhancements to his system.

It will be quite a bit of work to get this working but probably a LOT cheaper than the I-cube and possibly a lot more flexible.

MoRpH:
Well I have a midi keyboard, toaster and a commandpost and they are all pretty solid, just wish some VJing app would support the commandpost I mean its a pretty wikkid video controller (perfect actually). Also I'm definately getting a mixman controller and will report on it as soon as I play with it abit, it would also make a wikkid video controller for the more Skratch VJ ppl out there :)
I am definately interested in using DirectInput based controllers for VJ software and making interfacing software for them, especially the scratch thingy. I think what elbows proposed (at least what I picked up from it) goes a bit further than this, more geared towards output as well as input, interfacing with sensors, motors, for doing mad installations and such. DirectInput devices would be a great way to control software which (besides making visuals) controls such a system. My idea of this is that it should become like a LEGO system for interactive installations.

[robotfunk]

I think these two modules made by doepfer would make great blocks in this system:

http://www.doepfer.de/ctm.htm
can convert 64 switches and 4 pots to MIDI, making great inputs for sensors.

http://www.doepfer.de/mtc.htm
is the opposite, takes MIDI in and can control up to 64 relays, lamps, motors, electromagnets, magnetic valves, or what have you.

The price is right as well.

[elbows]

Excellent find robotfunk :) Those doepfer devices are a great way to jump ahead to the sensor development stage and not have to worry about getting bogged down in interface development for now.

Yeah Ive already studied the midibox stuff in quite some detail, does seem like a great DIY platform, I wonder if Thorsten would share the source code with people looking to develop it further?

I confess I am more interested in the input side rather than the output, but probably only because I had not thought about physical output devices much before, mechanics and robotics etc isnt really my area of knowledge at all, but I will think about that more now :)

Funnily enough I do actually know someone who definately has the skills to program microcontrolers etc for our needs if we started from scratch. He's done lots of PIC development work and building PCI cards that interface to hardware in arcade machies of the pub quiz variety. I think he's doing something with I2C busses at the moment, and he was designing a midi controller with another friend years ago before cheap ones were available, dont think they finished it though. Problem is he gets paid quite a lot by his current company but hates the work, he is really interested by these ideas but I dont know how much of his time I can get unless there is a plan to earn reasonable money from it.

Anyway as a first step I am going ahead with I-CubeX purchase for total evaluation and to get on with sensor related fun, it is expensive and I dont really have the money, but I am silly so I'll do it anyway :D

[ToddGraft]

Have a look at this it might help.
www.showmagic.com
This is an integral software package, cueing events with RS232
and midi.
you can use this in conjunction with a external DMX lighting desk
for lighting, sound and video cues.

[spark]

also don't forget lace's work, he's totally on it from a realistic DIY perspective, and he's got it working from robot input to dancing robot output.

www.vutag.com
www.vutag.com/robot/index.htm

for my tuppence, it has to be midi - directinput is a shortcut that will be too limiting. i'm looking for a lego-like midi-input system that i can build and evolve as a dedicated interface for vdmx. my hopes for the surface-one are fading since i've seen the dimensions and some higher quality pics (it looks really ropey close up).

toby

[katascope]

Very cool, the midibox system looks incredible. What would it take to use pressure-sensitive pads like these?

http://www.tactex.com/products.htm

[loboy]

This project may soon leave the drawing board after months of collecting dust and cat hair under my bed. I finally have the right parts I need and hopefully next week I can begin to build a mock-up version 2 a of turntable capable of jog shuttling through video clips. The idea is to be able to run clips and have comfortable control through the forward and reverse of the clips, manipulation happening on the turntable, as a scratch.

My first version didn't work that well because of the turntable I was using, but fornately my roommate just got some spanky new Techniques and he gave me his old busted one. I had opened up the table and inserted one of those USB laser mouses. I freed the laser from the casing of the mouse and installed it directly under the hood of the spinning table, so any motion of the table was controlling the mouse through the USB port.

My original original version was using a trackball mouse, but times have changed and this system works ten times more accurately and friction free. I still would love to pickup a USB jog shuttle and mess around with that, but the installation of the jog into the turntable may have disastrous results with wear and tear of use.

As for the software controlling the video clips, that is another project unto itself. I have been using this crappy hack of Quicktime (I'm on a Mac) aptly named QTurntable. It's a half-ass way for me to concentrate on developing the actually unit rather than the software.

The kind of software I really am going for is something that can database a lot of samples and crunch and analyse each sample's frames while taking information from the USB port. The problem here is I don't think USB is doable at such extreme rates of data transfer (am I wrong??), firewire is probably the correct solution. The video, after manipulation, would then be pushed out through S-video to whatever is connected.

Any suggestions on similar hardware and or software to use? I have seen the Vinyl Video system and that other one, I forget, but it plays mp3s off of turntables. That system maybe adaptable?? I may begin learning some more programming if need be.

[robotfunk]

Originally posted by katascope
Very cool, the midibox system looks incredible. What would it take to use pressure-sensitive pads like these?

http://www.tactex.com/products.htm

big moolah
they have a ready made midi controller for controlling MAX/MSP
and m-audio made a midi controller with the stuff

when i emailed tactex about getting some samples to test with they ask me how many thousand I needed. When I told them i just needed one to test with they told me to get the ready made MAX controller . sheesh.

on a hardware level you'd need 3 analog inputs (sacrifice 3 knobs/faders on a midibox for example) to get the x y and z info of the tactex

[elbows]

Yeah unfortunately they are geared towards supplying large OEMs so Im not surprised they didnt want to know :(

loboy that sounds like a very interesting project - think I might butcher a mouse or 2 tonight and have a play around :D I cant say I understand 100% what you want your software to do, and Im not sure why USB bandwidth would be an issue, you are only using USB to get the mouse data yes? Perhaps if you could explain again but just state the goal in terms of output and control.

Im afraid Im a PC user so my Mac knowledge is limited, but on a PC if I wanted to be able to scratch video back and forth using your hardware, the I could just use something like visualJockey which accepts a mouse as an input device.

[loboy]

Originally posted by elbows
loboy that sounds like a very interesting project - think I might butcher a mouse or 2 tonight and have a play around :D I cant say I understand 100% what you want your software to do, and Im not sure why USB bandwidth would be an issue, you are only using USB to get the mouse data yes? Perhaps if you could explain again but just state the goal in terms of output and control.

objective: vinyl record motion (forward, reverse, still) would control video sample.

ex: the turntable is spinning the record normally controlling the video sample playing forwards. I pull the record back on the table and and the video sample gets pulled back as well (reversing it in realtime). I scratch the record back and forth and the video sample gets pulled back and forth. Just as a jog shuttle works when editting video linearly.

Since the unit would be USB it would feed the data of this motion to the software which would in turn would process it and output the manipulated video.

Ideally I would love to alter an entire movie in this fashion. Instead of remixing two records, you could remix two movies. Oh the possibilities! Spinning DVDS!

The thing is the amount of frames the CPU is going to have to chug through. Don't most visual sampling applications digest all the frames of a given sample? Have to look into that.

The whole database thing is just a library of samples easily accessible through an interface on the turntable rather than on a computer.

Again, ideally the unit would become self-contained. Meaning that the computer is located inside the turntable and you wouldn't have to mess with a seperate console. The unit would interpret the motion, manipulate the pre-selected sample, and output through S-video. There would be an internal hard drive for storing only the OS, software, and video samples. The unit would also have a firewire port for importing more samples from either another unit, computer, or drive, and a video in port for live manipulation and or recording of new samples. Everything self-contained in one unit a little bigger than a turntable.

This idea has stretched in mind over the past couple of years and the more and more I think of it, the unit shouldn't just port its video, it should straight up project it. So it's a hybrid video manipulating turntable/projector capable of importing,exporting, and recording new samples.

whoa. our kids are gonna be crazy. divulging. man, when will we have holograms?

[elbows]

Sounds good to me :)

I'l have a play with visualJockey tonight - its mouse control will do nearly exactly what you need, only tricky issue is calibrating things so that normal turntable speed = plays clip at normal framerate.

What does the QTurntable software do at the moment?

USB wont be a problem for what you want, as long as the mouse itself samples at a high enough frequency which Im sure it does.

Returnign to visualJockey again, it has the ability to attach the mouse to a huge variety of clip parameters, effects etc so you could use the turntables for all sorts of control. There is a live video input and also some buffering/delay effects so you could capture live video samples into memory and manipulate them with the turntable. Theres loads of other software which does similar things using midi, and it is possible to get software for the PC that converts mouse->midi so that opens up lots of other software possibilities.

PC motherboards with firewire etc onboard are now available in quite small sizes so that wont be too much of a problem. Whether the CPU can handle high res video in a fast enough way is an issue, certainly it can be done now by making careful hardware & codec choices, but I would expect a little teething trouble in this area.

I guess long term you would ideally need to write custom software for this to provide easy clip selection etc, or at least write a custom front end for existing software.

I wouldnt build the projector into the device though, I think thats going one step too far, would add massive cost to the system and would create inflexibilities in how people use the device and where they are located when performing.

[loboy]

Originally posted by elbows
What does the QTurntable software do at the moment?

Not much at all, just provides a platform for me to test my mock-up unit in, mouse control and frame rates. Very chincy and bare bones.

Definitely have to check out visualJockey! I know VDMX has a feature like this, but it lags in the control parameters I am looking for and its overly feature rich.

I guess long term you would ideally need to write custom software for this to provide easy clip selection etc, or at least write a custom front end for existing software.

whoa. That's what I thought. time, time, tickin in my head.

I wouldnt build the projector into the device though, I think thats going one step too far, would add massive cost to the system and would create inflexibilities in how people use the device and where they are located when performing.

Definitely noted. The projector thing is sort of fantastic. The thing is, I like to think of the the projection emanating from the performer. It becomes more theatrical in presentation. You could have battle or collaborative performances on a large translucent center panel, each VJ on either side of the panel, or an octagonal setup with performers on the inside projecting out or vice versa. The battle/collaborative projections would overlay each other due to the translucency of the panel.

in my dreams...

[MoRpH]

loboy check out this controller http://www.vjforums.com/showthread.php?s=&threadid=621 basically what you after full forward/backward/scratch plus 16 triggers, Xfader, joystick for FX and misc other buttons..... the hardware is out there, now we just need the VJ software to use it :)

[loboy]

Morph,

right on. how come I haven't seen this thing?

seems a little small, and too bad no mac version.

yes, yes, yes! where are the vj apps for this thing?

the other thing is you can't even mix your mp3s, you have to convert them over to .wav format! I have a huge library of . wav files but they are saved on 5 inch floppies! I guess the wav format is easier for the computer to handle when mixing and "scratching" on this thing.

Who would use the generic crap that comes with the software anyway? I don't want to sit in my bedroom and mix cheesy samples of a handclap and corny sythesizer riffs.

The controller is there. The marketing dept. at Mixman needs to update its target audience and more money pushed into R&D for video. I would buy this thing, too bad no Mac version. If I could get all the specs and info on the software I might consider.

I have used older Mixman software on my roommates crap ass IBM clone and what I have learned with such software is that it generally stinks. That was a couple years ago though. The facade of cool graphics only goes so far, I like software that can crunch and get the job done easily and thoroughly, without that edge of cool.

Still seems like a toy.

Mixman?? Are you listening??

Thanks Morph for the link.

[spark]

// oops bit of a cross post here, this is mostly talking about the mixman toy on the other hardware thread... sorry folks

hmm. looks like a toy, but toys are cheap and easy to replace when trashed live...

also the smaller the better, imho - the midiman surface one is massively bigger than my lappy, and that takes the piss.

ok obviously we don't care two hoots about the software, but its USB so it is theoretically PC and Mac compatible, and given the cheap nature of the unit its going to have simple as possible internals... maybe a standard controller chip with a few buttons/pots wired in to it... so the point being it shouldn't be too hard for someone to reverse engineer the driver - ha ha ok not 5 minutes work i realise.

but yeah. i want midi-lego!

toby

[MoRpH]

Yeah you could hammer the mixman unit live and just replace it :)

Yeah if its USB you could use it with the mac you just need your software to support it.

As for hacking the driver, there is no need if software programmers using directX would use the directInput setup with the mixman unit being a HID controller, the control data is already going into directX :)

Now if you wanted it to generate midi data to connect THAT to an application via say midiyoke, we gotta get someone to do a directInput to midi app :)

[loboy]

I looked into DirectX, yet I am on a Mac, so I will begin the inevitable journey into the abyss of Quicktime. Is this a bad thing, if so, someone stop me with three good reasons.

I have started to dig into the Quicktime developer's resources. hmm, it will be long and treacherous but I'm in it to win it. Hopefully this is the right road, got some books on USB and MIDI and the like.

Sorry, this is turning into a software post.

I may go ahead and purchase the DM2. Still wavering. I want to crack that thing open and smell sweet soldering smoke.

I was re-reading the posts and I am thinking of returning to the MIDI route, this is probably best knowing that most of the VJ apps out there can interpret MIDI data. Am I correct?

I am sort of newbie to all this.

[spark]

if you're going to do it, here's my tuppence:

go the midi route: it'll give the device the biggest market.

pure 2 channel videoscratching / crossfading would bore me senseless and not really deliver that interesting a mix... its all in the combination of effects, mixing modes, clip triggers etc

toby

[MoRpH]

Hmmm just like midi learn, I was hoping for a HID/DirectInput learn function..... now that would be nice and then you could assign the controller to whatevery you want, I agree that DirectInput to midi would be good for apps that already have midi, however there are apps out there than have not yet or do not what to go down that path, I think using directInput with a device like this would be an acceptable alternative.

AS for mac/quicktime I have no idea and it isn't really something I'm ready to look in to @ the momment considering a) I don't have a mac b) most software developers I have contact with are working on PC c) mac OS b4 OSX was poo.

[spark]

yeah morph, i just _loved_ dos... wa ha ha ha snicker snicker...

[loboy]

What are most performing vjs using mac or windows? In most setups I've seen are Macs. Maybe they are just more stylish to photograph and I've only been in the more artsy fartsy circle of vjing.

aaaaaaaaaaaaa!

focus!

pure 2 channel videoscratching / crossfading would bore me senseless

true!!

I am going to be moving out of this thread soon with my updates, cause looks like I'm going on to be concentrating on this soft project.

Any hardware notes will be posted here, though.

okay, so here is my software interface mock-up. take a look, i know its pretty basic right now, but any feedback is well appreciated.

http://community.audiovisualizers.com/cgi/gallery.php?user=l0b0y&gid=4&imgid=1024

[MoRpH]

:( spark I'm not getting into a platform war here..... I don't have the time or inclination. Lets try to keep this thread OT. :)

[spark]

don't worry i wasn't going to... hence the one-liner. but it's you who made the dig, no? don't take offense when these things come back at ya...

as for software development - i suggest you speak to some people who've been developing their own recently, like phluxm who did it for a masters degree (no really!). you might find that to get it really good, ie worth developing for the end result rather than process, its a massive undertaking that will take years to get 'just so' when $200 can buy you the state of the art, now. maybe think of a real niche, or find an open-source project or plug ins for existing apps as a more realistic project... not to put a downer on it, but i've often bitten off more than i can chew with this kind of thing...

i personally am keeping with developing hardware as the next major step... i have a lovely laptop and amazing software, but only a frigging mouse to control it with > thats the weak link with nothing out there.

toby

[MoRpH]

OK OK yeah I did, just playing :P (allocating RAM per application is down right arcane though)

As for software I don't think its that hard, I mean software like SVi already has video scratch and play @ different speeds, just no appopriate controller, this is where (svi is a directX beast) the mixman controller comes in. I think both sides are there, just with different targets (audio and video) we just need to bring them together :)

[spark]

you see, there you go again! you just can't help yourself!

its not that softs ain't hard, but getting something that you can use as your main tool 'cos its better that something already out there... is. how long has SVI been in development? years?

but yup, as you say, the controller is the missing link... at the mo i'm just stumped over a lappy trying to be precise with a graphite tablet... not so good...

ps. those cheap tablets are excellent hardware controllers for scratching video and effect parameters, and are ultra thin so pack away with your lappy nicely.

[loboy]

I am sticking with the solely the hardware aspect, just making a slightly better platform for myself to use my hardware. I don't have any plans for a software product going to market!

I'll be using MAX 4/MSP 2 wifh Jitter.
It's got everything I need.

As for the hardware, I may have some pics up tonight of my table so far.

[MoRpH]

:D yep I just can't help it :p

Hmm as I said there are plenty of good controllers (IE mixman, commandpost, etc....) its connecting them upto the software thats the issue, also thankfully its the easier problem to solve if we can get programmers to embrace the hardware