Audio to MIDI conversion?

[mkl]

Hiya

Does anyone know of software that can convert audio to a midi stream?

I was talking with the Vj at a club I attended over the weekend, he mentioned this software, but I am unable to recall the specifics (like the name ;))

Analysing audio data, into specific instruments and notes, would seem beyond the ability of present techniques. I guess this software simply encodes in MIDI the familiar frequency analysis tools.

Mkl

[elbows]

Could be this software:

http://www.digital-ear.com/

I tried it a while ago - seemed quite good but I had some crashing with the demo, never found out why really.

Definately is quite surprisingly powerful really, far from perfect but it tries hard and should be quite useable for VJ stuff or other tasks that dont require near-perfect replication of all the musical info.

[ElektroMotif]

Foreseeable problems with this is that a DSP algorhythm capable of taking samples of incoming sound data and then converting them to midi data is going to be a very complicated procedure

A composite audio waveform from 20Hz to 20KHz is riddled with harmonics and phase differentials and to convert these to midi note data would......I beleive ........ be nigh on impossible

If what you are after is mean program power analysis of any frequency range then you really don't have to use midi at all........quite responsive sound system options are available in Visual Jockey R3.2

I'm not sure from your post what you are trying to acheieve

With due respect Elbows....... I haven't had time to check out the link at the time of this post

HTH

[HuoLong]

if you like to "program", PD does this easily with objects like [bonk~] "beat detection" and [fiddle~] audio2midi notes...
and of course it has the whole palette of FFT-tools, but it's not easy for beginners...

[robotfunk]

my sentiments exactly. There are (crude) audio to midi converter apps but why would someone into visualizing want one?
The current state of such apps is that a monophonic melody can be picked up almost reliably.
In music it can be handy to get a melody or rhythm into midi. The main use seems to be to reproduce music into midi that the user cant play manually, because he can't play the keyboard well, or plays another instrument entirely, or sing for example.

These apps tend to be consisting of an envelope follower ( rhythm tracker) and harmonic spectrum analysis ( melody tracker)
either component alone is much more useful to use in visual apps than an all in one thing that adds a (for visual purposes) unnecessary step of conversion to MIDI. unless you have a big plan that calls for this that is.

I dont know of any apps using MIDI to transfer FFT (harmonic content) data and there wouldnt be much of a point .. unless again you have a really great idea that calls for its use.

[Meierhans]

...but why not just work together with people doing their stuff live on stage over midi? ;)

I know there are many situation where you can`t do so, but if you find some people that perform live all these reconversion problems are gone.

Another dimension of syncronism - hard to handle, but also a new world of possibilities.

Its such a pitty: Most people doing visual to computer generated music, and most of them sync to the audio signals that are based on beats and melodys that are laying around somewhere on the world in midi format....

Ok, jumpback to topic :
:D

I would be happy if there was a tool that just gives me a stable (!!!!!!!) midi clock / timecode and (perhabs one trigger for base and one for highhat would`t be to bad too) to non-straight audio signal. (4 to the floor...)
Anybody knows a realtime /fast reacting /non beat loosing program that is easy to handle?

Something that is intelligent enough to "feel" the beat?
(yes, im so lazy, i know :p )

[elbows]

Well Digital Ear is still worth playing with I found, was quite interesting.

I cant remember what my original mission was when I got the demo - I think I might have been wanting to turn the pitch of my voice into midi cc data.

Might well be worth trying for beat detection too, if unsatisfied by current vj software, nothing to lose.

Personally my dream is to do the aforementioned show linked up to live musicians, and I have been making contacts in that regard, but as I have the musical ability of a deaf rabbit, I should be teaming with an existing group/band really I guess.

PD may indeed prove a beter solution to some of these problems, I just have to find the time some day to sit down and program with it properly, same with eyesweb.

[mkl]

Yes! Digital Ear is the software I heard about. I have just checked it out. It seems sufficient for converting a single instrument, well recorded sample to MIDI, but totally fails if there's anything more complicated. This is what I expected, though I was told the Digital Ear is capable of much more.

I think it could be possible to do audio - midi conversion, by obtaining detailed analysis of each instrument and the space acoustics before a performance.

Meierhans, I totally agree! Just a short while ago I posted - http://www.vjforums.com/showthread.php?postid=2836#post2836 - about a project I've been looking into.

[elbows]

Aha you are talking about analying live performances then?

Well, thats a lot more realistic. Firstly get as many of the instruments as possible directly hooked up to midi.

I know for drums its easy, you just get some midi pickups which are effectively microphones which hear the beat and convert to midi with minimal delay. A trigger for each part of the drumkit ensures a pretty good midi interpretation of the drums.

I havent got round to researching other instruments yet, but I know wsome midi wind instruments are around. Theres lots of stuff Id guess.

Then for other instruments and sound sources which cant be directly midi'd, you need a bank of PC's running digital-ear or equivalent, each analying one instrument.

It would end up quite an expensive setup really, its something Im working towards slowly over the next few years. The other area Im interested in is live video analysis, whcih would be converting eg dancers into midi data. Now, I see this technology improving, and along with more cpu power it may even be practical to read certain musicians using video convcerted to midi, for a few instruments, or to create all sorts of invisible instruments, entering jean michelle jarr (spelling!?!) laser harp type territory.

For now Im starting with drums because a friend is a drummer, and its the easiest to do.

In terms of the other side of things, the visual output side, so far Im using visualJockey because the number of variables it can control through midi enables a good expressive output other than just triggering clips, adjusting levels, clip speed etc.

midivid makes uses of midi velocity information and is good at triggering clips fast, just a shame there arent more effects for it.

Someone wrote quite a fun midi-> graphics program, it never really got up to scratch in terms of providing sleek graphics that vjs would want to use, but it is worth looking at quickly to see a midi song come to life visually. I cant be sure of its name, I think its 3D Midi.

I am so slow at doing this stuff though, Ive been thinking about it for several years now but dont seem to have got so far.

[robotfunk]

Meiherhans:
I would be happy if there was a tool that just gives me a stable (!!!!!!!) midi clock / timecode and (perhabs one trigger for base and one for highhat would`t be to bad too) to non-straight audio signal. (4 to the floor...)

the only algo that really impressed me that does this is http://www.cs.tut.fi/~klap/iiro/overview2001/tatum.html

it is available as a pd patch and eats up an entire computer.
not easy but this baby chases anything. I have red sound voyager beat extractor that does it but i hardly use it , find it much easier just tapping in a tempo manually, much more reliable anyway.

[hyperdimensional]

The other area Im interested in is live video analysis, whcih would be converting eg dancers into midi data

I take it you've seen applications like big-eye, eyesweb and softVNS (for max)?

As far as Midi pickups, you could try piezo contact mikes (a couple of pennies from Maplin) and a MIDI drum brain (check out www.paia.com if you want to build ur own, or alesis and roland also sell them). Roland also used to do a midi guitar pickup but i'm not sure how reliable that is.

[elbows]

Yes Im aware of the cool stuff of which you speak, but cheers anyway. I wish there were a few more eyesweb users round here, but then again I havent touched it for a few months so if I want stuff to happen with it I better get on with it instead of just yapping about it ;)

I seem to have about 5 affordable possible options for the midi sensor stuff, in terms of DIY projects Ive seen on the web.

I really need my boss to give me 2 months off work, Im never gonna get this stuff done otherwise. 24 hours just arent enough in a day anymore :(