Codec bending? Data trashing? Forcible Glitch art?

[KillingFrenzy]

There have been a couple mentions of tweaking codecs to get intentional flaws.

I'm wondering if anyone has played with the idea of taking the raw data of video files and mashing it up.

If anybody has thoughts on how you could do this in Max/MSP/Jitter, I'd be interested.

I mean, if I could dump the whole file into a matrix and alter it.
It seems like shuffling about bits and pieces of it might get interesting.
I'm not talking about spatially grabbing pieces and just rebuilding them... I'm talking about messing up the file structure to force codec glitches.
Or creating a "player" that is intentionally mislead into reading random parts of the file instead of the proper ones.
I don't want mush, I just want damage.
and I want my damage able to be variable in some way I could sync to tunes or control.

[vjpixylight]

cool..
all I know about this is that when you forward fast while a DV tape is playing, it gives you digital dropout artifacts(like scan problems without the analog looking distortion)..

[DrEskaton]

that's a cool idea KF but to do what you want i think you'd have to get much lower access to the file structure of a movie than that MAX/MSP provides.

why not take an open source movie player like mplayer or videolan and hack it's codec drivers? I imagine they lots of parameters for frame lengths etc and if you just change the parameters to 'incorrect' values and try to play a clip you'll get interesting results.....

[Meierhans]

I would not try this live... easiely hangs the whole PC. I did **** around with some xvid files and a hex editor, most times it just crashed the player or led to 1 FPS, sometimes it looked nice. Maybe a tool that confuses a bit more sensefull the keyframes or vectors, exchange them so f.e. movement information if applied on the wrong pictures would be better. The second problem with pure damage is that you will get problems rendering into something playable, VirtualDub may say: This file is corrupt.... and just don`t render.
So in general the structure of the file should stay untouched, only content should be changed. Hard to tell what is what in a hex editor.

[caine]

i've been using glitch browser for this sort of thing. export a clip as jpegs from quicktime, upload them to my webspace as one page with all the images on it, open the page in glitch browser, download all the pictures, import to quicktime, export picture sequence as a .mov file. a painstakingly long process, but its wicked!
i contacted them to ask if the software could be built into a freeframe plugin or just a standalone version to use on my computer offline, but never heard anything back from them, which is a pitty, cos i feel bad abusing their servers so much!

[glitchism]

i'v tried some hex editing, simple delete, copie paste stuff from the text, but not always succesful. What Meierhans said, its was just 1 frame visual.

Then I worked with Benjamins Corrupt. http://www.recyclism.com
Importing frame sequences and end up haven't the corrupted style videos. I asked him to look at a video plugin version.

There's also the Glitch-alike max patch from Vade, if you didn't knew. But thats doesn't do anything with the data. http://abstrakt.vade.info/?p=48

I'm also very interested in working with video data and max/msp....

[johnnylocust]

There's a "feature" in DirectShow that lets you induce temporal artifacts when decoding interframe codecs. (mpeg2, mpeg4, wmv, etc). If you have Graphedit you can see how it works:

Run GraphEdit, select "Render Media File", and choose a clip (preferably mpeg4). You'll see something like this:


Insert a SampleGrabber filter between the Splitter and the Video codec filter:


Press play. It will look normal until you grab the seek bar and move it around:


This will work for just about every interframe codec out there. The reason it does this, is that the SampleGrabber filter doesn't pass the discontinuity flag to the decoder, so even if you seek to a new position, the codec has no idea you did so. It will then use the most recent frame as a keyframe to decode the interframes.

You can do this programatically very easy too. I modified the PlayWin example from the DirectShow.Net lib to illustrate:



Glitch example and source (requires .Net 2.0):
Attachment 378
Attachment 379

[dePICT]

Or, how about a way to generate random values to then input into a freeframe plugin chain. With a range plugins values could be generated once and view result or constantly updated.

The unpredictability and scrambling of parameters may result in what you want K.F.

[glitchism]

Uncle nice, johnnylocust! thanks! that will do the trick for dvd glitch-alikes. very simple and useful. thanks again!

[johnnylocust]

No problem. It originally showed up as a heinous bug in one of my apps. I never would have guessed it would turn out to be usefull