decoding video

[levon]

hello, a friend of mine has become iterested in me becoming a VJ and video software, and as he is a coder has decided to try writing his own VJing app specilised for me.

i was wondering if people have any links to info about decoding video, he said he is having trouble finding info on coding this aspect of the software.

thanks guys, and once this software is working properly it will be free for everyone who wants it.

oh and btw, please no coments about how he shouldnt waste his time writing 'another' free app, cause there are so many out there already, yes i and he knows that, but hes doing this as a practical learning exercise :D

edit: YAY 100 posts :D

[djMidnight]

PC or MAC? If it's on the PC, he'll want to look up:

AVIFileOpen
AVIStreamOpen
AVIStreamBeginStreaming
AVI... (etc)
DrawDib

Look here: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/multimed/htm/_win32_multimedia_functions.asp

It's possible to use DirectShow as well, but that I couldn't describe in a few functions, and it's also asynchronous, which makes it a PITA for triggering without lag. If anyone has found a way to trigger DirectShow quickly, I'd love to hear it.

Jason

[johnnylocust]

Well, you actually can trigger directshow quickly. You just have to want to be able to do this pretty bad though. Things to do to make quick(er) triggering in DirectShow:

Limit clips to mpeg compression and containers. Almost all graphics cards, and some motherboards have built-in mpeg 1, 2 and 4 acceleration. This also avoids the problem of audio and video desyncronizing. (Limiting yourself to mpeg however defeats the purpose of using Directshow in the first place.)

Only render the video stream. You can tell directshow to only build a graph for the video stream. If it doesn't have to worry about audio, it'll zoom right along. (Not likely anyone's gonna use audio in thier clips anyhow)

Prebuild filtergraphs. This is actually not as stagering a task as one might think. If you have 100 filtergraphs prebuilt, but only 4 or 5 running, there's not that much impact on memory or CPU usage.

Caching filters. If you do destroy a graph, just disconnect the filters, and reuse them when building new graphs. This cuts down greatly on preroll time.

Avoid intelligent connect graph building. Only do this once and store the structure of a filtergraph for a particular clip in an XML document. This can be done for all clips before hand to avoid intelligent-connect graph building during performances. Manually building a graph from known filter structures and cached filters can take up to 99% less time than letting directshow figure it out itself.

Put small clips in physical memory. You can actually stream video-clips from physical memory using Black Trip's AsyncFileSource Filter
AsyncFileSource Filter (http://www.progdigy.com/boards/index.php?act=ST&f=1&t=1721&hl=tmemorystream&) . Doing so free's up access to hard drive seeks.

2 hard drives! I can't stress this enough. Media should never be stored on the same disc as system and program files. They're a dime a dozen, and it's not tranfer times that kill a presentation, but seeking. Even a USB2 harddrive can provide more than adequite bandwidth for many simultaneous video clips, just make sure to de-frag to cut down on excessive seeking.

This list could go on forever. I'm not exactly sure what other methods people use for rendering video. Quicktime's API? Opensource codecs? I've heard OpenML is pretty nice, but haven't seen any good implimentations yet.

Hey levon, what language is your bud using?

[djMidnight]

Originally posted by johnnylocust
Well, you actually can trigger directshow quickly.... (snip)

The things you mention would make the clip LOAD faster, but I haven't found a way to tell DirectShow when I hit a key to instantly display a frame, short of running the graph to the first frame to decode it, and then pausing it. This means that if I have 100 plugins loaded, I have 100 "first frames" decoded in RAM, which isn't ideal.

With the AVIFile model, I can pre-load the AVI frame data into memory (compressed). When the user hits a key, I can issue the read for the next frame asyncronously, then decode the current frame to a framebuffer and display it synchronously. The AVI interfaces allow for finer control. That's the bit I haven't figured out how to do in DShow, so if you have any tips for that I'd love to hear them. The downside to the AVI model is that there's no hardware support, which is why I'm interested in getting the DShow model to bend to my will. I'm hoping that using the hardware decoding won't make the latency worse. There are also ways to get the hardware to keep the decoded samples in VRAM, which I believe the ATI video shader sample implements.

Your tips are great - I'm going to revisit my code and see how much of this stuff I can implement in my current model. It's a bit abstracted, so things like re-using old graphs may be tricky, but I might be able to work something out.

Thanks!
Jason

[levon]

hehe, sorry forgot to say it would be on windows. and i think he said he would code it in dephi, though might just be C.

ill give him that link, thanks for all your help.

ill keep you posted about how the app is going.

[johnnylocust]

short of running the graph to the first frame to decode it, and then pausing it
yup, that's one of the ugly truths. You Don't have to run it and then pause it though. Just issue a Pause command right after building the graph. It allows for instantanious starting of a graph when you tell it. (assuming it's settled down into a paused state)