VJing in HD - when?

[gpvillamil]

Lately I've been focusing more on shooting & editing video, in HD. Really interesting, once you get used to HD resolution, anything less looks quite coarse...

I was wondering about the implications for VJing - right now, a lot of VJs are working at the QVGA resolutions (320x240), and with a bit of a stretch, the full NTSC/PAL resolutions.

I've been able to drive HD resolution from Pilgrim, from rendered 3D scenes, and even some short tests using HD footage. But it does put some load on the system...

Also, using HD means running a VGA cable straight to the projector, which means no mixer, and hence no possibility of mixing with DVD or other second source. I suppose the fancy Edirol mixer (V440 HD) could do it, but at a price!

As people get used to HD resolutions, what are the implications for VJing?

How interested are promoters, bands, etc?

Is anyone already experimenting with HD resolutions at gigs?

What kind of changes to equipment? To preparing clips?

[deepvisual]

I heard a rumour that the frame sequence in HDV is not linear. The implication of this is that just to do 2 simple cuts within half a second you would need 4 channels of real time video as HDV needs to buffer two lots of video for each cut.
I think until a new format supercedes HDV, this just isn't going to happen in the VJ world, not at the computer level anyway.

[sleepytom]

its totally possible to VJ in HD now. however it is very expensive due to the stupidly high costs of HD projection (projectorcentral.com lists only 20 projectors capable of full res 1900x1200 and half of them are low brightness home cinema projectors)

this makes 1080i impractical for the time being, however 1280x720 is more realistically achievable (both in terms of playback and projection) - mixers are still expensive and are a necessity for working at these resolutions (even fast computers are not really capable of smooth multilayer HD playback leaving you with the need to use multiple computers to achieve interesting compositions).

but for a budget of 15k you could be mixing at 720p which puts it into the realms of possibility for tours and the like.

creatively the higher res offers many nice new options - for starters there is the widescreen aspect ratio - this is better suited to a lot of venues which do not have the high ceilings required for large 4:3 screens. it also gives a nicer shape for compositing imho. The high resolution of HD gives many create options for picture in picture style composition of lower resolution clips, it also allows for much better use of non fullscreen techniques where much of the image content is black. both of these factors are good for creativity.

[sleepytom]

I heard a rumour that the frame sequence in HDV is not linear. The implication of this is that just to do 2 simple cuts within half a second you would need 4 channels of real time video as HDV needs to buffer two lots of video for each cut.
I think until a new format supercedes HDV, this just isn't going to happen in the VJ world, not at the computer level anyway. your confusing two seperate issues here - the long GOP structure of HDV mpeg2 has no impact upon realtime mixing - the HD component out of a Z1 is real 1080i (its upscaled inside the camera from the 1400x1080 CCD) when plugged into an HD mixer you can cut just the same as with SD video.

HDV is not really suited to computer based realtime playback in VJ software - but you can render other HD formats that will work very well indeed (i have had some success with 720p MJPEG for example)

I suspect that we will start to see a lot of canon HD1's at bigger events as the camera has SDI out and genlock / timecode inputs - the possibilty to intergrate the camera into an existing SDI event workflow should make it a popular choice.

[RayV]

http://www.vjforums.com/showthread.php?t=12593&highlight=Gary+Eva

[gpvillamil]

...
HDV is not really suited to computer based realtime playback in VJ software - but you can render other HD formats that will work very well indeed (i have had some success with 720p MJPEG for example)
...

Yep, me too. MJPEG at 720p30 or even MJPEG at 720p60 has worked quite well.

I have done some tests at weird resolutions as well, such as 1920x480 - also perfectly functional.

[gpvillamil]

http://www.vjforums.com/showthread.php?t=12593&highlight=Gary+Eva

Interesting intro, but some misconceptions present, eg. the Panasonic cam is not HDV, it is DVCAM-PRO HD.

Also confusion between HD (resolution) and HDV (file format & codec). Resolume can't play HDV (I've tried), it CAN play at HD resolution using MJPEG or Cineform codecs.

Pilgrim, vvvv and probably VJamm can all handle HD resolutions.

[many2]

To overcome the limit of resolution of most projectors, you can always use more than one, and the same thing apply to your video sources, whatever they might be - arrays of computers, of media servers, of DVD players, whatever.

The Hippotizer HD is also a good solution as you don't need any extra mixer to do your show in HD and you can mix HD content stored on the unit itself as well as apply real-time effects, for an all-in-one solution.

[sleepytom]

Yeah i was going to mention the hippo hd - its a fairly powerful system but expensive to. its also unable to mix external HD sources being limited in its HD functionality to internally stored mpeg2 clips.

for the cost of the hippo HD you can buy an edirol hd mixer or an analogway eventix which gives a more flexible HD mixing solution for live cameras or other external sources.

its very much horses for courses and the hippo is great for certain situations - if you don't need external sources and can cope with having a computer as the last thing before the screen then the hippo is definitely in the running.

[Rovastar]

I could google this but what refreesh rate is HDTV is it still like a poor 50/60 fps?

[kostya]

I could google this but what refreesh rate is HDTV is it still like a poor 50/60 fps?

Why is it poor? Even if computer is capable of higher FPS I do not think HDTV content is being shot at framerates higher then 60FPS. Also I do crossfade of 2 HDTV videos on a single computer in my installations and do not see any artefacts. And my computer is far from being most powerful. I think that on purposely built rig one should not have any problems mixing 4 layers of HD

[Rovastar]

I had just hoped it would be more.

In the future vision mixers running at HD will be standard and whilst the res I welcome for input from a PC the refresh rate I may be disapointed in as tbh I like to work in 70+ fps to get the best responsive times.

[michela]

Talking about weird resolutions and testing, anyone London-based interested in taking some of the HD rez plates from my short re-mixable film Sanctuary for a test run? The whole film library will be released (when we finish the bloody thing!) but looking to get more locals involved in parallel with post.

It was shot on 35mm at 2:3:5 to 1 aspect ratio, telecined to HDCAM 1920x1080 (so bit of letterboxing) and without having any of the latest gear I'm curious to see what's achieveable in real time atm.

Sanctuary
http://modfilms.com/sanctuary

Cheers
.M.

[alangeering]

I could google this but what refreesh rate is HDTV is it still like a poor 50/60 fps?

Why would you want information your eyes couldn't take in.
The only need for higher frame rates would be to mess with people's heads (assuming sumbliminal messaging works).

60 fps is enough.

Remember that film is only 24 fps.

People who've spent time reading tech news about computer graphics cards for gaming have become obsessed with frames per second as a performance measure. Really all that's important is that you have more than the required (approx) 60 fps. 135 fps in a PC game is just showing off, you can't actually see the extra frames (your monitor probably wont show more than 85 anyway).

[InsideUsAll]

Why would you want information your eyes couldn't take in.
60 fps is enough.

although essentially I agree that 60 fps is enough for most applications, I also agree that it would have been nice to see the HD standard adopt a higher frame rate.

The human visual system can apparently only handle input of around 50 fps, but if your jumping up and down, or rotating your head you can still catch artefacts that you wouldn't see at higher refresh rates.. like the RGB effect seen in DLP projectors when you turn your head quickly... maybe i'm talking crap, but would that not be avoided if the scan rate was higher?

I have had a few drinks this evening so excuse me if i'm talking nonsense!

[sleepytom]

as the majority of HD is going to be digital broadcast TV then we have to look at the trade off between datarate and increased frame rate and frame size. Whilst 2MB/s can look pretty good at PAL you start to get some nasty artifacts when you up the frame size to 1900x1080 - if you try and do 85FPS at such high resolutions you massively increasing the bandwidth required to get a good signal.
the TV companies only have a limited amount of bandwidth available and governments are not suddenly going to make huge increases to the size of the RF spectrum assigned to TV - thus their is no benefit to be had from increasing the framerate beyond 60FPS - even non broadcast situations have difficulty maintaining 60FPS @ 1900x1080 without resorting to savage compression ratios, hence why 1080p is a very rare form of HD...

[Meierhans]

I guess the fastest software to do HD mixing is MXwendler right now. I was really amazed to see HOW performand it is, did mix 3 layers of 720p with effects at 25 fps.
To be honest, the difference between 720p and 1080p is visible, at least at big screens, but it not that big. For me 720p is enough right now and the data rate of 1080p limits alot.
I am a Quake player, and I can feel the difference between 125 and 85 fps. But I guess if you show it to me without a mouse in my hand I can not see it.
I?m very happy with the change from SD to 720p in the moment.

Completly off topic (for all those who are internet addicted and check every post...)

Check out: http://musicv2.com/browse.mv2

The mixes and songs there seem to play only in their player... but if you look inside HTML code and search for MP3 (at 5 result) comes a link to a very nice downloadable 192 Kbit MP3 file. Some of them are very nice!

For your love.

[alangeering]

I am a Quake player, and I can feel the difference between 125 and 85 fps. But I guess if you show it to me without a mouse in my hand I can not see it.

It's more likly that you see the difference between an average of 125 fps and an average of 85 fps.

The real time fps varies a lot. The "125 fps" system probably varies between 80 and 140 fps whilst the "85 fps" system probably varies between 45 and 100 fps depending on the ammount of change and the complexity of a scene and movement.

What you're noticing is probably the difference between minimums. 45 fps and 80 fps.

In broadcast they number of frames (or fields) per second is not changed during a transmission.

[Rovastar]

Why would you want information your eyes couldn't take in.
The only need for higher frame rates would be to mess with people's heads (assuming sumbliminal messaging works).

60 fps is enough.

Remember that film is only 24 fps.

People who've spent time reading tech news about computer graphics cards for gaming have become obsessed with frames per second as a performance measure. Really all that's important is that you have more than the required (approx) 60 fps. 135 fps in a PC game is just showing off, you can't actually see the extra frames (your monitor probably wont show more than 85 anyway).

But you can take in more that 60fps true not 60 seperate frame and tell what is happening but still you can tell. Most people that play computer games can tell you that.

It is very clear when reducing the refresh from 85fps to 60 or 50. It still pains me to reduce the rate this much stuff just isn't as responsive.

I also dispute how much you think the average fps to vary the fps counter today are normaly accurate to within the last few frames.

[many2]

There is a huge difference between what you get when you compare real-world video footage with computer generated real-time graphics at the same frame rate. Video footage at 60 frames per second is very convincing. 3d video games running at the same frame rate are barely acceptable.

Why is it that way ? It is a question of how long each those frame are representing as quantities of time. When you shoot at 60 fps on your camera, each frame you capture is not an absolute instant in time but a certain period of time. If we assume perfect conditions for sake of comprehension, you can imagine each frame as a picture taken with an exposure time of 1/60 of a second. This means that a very fast moving object will create a motion blurred trace on the image covering the entire area covered by the object while it was moving during that 1/60 of a second.

When you render a frame on a computer game what you get is a picture of an absolute instant, 1/(infinite) of a second. You get what the world is looking like at a given moment, not a given period of time. You get a picture of how each object is looking like at moment A and then at moment B, not what it was looking like between moment A and moment B. Sure, there are tricks to simulate motion blur in some games, but this still isn't enough to make you believe that a framerate of 60 fps is more than enough. You need higher framerates in computer games so that human vision creates the motion blur by itself.

[alangeering]

Also traditionaly the persistance of phospher screen on TV was higher than that of a PC monitor. This means that lower frame are usable as they don't appear to flicker. Using video refresh rates on a PC monitor might get annoying.

LCDs are a different kettle of fish!

For the record: I reject game type fps as being relevent to video

Note: Top end graphics cards have a genlock function so thay can work within a broadcast environment for logo insertion, etc. This allows them to put out exactly 30/50/60 fps (and in sync) like the rest of the video devices. Whether its capable of 90 or 150 fps won't matter as long as it can meet the 30/50/60.

Rova: I was not saying the measure was inaccurate. I was saying that over the period of time the fps is averaged there can be a large variance. Personally if I was on a review site for a game I would very much like to know the minimum frame rate at any point, or at least the standard deviation or variance. Why? because it will give me an idea of how the PC deals with the hardest part of the demo.

[DrEskaton]

I'd like video content encoded at 100fps just so you could retime live and get much smoother results. 25 samples per second is why video scratching will never look good, there's just not enough there to work with.

even if the end result was only output at say 50p to a projector it would be great to have over sampling frame rate for the content and a realtime play speed manipulator.

failing that how about optical flow retiming in realtime running in the GPU on vj software? eg slowdown a clip by generating new frames using vector motion analysis not simply by blending or "stepping".

(i give it another five years).

[many2]

failing that how about optical flow retiming in realtime running in the GPU on vj software? eg slowdown a clip by generating new frames using vector motion analysis not simply by blending or "stepping".
(i give it another five years).

That would be so nice ! I believe that the first step to get there is using the GPU for non real-realtime optical flow calculations as, as far as I understand, they are much better suited for the task than standard CPUs. There was an article on the subject written by Pete Warden last year :
http://petewarden.com/notes/archives/2005/05/gpu_optical_flo.html

As for the speed scratching potential of videos shot at high framerates, you are 100% correct. If you have the opportunity, try to play with a high-speed camera - it is a lot of fun.

[Meierhans]

I agree that game FPS can`t be directly compared to video FPS. Recording at 1/10000 shutter will even make 60 fields look a bit stroboscopic. (Which can even at 25 FPS look very nice, like "28 days after" shows - if used in the right context.) I just took game FPS because I do not have many other sources that deliver 125 fps. I tend to limit my FPS a bit lower than my PC can do, so it stays more stable and game is better to control.
What I ment was that there is a limit your eyes can can process, even at 1/infinite shutter. I guess its somewhere between 85 and 100 FPS, maybe a bit lower for people that are not used to these kind of games. ;)