updated: Wed Jan 11 12:44:53 CET 2006 This is a small document trying to explain the confusing world of encoding, processing and reproducing colour information. In pf, the bitmap/* type cannot be used for computation. For this the internal image/* type is used, while bitmap/* is for storage and i/o (v4l, quicktime, xv, yuv4mpeg, ...). The preferred encoding is bitmap/i420, which seems to be the 'most default' encoding. Colour spaces ------------- Some theory about colour spaces. check http://www.poynton.com/Poynton-video-eng.html http://www.poynton.com/papers/Discreet_Logic/index.html terminology used: R'G'B' gamma corrected (nonlinear RGB) Y'CbCr computed from R'G'B' Y' = .299 R' + .587 G' + .114 B' Cb = .564 (B' - Y') Cr = .713 (R' - Y') ( this is CCIR 601. there's also mention of CCIR 656 in http://www.fourcc.org/fccyvrgb.php ) in words: Luma (Y) is a weighted average of gamma-corrected RGB. The components Cb and Cr encode how much the blue and red channels differ from grayscale. So the latter two 'augment' gray scale video to colour video. The reason of existence of YUV video is to make colour TV compatible with black and white TV. Think of the Cr and Cb channels as an upgrade. (An interesting analogue here is FM radio, where a stereo transmission is coded as mono + difference. In fact, a lot of coding schemes use this approach.) The interesting part is that the human eye seems to be less sensitive to this colour information. This is a happy coincidence. It can be exploited by storing the Cr and Cb information with lower resolution. (For analog TV signals less bandwidth is allocated.) To summarize: YUV (aka. TV/video) is a series of hacks to enable compatibility of colour and B&W TV, and it is easier to handle since it requires less bandwidth. The correct name for YUV is Y'CbCr, indicating it is derived from gamma-corrected RGB (denoted as R'G'B') On to gamma correction. The relationship between perceived (coloured) light intensity and (physical) light power is not linear. Another strange coincidence is that this non-linearity is largely compensated by the nonlinearity of a CRT display. Another thing mid-20th century TV engineers got away with! The conclusion we might draw here is that the R'G'B and Y'CbCr signals actually carry perceived intensity, not physical light power. A very interesting observation is that it is possible to do simple operations directly on R'G'B or Y'CbCr video. This means the operations on PERCEIVED quantities instead of PHYSICAL ones. This is not at all trivial! More so, in computer graphics (CG), one always works with the physical, non gamma-corrected RGB values to simulate physical processes of light mixing, and most CG generated images need to be compensated (gamma-corrected) so they actually display correctly on a TV. The real hairy stuff pops up when combining CG and recorded material (usually presented in Y'CbCr). If you don't get gamma right in those cases, composed images will look artificial. There's a lot to take in here, and a million ways to get it wrong.. I'll try to simplify by presenting 2 kinds of images that occur most of the time. * rgb: usually computer generated, not gamma corrected RGB * yuv: usually recorded by camera, gamma corrected Y'CrCb video, displayable on TV Now, sometimes rgb can be gamma-corrected (R'G'B'), but i've never encountered a yuv format that's not gamma corrected, unless it was computed (by mistake) from non gamma-corrected RGB. Then there's this note from http://www.fourcc.org/fccyvrgb.php ( i suppose Y'CbCr is meant by YCrCb ) The reason your first set of coefficients is incorrect, and related to your clamping comment, is because the defined range for Y is [16,235] (220 steps) and the valid ranges for Cr and Cb are [16,239] (235 steps) These are normalized ranges and when the data is used, 16 is added to Y and 128 is added from Cr and Cb to de-normalize them. The reason for this is that control and colorburst information is stored in the components along with the luminence and chromiance data which is why the full range of [0,255] is not used. There is no "rumor" about this being correct. It is a defined standard for YCrCb video. I wouldn't be surprised if certain PC programs are using a variant where the entire range of [0,255] is used but technically it is not YCrCb. If you use YCrCb for video purposes, its probably going to be CCIR 601 compliant. I don't know why you would use YCrCb for anything else as many rendering operations are extremely expensive to do in the non-linear YCrCb color space relative to the linear RGB color space. image/s16/* contains signed integers. What is actually contained in those integers is upto you. Converting a movie file or video input to image/s16/* will provide gamma-corrected Y'CrCb. So, technically, pf needs to have some kind of gamma compensation next to proper YUV/RGB conversion. I've never run into a case where i need to be so 'correct', but in theory, when doing 'ordinary' video work, you need to pay attention to this. FIXME: add proper gamma + CSC pf formats ---------- The best supported luma/chroma format in pf is i420, this is also called 4:2:0 subsampled planar Y'CbCr. This is the format used in the YUV4MPEG stream format, which seems to be a de-facto standard, supported by pf. It is a planar format, with plane ordering Luma (Y'), Chroma Blue (Cb) and Chroma Red (Cr), where the 2 colour planes are 2x2 subsampled. Y' is encoded as unsigned 8 bit value 0-255, while Cb and Cr are encoded as unsigned values 0-255, with 128 equal to zero. Chroma values in image/* packets are always encoded as signed. Where Cr=Cb=0 means grey, while in bitmap/* packets, they are encoded as unsigned where Cr=Cb=128 is grey. links http://en.wikipedia.org/wiki/YCbCr http://www.poynton.com/Poynton-video-eng.html http://www.fourcc.org/fccyvrgb.php