Igor,
I'm also interested in the exact conversion done by the 3D GPU when using glTextDirectVIV. I checked the sections you referenced, but I cannot see how this would help to answer Lin Ya's and my question.
2.4.1 Color space conversion from YUV to RGB:
nice example, but it neither mentions how the conversion is done by the G2D API (which by the way Lin Ya already found) nor does it even mention the 3D GPU
Sect.13.1.1.6 of Chapter 13 Graphics Processing Unit (GPU):
also nice to know that some binaries are proprietary and not all sources are available, but currently I'm not interested in any sources. I just want to know the conversion factors used by the GPU which I refuse to believe is in any way "confidential".