簡單記錄一下這兩天用Texture實現渲染YUV420P的一些要點。
在視頻播放的過程中,有的時候解碼出來的數據是YUV420P的。表面(surface)通過設置參數是可以渲染YUV420P的,但Texture紋理似乎不支持直接渲染YUV420P。表面(surface)用法似乎比較單一,通常用來顯示數據,用Texture的話就可以用上D3D的許多其他功能,做出一些效果。當然,這看個人需求,通常而言顯示視頻數據用表面就夠了。
視頻播放過程中幀與幀之間是有固定時間間隔的。若解碼解出來的是YUV420P的數據,要用Texture渲染的話,就需要把數據轉為RGB32的(應該是要轉成RGB32的,沒做過詳細調查,看到的例子中紋理所用的數據都是RGB32的),如果這個過程交給CPU去做的話,會提高CPU的占用率,用GPU來做這項工作則就釋放了CPU的一部分壓力。
本文考慮的方式是用三層紋理分別存放YUV420P的Y、U、V分量(這個詞不知道用對沒有),然後像素著色器把三個分量的數據計算成RGB32的數據然後顯示。這是本文的核心內容。
像素著色器的HLSL代碼如下:
sampler YTex; sampler UTex; sampler VTex; struct PS_INPUT { float2 y : TEXCOORD0; float2 u : TEXCOORD1; float2 v : TEXCOORD2; }; float4 Main(PS_INPUT input):COLOR0 { float y = tex2D(YTex,input.y).r; float u = tex2D(UTex, input.u.xy / 2).r - 0.5f; float v = tex2D(VTex,input.v.xy / 2).r - 0.5f; float r = y + 1.14f * v; float g = y - 0.394f * u - 0.581f * v; float b = y + 2.03f * u; return float4(r,g,b, 1); }
HLSL代碼可以直接寫在txt文件中,sampler可視作標識紋理層和采樣級的對象,Direct3D將把每一個sampler對象唯一地與某一紋理層關聯起來。具體的HLSL語法請自行查資料,我也是粗略知道是怎麼回事,就不誤人子弟了。在代碼中通過調用D3DXCompileShaderFromFile函數可以從文件編譯像素著色器。但實際上,我個人不是很喜歡這種把代碼放在一個單獨文件裡面的做法,這種代碼應該盡可能的編進exe裡面。但是我還只是初步了解D3D,不知道怎麼把它編進exe裡面,如果有人知道,還望指教。
ID3DXBuffer* shader = 0; ID3DXBuffer* errorBuffer = 0; hr = D3DXCompileShaderFromFile( "ps_multitex.txt", 0, 0, "Main", // entry point function name "ps_2_0", D3DXSHADER_DEBUG, &shader, &errorBuffer, &MultiTexCT); // output any error messages if( errorBuffer ) { ::MessageBox(0, (char*)errorBuffer->GetBufferPointer(), 0, 0); d3d::Release<ID3DXBuffer*>(errorBuffer); } if(FAILED(hr)) { ::MessageBox(0, "D3DXCompileShaderFromFile() - FAILED", 0, 0); return false; } // // Create Pixel Shader // hr = Device->CreatePixelShader( (DWORD*)shader->GetBufferPointer(), &MultiTexPS); if(FAILED(hr)) { ::MessageBox(0, "CreateVertexShader - FAILED", 0, 0); return false; } d3d::Release<ID3DXBuffer*>(shader);
以上代碼中,D3DXCompileShaderFromFile函數從文件ps_multitex.txt編譯HLSL代碼;參數Main是HLSL代碼的入口函數,如上一點代碼中所見。這個入口函數可以是自定義的其他的,但要注意保持一致;ps_2_0表示像素著色器的版本。CreatePixelShader函數創建像素著色器。
創建紋理層。本文實現YUV420P渲染的方法采用了三層紋理,每層紋理分別存放Y、U、V數據。
Device->CreateTexture ( Width, Height, 1, D3DUSAGE_DYNAMIC, D3DFMT_L8, D3DPOOL_DEFAULT, &YTex, NULL ) ; Device->CreateTexture ( Width / 2, Height / 2, 1, D3DUSAGE_DYNAMIC, D3DFMT_L8, D3DPOOL_DEFAULT, &UTex, NULL ) ; Device->CreateTexture ( Width / 2, Height / 2, 1, D3DUSAGE_DYNAMIC, D3DFMT_L8, D3DPOOL_DEFAULT, &VTex, NULL ) ;
sampler與紋理的關聯
// // Get Handles // YTexHandle = MultiTexCT->GetConstantByName(0, "YTex"); UTexHandle = MultiTexCT->GetConstantByName(0, "UTex"); VTexHandle = MultiTexCT->GetConstantByName(0, "VTex"); // // Set constant descriptions: // UINT count; MultiTexCT->GetConstantDesc(YTexHandle, &YTexDesc, &count); MultiTexCT->GetConstantDesc(UTexHandle, &UTexDesc, &count); MultiTexCT->GetConstantDesc(VTexHandle, &VTexDesc, &count); MultiTexCT->SetDefaults(Device);
設置紋理/sampler的狀態,這一部分我是在渲染的時候做的,也可以直接寫在HLSL代碼中。在後面渲染部分還會見到這些代碼,其實是同一段代碼,我只是為了表述紋理與sampler關聯的一個整體過程,把它預先從渲染部分截了出來,希望不會造成誤解。
// Y tex Device->SetTexture( YTexDesc.RegisterIndex, YTex); Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_MAGFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_MINFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_MIPFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_ADDRESSU, D3DTADDRESS_BORDER); Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_ADDRESSV, D3DTADDRESS_BORDER); // U tex Device->SetTexture( UTexDesc.RegisterIndex, UTex); Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_MAGFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_MINFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_MIPFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_ADDRESSU, D3DTADDRESS_BORDER); Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_ADDRESSV, D3DTADDRESS_BORDER); // string tex Device->SetTexture( VTexDesc.RegisterIndex, VTex); Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_MAGFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_MINFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_MIPFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_ADDRESSU, D3DTADDRESS_BORDER); Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_ADDRESSV, D3DTADDRESS_BORDER);
獲得YUV420P數據。本文直接讀取的YUV420P數據。
打開文件代碼:
if((infile=fopen("test_yuv420p_320x180.yuv", "rb"))==NULL){ printf("cannot open this file\n"); return false; }
讀取數據並將數據copy到紋理中:
if (fread(buf, 1, Width*Height*3/2, infile) != Width*Height*3/2){ // Loop fseek(infile, 0, SEEK_SET); fread(buf, 1, Width*Height*3/2, infile); } // // Render // Device->Clear(0, 0, D3DCLEAR_TARGET | D3DCLEAR_ZBUFFER, 0xffffffff, 1.0f, 0); plane[0] = buf; plane[1] = plane[0] + Width*Height; plane[2] = plane[1] + Width*Height/4; D3DLOCKED_RECT d3d_rect; byte *pSrc = buf; //Locks a rectangle on a texture resource. //And then we can manipulate pixel data in it. LRESULT lRet = YTex->LockRect(0, &d3d_rect, 0, 0); if (FAILED(lRet)){ return false; } // Copy pixel data to texture byte *pDest = (byte *)d3d_rect.pBits; int stride = d3d_rect.Pitch; for(int i = 0;i < Height;i ++){ memcpy(pDest + i * stride,plane[0] + i * Width, Width); } YTex->UnlockRect(0); D3DLOCKED_RECT d3d_rect1; lRet = UTex->LockRect(0, &d3d_rect1, 0, 0); if (FAILED(lRet)){ return false; } // Copy pixel data to texture byte *pDest1 = (byte *)d3d_rect1.pBits; int stride1 = d3d_rect1.Pitch; for(int i = 0;i < Height/2;i ++){ memcpy(pDest1 + i * stride1 / 2,plane[1] + i * Width / 2, Width / 2); } UTex->UnlockRect(0); D3DLOCKED_RECT d3d_rect2; lRet = VTex->LockRect(0, &d3d_rect2, 0, 0); if (FAILED(lRet)){ return false; } // Copy pixel data to texture byte *pDest2 = (byte *)d3d_rect2.pBits; int stride2 = d3d_rect2.Pitch; for(int i = 0;i < Height/2;i ++){ memcpy(pDest2 + i * stride2 / 2,plane[2] + i * Width / 2, Width / 2); } VTex->UnlockRect(0);
渲染:
Device->BeginScene(); Device->SetPixelShader(MultiTexPS); Device->SetFVF(MultiTexVertex::FVF); Device->SetStreamSource(0, QuadVB, 0, sizeof(MultiTexVertex)); // Y tex Device->SetTexture( YTexDesc.RegisterIndex, YTex); Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_MAGFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_MINFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_MIPFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_ADDRESSU, D3DTADDRESS_BORDER); Device->SetSamplerState(YTexDesc.RegisterIndex, D3DSAMP_ADDRESSV, D3DTADDRESS_BORDER); // U tex Device->SetTexture( UTexDesc.RegisterIndex, UTex); Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_MAGFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_MINFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_MIPFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_ADDRESSU, D3DTADDRESS_BORDER); Device->SetSamplerState(UTexDesc.RegisterIndex, D3DSAMP_ADDRESSV, D3DTADDRESS_BORDER); // string tex Device->SetTexture( VTexDesc.RegisterIndex, VTex); Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_MAGFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_MINFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_MIPFILTER, D3DTEXF_LINEAR); Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_ADDRESSU, D3DTADDRESS_BORDER); Device->SetSamplerState(VTexDesc.RegisterIndex, D3DSAMP_ADDRESSV, D3DTADDRESS_BORDER); Device->DrawPrimitive(D3DPT_TRIANGLELIST, 0, 2); Device->EndScene(); Device->Present(0, 0, 0, 0);
D3D Texture紋理渲染YUV420P的主要就是以上一些內容。完整工程代碼:http://download.csdn.net/download/qq_33892166/9702415
本文的HLSL代碼寫出來的像素著色器畫面有些偏黃,知道如何優化的朋友還請指教。