In the previous entry we built up a basic sample that loads a web page and uploads it to a texture, which we then rendered to a full screen triangle. In this entry we're going to work on optimizing that process a bit, and making it so that our texture updates whenever the source updates.

Introduction

One of the problems with our current mechanism is that we are not handling page updates. I.e. if the page has animations, images that load after some time, and other similar conditions then our image will not be similarly updated with this new information. This poses a problem for us if we're going to use something like Awesomium for a game UI.

The solution is to not stop updating the web client, and to update the image every time it changes. There are, however, a few problems with this:

• Updating an entire image is slow, especially if the image takes up the entire screen.
  
• Rarely does the entire page change, so why update the entire image when only a small portion of it has changed?
  

We can solve this easily enough by simply knowing which parts of the image need to be changed. Which is where the Awesomium::Surface and Awesomium::SurfaceFactory come into play.

The Surface Factory

Awesomium provides us with the ability to provide it with a custom surface to render to. The rendering code then simply calls to the surface and asks it to blit certain rectangles of data, which we can then translate into the appropriate texture updates. In order for Awesomium to construct one of our surfaces it needs us to provide it with a factory instance capable of constructing the surface. This is where our D3DSurfaceFactory comes in.

The D3DSurfaceFactory is a simple factory which we create an instance of and passing it the appropriate Direct3D context to use for updating textures created from the factory. As you can see below, the implementation of the creation and release methods are fairly trivial, being mostly there to simply pass through any necessary state:
virtual Awesomium::Surface * CreateSurface(Awesomium::WebView * view, int width, int height) { return new D3DSurface(m_context, view, width, height);}virtual void DestroySurface(Awesomium::Surface * surface) { delete surface;}With this state passed through we can move onto the meat of our changes... the D3DSurface.

The D3DSurface

The D3DSurface is our implementation of the Awesomium::Surface interface. The Surface interface expects us to provide two methods: Paint, which is called whenever a rectangle of the surface needs to be updated, and Scroll, which is used to scroll a portion of the view. In our case we actually don't care about scrolling, and so we'll leave this method blank. Our paint method, as can be seen below, is fairly trivial:
virtual void Paint(unsigned char *srcBuffer, int srcRowSpan, const Awesomium::Rect &srcRect, const Awesomium::Rect &destRect) { auto box = CD3D11_BOX(destRect.x, destRect.y, 0, destRect.x + destRect.width, destRect.y + destRect.height, 1); // 4 bytes per pixel, srcRowSpan is already in bytes. auto startingOffset = srcRowSpan * srcRect.y + srcRect.x * 4; m_context->UpdateSubresource(m_texture, 0, &box, srcBuffer + startingOffset, srcRowSpan, 0);}All this really does is convert the destination rectangle into a box, and then calculate the appropriate starting byte in the source buffer. We then simply pass this on through to UpdateSubresource, which does the bulk work of copying our data and uploading it to the GPU.

Miscellaneous other Bits

As a final set of pieces for this demo, we want to render our HTML over somethimg. In this case, that colorful triangle from the first of this series.

To do this we need to change how our Awesomium renders, and also setup a blend state so that our back buffer is blending data instead of overwriting it. Configuring Awesomium views to render transparent turns out to be trivial:
m_view->SetTransparent(true);Configuring a blendstate is equally as simple:
blendDesc.AlphaToCoverageEnable = false;blendDesc.IndependentBlendEnable = false;blendDesc.RenderTarget[0] = { true, D3D11_BLEND_SRC_ALPHA, D3D11_BLEND_INV_SRC_ALPHA, D3D11_BLEND_OP_ADD, D3D11_BLEND_ONE, D3D11_BLEND_ZERO, D3D11_BLEND_OP_ADD, D3D11_COLOR_WRITE_ENABLE_ALL};device->CreateBlendState(&blendDesc, &m_blendState);With these in place we can now render our colorful triangle:
context->OMSetBlendState(m_blendState, nullptr, ~0);context->IASetPrimitiveTopology(D3D11_PRIMITIVE_TOPOLOGY_TRIANGLELIST);context->IASetIndexBuffer(nullptr, (DXGI_FORMAT)0, 0);context->IASetVertexBuffers(0, 1, &m_vertexBuffer.p, &Vertex::Stride, &Vertex::Offset);context->IASetInputLayout(m_inputLayout);context->VSSetShader(m_triangleVS, nullptr, 0);context->PSSetShader(m_trianglePS, nullptr, 0);context->Draw(3, 0);and then our UI:
context->IASetVertexBuffers(0, 0, nullptr, nullptr, nullptr);context->IASetInputLayout(nullptr);context->VSSetShader(m_vertexShader, nullptr, 0);context->PSSetShader(m_pixelShader, nullptr, 0);context->PSSetSamplers(0, 1, &m_sampler.p);if (m_surface) m_surface->Bind();context->Draw(3, 0);Simple enough eh?

An Example Game UI

With our triangle rendered and our UI being displayed over it, it is time to come up with a UI. For this particular case I'm using the jquery-ui kit, and jquery. I've built a pretty simple HTML web page to showcase what you can do... at the top of it is a progress bar that indicates something, perhaps the health of a boss monster. To the right is a set of quest goals for the currently selected quest. And lastly at the bottom we have a button. It doesn't do anything yet, and perhaps we'll touch on getting that working next time.
jQuery UI Progressbar - Default functionality html, body { margin: 0; padding: 0; height: 100%; } #wrapper { min-height: 100%; position: relative; } #content { padding: 10px; padding-bottom: 80px; /* Height of the footer element */ } #footer { width: 100%; height: 80px; position: absolute; bottom: 0; left: 0; } .center { width: 100%; text-align: center; } $(function () { $('#progressbar').progressbar({ value: 100 }); $("#button").button(); $("#quest-button").button(); $('#quest-tracker').menu(); $('#quest-show-hide').menu(); })

• Show/Hide Quests

• A quest goal!
• Another quest goal!

A Skill Button

Lastly, we can send a message to our view anytime we desire using javascript. This allows us to update our boss health as the player hits a key, and the progress bar will change it's value, redrawing only that portion of the screen:
LRESULT OnKeyUp(unsigned message, WPARAM wParam, LPARAM lParam, BOOL & handled) { if (wParam == 'A' && m_view) { --m_bossHealth; UpdateBossHealth(); } return DefWindowProc(message, wParam, lParam);}void UpdateBossHealth() { auto javascript = std::string("$('#progressbar').progressbar({ value: ") + std::to_string(m_bossHealth) + "}); "; m_view->ExecuteJavascript(Awesomium::ToWebString(javascript), Awesomium::WSLit(""));}Now, every time we hit the 'A' key, the boss's health will decrease.

Full Sample

#define NOMINMAX#include #include #include #include #include #include #include #include #include #include #pragma comment(lib, "d3d11.lib")#pragma comment(lib, "awesomium.lib")#include #include #include #include #include #include #include #ifdef UNICODEtypedef wchar_t tchar;typedef std::wstring tstring;templatetstring to_string(T t) { return std::to_wstring(t);}#elsetypedef char tchar;typedef std::string tstring;templatetstring to_string(T t) { return std::to_string(t);}#endifstruct Vertex { float position[4]; float color[4]; float texCoord[2]; static const unsigned Stride = sizeof(float) * 10; static const unsigned Offset = 0;};void ThrowIfFailed(HRESULT result, std::string const & text) { if (FAILED(result)) throw std::runtime_error(text + "");}class RenderTarget {public: RenderTarget(ID3D11Texture2D * texture, bool hasDepthBuffer) : m_texture(texture) { CComPtr device; texture->GetDevice(&device); auto result = device->CreateRenderTargetView(m_texture, nullptr, &m_textureRTV); ThrowIfFailed(result, "Failed to create back buffer render target."); m_viewport = CD3D11_VIEWPORT(m_texture, m_textureRTV); result = device->CreateTexture2D(&CD3D11_TEXTURE2D_DESC(DXGI_FORMAT_D32_FLOAT, static_cast(m_viewport.Width), static_cast(m_viewport.Height), 1, 1, D3D11_BIND_DEPTH_STENCIL), nullptr, &m_depthBuffer); ThrowIfFailed(result, "Failed to create depth buffer."); result = device->CreateDepthStencilView(m_depthBuffer, nullptr, &m_depthView); ThrowIfFailed(result, "Failed to create depth buffer render target."); } void Clear(ID3D11DeviceContext * context, float color[4], bool clearDepth = true) { context->ClearRenderTargetView(m_textureRTV, color); if (clearDepth && m_depthView) context->ClearDepthStencilView(m_depthView, D3D11_CLEAR_DEPTH, 1.0f, 0); } void SetTarget(ID3D11DeviceContext * context) { context->OMSetRenderTargets(1, &m_textureRTV.p, m_depthView); context->RSSetViewports(1, &m_viewport); }private: D3D11_VIEWPORT m_viewport; CComPtr m_depthBuffer; CComPtr m_depthView; CComPtr m_texture; CComPtr m_textureRTV;};class GraphicsDevice {public: GraphicsDevice(HWND window, int width, int height) { D3D_FEATURE_LEVEL levels[] = { D3D_FEATURE_LEVEL_11_1, D3D_FEATURE_LEVEL_11_0, D3D_FEATURE_LEVEL_10_1, D3D_FEATURE_LEVEL_10_0, }; DXGI_SWAP_CHAIN_DESC desc = { { width, height, { 1, 60 }, DXGI_FORMAT_R8G8B8A8_UNORM, DXGI_MODE_SCANLINE_ORDER_UNSPECIFIED, DXGI_MODE_SCALING_UNSPECIFIED }, { 1, 0 }, DXGI_USAGE_BACK_BUFFER | DXGI_USAGE_RENDER_TARGET_OUTPUT, 1, window, TRUE, DXGI_SWAP_EFFECT_DISCARD, 0 }; auto result = D3D11CreateDeviceAndSwapChain( nullptr, D3D_DRIVER_TYPE_HARDWARE, nullptr, D3D11_CREATE_DEVICE_DEBUG | D3D11_CREATE_DEVICE_BGRA_SUPPORT, levels, sizeof(levels) / sizeof(D3D_FEATURE_LEVEL), D3D11_SDK_VERSION, &desc, &m_swapChain, &m_device, &m_featureLevel, &m_context ); ThrowIfFailed(result, "Failed to create D3D11 device."); } void Resize(int width, int height) { if (m_renderTarget) m_renderTarget.reset(); auto result = m_swapChain->ResizeBuffers(1, width, height, DXGI_FORMAT_UNKNOWN, 0); ThrowIfFailed(result, "Failed to resize back buffer."); CComPtr backBuffer; result = m_swapChain->GetBuffer(0, __uuidof(ID3D11Texture2D), reinterpret_cast(&backBuffer)); ThrowIfFailed(result, "Failed to retrieve back buffer surface."); m_renderTarget = std::make_unique(backBuffer, true); } void SetAndClearTarget() { static float color[] { 0, 0, 0, 0}; if (!m_renderTarget) return; m_renderTarget->Clear(m_context, color); m_renderTarget->SetTarget(m_context); } void Present() { m_swapChain->Present(0, 0); } ID3D11Device * GetDevice() { return m_device; } ID3D11DeviceContext * GetDeviceContext() { return m_context; }private: D3D_FEATURE_LEVEL m_featureLevel; CComPtr m_device; CComPtr m_context; CComPtr m_swapChain; std::unique_ptr m_renderTarget;};class D3DSurface : public Awesomium::Surface {public: D3DSurface(ID3D11DeviceContext * context, Awesomium::WebView * view, int width, int height) : m_context(context), m_view(view), m_width(width), m_height(height) { CComPtr device; context->GetDevice(&device); auto result = device->CreateTexture2D(&CD3D11_TEXTURE2D_DESC(DXGI_FORMAT_B8G8R8A8_UNORM, width, height, 1, 1), nullptr, &m_texture); result = device->CreateShaderResourceView(m_texture, nullptr, &m_textureView); } virtual void Paint(unsigned char *srcBuffer, int srcRowSpan, const Awesomium::Rect &srcRect, const Awesomium::Rect &destRect) { auto box = CD3D11_BOX(destRect.x, destRect.y, 0, destRect.x + destRect.width, destRect.y + destRect.height, 1); // 4 bytes per pixel, srcRowSpan is already in bytes. auto startingOffset = srcRowSpan * srcRect.y + srcRect.x * 4; m_context->UpdateSubresource(m_texture, 0, &box, srcBuffer + startingOffset, srcRowSpan, 0); } virtual void Scroll(int dx, int dy, const Awesomium::Rect &clip_rect) { } void Bind() { m_context->PSSetShaderResources(0, 1, &m_textureView.p); } virtual ~D3DSurface() { }private: CComPtr m_textureView; CComPtr m_texture; ID3D11DeviceContext * m_context; Awesomium::WebView * m_view; int m_width; int m_height;};class D3DSurfaceFactory : public Awesomium::SurfaceFactory {public: D3DSurfaceFactory(ID3D11DeviceContext * context) : m_context(context) { } virtual Awesomium::Surface * CreateSurface(Awesomium::WebView * view, int width, int height) { return new D3DSurface(m_context, view, width, height); } virtual void DestroySurface(Awesomium::Surface * surface) { delete surface; }private: ID3D11DeviceContext * m_context;};class MainWindow : public CWindowImpl {public: DECLARE_WND_CLASS_EX(ClassName, CS_OWNDC | CS_HREDRAW | CS_VREDRAW, COLOR_BACKGROUND + 1); MainWindow(Awesomium::WebCore * webCore) : m_webCore(webCore), m_view(nullptr, [](Awesomium::WebView * ptr) { ptr->Destroy(); }), m_isMaximized(false), m_surface(nullptr) { RECT rect = { 0, 0, 800, 600 }; AdjustWindowRectEx(&rect, GetWndStyle(0), FALSE, GetWndExStyle(0)); Create(nullptr, RECT{ 0, 0, rect.right - rect.left, rect.bottom - rect.top }, WindowName); ShowWindow(SW_SHOW); UpdateWindow(); } void Run() { MSG msg; while (true) { if (PeekMessage(&msg, 0, 0, 0, PM_REMOVE)) { if (msg.message == WM_QUIT) break; TranslateMessage(&msg); DispatchMessage(&msg); } else { Update(); } } } void Update() { auto context = m_device->GetDeviceContext(); m_webCore->Update(); if (m_view->IsLoading()) { m_isLoading = true; } else if (m_isLoading) { m_isLoading = false; UpdateBossHealth(); m_webCore->Update(); m_surface = static_cast(m_view->surface()); } m_device->SetAndClearTarget(); context->OMSetBlendState(m_blendState, nullptr, ~0); context->IASetPrimitiveTopology(D3D11_PRIMITIVE_TOPOLOGY_TRIANGLELIST); context->IASetIndexBuffer(nullptr, (DXGI_FORMAT)0, 0); context->IASetVertexBuffers(0, 1, &m_vertexBuffer.p, &Vertex::Stride, &Vertex::Offset); context->IASetInputLayout(m_inputLayout); context->VSSetShader(m_triangleVS, nullptr, 0); context->PSSetShader(m_trianglePS, nullptr, 0); context->Draw(3, 0); context->IASetVertexBuffers(0, 0, nullptr, nullptr, nullptr); context->IASetInputLayout(nullptr); context->VSSetShader(m_vertexShader, nullptr, 0); context->PSSetShader(m_pixelShader, nullptr, 0); context->PSSetSamplers(0, 1, &m_sampler.p); if (m_surface) m_surface->Bind(); context->Draw(3, 0); m_device->Present(); }private: BEGIN_MSG_MAP(MainWindow) MESSAGE_HANDLER(WM_DESTROY, [](unsigned messageId, WPARAM wParam, LPARAM lParam, BOOL & handled) { PostQuitMessage(0); return 0; }); MESSAGE_HANDLER(WM_CREATE, OnCreate); MESSAGE_HANDLER(WM_SIZE, OnSize); MESSAGE_HANDLER(WM_EXITSIZEMOVE, OnSizeFinish); MESSAGE_HANDLER(WM_KEYUP, OnKeyUp); END_MSG_MAP()private: LRESULT OnCreate(unsigned message, WPARAM wParam, LPARAM lParam, BOOL & handled) { try { RECT rect; GetClientRect(&rect); m_device = std::make_unique(m_hWnd, rect.right, rect.bottom); auto device = m_device->GetDevice(); tstring filename(MAX_PATH, 0); GetModuleFileName(GetModuleHandle(nullptr), &filename.front(), filename.length()); filename = filename.substr(0, filename.find_last_of('\\')); SetCurrentDirectory(filename.c_str()); std::vector vs(std::istreambuf_iterator(std::ifstream("FullScreenTriangleVS.cso", std::ios_base::in | std::ios_base::binary)), std::istreambuf_iterator()); auto result = device->CreateVertexShader(&vs.front(), vs.size(), nullptr, &m_vertexShader); ThrowIfFailed(result, "Could not create vertex shader."); std::vector ps(std::istreambuf_iterator(std::ifstream("FullScreenTrianglePS.cso", std::ios_base::in | std::ios_base::binary)), std::istreambuf_iterator()); result = device->CreatePixelShader(&ps.front(), ps.size(), nullptr, &m_pixelShader); ThrowIfFailed(result, "Could not create pixel shader."); result = device->CreateSamplerState(&CD3D11_SAMPLER_DESC(CD3D11_DEFAULT()), &m_sampler); ThrowIfFailed(result, "Could not create sampler state."); vs.assign(std::istreambuf_iterator(std::ifstream("TriangleVS.cso", std::ios_base::in | std::ios_base::binary)), std::istreambuf_iterator()); result = device->CreateVertexShader(&vs.front(), vs.size(), nullptr, &m_triangleVS); ThrowIfFailed(result, "Could not create vertex shader."); ps.assign(std::istreambuf_iterator(std::ifstream("TrianglePS.cso", std::ios_base::in | std::ios_base::binary)), std::istreambuf_iterator()); result = device->CreatePixelShader(&ps.front(), ps.size(), nullptr, &m_trianglePS); ThrowIfFailed(result, "Could not create pixel shader."); std::vector inputElementDesc = { { "SV_POSITION", 0, DXGI_FORMAT_R32G32B32A32_FLOAT, 0, 0 }, { "COLOR", 0, DXGI_FORMAT_R32G32B32A32_FLOAT, 0, 4 * sizeof(float) }, }; result = device->CreateInputLayout(&inputElementDesc.front(), inputElementDesc.size(), &vs.front(), vs.size(), &m_inputLayout); ThrowIfFailed(result, "Unable to create input layout."); // Hard coded triangle. Tis a silly idea, but works for the sample. Vertex vertices[] = { { { 0.0f, 0.5f, 0.5f, 1.0f }, { 1.0f, 0.0f, 0.0f, 1.0f }, { 0.5f, 1.0f } }, { { 0.5f, -0.5f, 0.5f, 1.0f }, { 0.0f, 1.0f, 0.0f, 1.0f }, { 0.0f, 0.0f } }, { { -0.5f, -0.5f, 0.5f, 1.0f }, { 0.0f, 0.0f, 1.0f, 1.0f }, { 1.0f, 0.0f } }, }; D3D11_BUFFER_DESC desc = { sizeof(vertices), D3D11_USAGE_DEFAULT, D3D11_BIND_VERTEX_BUFFER }; D3D11_SUBRESOURCE_DATA data = { vertices }; result = device->CreateBuffer(&desc, &data, &m_vertexBuffer); ThrowIfFailed(result, "Failed to create vertex buffer."); D3D11_BLEND_DESC blendDesc; blendDesc.AlphaToCoverageEnable = false; blendDesc.IndependentBlendEnable = false; blendDesc.RenderTarget[0] = { true, D3D11_BLEND_SRC_ALPHA, D3D11_BLEND_INV_SRC_ALPHA, D3D11_BLEND_OP_ADD, D3D11_BLEND_ONE, D3D11_BLEND_ZERO, D3D11_BLEND_OP_ADD, D3D11_COLOR_WRITE_ENABLE_ALL }; device->CreateBlendState(&blendDesc, &m_blendState); m_surfaceFactory = std::make_unique(m_device->GetDeviceContext()); m_webCore->set_surface_factory(m_surfaceFactory.get()); m_view.reset(m_webCore->CreateWebView(rect.right, rect.bottom, nullptr, Awesomium::kWebViewType_Offscreen)); m_view->SetTransparent(true); Awesomium::WebURL url(Awesomium::WSLit(URL)); m_view->LoadURL(url); m_device->Resize(rect.right, rect.bottom); } catch (std::runtime_error & ex) { std::cout << ex.what() << std::endl; return -1; } return 0; } LRESULT OnSizeFinish(unsigned message, WPARAM wParam, LPARAM lParam, BOOL & handled) { try { RECT clientRect; GetClientRect(&clientRect); m_device->Resize(clientRect.right, clientRect.bottom); if (m_view->IsLoading()) m_view->Stop(); m_surface = nullptr; m_view.reset(m_webCore->CreateWebView(clientRect.right, clientRect.bottom, nullptr, Awesomium::kWebViewType_Offscreen)); m_view->SetTransparent(true); Awesomium::WebURL url(Awesomium::WSLit(URL)); m_view->LoadURL(url); } catch (std::runtime_error & ex) { std::cout << ex.what() << std::endl; } return 0; } LRESULT OnSize(unsigned message, WPARAM wParam, LPARAM lParam, BOOL & handled) { if (wParam == SIZE_MAXIMIZED) { m_isMaximized = true; return OnSizeFinish(message, wParam, lParam, handled); } else { if (m_isMaximized) { m_isMaximized = false; return OnSizeFinish(message, wParam, lParam, handled); } } return 0; } LRESULT OnKeyUp(unsigned message, WPARAM wParam, LPARAM lParam, BOOL & handled) { if (wParam == 'A' && m_view) { --m_bossHealth; UpdateBossHealth(); } return DefWindowProc(message, wParam, lParam); } void UpdateBossHealth() { auto javascript = std::string("$('#progressbar').progressbar({ value: ") + std::to_string(m_bossHealth) + "}); "; m_view->ExecuteJavascript(Awesomium::ToWebString(javascript), Awesomium::WSLit("")); }private: std::unique_ptr m_device; std::unique_ptr m_surfaceFactory; std::unique_ptr m_view; Awesomium::WebCore * m_webCore; D3DSurface * m_surface; CComPtr m_pixelShader; CComPtr m_vertexShader; CComPtr m_sampler; CComPtr m_blendState; CComPtr m_vertexBuffer; CComPtr m_trianglePS; CComPtr m_triangleVS; CComPtr m_inputLayout; int m_bossHealth = 100; bool m_isLoading; bool m_isMaximized;private: static const tchar * ClassName; static const tchar * WindowName; static const char * URL;};const tchar * MainWindow::WindowName = _T("DX Window");const tchar * MainWindow::ClassName = _T("GameWindowClass");const char * MainWindow::URL = ""; // Was: file : ///./Resources/UIInterface.html (remove spaces)int main() { Awesomium::WebCore * webCore = Awesomium::WebCore::Initialize(Awesomium::WebConfig()); { MainWindow window(webCore); window.Run(); } Awesomium::WebCore::Shutdown();}