Vulkan WSI, Oculus and SDL2

In today's post I describe how I deal with the Window System Integration in the Vulkan API. So far I have implemented support for Oculus Rift PC headsets and SDL2 widgets for my renderers.

I enjoy that Vulkan can be used without an access to a desktop system. The details of the desktop have been entirely abstracted behind the WSI.

I think the common ways to implement WSI integration into Vulkan are going to be a bit different from how they meant it to be.


Both SDL2 and Oculus PC SDK require access to variety of instance and device extensions to function. So they have to influence the initialization of the Vulkan Device.

I thought out they are very much like sidekicks so I call them so. The sidekick is not necessarily doing WSI, but any sidekick extends the Vulkan interface somehow.

My Vulkan programs begin like this:

window = sdl.Window()
hmd = vr.HMD()

gpu = GPU(window.wsi, hmd.wsi)

In the future you will be ideally able to initialize it like this:

gpu = GPU(sdl.wsi, vr.wsi)

For now with SDL we have to pass the SDL Window first, because we would have to know which window engine is going to be used: Win32?, Wayland?, Mir?, Xcb? X11? Only the SDL_SysWMinfo provides that information and you have to call it with the window.

On the Oculus PC SDK, the WSI can be deployed when you have a session, but the Oculus session is associated to a single HMD. I do not know what is the status of their multiple-HMD support.

Even with this system it's possible to render into multiple windows because the queue that can support one surface can support them all, but I don't think it's a nice option to leave the possibility there so for now we have to open all the SDL windows that we are rendering to and pass their sidekicks into the GPU initializer.

Perhaps the situation improves with newer SDL releases. A function like SDL_GetPlatformInfo that can be called after SDL_Init would be sufficient.

Note that if you have to restart the Vulkan Device because of this, it means your app loses all the memory references to the GPU memory. That's not a very nice thing to prepare for.

Sidekick interface

The Sidekick interface here is constructed like this:

instance_support (vk, extensions)
setup (vk, instance)
device_support (vk, dev, extensions)
queue_support (vk, dev, queueFamilyIndex)

During the GPU instantiation, the first function called for the sidekicks is the instance_support. The function returns the list of instance extensions that the sidekick needs to function. The input extensions is provided so that the sidekick can optionally include extensions if the ideal setup of extensions is not available.

SDL requires the extension VK_KHR_surface and one of the VK_KHR_*_surface to function.

At the moment Oculus Rift PC SDK is not providing a list of required instance extensions, but I have heard that they are going to provide function to retrieve the necessary instance extensions in the 1.18 SDK.

The setup is called right after the Vulkan instance has been created. This allows the sidekicks to prepare for selecting the physical device.

SDL sidekick initializes the surface at this point and uses the SDL_GetWindowWMInfo to create a SurfaceKHR object, whereas Oculus Rift sidekick has to call the ovr_GetSessionPhysicalDeviceVk to retrieve a reference to the PhysicalDevice they want you to use.

I do not think the ovr_GetSessionPhysicalDeviceVk is entirely satisfactory as a solution, but it will do for now. I would prefer if it was more like the Get*SupportKHR -functions.

Next your application needs to select the suitable device for the work.

I have set up my application to select a queue with both graphics and compute support. I think that's generally reasonable to do and it has not caused any trouble so far.

The device_support will again receive a list of extensions that are available, and has to return a list of the extensions that it wants to activate. If the returned extension list is a superset of the extensions, the application will fail. The sidekick has to return null in order to discard the device as an option.

In the device_support the SDL sidekick returns the KHR swapchain extension whereas the Rift sidekick returns the external_memory related extensions.

The queue_support selects among the queues. The SDL queries the vkGetPhysicalDeviceSurfaceSupportKHR at this point. That function requires a surface that was created in the setup -call. There are also platform-specific support queries that do not require a surface as an argument.

Note that the SurfaceKHR object stays the same even if the window is resized, so you don't need to adjust anything in the GPU sidekicks to allow resizable windows.

Swapchain / RenderTarget setup

After we have obtained the GPU instance, this whole system is very much ready for action. With SDL you obtain the surface from the WSI sidekick and create swapchain like usual. Oculus has the ovr_CreateTextureSwapChainVk which requires few parameters. This is a bit annoying to fill up because the image format and type for the swapchain are described with Oculus parameters, rather than Vulkan's parameters. You may need to translate across the two and that is not nice.

This is pretty much it. Despite that Oculus Rift is using a custom swapchain object, you can retrieve the images, width, height, format, samples like usual and pass them into the renderer which you prefer. Rift SDK provides the ovr_GetTextureSwapChainCurrentIndex and ovr_CommitTextureSwapChain that provide the same functionality as what the vkAcquireNextImageKHR and vkQueuePresentKHR provide.

When designing your renderers you want to acknowledge that you're possibly rendering two eyes with one render pass, and then either rendering into the window with a second render pass or mirroring it in from either one of the rendered eyes.