Vulkan WSI, Oculus and SDL2
In today's post I describe how I deal with the Window System Integration in the Vulkan API. So far I have implemented support for Oculus Rift PC headsets and SDL2 widgets for my renderers.
I enjoy that Vulkan can be used without an access to a desktop system. The details of the desktop have been entirely abstracted behind the WSI.
I think the common ways to implement WSI integration into Vulkan are going to be a bit different from how they meant it to be.
Sidekicks
Both SDL2 and Oculus PC SDK require access to variety of instance and device extensions to function. So they have to influence the initialization of the Vulkan Device.
I thought out they are very much like sidekicks so I call them so. The sidekick is not necessarily doing WSI, but any sidekick extends the Vulkan interface somehow.
My Vulkan programs begin like this:
window = sdl.Window()hmd = vr.HMD()gpu = GPU(window.wsi, hmd.wsi)
In the future you will be ideally able to initialize it like this:
gpu = GPU(sdl.wsi, vr.wsi)For now with SDL we have to pass the SDL Window first,
because we would have to know which window engine is going
to be used: Win32?, Wayland?, Mir?, Xcb? X11? Only the
SDL_SysWMinfo provides that information and you have to
call it with the window.
On the Oculus PC SDK, the WSI can be deployed when you have a session, but the Oculus session is associated to a single HMD. I do not know what is the status of their multiple-HMD support.
Even with this system it's possible to render into multiple windows because the queue that can support one surface can support them all, but I don't think it's a nice option to leave the possibility there so for now we have to open all the SDL windows that we are rendering to and pass their sidekicks into the GPU initializer.
Perhaps the situation improves with newer SDL releases.
A function like SDL_GetPlatformInfo that can be called
after SDL_Init would be sufficient.
Note that if you have to restart the Vulkan Device because of this, it means your app loses all the memory references to the GPU memory. That's not a very nice thing to prepare for.
Sidekick interface
The Sidekick interface here is constructed like this:
instance_support (vk, extensions)setup (vk, instance)device_support (vk, dev, extensions)queue_support (vk, dev, queueFamilyIndex)
During the GPU instantiation, the first function called for
the sidekicks is the instance_support. The function
returns the list of instance extensions that the sidekick
needs to function. The input extensions is provided so
that the sidekick can optionally include extensions if the
ideal setup of extensions is not available.
SDL requires the extension VK_KHR_surface and one of the
VK_KHR_*_surface to function.
At the moment Oculus Rift PC SDK is not providing a list of
required instance extensions, but I have heard that they are
going to provide function to retrieve the necessary instance
extensions in the 1.18 SDK.
The setup is called right after the Vulkan instance has
been created. This allows the sidekicks to prepare for
selecting the physical device.
SDL sidekick initializes the surface at this point and uses
the SDL_GetWindowWMInfo to create a SurfaceKHR object,
whereas Oculus Rift sidekick has to call the
ovr_GetSessionPhysicalDeviceVk to retrieve a reference to
the PhysicalDevice they want you to use.
I do not think the ovr_GetSessionPhysicalDeviceVk is
entirely satisfactory as a solution, but it will do for now.
I would prefer if it was more like the Get*SupportKHR
-functions.
Next your application needs to select the suitable device for the work.
I have set up my application to select a queue with both graphics and compute support. I think that's generally reasonable to do and it has not caused any trouble so far.
The device_support will again receive a list of extensions
that are available, and has to return a list of the
extensions that it wants to activate. If the returned
extension list is a superset of the extensions, the
application will fail. The sidekick has to return null
in order to discard the device as an option.
In the device_support the SDL sidekick returns the KHR
swapchain extension whereas the Rift sidekick returns the
external_memory related extensions.
The queue_support selects among the queues. The SDL
queries the vkGetPhysicalDeviceSurfaceSupportKHR at this
point. That function requires a surface that was created in
the setup -call. There are also platform-specific support
queries that do not require a surface as an argument.
Note that the SurfaceKHR object stays the same even if the
window is resized, so you don't need to adjust anything in
the GPU sidekicks to allow resizable windows.
Swapchain / RenderTarget setup
After we have obtained the GPU instance, this whole system
is very much ready for action. With SDL you obtain the
surface from the WSI sidekick and create swapchain like
usual. Oculus has the ovr_CreateTextureSwapChainVk which
requires few parameters. This is a bit annoying to fill up
because the image format and type for the swapchain are
described with Oculus parameters, rather than Vulkan's
parameters. You may need to translate across the two and
that is not nice.
This is pretty much it. Despite that Oculus Rift is using a
custom swapchain object, you can retrieve the images, width,
height, format, samples like usual and pass them into the
renderer which you prefer. Rift SDK provides the
ovr_GetTextureSwapChainCurrentIndex and
ovr_CommitTextureSwapChain that provide the same
functionality as what the vkAcquireNextImageKHR and
vkQueuePresentKHR provide.
When designing your renderers you want to acknowledge that you're possibly rendering two eyes with one render pass, and then either rendering into the window with a second render pass or mirroring it in from either one of the rendered eyes.