Vulkan API is liberating the graphics from the desktop
Hardware accelerated 3D graphics and desktop environments wound up together when they took off during the 1990s. Back then it was expected that the application would render into a window and the window manager would figure out where the window is on one of the displays.
After two decades of incremental progress, the GPUs have become generic purpose tools capable of much more than mere triangle rasterization. They can do image and audio processing, solve massive equations, run simulations.
Until now the web servers have had to work around the requirement to have a desktop in order to use GPU hardware. The requirement of having a running desktop environment before you can utilize GPU is slowly turning into a developmental brake we cannot afford to have.
In the Oculus DK1, the window manager grabbed the HMD although it is not aware of how to present content properly in a HMD.
Vulkan API is finally separating the GPUs from the desktop environments. About a year ago I submitted an issue into KhronosGroup and they provided some new extensions that help a lot.
The new extensions on Nvidia cards
Nvidia GTX960 is a recent graphics card. After the latest beta driver upgrade (375.27.13), the vulkan library lists the following instance and device extensions:
instance extensionsVK_KHR_surfaceVK_KHR_xcb_surfaceVK_KHR_xlib_surfaceVK_KHR_displayVK_EXT_debug_reportVK_KHR_get_physical_device_properties2VK_KHX_device_group_creationVK_KHX_external_memory_capabilitiesVK_KHX_external_semaphore_capabilitiesVK_EXT_display_surface_counterVK_EXT_direct_mode_displayVK_EXT_acquire_xlib_displaydevice extensionsVK_KHR_swapchainVK_KHR_display_swapchainVK_KHR_descriptor_update_templateVK_KHR_maintenance1VK_KHR_push_descriptorVK_KHR_shader_draw_parametersVK_KHR_sampler_mirror_clamp_to_edgeVK_KHX_device_groupVK_KHX_external_memoryVK_KHX_external_memory_fdVK_KHX_external_semaphoreVK_KHX_external_semaphore_fdVK_KHX_multiviewVK_EXT_discard_rectanglesVK_EXT_shader_subgroup_ballotVK_EXT_shader_subgroup_voteVK_EXT_display_controlVK_NV_dedicated_allocationVK_NV_glsl_shaderVK_NV_sample_mask_override_coverageVK_NV_viewport_array2VK_NV_viewport_swizzleVK_NV_geometry_shader_passthroughVK_NVX_device_generated_commandsVK_NVX_multiview_per_view_attributes
I did some checks and verified that these extensions are
listed even if I run service stop lightdm to stop my
window system and try things out from the linux console.
They are still working on this while I am writing, but lets go through some of these new extensions and what they're bringing to the table.
Display control and rendering
VK_KHR_displayVK_EXT_display_surface_counterVK_EXT_direct_mode_displayVK_EXT_display_swapchainVK_EXT_display_controlVK_EXT_discard_rectangles
These extensions allows to enumerate the displays on the system, retrieve device display events with callbacks, obtain exclusive access to the display and provide swapchains so you can render to them.
Inter-process GPU resource sharing
VK_KHX_external_memory_capabilitiesVK_KHX_external_semaphore_capabilitiesVK_KHX_external_memoryVK_KHX_external_memory_fdVK_KHX_external_semaphoreVK_KHX_external_semaphore_fd
These extensions let you allocate external memory objects
and share them between processes by passing file handles
around. There are _win32 variations of the extensions for
Windows operation.
Multiple view render passes
VK_KHX_multiviewThis extension allows you to define a render pass that has
multiple views associated to it. The ViewIndex is exposed
to the shaders. It lets you render multiple, slightly
similar views in a single render pass.
Common entry points for querying device features
VK_KHR_get_physical_device_properties2This one doesn't provide any new features, but some of the earlier extensions depend on it and it grabbed my attention.
Some of these new extensions provide new feature bits, and
to do that they extend the existing functions to do it. This
is a fairly interesting detail because these new functions
use the pNext -fields to retrieve information rather than
send it.
Vulkan and these new extensions provide an opportunity for several revolutions in how we use graphics cards.
Revolution for the virtual reality
All HMD users & vendors can benefit because the details of acquiring the HMD display becomes well-documented. This makes it a lot easier to provide portable and versatile VR drivers.
You can control the displays without a desktop environment, therefore you will no longer require a display or a keyboard in your PC. You can enter directly into the VR environment by wearing the headset.
Revolution for the web servers
You have been able to use OpenCL & Cuda for years on web servers to use GPU computing. Vulkan provides access to the full graphics pipeline. Therefore using the GPU capabilities on servers is no longer different than using the same features on the desktop computers.
Revolution for the infobooths and advertising displays
Vulkan makes it trivial to configure a Linux PC to reboot one or more of its connected displays to run a highly interactive infobooth or an advertising display. Especially if you leave the desktop environment away from them and make them remote-accessible by default. That makes it easier to maintenance them anyway.
From now on the best solution is the obvious one and anyone can come up with it. Great times for industries that have to install and operate such systems in their facilities.
Revolution for the desktop and user interface design
Because we no longer depend on an existing window manager, it lets us create our own. It will fragment the desktop, but it also opens a whole new space for user interface designers.
Believe me or not, but the desktop has not been perfected in the design. The whole idea that an application opens a window feels a bit flawed psychologically because the application usually opens it at the wrong time and into the wrong place. It is a small nuisance that appears every day when you work on a computer.
An application takes more than a second to start up. When it finally starts up it jumps in front of you no matter what you were doing and interrupts you. If you are doing something in the meanwhile you risk giving a wrong command to the application that pops up. Giving the window control to the user would eliminate that one little nuisance from the day.
If you combine that with a concept of having a session that can be saved into a file and opened up later, it might give a significant productivity boost for today's office workers.
Positive touch
I may have missed something here, but overall these extensions have brought up my confidence that Vulkan is ending up to be a great advance in the GPU computing.
The additional responsibilities offloaded to the user is offset by the positive effects that come when some no longer needed legacy is erased. The possibility to straightforwardly do cross-platform window compositors or video streaming systems make the Vulkan a sought-after interface in the future.