Vulkan API is liberating the graphics from the desktop
Hardware accelerated 3D graphics and desktop environments wound up together when they took off during the 1990s. Back then it was expected that the application would render into a window and the window manager would figure out where the window is on one of the displays.
After two decades of incremental progress, the GPUs have become generic purpose tools capable of much more than mere triangle rasterization. They can do image and audio processing, solve massive equations, run simulations.
Until now the web servers have had to work around the requirement to have a desktop in order to use GPU hardware. The requirement of having a running desktop environment before you can utilize GPU is slowly turning into a developmental brake we cannot afford to have.
In the Oculus DK1, the window manager grabbed the HMD although it is not aware of how to present content properly in a HMD.
Vulkan API is finally separating the GPUs from the desktop environments. About a year ago I submitted an issue into KhronosGroup and they provided some new extensions that help a lot.
The new extensions on Nvidia cards
Nvidia GTX960 is a recent graphics card. After the latest beta driver upgrade (375.27.13), the vulkan library lists the following instance and device extensions:
instance extensions VK_KHR_surface VK_KHR_xcb_surface VK_KHR_xlib_surface VK_KHR_display VK_EXT_debug_report VK_KHR_get_physical_device_properties2 VK_KHX_device_group_creation VK_KHX_external_memory_capabilities VK_KHX_external_semaphore_capabilities VK_EXT_display_surface_counter VK_EXT_direct_mode_display VK_EXT_acquire_xlib_display device extensions VK_KHR_swapchain VK_KHR_display_swapchain VK_KHR_descriptor_update_template VK_KHR_maintenance1 VK_KHR_push_descriptor VK_KHR_shader_draw_parameters VK_KHR_sampler_mirror_clamp_to_edge VK_KHX_device_group VK_KHX_external_memory VK_KHX_external_memory_fd VK_KHX_external_semaphore VK_KHX_external_semaphore_fd VK_KHX_multiview VK_EXT_discard_rectangles VK_EXT_shader_subgroup_ballot VK_EXT_shader_subgroup_vote VK_EXT_display_control VK_NV_dedicated_allocation VK_NV_glsl_shader VK_NV_sample_mask_override_coverage VK_NV_viewport_array2 VK_NV_viewport_swizzle VK_NV_geometry_shader_passthrough VK_NVX_device_generated_commands VK_NVX_multiview_per_view_attributes
I did some checks and verified that these extensions are
listed even if I run
service stop lightdm to stop my
window system and try things out from the linux console.
They are still working on this while I am writing, but lets go through some of these new extensions and what they're bringing to the table.
Display control and rendering
VK_KHR_display VK_EXT_display_surface_counter VK_EXT_direct_mode_display VK_EXT_display_swapchain VK_EXT_display_control VK_EXT_discard_rectangles
These extensions allows to enumerate the displays on the system, retrieve device display events with callbacks, obtain exclusive access to the display and provide swapchains so you can render to them.
Inter-process GPU resource sharing
VK_KHX_external_memory_capabilities VK_KHX_external_semaphore_capabilities VK_KHX_external_memory VK_KHX_external_memory_fd VK_KHX_external_semaphore VK_KHX_external_semaphore_fd
These extensions let you allocate external memory objects
and share them between processes by passing file handles
around. There are
_win32 variations of the extensions for
Multiple view render passes
This extension allows you to define a render pass that has
multiple views associated to it. The
ViewIndex is exposed
to the shaders. It lets you render multiple, slightly
similar views in a single render pass.
Common entry points for querying device features
This one doesn't provide any new features, but some of the earlier extensions depend on it and it grabbed my attention.
Some of these new extensions provide new feature bits, and
to do that they extend the existing functions to do it. This
is a fairly interesting detail because these new functions
pNext -fields to retrieve information rather than
Vulkan and these new extensions provide an opportunity for several revolutions in how we use graphics cards.
Revolution for the virtual reality
All HMD users & vendors can benefit because the details of acquiring the HMD display becomes well-documented. This makes it a lot easier to provide portable and versatile VR drivers.
You can control the displays without a desktop environment, therefore you will no longer require a display or a keyboard in your PC. You can enter directly into the VR environment by wearing the headset.
Revolution for the web servers
You have been able to use OpenCL & Cuda for years on web servers to use GPU computing. Vulkan provides access to the full graphics pipeline. Therefore using the GPU capabilities on servers is no longer different than using the same features on the desktop computers.
Revolution for the infobooths and advertising displays
Vulkan makes it trivial to configure a Linux PC to reboot one or more of its connected displays to run a highly interactive infobooth or an advertising display. Especially if you leave the desktop environment away from them and make them remote-accessible by default. That makes it easier to maintenance them anyway.
From now on the best solution is the obvious one and anyone can come up with it. Great times for industries that have to install and operate such systems in their facilities.
Revolution for the desktop and user interface design
Because we no longer depend on an existing window manager, it lets us create our own. It will fragment the desktop, but it also opens a whole new space for user interface designers.
Believe me or not, but the desktop has not been perfected in the design. The whole idea that an application opens a window feels a bit flawed psychologically because the application usually opens it at the wrong time and into the wrong place. It is a small nuisance that appears every day when you work on a computer.
An application takes more than a second to start up. When it finally starts up it jumps in front of you no matter what you were doing and interrupts you. If you are doing something in the meanwhile you risk giving a wrong command to the application that pops up. Giving the window control to the user would eliminate that one little nuisance from the day.
If you combine that with a concept of having a session that can be saved into a file and opened up later, it might give a significant productivity boost for today's office workers.
I may have missed something here, but overall these extensions have brought up my confidence that Vulkan is ending up to be a great advance in the GPU computing.
The additional responsibilities offloaded to the user is offset by the positive effects that come when some no longer needed legacy is erased. The possibility to straightforwardly do cross-platform window compositors or video streaming systems make the Vulkan a sought-after interface in the future.