Hello everyone and welcome to the fifth chapter of Introduction to Metal Compute! We made a lot things in the previous parts. We created a simple image editing app that is able to open, preview, adjust and export images. To do that, we wrote an image editing Metal shader kernel, created an encoder for it, learned, how to convert images to textures and pass the data to the GPU while dispatching the commands to it. The aim of this article is to encourage you to use more “Swifty” way of writing Metal related code. Also, we will migrate from
CAMetalLayer for previewing the result.
Vanilla Metal provides an access to work with device’s GPU. It practically does not add any abstraction and allows you to work in the same paradigm in which the hardware works. Being a low-level API, on one hand, Metal provides an ability to have a fine grained control over the hardware, and on the other hand, it introduces a little bit of complexity and redundancy in some cases. While writing metal pipeline, we operate such concepts as device, command queue, command buffer, command encoder, library, function and more. Some of these objects are created once and can be reused, others need to be initialised on every kernel dispatch. Some of them need to be initialised with their corresponding descriptors and some are not. In some places the API is throwable and returns optionals in the other. In general it feels like Metal was written for Objective-C users without any extra adaptation for Swift.
With all these thoughts in mind Alloy was born. This framework’s purpose is to simplify Metal development on Swift, make the code cleaner and consistent without changing the main paradigm of low-level control over how things work. It provides nano-tiny layer over vanilla Metal API, that hides the majority of redundant explicity in the Metal code, while not limiting a flexibility a bit. Originally Alloy written in 2018 by my colleague Andrey Volodin and what concerns me, I was one of the main contributors to it for the last few years.
To make code more consistent, the Alloys API is designed to be throwable in those places where vanilla Metal is either throwable or returns optionals. A lot of extensions were added to device, texture, command queue and other classes to reduce the number of repeating boilerplate code.
The device has been upgraded and by using it you are able to:
- allocate a heap without a descriptor:
- create a texture with few lines of code:
- allocate a buffer with a value:
… and more!
We extended the command queue with two convenience function that allow you to:
- dispatch a command buffer in async manner:
- dispatch a command buffer synchronously:
Now encoding the commands to command buffer can be done by calling just one function. Also, you don’t need to worry about committing the work. You can easily encode:
- a compute command:
- a render command:
- a blit command:
Compute Command Encoder
Remember how you passed data to shaders via compute command encoder? If you needed to pass any value, you needed to calculate it’s size in bytes and pass a reference to the value. Now you can just call:
or if you need to pass an array:
Also you can set a number of textures just by calling:
One of key things is that now you don’t need to write threadgrop size computations by hand and the code can be reduced just to:
What concerns textures, now you are able to create images from them by calling:
and pixel buffers:
Now it is easy to get
region and a
descriptor of a texture as well as to create it’s empty copy:
The only new concept that Alloy introduces is
MTLContext. The context is an object that is designed to be injected across the app. Internally, the context holds references to such objects that remain the same over the whole metal pipeline lifecycle (device, command queue, library cache and texture loader) and provides a convenience API to maintain it. With the help of context you can:
- create a texture form
- create a shaders library for a given bundle:
- do everything that a device and command queue can.
Also it is important to notice that this framework provides a set of handwritten utility kernels that are commonly used in image processing:
And yet we have covered just a little part of all the extensions that Alloy has. Alloy is a production ready tool and it is used in the development of all Prisma’s apps. I highly recommend you to give this framework a try, and I am sure, you won’t return to vanilla Metal any more 🙂.
Let’s migrate our existing codebase to Alloy and see it in action. First, add is as a dependency to the project.
Adjustments.swift and replace
import Metal with
Now let’s modify the class constructor. First thing is
deviceSupportsNonuniformThreadgroups. Make this property a constant by replacing
letand init it this way:
The function constants can be created much cleaner:
To crate a pipeline state now you don’t need to init with a function. Instead, pass the function name directly to the pipeline state constructor:
The final variant of the
Adjustments init looks like this:
Next stop is the encoding function. Currently, it looks large, explicit and takes about 39 lines of code. Thanks to Alloy’s extensions over command buffer and command encoder, we can significantly reduce the amount of the code. Replace the encoding with the following:
As you can see, now you don’t need to create encoder by hand, instead we are using Swift’s closures which looks much cleaner. Now let’s add the encoding logic:
- set the label
- set the textures:
- set the floats:
- dispatch the command:
The result function looks like this:
and it takes only 16 lines of code! Note that we don’t need to worry about the
encoder.endEncoding(), because Alloy does it under the hood and reduces the amount of redundant code just by changing the way you call it.
Next, delete the
TextureManager file. All its logic now can be replaced by Alloy’s
MTLContext. Navigate to the
ViewController. Import Alloy and replace
textureManager properties with:
ViewContoller’s constructor should be modified:
as well as the calling of this constructor in
Note, how easy it is to initialise Adjustments with just one line of code.
handlePickedImage’s texture creation logic needs to be replaced with:
Creation textures from images and making matching textures is now super easy.
Knowing that the context holds a command queue inside and the textures now are able to output images, we can delete the command queue property from the class and replace the
redraw function with the following:
Now it looks much better and cleaner. And the bonus is that overall we reduced the number of lines of code by 122, which is very cool 🎉.
Creation an image by every slider change is extremely inefficient: each time the system allocates memory, copies the texture bytes to it, creates an image, sets it to the image view and draws the layer to the screen. The better approach is just to render the texture directly to
CAMetalLayer. In order to do that we will use Texture View. Internally this little framework renders two triangles with a texture stretched over them.
Let’s add it as a dependency.
TextureView in the
ViewController. Replace the image view with a texture view:
- the property:
- in the
- in the
Replace the getting result image logic of
share function with the following:
handlePickedImage replace the image view code with:
And finally, the the redraw function should look like this:
Nice! Now we have an image editing app with an end-to-end Metal pipeline 🤘. The source code of the result project can be found here. In the next chapter we will learn how to use another cool tool for kernel encoder codegeneration.