What's New in iOS 9 and OS X 10.11

This chapter summarizes the new features introduced in iOS 9 and OS X 10.11.

Feature Sets

All devices that support Metal conform to a feature set value described inListing 10-1.

Listing 10-1 Metal Feature Sets

All OS X devices that support Metal support theOSX_GPUFamily1_v1feature set.

iOS devices that support Metal support a feature set determined by their GPU and OS versions. See theMTLFeatureSetreference andiOS Device Compatibility Referencefor more information.

To find out which feature set is supported by a device, query thesupportsFeatureSet:method of aMTLDeviceobject.

Device Selection

Use theMTLCreateSystemDefaultDevicefunction to obtain the preferred GPU for your app.OSX_GPUFamily1_v1feature set devices may have multiple GPUs, which you can obtain by calling theMTLCopyAllDevicesfunction. To further obtain the characteristics of a single GPU in a multi-GPU system, query theheadlessproperty to find out if the GPU is attached to a display and query thelowPowerproperty to find the lower-power GPU in an automatic graphics switching system.

You can query specific render and compute characteristics with the newsupportsTextureSampleCount:method andmaxThreadsPerThreadgroupproperty added to theMTLDeviceprotocol.

Resource Storage Modes and Device Memory Models

TheOSX_GPUFamily1_v1feature set includes support for managing resources on GPUs that contain discrete memory. Resource memory allocation is explicitly handled by selecting an appropriate storage mode value from theListing 10-2enum for textures andListing 10-3enum for buffers.

Listing 10-2 Texture Storage Modes

Listing 10-3 Buffer Storage Modes

iOS feature sets only support shared and private storage modes. A description of the three new resource storage modes is summarized in the following sub-sections.

Shared

Resources allocated with the shared storage mode are stored in memory that is accessible to both the CPU and the GPU. On devices with discrete memory, these resources are accessed directly from CPU local memory rather than being copied to GPU memory.

In iOS feature sets, this is the default storage mode for textures. In theOSX_GPUFamily1_v1feature set, textures cannot be allocated with the shared storage mode.

Private

Resources allocated with the private storage mode are stored in memory that is only accessible to the GPU.

Buffers and textures allocated with private storage mode can only be accessed by the GPU. Thecontentsproperty of a buffer returnsNULLand theMTLTexturemethods listed below are illegal to call:

• replaceRegion:mipmapLevel:withBytes:bytesPerRow:

• replaceRegion:mipmapLevel:slice:withBytes:bytesPerRow:bytesPerImage:

• getBytes:bytesPerRow:fromRegion:mipmapLevel:

• getBytes:bytesPerRow:bytesPerImage:fromRegion:mipmapLevel:slice:

To access a private resource, your app may perform one or more of the following actions:

• Blit to or from the private resource.

• Read from the private resource from any shader function.

• Render to the private resource from a fragment shader.

• Write to the private resource from a compute function.

Managed

On GPUs without discrete memory, managed resources have only a single memory allocation accessible to both the CPU and GPU. On GPUs with discrete memory, managed resources internally allocate both CPU-accessible and GPU-accessible memory.

Managed textures are not available in iOS feature sets; useMTLStorageModeSharedinstead.

Any time your app uses the CPU to directly modify the contents of a buffer, you must make a call to thedidModifyRange:method to notify Metal that you have changed the contents of the resource indicated by the specific range.

If you use the GPU to modify the contents of a managed resource and you wish to access the results with the CPU, then you must first synchronize the resource using thesynchronizeResource:method orsynchronizeTexture:slice:level:method.

Choosing a Resource Storage Mode

Generally, there are four main scenarios to consider when choosing a storage mode:

• ChooseMTLResourceStorageModePrivatefor resources that are only ever read from and/or written to by the GPU. For example—render target textures (a particularly common and important case).

• ChooseMTLResourceStorageModeManagedorMTLResourceStorageModePrivatefor resources that are initialized once and used many times in the future.

• ChooseMTLResourceStorageModeManagedfor resources that are populated per-frame by the CPU and then read by the GPU. For example—shader constant buffers or dynamic vertex data.

• ChooseMTLResourceStorageModeSharedfor resources written to by the CPU and then blitted into GPU memory (or, for CPU reads from GPU memory). For example—staging buffers.

Setting and Querying a Resource Storage Mode

For textures, set the desired storage mode with thestorageModeproperty of aMTLTextureDescriptorobject. The default storage mode for textures isMTLStorageModeSharedin iOS feature sets andMTLStorageModeManagedin theOSX_GPUFamily1_v1feature set.

For buffers, set the desired storage mode by passing in the respectiveMTLResourceOptionsvalue in any of thenewBufferWithLength:options:,newBufferWithBytes:length:options:, ornewBufferWithBytesNoCopy:length:options:deallocator:methods ofMTLDevice. The default storage mode for buffers isMTLStorageModeShared, butOSX_GPUFamily1_v1feature set apps may benefit from increased performance by explicitly managing their buffers with the managed or private storage modes.

The storage mode of a resource, for either a texture or a buffer, can be queried with thestorageModeproperty of aMTLResourceobject.

Textures

Hardware support for different texture types and pixel formats is a key capability difference between feature sets. This section lists the major texture additions to the framework; for a more detailed discussion, see the code listings and comparison tables in theMTLPixelFormatreference andMetal Feature Set Tableschapter.

Compressed Textures

TheiOS_GPUFamily2_v1,iOS_GPUFamily2_v2, andiOS_GPUFamily3_v1feature sets add support for ASTC textures. These pixel formats are listed inListing 10-4.

Listing 10-4 ASTC Pixel Formats

TheOSX_GPUFamily1_v1feature set supports BC textures instead. The new pixel formats are listed inListing 10-5.

Listing 10-5 BC Pixel Formats

PVRTC Blit Operations

TheMTLBlitCommandEncoderprotocol contains a newMTLBlitOptionvalue that enables copying to or from a texture with a PVRTC pixel format. Two new methods support these operations by passing theMTLBlitOptionRowLinearPVRTCvalue into theiroptionsparameter:

• To copy PVRTC data from a buffer to a texture, usecopyFromBuffer:sourceOffset:sourceBytesPerRow:sourceBytesPerImage:sourceSize:toTexture:destinationSlice:destinationLevel:destinationOrigin:options:

• To copy PVRTC data from a texture to a buffer, usecopyFromTexture:sourceSlice:sourceLevel:sourceOrigin:sourceSize:toBuffer:destinationOffset:destinationBytesPerRow:destinationBytesPerImage:options:

PVRTC blocks are arranged linearly in memory in row-major order, similar to all other compressed texture formats. PVRTC pixel formats are only available in iOS feature sets.

Depth/Stencil Render Targets

TheOSX_GPUFamily1_v1feature set does not support separate depth and stencil render targets. If these render targets are needed, use one of the newly introduced depth/stencil pixel formats to set the same texture as both the depth and stencil render target. The combined depth/stencil pixel formats are listed inListing 10-6.

Listing 10-6 Depth/Stencil Pixel Formats

All feature sets support theMTLPixelFormatDepth32Float_Stencil8pixel format. Only some devices that support theOSX_GPUFamily1_v1feature set also support theMTLPixelFormatDepth24Unorm_Stencil8pixel format. Query thedepth24Stencil8PixelFormatSupportedproperty of aMTLDeviceobject to determine whether the pixel format is supported or not.

Textures with a depth, stencil, or depth/stencil pixel format can only be allocated with the private storage mode. To load or save the contents of these textures, you must perform a blit operation. TheMTLBlitCommandEncoderprotocol contains the newMTLBlitOptionvalues that define the behavior of a blit operation for textures with a depth, stencil, or depth/stencil pixel format:

• Use theMTLBlitOptionNonevalue to blit the contents of a texture with a depth-only or stencil-only pixel format.

• Use theMTLBlitOptionDepthFromDepthStencilvalue to blit the depth portion of a texture with a combined depth/stencil pixel format.

• Use theMTLBlitOptionStencilFromDepthStencilvalue to blit the stencil portion of a texture with a combined depth/stencil pixel format.

Two new methods support these operations with theiroptionsparameter:

• To copy depth or stencil data from a buffer to a texture, usecopyFromBuffer:sourceOffset:sourceBytesPerRow:sourceBytesPerImage:sourceSize:toTexture:destinationSlice:destinationLevel:destinationOrigin:options:

• To copy depth or stencil data from a texture to a buffer, usecopyFromTexture:sourceSlice:sourceLevel:sourceOrigin:sourceSize:toBuffer:destinationOffset:destinationBytesPerRow:destinationBytesPerImage:options:

Note: To blit the contents of a texture with a depth-only or stencil-only pixel format, you may forgo the methods that take aMTLBlitOptionvalue and instead use thecopyFromBuffer:sourceOffset:sourceBytesPerRow:sourceBytesPerImage:sourceSize:toTexture:destinationSlice:destinationLevel:destinationOrigin:andcopyFromTexture:sourceSlice:sourceLevel:sourceOrigin:sourceSize:toBuffer:destinationOffset:destinationBytesPerRow:destinationBytesPerImage:methods.

Cube Array Textures

TheOSX_GPUFamily1_v1feature set adds support for cube array textures with theMTLTextureTypeCubeArraytexture type value. Similarly, support for this texture type has been added to the Metal shading language with thetexturecube_arraytype. The maximum length of a cube array texture is341(2048 divided by 6 cube faces).

Texture Usage

TheMTLTextureDescriptorclass contains the newusageproperty that allows you to declare how a texture will be used in your app. MultipleMTLTextureUsagevalues may be combined with a bitwise OR (|) if the texture will serve multiple uses over its lifetime. AMTLTextureobject can only be used in the ways specified by itsusagevalue(s) (an error will occur otherwise). TheMTLTextureUsageoptions and their intended use are described as follows:

• MTLTextureUsageShaderReadenables loading or sampling from the texture in any shader stage.

• MTLTextureUsageShaderWriteenables writing to the texture from compute shaders.

• MTLTextureUsageRenderTargetenables using this texture as a color, depth, or stencil render target in a render pass descriptor.

• MTLTextureUsagePixelFormatViewindicates that the texture will be used to create a new texture with thenewTextureViewWithPixelFormat:ornewTextureViewWithPixelFormat:textureType:levels:slices:methods.

Specifying and adhering to an appropriate texture usage allows Metal to optimize GPU operations for a given texture. For example—set the descriptor’susagevalue toMTLTextureUsageRenderTargetif you intend to use the resulting texture as a render target. This may significantly improve your app’s performance (with certain hardware).

If you don’t know what a texture will be used for, set the descriptor’susagevalue toMTLTextureUsageUnknown. This value allows your newly created texture to be used everywhere, but Metal will not be able to optimize its use in your app.

Important: The default usage value in theOSX_GPUFamily1_v1feature set isMTLTextureUsageShaderRead. In the iOS feature sets the default value isMTLTextureUsageUnknown, but you should always aim to determine and set specific texture usages. Do not rely onMTLTextureUsageUnknownfor the best performance.

Listing 10-7shows you how to create a texture with multiple uses.

Listing 10-7 Specifying a Texture’s Usage

Detailed Texture Views

TheMTLTextureprotocol adds the extendednewTextureViewWithPixelFormat:textureType:levels:slices:method that allows you to specify a new texture type, base level range, and base slice range for a new texture view (in addition to the pixel format parameter that was already supported by thenewTextureViewWithPixelFormat:method). Textures created with these texture view methods can now query their parent texture’s attributes with theparentTexture,parentRelativeLevel, andparentRelativeSliceproperties (in addition to querying other attributes already supported by thepixelFormatandtextureTypeproperties). For details on texture view creation restrictions, such as valid casting targets, seeMTLTexture Protocol Reference.

Note: Similar to querying parent texture attributes, a new texture created from a buffer with thenewTextureWithDescriptor:offset:bytesPerRow:method can now query its source buffer attributes with thebuffer,bufferOffset, andbufferBytesPerRowproperties.

IOSurface Support

TheOSX_GPUFamily1_v1feature set adds support for IOSurfaces. Use thenewTextureWithDescriptor:iosurface:plane:method to create a new texture from an existing IOSurface.

Render Additions

Render Command Encoder

There are several graphics API additions to theMTLRenderCommandEncoderclass. The main new features are summarized below:

• The methodsetStencilFrontReferenceValue:backReferenceValue:allows front-facing and back-facing primitives to use different stencil test reference values.

• Depth clipping is supported with theMTLDepthClipModeenum andsetDepthClipMode:method.

• Counting occlusion query is supported in theOSX_GPUFamily1_v1andiOS_GPUFamily3_v1feature sets with the newMTLVisibilityResultModeCountingvalue that can be passed into thesetVisibilityResultMode:offset:method.

• Texture barriers are supported in theOSX_GPUFamily1_v1feature set by calling thetextureBarriermethod between same-texture write and read operations.

• Base vertex and base instance values are supported in theOSX_GPUFamily1_v1andiOS_GPUFamily3_v1feature sets with thedrawPrimitives:vertexStart:vertexCount:instanceCount:baseInstance:anddrawIndexedPrimitives:indexCount:indexType:indexBuffer:indexBufferOffset:instanceCount:baseVertex:baseInstance:methods. Similarly, support for these drawing inputs has been added to the Metal shading language by providing the new[[ base_vertex ]]and[[ base_instance ]]vertex shader inputs.

• Indirect drawing is supported in theOSX_GPUFamily1_v1andiOS_GPUFamily3_v1feature sets with the argument structures listed inListing 10-8. Use these structs alongside thedrawPrimitives:indirectBuffer:indirectBufferOffset:anddrawIndexedPrimitives:indexType:indexBuffer:indexBufferOffset:indirectBuffer:indirectBufferOffset:methods, respectively.

Listing 10-8 Indirect Drawing Argument Structures

• Shader constant updates can now be performed more efficiently by setting a vertex buffer once and then simply updating its offset inside your draw loop, as shown inListing 10-9. If your app has a very small amount of constant data (tens of bytes) you can instead use thesetVertexBytes:length:atIndex:method so the Metal framework can manage the constant buffer for you.

Listing 10-9 Shader Constant Updates

Layered Rendering

TheOSX_GPUFamily1_v1feature set adds support for layered rendering with new APIs in theMTLRenderPassDescriptorandMTLRenderPipelineDescriptorclasses. Layered rendering enables a vertex shader to render each primitive to a layer of a texture array, cube texture, or 3D texture, specified by the vertex shader of its first vertex. For a 2D texture array or a cube texture, each slice is layer; for a 3D texture, each depth plane of pixels is a layer. Load and store actions apply to every layer of the render target.

To enable layered rendering, you must configure your render pass and render pipeline descriptors appropriately:

1. Set the value of therenderTargetArrayLengthproperty to specify the minimum number of layers available across all render targets. For example—set this value to6if you have a 2D texture array and a cube texture, each with a minimum of 6 layers.

2. Set the value of theinputPrimitiveTopologyproperty to specify the primitive type being rendered. For example—set this value toMTLPrimitiveTopologyClassTrianglefor cube-based shadow mapping. The full enum declaration containing all primitive type values is listed inListing 10-10

3. Additionally, the value ofsampleCountmust be 1 (this is the default value).

Listing 10-10 Primitive Topology Values

Compute Additions

There are also a few compute API additions to both existing and new classes, as summarized below:

• Indirect processing is supported in theOSX_GPUFamily1_v1andiOS_GPUFamily3_v1feature sets with the argument structure listed inListing 10-11. Use this struct alongside thedispatchThreadgroupsWithIndirectBuffer:indirectBufferOffset:threadsPerThreadgroup:method.

Listing 10-11 Indirect Processing Argument Structures

• The newMTLComputePipelineDescriptorclass specifies the compute configuration state used during a compute operation pass. This descriptor object is used to create aMTLComputePipelineStateobject. This new class is the compute counterpart to the previously-introducedMTLRenderPipelineDescriptorclass.

Supporting Frameworks

Metal provides two new frameworks that help you build Metal apps in a much easier and more powerful way.

MetalKit

The MetalKit framework provides a set of utility functions and classes that reduce the effort required to create a Metal app. MetalKit provides development support for three key areas:

• Texture loading helps your app easily and asynchronously load textures from a variety of sources. Common file formats such as PNG and JPEG are supported, as well as texture-specific formats such as KTX and PVR.

• Model handling provides Metal-specific functionality that makes it easy to interface with Model I/O assets. Use these highly-optimized functions and objects to transfer data efficiently between Model I/O meshes and Metal buffers.

• View management provides a standard implementation of a Metal view that drastically reduces the amount of code needed to create a graphics-rendering app.

The MetalKit framework is available in all Metal feature sets. To learn more about the MetalKit APIs, seeMetalKit Framework Reference.

Metal Performance Shaders

The Metal Performance Shaders framework provides highly-optimized compute and graphics shaders that are designed to integrate easily and efficiently into your Metal app.

Use the Metal Performance Shader classes to achieve optimal performance for all supported devices, without having to target or update your shader code for specific GPU families. Metal Performance Shader objects fit seamlessly into your Metal app and can be used with resource objects such as buffers and textures.

Common shaders provided by the Metal Performance Shader framework include:

• Gaussian blur.

• Image histogram.

• Sobel edge detection.

The Metal Performance Shaders framework is available in theiOS_GPUFamily2_v1,iOS_GPUFamily2_v2, andiOS_GPUFamily3_v1feature sets. To learn more about the Metal Performance Shaders APIs, seeMetal Performance Shaders Framework Reference.