June 7, 2017, 5:33 a.m.

iOS 11 ARKit Tutorial (with Demo Project)

by farice

Apple has just announced their developer platform for augmented reality, ARKit. Users will be able to get their hands on these apps when iOS 11 is released in the Fall. The good news is, for us developers, we can get our hands on the new tools now.

I made a straightforward AR first-person shooter, using ARKit and SceneKit, and added the project to my GitHub. If you get lost or you don't have much time, it may be useful to simply clone this project and add on to what already exists. This guide will cover all of the steps to make an app like this, and hopefully will provide enough insight for you to get started on your own iOS AR project. This GIF demonstrates what it looks like in action.

Getting Started

Requirements:

  • Xcode 9 beta

  • iOS device with iOS 11 beta installed

You can download these here (as long as you are part of the Apple Developer Program).

To begin, create a new project in Xcode 9 (single view app) and name it anything that suits you. Open up Main.storyboard and drag in an ARKit Scenekit View to cover up your single view controller. You may need to drag the edges or set constraints so that the view covers the entire screen. Next, ctrl + drag that added View to ViewController.swift so that it's a class variable. I named it sceneView. That's it for the storyboard!

Note: In your Info.plist open as source and add the following information to give your app permission to use the camera:

<key>NSCameraUsageDescription</key>
<string>This application will use the camera for Augmented Reality.</string>

ARKit

Now on to the good stuff. Open up ViewController.swift and import ARKit. Also, designate your view controller as a ARSCNViewDelegate and SCNPhysicsContactDelegate. We'll need the first subclass in order to do anything with our ARSCNView. The second is an optional subclass that's important to our FPS so we can understand when our "bullets" hit our "ships". All together, you should have a standard setup that looks something like this:

//  ViewController.swift

import UIKit
import ARKit

class ViewController: UIViewController, ARSCNViewDelegate, SCNPhysicsContactDelegate {

    @IBOutlet weak var sceneView: ARSCNView!

    override func viewDidLoad() {
        super.viewDidLoad()
        // Do any additional setup after loading the view, typically from a nib.
    }

}

Now, in viewDidLoad() let's tell our ARSCNView that this class is its delegate

// Set the view's delegate
sceneView.delegate = self

// Show statistics such as fps and timing information
sceneView.showsStatistics = true

// Create a new (and empty) scene
let scene = SCNScene()

// Set the scene to the view
sceneView.scene = scene
sceneView.scene.physicsWorld.contactDelegate = self

And in viewWillAppear() let's configure the session

// Create a session configuration
let configuration = ARWorldTrackingSessionConfiguration()
configuration.planeDetection = ARWorldTrackingSessionConfiguration.PlaneDetection.horizontal

// Run the view's session
sceneView.session.run(configuration)

Checkpoint 1: Okay great! If you build and run on your device you should get something like a camera viewer with the frame rate at the bottom. Not very exciting!

Let's add a floating cube. To do that, we can create a separate class (in the same file or in a separate one, your choice). It's worth mentioning that ARKit is compatible with SpiteKit, Unity, Unreal and SceneKit. All of these serve the purpose of developing 2D and 3D graphics. However, SceneKit is remarkably simple so I chose it to develop the basic 3D graphics that this app uses.

Hence, because we're using SceneKit, our ship will be a subclass of SCNNode. We'll want to give it a simple box shape so it'll have SCNBox geometry. Furthermore, we give it a physics body so we can register contact in the future.

import UIKit
import SceneKit

class Ship: SCNNode {
    override init() {
        super.init()
        let box = SCNBox(width: 0.1, height: 0.1, length: 0.1, chamferRadius: 0)
        self.geometry = box
        let shape = SCNPhysicsShape(geometry: box, options: nil)
        self.physicsBody = SCNPhysicsBody(type: .dynamic, shape: shape)
        self.physicsBody?.isAffectedByGravity = false
        self.physicsBody?.categoryBitMask = CollisionCategory.ship.rawValue // See below for what CollisionCategory is! 
        self.physicsBody?.contactTestBitMask = CollisionCategory.bullets.rawValue
     }
}

You'll notice that your code no longer compiles because there's some mysterious struct called CollisionCategory that I popped in. Well, earlier than we might have expected, it's time to understand collisions.

A Brief Bit Mask Excursion

Let's talk about collisions for a moment. It's important that our app understands when bullets hit ships in order to respond accordingly. Overall, we're fine with "physics" occurring when bullets strike bullets or ships strike ships. In other words, bullets should still bounce off one another and ships the same. However, we want our delegate to only handle ship-bullet collisions in order to trigger an explosion (recall SCNPhysicsContactDelegate from earlier). Hence, we could create a table like this:

//  Contact should be delegated by SceneKit? T/F
       Bullet | Ship
Bullet  False | True
Ship     True | False

Bit masks provide a straightforward way to record a table like this. Let's talk about integers for a moment. Generally, integers are 4 bytes, but we'll pretend they're just a byte without loss of generality. Hence, if we were to give each type of node a unique integer identifier it'd look like just a chain of bits. So, let's say our first type of node has the identifier 1. At the bit level this looks like:

0 0 0 0 0 0 0 1

Hence, let's give our second type of node an identifier with no overlap to the second:

0 0 0 0 0 0 1 0

Well, if we do a bitwise AND between these too identifiers we get a chain of zeros across the board. Interesting. Well, it turns out that SceneKit gives each node a categoryBitMask and when it strikes another type of node it does a bitwise AND with the second node's contactTestBitMask to determine if a collision should be delegated. If the result has any bits that aren't zeroes, a "contact" is recorded and handled.

Hence, let's consider a struct with the same idea in mind (put this struct in a separate file or below one of your classes):

struct CollisionCategory: OptionSet {
    let rawValue: Int

    static let bullets  = CollisionCategory(rawValue: 1 << 0)
    static let ship = CollisionCategory(rawValue: 1 << 1)
}

The exact same two identifiers as shown above. Now, let's refer back to our earlier code in Ship.swift.

self.physicsBody?.categoryBitMask = CollisionCategory.ship.rawValue
self.physicsBody?.contactTestBitMask = CollisionCategory.bullets.rawValue

Now, imagine two ships hit one another: bitwise AND is performed between (1 << 0) & (1 << 1) == 0. So, "contact" is not observed. Assuming that we give our bullets the category mask 1 << 0, we'll then have bullets not contacting one another and bullets contacting ships.

Note the distinction between contactTestBitMask and collisionBitMask. The former decides whether two physics bodies coming in contact should be delegated. The latter decides whether a physical, visual collision should occur.

Let this all sink in for a bit. Once you have many categories of nodes with various collision behaviors, understanding all of this will make your life much easier.

Bonus: Because two's complement is the universal signed integer representation, if you set all of the contact bit masks to -1 (all the bits are 1), then all collisions will be delegated. In fact, the default value is -1 and by virtue of the fact that we haven't changed the collisionBitMask, all collisions are occurring (but not all are delegated)

Back to the ViewController

Okay, now that we have our Ship class together, let's make one appear on the screen. In viewDidLoad, add the function call self.addNewShip() with that method defined as

func addNewShip() {
    let cubeNode = Ship()

    let posX = floatBetween(-0.5, and: 0.5)
    let posY = floatBetween(-0.5, and: 0.5  )
    cubeNode.position = SCNVector3(posX, posY, -1) // SceneKit/AR coordinates are in meters
    sceneView.scene.rootNode.addChildNode(cubeNode)
}

Well, we want the ship to appear the same distance from the user each time, but in a random position in the plane 1 meter ahead of the user. So, all that the above method does is create a ship, place it as specified, then add the node to our sceneView. Hence, floatBetween is simply a method to return a random Float in some range

func floatBetween(_ first: Float,  and second: Float) -> Float {
    return (Float(arc4random()) / Float(UInt32.max)) * (first - second) + second
}

Checkpoint 2: Nice, now build again and you should see a cube appear 1m in front of you. Pan around to admire the AR you've leveraged--it's genuinely cool!

Now let's shoot that ship down:

Return to the storyboard and add a tap gesture recognizer to the view controller. ctrl + drag in to add an action (no outlet necessary) to ViewController.swift. This will let us know when the user is attempting to fire a bullet. We'll fill this action as follows

@IBAction func didTapScreen(_ sender: UITapGestureRecognizer) { // fire bullet
    let bulletsNode = Bullet()
    bulletsNode.position = SCNVector3(0, 0, -0.2) // SceneKit/AR coordinates are in meters

    let bulletDirection = self.getUserDirection()
    bulletsNode.physicsBody?.applyForce(bulletDirection, asImpulse: true)
    sceneView.scene.rootNode.addChildNode(bulletsNode)

}

Evidently, we need to define what a Bullet is. The above method looks simple enough though. We're creating a node, placing it right in front of the camera, and then applying a force in the direction the user has their camera facing. Let's define the Bullet class, which looks awfully similar to the Ship one

class Bullet: SCNNode {
    override init () {
        super.init()
        let sphere = SCNSphere(radius: 0.025)
        self.geometry = sphere
        let shape = SCNPhysicsShape(geometry: sphere, options: nil)
        self.physicsBody = SCNPhysicsBody(type: .dynamic, shape: shape)
        self.physicsBody?.isAffectedByGravity = false
        self.physicsBody?.categoryBitMask = CollisionCategory.bullets.rawValue
        self.physicsBody?.contactTestBitMask = CollisionCategory.ship.rawValue
    }
    }

We still need the method for getting the user's direction

func getUserDirection() -> SCNVector3 {
    if let frame = self.sceneView.session.currentFrame {
    let mat = SCNMatrix4FromMat4(frame.camera.transform)
        return SCNVector3(-1 * mat.m31, -1 * mat.m32, -1 * mat.m33)
    }
    return SCNVector3(0, 0, -1)
}

Checkpoint 3: If we build now, we can shoot bullets (spheres, really) at our ships (cubes, really). However, we note that the spheres will simply bounce off of the cubes (and send the cubes flying which is pretty cool).

We need some destruction! Here's where we implement the contact delegate we mentioned at the beginning.

// MARK: - Contact Delegate

func physicsWorld(_ world: SCNPhysicsWorld, didBegin contact: SCNPhysicsContact) {
        contact.nodeA.removeFromParentNode()
        contact.nodeB.removeFromParentNode()
        print("Hit ship!")
        self.addNewShip()

}

Because we prevented bullet-bullet and ship-ship collisions, we know that all contact is the result of a bullet-ship collision. Note that we'd need a more nuanced function if this wasn't the case. However, because the app is as simple as it is, we can simply remove the two nodes and call our addNewShip method. Ta-da!

Final Checkpoint: Run your app and enjoy shooting the cubes! It's actually pretty exciting. Feel free to extend this into a genuine game.

Well, as free as the Apache license allows anyway.

Conclusion

This concludes my short guide on Apple's new ARKit. I hope you enjoyed it and learned something too! Comment below if you have any questions.

This guide is still a work in progress. This means there are no plans to add more code, but there are plans to add more explanations to clarify things further. Thanks for your support.


Add comment
June 7, 2017, 9:59 a.m.
farice

Let me know if I can help in any way!

June 27, 2017, 11:49 a.m.
satish

Hi There, it was helpful.
I am just starting with ARKit and your guide is helpful to me.

It will be great if you can help me some more.
Actually i am trying to calculate the height of the actual object, how can i use ARKit and other new features for the same?

Thanks in advance.

July 2, 2017, 5:59 p.m.
PaulProg

Where does the Apache license come into play? SceneKit?

Oct. 8, 2017, 10:55 a.m.
reelmark

hi,

what is your email address?

regards,
facorp