iOS 11 introduces ARKit, a new framework that allows you to easily create unparalleled augmented reality experiences for iPhone and iPad. By blending digital objects and information with the environment around you, ARKit takes apps beyond the screen, freeing them to interact with the real world in entirely new ways.
What is ARKit?
ARKit allows developers to tap into the latest computer vision technologies to build detailed and compelling virtual content on top of real-world scenes for interactive gaming, immersive shopping experiences, industrial design and more.
ARKit is Apple’s new ‘Developer Platform‘, allowing programmers to build apps that blend the real world with augmented digital objects. ARKit runs on the Apple A9, A10, and A11 processors. These processors deliver breakthrough performance that enables fast scene understanding and lets you build detailed and compelling virtual content on top of real-world scenes. You can take advantage of the optimizations for ARKit in Metal, SceneKit, and third-party tools like Unity and Unreal Engine.
- Game Engines: Unity, Unreal
- X-Code Library: SceneKit, MetaAPI
- Devices: iPhone SE, iPhone 6s to iPhone 8Plus, iPhone X, iPad Pro, 9.7-inch iPad 2017
- Fast, stable motion tracking
- Plane estimation with basic boundaries
- Ambient lighting estimation
- Scale Estimations.
- SLAM tracking and sensor fusion
Since ARKit will be available on iPads and iPhones, Apple CEO Tim Cook claims it will represent the world’s largest AR platform. Compare it to Google ARCore for example, Google doesn’t include ARCore on every Android device yet; it only available for 5-6 devices till date.
Why is ARKit?
Augmented reality is huge, and it’s only getting bigger
- Apple has invested heavily in augmented reality in recent years, making several VR/AR company acquisitions including PrimeSense, FaceShift, and Metaio.
- Apple’s much-anticipated “iPhone X”, have 3D sensing capabilities but only in front camera, in future Apple may include same depth sensing capability in rare camera also.
- Basically, Apple is betting on developers wanting to build AR experiences for the millions of iOS users out there
With every ARKit-infused app, the digital objects were grounded in the real world. We could walk around them and interact, as long as we were holding an iOS device. It’s true, not every ARKit app demo was knock-your-socks-off amazing, but they all speak to a core change in iOS app experience. A breaking down of the wall between the virtual and real to reveal a new truth in gaming, entertaining, utility and digital life.
Features offered by ARKit
By using the Built-in camera, Processors, and Motion Sensors found in iOS devices, virtual content appears on top of real-world scenes, and users don’t need any special equipment to enjoy them because ARKit does the heavy lifting.
- Other AR software often requires some sort of physical tracking mechanism (like an ImageTarget Or QR code), which inevitably becomes a major piece of friction if you are trying to get anyone to use this stuff.
- Another incredible aspect of ARKit is how it handles lighting adjustments in real time
- ARKit comes with sophisticated motion tracking, which Apple called “fast and stable motion tracking.” This will ultimately allow for a more immersive experience, as objects will appear to be intrinsically part of the real space.
It will take some time to go through ARKit’s full potential, but Apple demoed a couple AR experiences created with the tech to show a bit of what’s possible.
In a demonstration of software produced by the new ARKit, the software identified a table surface and applied a virtual coffee cup properly scaled to the surface. Following a lamp’s addition to the surface, the lighting model adjusted dynamically as Federighi moved the lamp around the cup.
How ARKit Works
The valuable information available on Apple’s ARKit developer page sheds more light on this platform. we can categorize the working of ARKit in following aspects.
- Visual-Inertial Odometry (VIO): which will let ARKit track the world, and for this, it will fuse camera sensor data with CoreMotion data. According to Apple, “these two inputs allow the device to sense how it moves within a room with a high degree of accuracy, and without any additional calibration.”
- Scene Understanding and Lighting Estimation: ARKit analyses the scene based on the camera view, and then find “horizontal planes like tables and floors, and can track and place objects on smaller feature points as well.” The ARKit also relies on the camera sensor to figure out the amount of light in a room, and will then apply the correct light to the virtual objects.
- High-Performance Hardware and Rendering Optimizations: ARKit runs on the Apple A9 and A10 processors. These processors deliver breakthrough performance that enables fast scene understanding and lets you build detailed and compelling virtual content on top of real-world scenes. You can take advantage of the optimizations for ARKit in Metal, SceneKit, and third-party tools like Unity and Unreal Engine.
Supporting Hardware & Technology
ARKit takes advantage of the iPhone’s camera, motion sensors, and graphics processors to allow developers create apps and games that combine the real world with the virtual world. From developing to run AR app using ARKit following are the prerequisite:
To Run on the device:
- Operating System: iOS 11
- Hardware Support: A9, A10 or A11 chip Only.
These include: iPhone SE, iPhone 6s to iPhone 8 Plus, iPhone X, iPad Pro, 9.7-inch iPad 2017
- X-Code Version: Min. X-Code 9
- AR SDK: ARKit SDK
- Special Plugins: SceneKit or SpriteKit
- Game Engine: Unity 3D & Unity ARKit Plugin (Optional)
- Apple devices with only hardware with the higher than A9 processors able to use the augmented reality software. which means that the iPhone 6s plus families can use the technology.
- For iPads, the situation is a bit more restrictive with the 2017 iPad, and the iPad Pros the only compatible models.
- Apple like tech giant is now investing in and promoting AR validates the market that other AR SDK like Vuforia, Wikitude has been working hard to create for nearly a decade.
Without a doubt, Apple’s announcement will contribute to boosting AR usage to an unprecedented worldwide scale and we are looking forward to what lays ahead.
With the release of ARKit to developers, Apple is bringing augmented reality to the masses on iPhones, iPads. In comparison, Microsoft’s HoloLens is still a far more expensive. Rival Google’s ARCore, a specialized AR platform, is available on a handful of devices. Apple isn’t even asking to invest in a special hardware to experience augmented reality apps instead, it is pushing developers to create the apps.
With this release, Apple has officially joined the rising AR scene, alongside Google and Facebook, which have also unveiled augmented reality tools of their own. This means that the large user base of iPhone users, all with access to cutting-edge augmented reality software, means it’s inevitable that we’re going to some interesting iOS exclusives in the coming months.