Apple ARKit 2.0 & Short Summary of WWDC-2018

Next generation of ARKit

Apple announced at WWDC today the next generation of ARKit, which introduces a new multiperson mode for AR games and another mode that allows for virtual objects to be placed in an area and remain in place. ARKit 2.0 also comes with improved face tracking, 3D object detection, and realistic rendering.

Apple began WWDC talking all about the update it’s delivering to its augmented reality platform, introducing ARKit 2 with iOS 12, which brings following big tech upgrades.

With these massive updates, ARkit became a platform that allows developers to integrate shared experiences, persistent AR experiences tied to a specific location, object detection, and image tracking to make AR apps even more dynamic. Apple is also unveiling the Measure app for iOS, which uses AR to quickly gauge the size of real-world objects, as well as a new open file format with iOS 12, usdz, which is designed to more deeply integrate AR throughout iOS and make AR objects available across the ecosystem of Apple apps.

Object Detection and Tracking

ARKit 2 offers full 3D object detection, not just flat surface detection. ARKit 1.5 added support for 2D image detection, letting you trigger an AR experience based on 2D images like posters, artwork, or signs. ARKit 2 extends this support to offer full 2D image tracking, so you can incorporate movable objects like product boxes or magazines into your AR experiences. ARKit 2 also adds the ability to detect known 3D objects like sculptures, toys, or furniture.

Onstage, Lego showed how the new ARKit could be used to combine physical and digital worlds. In the demo, an existing Lego structure was scanned with an iPad, unlocking a digital overlay that allowed for dropping in characters, additional buildings, and objects like cars. Bubbles also appeared over characters and objects, indicating they could be tapped on to further gameplay and go on missions with up to four friends.

Apple also said that it has teamed up with Pixar to create a new AR format, called USDZ. The new format is a zero compression, unencrypted zip archive supported across iOS, so objects in AR can be shared and experienced in Safari, Mail, or anything else that’s part of the Apple ecosystem. Adobe also announced it will have native USDZ support in Creative Cloud.

Shared AR Experiences

The multiperson mode will allow two or more people to share data to see the same object or play the same game within a virtual environment. At Google I/O, we saw a new technology called Cloud Anchors, which matches up what a pair of smartphone users are seeing in the cloud to allow for multiplayer. It’s not clear how Apple is doing this, but what’s interesting about Google’s approach is that Cloud Anchors allows for multiplayer across platforms so Android and iOS users can pair up easily, as well.

Multiplayer is a big update for AR at Apple; the multiplayer addition will be big for gaming and Apple will certainly try to bring the functionality to other categories moving forward. These new features supposedly give a taste of what’s in store for Apple’s AR headset, coming as soon as 2020.

Persistent AR Experiences

Now your apps can provide AR experiences that persist between sessions, and can be resumed at a later time. Users can start an AR puzzle on a table and come back to it later in the same state, or tackle an interior decorating project over multiple days without having to start over.

The developer platform is based on technology that synthesized the relationship between the inertial sensors of Apple devices and their cameras to track their position in space. This positional tracking allows users to not only move around digital objects but fix those objects to physical places in an environment.

New app ‘Measure’

As part of the new AR update, Apple also introduced a new app called Measure, which allows for taking the dimensions of objects, measuring lines across surfaces, and detecting rectangles. When in the Measure app, simply hold it above an object, tap, and drag along a line to take a measurement.

The new app automatically provides the dimensions of objects like picture frames, posters, and signs, and can also show diagonal measurements, compute area and allow users to take a photo and share it with accurate dimensions right from their iPhone or iPad.

ARKit 2 and usdz support will be available this fall as part of a free software update for iPhone 6s and later, all iPad Pro models, iPad 5th generation and iPad 6th generation.

Short Summary of WWDC-2018 :

Leading iOS 12 are performance updates, as Apple dials back from its punishing feature push to build a leaner operating system that runs on every phone that can run iOS 11.
iOS 12 gets some “Time Well Spent” features, letting users more easily control notifications and screen time.
Siri acquires “shortcuts”, simple actions which can be launched verbally or with a button press, and chained together to produce mini-programs for power users.
Safari’s tracking prevention features have been upgraded: they’ll now lock Facebook out of tracking users through its comment boxes and like buttons, and prevent anyone from “fingerprinting” devices by looking at their configuration details.
From 2019, the Mac will be able to run iOS apps – four of which are launching this autumn, as Apple ports over News, Stocks, Home and Voice Memos.
Finally, the Mac. First off, the name. With Apple having followed a California location naming theme since it ran out of cats, the company’s gone from the mountains to the desert: macOS 10.14 is called Mojave.
The flagship consumer feature, Federighi demonstrates, is a dark mode. It makes your Mac dark. It looks good! It even changes the default desktop background from a daytime shot of the desert to a night-time one.

Apple ARKit-2.0, a Massive Update:

Apple has been delivering major AR advancements to developers at a rapid pace since ARKit shipped last fall, and this is already the third major release. ARKit gives developers the tools to create AR experiences that change the way iOS users see the world in both fun and productive ways.