Apple has barely mentioned augmented or virtual reality in its big keynotes lately, however at WWDC 2023 earlier this month, the company quietly released probably one of the best 3D room-mapping tools for mobile AR yet.
Called RoomPlan, the ARKit Swift API uses the camera and LiDAR scanner on recent iPhones and iPads to create a 3D floor plan of a room, including key characteristics such as dimensions and types of furniture.
It’s not for consumers (yet) though. Apple says it’s aiming to appeal to professionals like architecture and interior designers for conceptual exploration and planning, as well as developers of real estate, e-commerce, or hospitality apps; developers can integrate RoomPlan directly into their AR-capable apps.
When it was released earlier this month, Jonathan Stephens, Chief Evangelist at spatial computing company EveryPoint, took RoomPlan for a test drive to see what it could do. The results are pretty surprising.
Follow along as I do a series of structured @Apple RoomPlan tests and share my findings/notes in this thread.
First up, I tried tricking RoomPlan with a large mirror. Surprisingly it wasn't fooled! Also, it was way off on french doors height.#WWDC22 #AR #ARKit #AI @Scobleizer pic.twitter.com/R4hJbO57Km
— Jonathan Stephens (@jonstephens85) June 7, 2023
RoomPlan seems to be able to deal with a number of traditionally difficult situations, including the mirror seen above, but also messy spaces, open and closed doors, windows, and generally complex architecture. Still, Stephens’ house isn’t just a bunch of cube-shaped rooms, so there’s a few bits that just didn’t match up.
Test #2 – vaulted ceilings. I noticed that the wall shapes have to be rectangular. It could not follow the slant angle of the ceiling line. This made parts of my walls much taller than in reality.
It did a great job at picking out the desks and bedroom furniture. pic.twitter.com/fbu5B9L3Ds
— Jonathan Stephens (@jonstephens85) June 7, 2023
Vaulted ceilings, wall openings, multifloor areas like you might find in foyers were all a bit too difficult for RoomPlan to correctly digest. Although not perfect, it seems to at least autocorrect to some degree based on some assumptions of how things might best fit together.
Here is probably the coolest find so far. When I look top down, the walls correct themselves based on some assumptions from Apple. pic.twitter.com/KblqeLYm5x
— Jonathan Stephens (@jonstephens85) June 7, 2023
RoomPlan isn’t just for app integrations though. Apple says it outputs in USD or USDZ file formats which include dimensions of each component recognized in the room, such as walls or cabinets, as well as the type of furniture detected.
If you’re looking to finetune the scan, dimensions and placement of each individual components can be adjusted when exported into various USDZ-compatible tools, such as Cinema 4D, Shapr3D, or AutoCAD, Apple says.
We’re still no closer to learning when the company plans to release its rumored mixed reality headset or its full-fledged AR glasses, however either AR or MR headset would need extremely robust space-mapping capabilities. Seeing Apple make these sorts of strides using its existent platforms certainly shows they’re on the right track.
If you haven’t been following along with the Apple rumor mill, check out some of the links below regarding the company’s mixed reality headset, codenamed N301:
What We (think we) Know About N301 Mixed Reality Headset
- N301 will be a precursor to a long-rumored, full-fledged AR device which may be named Apple Glass
- N301 may pack a dozen cameras for room-scale tracking, hand-tracking, eye-tracking, and passthrough AR
- N301 could cost around $3,000
- Supply chain rumors suggest it includes MicroOLED displays targeting pixel density of 3,000ppi
- “as early as 2023” launch dates for N301 have been reported in the past, however later reports suggest a delay to 2023
A special thanks to Hrafn Thorisson for pointing us to the news!
,
What We (think we) Know About N301 Mixed Reality Headset
- N301 will be a precursor to a long-rumored, full-fledged AR device which may be named Apple Glass
- N301 may pack a dozen cameras for room-scale tracking, hand-tracking, eye-tracking, and passthrough AR
- N301 could cost around $3,000
- Supply chain rumors suggest it includes MicroOLED displays targeting pixel density of 3,000ppi
- “as early as 2023” launch dates for N301 have been reported in the past, however later reports suggest a delay to 2023
,
What We (think we) Know About N301 Mixed Reality Headset
- N301 will be a precursor to a long-rumored, full-fledged AR device which may be named Apple Glass
- N301 may pack a dozen cameras for room-scale tracking, hand-tracking, eye-tracking, and passthrough AR
- N301 could cost around $3,000
- Supply chain rumors suggest it includes MicroOLED displays targeting pixel density of 3,000ppi
- “as early as 2023” launch dates for N301 have been reported in the past, however later reports suggest a delay to 2023
,