Lego has worked extremely closely with Apple over the years, experimenting with unreleased iOS technology and showing it on stage at launch events like WWDC; That included some pretty heavy tinkering with the ARKit platform using augmented reality, which they incorporated several of their play sets into to add digital experiences to the physical toys.
But one of the most impressive integrations between iOS technology and physical Lego bricks just popped up on the App Store and was developed by a team of fans. The new Brickit app aims to surpass what the Lego group itself created with an app that uses computer vision technology to quickly understand a mountain of bricks.
All users have to do is randomly drop Legos on the floor in a single layer. From there, the app is able to quickly analyze and identify bricks in the collection and provide some fun little projects where users have all or most of the bricks to build. The most impressive element of the app is its speed – the app is capable of understanding hundreds of stones in a pile in seconds.
While unfortunately I don’t have access to a bunch of Legos right now, a Biomedarticles colleague demonstrated the app on iOS and had similar smooth results as the demo above, with a little extra load time between discovery and when users are able to scroll through proposed projects. In fact, as users navigate the directions, they are alerted to the area in the pile of bricks that has a particular piece of equipment they need.
What the Brickit team has done highlights the power of object recognition in the latest versions of iOS in a way that is surprisingly useful for this very, very niche use case.
However, the app is a bit limited as it is a third-party design. The App Store’s disclaimer page quickly makes it clear that this is not an app from the Lego Group and that its developers are only fans of the product, not employees of the company. Hopefully that’s enough to keep Lego from overzealously testing its lawyers, but given the impressive use of the app from Apple hardware, it really seems like the company is better off getting the app.
There’s a lot more Brickit could do with first-party access, mostly in terms of accessing integrations with existing libraries of Lego instructions. With Lego’s acquisition of BrickLink in 2019, it is clear that the company is aiming to gain more community fandom around aftermarket creations. Allowing the company to build a database of the actual bricks a user has in their possession and thus gain some insight into the collections of sets they own would undoubtedly be valuable data to Lego.
Currently, the Brickit app is limited to iOS, but the company’s website indicates that the team intends to have an Android app out and about by fall.