A Sketch of the Industrial Metaverse
NVIDIA Omniverse™ car factory of the future with digital twins of machinery and car assemblies
Into the Industrial Metaverse
Digital manufacturers are positioned to play a big role in the metaverse. To the uninitiated, a metaverse is an immersive online 3D virtual environment. The term was first coined by Author Neal Stephenson in his 1992 science-fiction novel “Snow Crash,” which depicts a virtual reality-based successor to the Internet. Gaming companies like Robolox and Epic Games have developed some of the initial (closed) metaverses, but industrial companies are not far behind. NVIDIA has created their Omniverse™ platform which is used to build factories of the future, autonomous vehicle simulation, and robotics applications. Unity has expanded beyond gaming and into manufacturing applications to shape the future of work. The core technologies powering the metaverse are familiar to Industry 4.0 participants: digital twins, augmented and virtual reality, and blockchain.
Note: Each header is clickable to find a detailed history of the news and companies behind each technology.
Digital Twin
Digital twin technology is core to the Industrial Metaverse. The origins of the concept date back to 2002 with the idea that “a digital informational construct about a physical system could be created as an entity on its own. This digital information would be a ‘twin’ of the information that was embedded within the physical system itself and be linked with that physical system through the entire lifecycle of the system.” The information can serve as a replacement for physical resources like time, energy, and material so that the process is more efficient throughout the whole product lifecycle. All objects in the metaverse must encode their information in a way that a virtual world can interpret and render. As digital twins evolve the physical condition of the real-world operating asset can be translated into the metaverse to enable experts across the globe to seamlessly interact, analyze, and service products and machines.
NVIDIA
Rev Lebaredian is vice president of simulation technology and Omniverse engineering at Nvidia. He is in charge of creating the “metaverse for engineers” that Nvidia has been building for years with a common standard for representing 3D data around Pixar’s Universal Scene Description (USD) format. USD enables many companies — there are now more than 700 companies and 70,000 3D designers building in the Omniverse — to integrate their 3D creations into a common format and run simulations for things like “digital twins.” BMW is building a digital twin of a factory, fully simulating it in the Omniverse so it can fix design flaws and then build it in the real world.
Unity
According to Lange, SVP of AI at Unity, users in markets like robotics, autonomous driving, drones, agriculture technology, and more are building simulations containing environments, sensors, and models with million-square-foot warehouses, dozens of robots, and hundreds of sensors. With these simulations, they can test software against realistic virtual worlds, teach and train robot operators, or try physical integrations before real-world implementation. This is all faster, more cost-effective, and safer, taking place in the metaverse.
Additionally, AutoDesk and Altair are the leaders 3D computer aided design and simulation software, and RoboDK is “a powerful and cost-effective simulator for industrial robots and robot programming.”
Augmented and Virtual Reality
Extended reality (XR), a universal term inclusive to immersive learning technologies, enables humans to interact with digital twins within the metaverse. Augmented reality (AR) was first developed in 1968 by a Harvard computer scientist, but it has struggled to find commercial applications. German car company, BMW, first used AR in a 2008 advertisement for the BMW Mini allowing interaction with a 2-dimensional magazine ad. Since then, companies like Google have tried to incorporate the technology into wearable headsets. Google Glass did not gain adoption in the consumer markets but found new life on the factory floor at AGCO as late as 2017. Most recently, during the COVID-19 pandemic, executives have been using mixed reality headgear to “visit” their manufacturing facilities across the globe. Some say they are never going back to the old way of jet setting the world. This is a precursor to how the industrial metaverse will shape industry moving forward.
Microsoft HoloLens
From improved onboarding and upskilling of employees to increased operational efficiency to reduced errors and waste, mixed reality is transforming manufacturing.
Meta, formerly Facebook, Oculus / Haptic technology
Keller’s goal is to invent soft, lightweight haptic gloves that address both sides of the AR/VR interaction problem — helping the computer to accurately understand and reflect the wearer’s hand movements, and reproducing a range of complex, nuanced sensations for the wearer such as pressure, texture, and vibration to create the effect of feeling a virtual object with your hands. To succeed, these gloves would need to be stylish, comfortable, affordable, durable, and fully customizable. You’d be able to pair them with your VR headset for an immersive experience like playing in a concert or poker game in the metaverse, and — eventually — they’d work with your AR glasses too. Far more than simply a peripheral device, these gloves would make the virtual world tangible.
Taqtile Manifest®
Knowledge when and where you need it. Manifest® gives deskless workers instant access to step-by-step guidance from your most experienced technicians and trainers anywhere, anytime.
Schneider Electric EcoStruxure Augmented Operator
EcoStruxure Augmented Operator Advisor puts real-time information at your fingertips, whenever and wherever it is needed. Our custom application improves operational efficiency thanks to augmented reality, enabling operators to superimpose data and virtual objects onto a cabinet, a machine, or a plant.
Blockchain: Composability, Decentralization, Web3
Blockchain is the latest frontier technology that can decentralize industrial operations and open access to a wider community. A key feature of blockchains is that they are composable which “allows anyone in a network to take existing programs and adapt or build on top of them.” According to the latest GitHub Octoverse report, the software projects that build on top of other projects and reuse code from other communities are thriving. If we extended this concept to digital twins, it creates opportunities for engineers to design and build novel assemblies or sub-assemblies of critical equipment like engines. Lastly, Web3 technologies can coordinate the interaction within and across blockchains to match suppliers and consumers in the metaverse with native payment rails and smart contracts.
IOTA Industry Marketplace
The Industry Marketplace is a vendor and industry-neutral platform, automating the trading of physical and digital goods and services. Building on specifications developed by the Plattform Industrie 4.0 (Germany’s central network for the advancement of digital transformation in manufacturing), it combines distributed ledger technology, immutable audit logs and standardized, machine-readable contracts to accelerate industrial automation and enable the “economy of things”.
IoTeX MachineFi
With IoTeX 2.0, machines will be able to deliver data, services, and intelligence to humans, businesses, and other machines and receive payment in a peer-to-peer fashion. Each machine can set its own terms and pricing for its unique menu of services, as well as select which counterparties they wish to authorize to use their services. This concept will sprout an army of useful and/or autonomous machines that can service society in a decentralized and transparent fashion.
Opportunities within the Industrial Metaverse
How the industrial metaverse plays out is anyone’s guess. New applications and business models we cannot yet conceive of will appear. There are a few opportunities where industrial firms should consider adopting the metaverse even at this early stage of the technology:
- Launching product pre-order campaigns. In the electric vehicle space, upstarts like Rivian have waiting lists into 2023. What if they took deposits through a smart contract on the Ethereum blockchain which upon deposit minted an non-fungible token (NFT) of their future vehicle. The depositor could then drive their digital Rivian R1T within a metaverse such as Decentraland or The Sandbox while they await physical delivery of their vehicle. This could serve two purposes: drive increased hype and demand for the product and provide opportunities for early product feedback and defect identification.
- Luxury good provenance and secondhand sale. Companies such as StockX have made a multi billion dollar businesses verifying authenticity for secondhand goods. What if Nike minted their limited editions as NFTs and created a smart contract so that they receive a cut of each future sale of that exclusive product? Artists such as Beeple, receive 10% each time their NFT changes hands after the initial sale.
- Coordinating lights out factories. Autonomous production may be a reality soon. How will it be coordinated? Web3 users could indicate their interest through depositing funds in a smart contract which could be programmed to produce a batch of goods after certain quantity is reached. The data from the production process can be uploaded to the blockchain to provide real-time indicators of where the product is in production lifecycle. Suppliers to the factory could latch on to signals of poor machine health and use virtual reality technology to resolve any equipment downtime and receive a fee for their service.
Visual Inspection
How Corning Borrowed Gorilla Glass Tech To Make Covid Vaccine Vials
Acoustic Monitoring
Assembly Line
BMW uses Nvidia’s Omniverse to build state-of-the-art factories
BMW has standardized on a new technology unveiled by Nvidia, the Omniverse, to simulate every aspect of its manufacturing operations, in an effort to push the envelope on smart manufacturing. BMW has done this down to work order instructions for factory workers from 31 factories in its production network, reducing production planning time by 30%, the company said.
Product customizations dominate BMW’s product sales and production. They’re currently producing 2.5 million vehicles per year, and 99% of them are custom. BMW says that each production line can be quickly configured to produce any one of ten different cars, each with up to 100 options or more across ten models, giving customers up to 2,100 ways to configure a BMW. In addition, Nvidia Omniverse gives BMW the flexibility to reconfigure its factories quickly to accommodate new big model launches.
BMW succeeds with its product customization strategy because each system essential to production is synchronized on the Nvidia Omniverse platform. As a result, every step in customizing a given model reflects customer requirements and also be shared in real-time with each production team. In addition, BMW says real-time production monitoring data is used for benchmarking digital twin performance. With the digital twins of an entire factory, BMW engineers can quickly identify where and how each specific models’ production sequence can be improved. An example is how BMW uses digital humans and simulation to test new workflows for worker ergonomics and efficiency, training digital humans with data from real associates. They’re also doing the same with the robotics they have in place across plant floors today. Combining real-time production and process monitoring data with simulated results helps BMW’s engineers quickly identify areas for improvement, so quality, cost, and production efficiency goals keep getting achieved.
Unity moves robotics design and training to the metaverse
“The Unity Simulation Pro is the only product built from the ground up to deliver distributed rendering, enabling multiple graphics processing units (GPUs) to render the same Unity project or simulation environment simultaneously, either locally or in the private cloud,” the company said. This means multiple robots with tens, hundreds, or even thousands of sensors can be simulated faster than real time on Unity today.
According to Lange, users in markets like robotics, autonomous driving, drones, agriculture technology, and more are building simulations containing environments, sensors, and models with million-square-foot warehouses, dozens of robots, and hundreds of sensors. With these simulations, they can test software against realistic virtual worlds, teach and train robot operators, or try physical integrations before real-world implementation. This is all faster, more cost-effective, and safer, taking place in the metaverse.
“A more specific use case would be using Unity Simulation Pro to investigate collaborative mapping and mission planning for robotic systems in indoor and outdoor environments,” Lange said. He added that some users have built a simulated 4,000 square-foot building sitting within a larger forested area and are attempting to identify ways to map the environment using a combination of drones, off-road mobile robots, and walking robots. The company reports it has been working to enable creators to build and model the sensors and systems of mechatronic systems to run in simulations.
High-tech potato-grading line ups profits for Cornish grower
Another problem caused by the older line relates to packers now wanting more specific size grading, particularly for the Gemson and Jazzy salad varieties the farm grows. They also prefer the crop to be delivered stone free, which is a challenge because of the region’s light loam soils having a relatively high small stone content. With their maincrop varieties, such as Electra, this can be addressed in the field, but it is much more difficult to remove on the harvester when using the narrow 28mm webs that are required for the salad crops.
Using Ventilation Simulation to Increase the Performance of HVAC Systems
For the first time, HVAC engineers are able to explore the full design space for HVAC product designs, not just at the component level but the spatial (room) level where the products are installed. This reduces cost and time by avoiding the trial-and-error characteristics typically seen in physical prototyping.
Inside X’s Mission to Make Robots Boring
It’s research by Everyday Robots, a project of X, Alphabet’s self-styled “moonshot factory.” The cafe testing ground is one of dozens on the Google campus in Mountain View, California, where a small percentage of the company’s massive workforce has now returned to work. The project hopes to make robots useful, operating in the wild instead of controlled environments like factories. After years of development, Everyday Robots is finally sending its robots into the world—or at least out of the X headquarters building—to do actual work.
Surge Demand
Cities compete for the next gold rush in semiconductors and electric vehicles. Deere workers ratify a new six year contract ending a five week strike. Car manufacturers are still grappling with a dearth of semiconductors and are redesigning their chips to be generic.