Virtual reality
and 3D scans

• Audi e-tron GT is the brand’s first model for which production planning was without any physical prototypes.
• Audi develops VR software for planning and workshops in a virtual space
• 3D scans use artificial intelligence and machine learning: 250,000 square meters (2,690,977.6 sq ft) of production space at Neckarsulm site already digitized

The Audi e-tron GT is the first four rings model for which production was planned entirely without physical prototypes. Multiple technical innovations made this possible, including three-dimensional building scans, machine learning processes and the use of virtual reality. All assembly processes, such as procedures and employee actions were tested and optimized in virtual spaces that model their real-world counterparts down to the finest detail. Virtual planning is now used across site boundaries, enabling digital, connected working without business trips or foreign assignments – and not just during the coronavirus pandemic. 3D scans and the planning in virtual spaces make processes more efficient and sustainable.

Where is virtual planning used and why are 3D scans so important for this?

A variety of prototypes are used in the conventional planning process for the production of a new Audi model. The vehicle prototypes are fabricated during the early planning phase as one-off models with hand-built parts. This is time consuming and cost intensive. Assembly Planning uses these prototypes to define and optimize the later production processes. What are the employee’s tasks? Where does a part have to be located for the employee to have optimal access to it? Can the employee hold and install the part by him- or herself? How does she have to move to do it? Are other parts in the way? What tools does he need? During production planning for the Audi e-tron GT, these questions were derived and answered entirely in the virtual world. Every step and every action was tested in the digital space using virtual reality. The goal of virtual planning is to ensure that during the later production of the vehicle, all processes are perfectly meshed and the cycles along the line are seamlessly coordinated. This requires that every detail of the production hall be modeled precisely and to scale. This is where 3D scans come into play. Using special hardware and software, they create a virtual reproduction of the physical production facility, including all equipment, tools and shelves. The Audi Böllinger Höfe at the Neckarsulm site, where the Audi e-tron GT is built, thus also exist in the digital world. And thanks to new, digital planning methods, the production of tomorrow can be planned virtually years in advance on the basis of this model.

How do 3D scans work and what role does artificial intelligence play?

A scanner – the hardware – is essential for generating the corresponding data. It is roughly two meters (6.6 ft) tall and is mounted on four wheels so that an employee can move it around the spaces. At the top is a LiDAR (Light Detection and Ranging) unit and three additional laser scanners as well as a camera. Two processes are conducted simultaneously while scanning a space: The wide-angle camera takes a picture of the space while the lasers precisely measure it and generate a three-dimensional point cloud of the surroundings. 250,000 square meters (2,690,977.6 sq ft) of production hall space at the Neckarsulm site alone has already been scanned using this technology. But it is the interaction between the hardware and software that takes the points, images and data sets generated and turns them into a usable overall image that can be used with the existing planning systems. The software used here is an in-house Audi development based on artificial intelligence and machine learning.

The point cloud and the photographs are combined to produce a three-dimensional, photo-realistic space similar to what is seen in Google Street View. Proportions and sizes are true to scale and correspond to reality. The software also automatically recognizes all objects, such as machines, shelves and systems in the space. It also learns automatically with each scan to recognize, distinguish and classify objects even more precisely. For example, the system distinguishes between a shelf and a steel beam. The position of the shelf can be changed in the program later and relocated in the virtual space. That of the steel beam cannot. These data enable a virtual walk-through of the scanned production facility from any starting point and can be used directly in planning processes.

Where is virtual planning with virtual reality already being used and what advantages does it offer?

The Audi e-tron GT is the brand’s first vehicle for which the assembly procedures and associated logistics processes were tested exclusively virtually and without any physical prototypes.

To do this, a holistic, virtual model of the planned assembly with vehicle data, material handling, equipment, tools and the planned processes is prepared as a so-called digital model. The 3D scans are one element of this. The digital model is the basis for further innovations, as Andrés Kohler, responsible for virtual assembly planning at Audi, explains. “Thanks to a VR solution developed here at Audi and the digital model, colleagues from all over the world can now meet in virtual spaces and find themselves in the middle of the production facility of tomorrow. They can look over the shoulders of digital workers as they perform the planned procedures. They can also experience and optimize the planned processes for any part variants in our application.” The results can then be used to train employees, also on the basis of the VR application.

These new possibilities are now being used in an increasing number of additional projects and at multiple sites. A 3P workshop (3P = Production Preparation Process), for example, was held at the Audi site in San José Chiapa, Mexico, and project team members from Ingolstadt also took part. As digital avatars and thus completely virtually, the experts discussed and planned the production of the Audi Q5 face lift and the new Q5 Sportback in VR.

All assembly procedures are jointly defined and tested in real time, as are ergonomic aspects or the exact arrangement of machines, shelves and parts along the assembly line. Audi is the Group lead for the development of the comprehensive VR solution including the digital model. The project is being continued across brand lines under the leadership of the four rings as a Group project and rolled out to more and more sites.

More than buildings and processes: How does virtual container planning work?

Virtual planning is not restricted to just processes and work procedures. Objects such as containers for the transport and storage of sensitive parts, called special load containers, can also be planned using this technology. These containers for individual, particularly sensitive parts of the Audi e-tron GT, such as electric modules or interior parts – were planned using Audi’s cross-site and cross-division virtually reality application rather than using multiple physical prototypes of iron and steel. Virtual container planning works like this: Since there are data sets for all parts, these can be loaded directly and to-scale in the VR application. As in the 3P workshops, multiple employees from different sites meet in a virtual space, where they use the part to check the perfect and tailored load carrier. Employees from Logistics, Assembly Planning, Occupational Safety, Quality Assurance, Material Flow Planning and also suppliers are involved in this process. They use digital pens to mark their changes on the virtual containers. The containers are loaded and unloaded, moved and measured during this process.

Optimal safety of the part during transport is one objective of this planning. But employees or a robot must also be able to easily grab the part and remove it from the load carrier. Once the virtual design is complete, the data are simply exported and the special load carrier manufactured.

What makes virtual planning so sustainable and ecological?

Sometimes less is more. There are thus three points that make virtual planning so sustainable:

• Fewer resources: The virtual planning of the Audi e-tron GT without physical prototypes saved not just time, but also materials and thus resources. It’s the same thing with the special load carriers and virtual container planning: Making prototypes of iron and steel requires resources and energy. Virtual planning makes this step superfluous in many cases.
• Less waste: Rather than in special load carriers, sensitive parts were often transported in universal load carriers with a custom protective lining. This protective lining is always purely disposable, however, and the use of custom containers makes it superfluous. Virtual planning thus directly reduces waste.
• Fewer business trips: Sustainability and environmental protection are the primary goal. But in times of the coronavirus pandemic, there are also health reasons for reducing the number of business trips as much as possible. Virtual planning is a major contributor here. Processes that once required a physical meeting are now possible in a virtual space.

What future perspectives does virtual planning open up?

The digital model is the basis for further possibilities in the virtual space. If you combine the possibilities of virtual planning including the digital model, 3D scans and the virtual reality application with those of 3D printing, 3P workshops could also be conducted in mixed reality in the future. Individual parts would then be produced immediately via a 3D printer and only slight amounts of resources. This enables the physical testing of individual elements in the virtual space, such as assessing the haptics and weight of the parts. A key step combining the advantages of both worlds. Virtual meetings and collaboration via avatars in the virtual world could increasingly replace foreign assignments and longer business trips. It is already possible today to use the spaces created by the 3D scanner for digital indoor navigation. And the positioning of machines and equipment in the space can be planned down to the centimeter using augmented reality.