The direct cooperation of humans and machines reaches an entirely new quality using human-robot interaction. In the future, employees in production processes will be able to determine the motion sequences of robots, which shall assist them in their tasks, in an independent, simple and precise way by applying the multi-modal strategy of interaction created by Fraunhofer IWU. Thus, robots can be programed intuitively via gestures and voice command without requiring any programing skills. Today, using robots in manufacturing still comes along with a high level of expenditure. Robot programers have to define the desired motion sequences using manufacturer-specific programing languages.

In the future, programing will become child’s play. In order to teach the robot the desired sequence of motions (teaching), SmartGesture makes it possible to track the motion path by simply using one’s finger (finger tracking) and fixing this path through a straightforward voice command. The robot can directly recognize these motion paths, thus demonstrating that it can exactly implement the desired motions. Programing via finger tracking and voice command can even take place remotely, i.e., in a location-independent manner. When the total motion path of the robot is determined, the system has even defined the safety zone: if a human comes too close to the robot, it slows down or brakes to a standstill. A teaching accuracy of approximately 0.5 cm is sufficient for many work tasks such as painting or polishing. This teaching accuracy can be achieved without additional means. If a higher accuracy is necessary, a fingertip attachment can considerably increase the level of precision.

Dr. Mohamad Bdiwi, head of the Cognitive Human-Machine Systems at Fraunhofer IWU, and his team of researchers include cobots as well as conventional industrial robots in the multi-modal human-robot interaction. Numerous application scenarios are possible not least because of combining intelligent path planning and creating safety zones in real time. Moreover, avoiding additional devices keeps the level of expenditure low. The only requirements comprise vision sensors with a RGB-D camera for detecting the finger and hand movements, and a voice recognition board. This setup opens up the potential for a wide range of applications. The objective lies in rendering robotics even more economic, which also makes it more interesting for smaller industrial production scenarios or the craft sector – right down to batch size 1. Mohamad Bdiwi: “Today, it is natural for desk and PC, notebook or tablet to go hand in hand. Why should not an easily programmable robot be part of every future workshop – or at least be a real option?” 

In particular, fascinating possible applications arise from combining robotics with an intelligent tool independent from the robot path. 

At automatica 2022, the Fraunhofer IWU robotics team demonstrates fast and high-quality robot-assisted painting of a piano keyboard cover with demanding geometry and sophisticated surface integrity. Without any user action, the RobFrame software and a digital twin optimize and validate the robot path in an environment for virtual commissioning to avoid collisions with the component to be painted or with other objects or persons. Additionally, the Dynarisk and SafeZone software packages ensure maximum safety with minimum space requirements. An AI solution assists in considering influencing variables such as the exact quality of the wood surface (grain, roughness), painting properties and light reflections of the wet paint (optical monitoring of the result) to achieve optimum painting results. “Using this approach, we decouple the robot path from the tool behavior that is equipped with its own sensors, which allows the tool to dispense the precise quantity of paint”, says Dr. Marcel Todtermuschke, head of the Business Unit for Agile Production Machines and Plants at Fraunhofer IWU. The module  XEIDANA® performs the final inspection of the painted surface as part of a 100% inspection, and also detects unknown errors in its capacity as an AI-based and adaptive tool. “We envision robots as indispensable assistants in the future craft sector, similar to journeymen”, explains Marcel Todtermuschke.

Saving time due to virtual commissioning

In addition to their provision of valuable input in optimizing robot paths, digital twins can also assist in the virtual commissioning of machines and plants. Especially when it comes to delays in the development process, virtual commissioning, e.g. using a simulation tower, helps ensure functions of machines or plants – while at the same time the hardware is being set up. This approach is possible even while working from home so no time is lost.

Tool changer: 3D printing with or without 3D printer

Fraunhofer IWU has a special asset to tackle the challenges of unstable global supply chains, dispersed production sites and scarce resources: the tool changer allowing for additive manufacturing of functional products with a mix of various materials – on one and the same machine. Four different tool heads can be used during the printing process due to the changing system in the prototype. The functionalized tool heads developed for the tool changer can also be employed using robot guidance in flexible manufacturing processes.

Safe and high-performance soft robotics for near-human tasks

Utilizing novel switchable stiffnesses in robotics, Fraunhofer IWU combines the required force with the necessary safety, thus closing the gap between high-performance industrial robotics and safe but inefficient soft robotics. The combination of supposedly contrary properties facilitates robots with unprecedented flexibility regarding possible applications and motions. At automatica 2022, the researchers present a first concept of a robot arm that will be able to assist humans in their direct environment, e.g. in near-human tasks in the fields of domestic work or nursing, but also in areas hard to access. In the future, such a robot could even carry out assembly or repair work in the household.