Fraunhofer IWU

Cognitive Teaming of Human and Cyber-Physical Production Systems: Fraunhofer IWU and TU Chemnitz pave the way for "human-centered" Industry 4.0

How do humans work together with machines? How can digital assistants support employees in the factory without overwhelming them with their complexity? How can technology designed around the needs and strengths of humans help them make their creativity more valuable? Or contribute to preserving valuable experiential knowledge of older employees and inspiring them for innovative production technology? Dr. habil. Franziska Bocklisch and her new group "Cognitive Teaming of Human and Cyber-Physical Production Systems" are investigating these questions at Fraunhofer IWU. They are assured that if the coexistence of humans and technology evolves into a true collaboration - a teaming - producing companies can still achieve significant efficiency potentials.

© Fraunhofer IWU
Franziska Bocklisch and Antje Ahrens (with eye tracker) during robot roll forming: Analyzing which process steps experienced employees pay particular attention to helps automate the process sensibly and preserve valuable experiential knowledge
© Professur Werkstoff- und Oberflächentechnik, TU Chemnitz
Operator monitoring and controlling a plasma coating system: High-resolution data is used for process modeling with transparent AI methods. This creates a cyber-partner of humans in complex decision-making. The heatmap visualizes the operator's gaze duration (green = short, red = long).

With Franziska Bocklisch, Fraunhofer IWU gets stronger in a field that is at first glance unusual for a production technology institute: cognitive psychology. It deals with aspects of human thinking and behavior such as perception, attention, memory, and decision-making. She investigates how sensory information is processed and turned into units of knowledge and how this expert knowledge influences the interpretation of information and the subsequent decision-making. Cognitive psychology also explores how humans solve complex problems thanks to creative strategies and can meaningfully reduce complexity. A presumed weakness of humans when facing a threatening overload is actually a great strength - they assess the situation, put it into a larger context, and draw on experiential knowledge: what helped me successfully handle a seemingly unsolvable task in a comparable previous situation? An important application area for the gained knowledge lies in the modern production technology. How the interaction between humans and technology can be made even more value-creating in the future is one of the priorities of the new research group.

Mutual coaching of humans and technology

Many innovations in robotics, artificial intelligence (AI), data analytics, and visualization technologies are shaping the modern industrial production (known as "Industry 4.0"). AI is precise and repetitively accurate - thanks to impressive computing power, it can handle massive amounts of data. However, these efficient assistance systems have sometimes increased the complexity of human work in production. The claim that technology should optimally support humans and enable them more freedom for value-creating creativity is therefore not yet fully realized.

The new team at Fraunhofer IWU still relies on the combination of the respective strengths of humans and technology (complementarity) and emphasizes the team concept as a prerequisite for further quality improvement in cooperation. Two essential characteristics of teamwork are shared knowledge and common goals. Easy-to-understand AI algorithms that fit the structure of human expertise and the approach of experts in a particular field can become true "cyber-assistants". In a kind of mutual coaching, technology "tells" humans how to solve a task even better, such as by relying on well-structured, relevant data provided by an assistance system. Conversely, employees could make an AI solution (that is not aware of all decision options) more stable. For instance, if they understand its functioning and create suitable decision-making foundations, the AI can "learn" in a new training loop based on knowledge and data.

Thinking of technology from a human perspective

If one takes the claim of thinking of assistance systems from a human perspective seriously, the core question should not be what is technically possible, but what the human can process nicely, and which solution provides actual assistance? To come back to the example of AI: for large quantities, the setup and adjustment effort of AI solutions is in an appropriate relationship to the support provided to employees. If the products change frequently, the maintenance effort is relatively high in comparison. Sometimes too high if a fallback solution for system failure is also needed. Experienced decision-makers and users know that even the best AI solution is not free of errors; those who understand its functioning can use it even more effectively.

Making human experience knowledge available

Too complex systems make education and training of new employees difficult. Sensibly developed and used, intelligent systems, on the other hand, support the securing of competencies and the transfer of experience knowledge. For instance, initial research results on the observation of manufacturing processes by experienced employees are promising. The evaluation enables precise inquiries and process descriptions of much higher quality, especially when it comes to training new employees about the priorities in their tasks. The researchers at Fraunhofer IWU have also observed and at various levels systematically described the robot roll forming, an incremental forming process of sheet metal and suitable for small quantities. An eye tracker followed the technical experts' vision and transferred it to a tablet (see Fig.1). A recording and detailed analysis of human gaze data then enabled precise inquiries about why a particular process section was executed and why certain aspects received (special) attention. This approach provides important starting points for creating cognitive assistance systems and target-oriented automation solutions.

Achieving common goals and gaining shared knowledge

In another research project at TU Chemnitz, the research group "Human-Cyber-Physical Systems" investigated how humans and AI can achieve common goals and acquire shared knowledge in a thermal coating process, so that the surface quality is appropriate and as little material as possible is consumed. The numerous adjustment options of a complex system and the diverse quality goals, which sometimes contradict each other, were modeled by the team in accordance with human expertise. The human decision-making process is considered, complemented precisely, and not replaced in this approach. Since the AI modeling can provide humans with a good understanding of complex patterns in the high-resolution technical process data, the first essential step towards "teaming partner" has already been taken (see Fig. 2).

It is important to firmly anchor the teaming concept for a better cooperation between humans and technology. Cognitive psychology expertise is rarely available in practical operations, it does not need to be, as long as science and industry work together in a transdisciplinary and cross-institutional manner, and for example, students are sensitized to design technology more from the perspective of humans. The Fraunhofer IWU and the Faculty of Mechanical Engineering at TU Chemnitz have been working successfully together in numerous research projects for many years. In the spirit of deepened cooperation, Franziska Bocklisch continues to lead the "Human-Cyber-Physical Systems" group at the Chair of Materials and Surface Engineering and contributes to the profile line "Human-Machine Teaming" of the Faculty of Mechanical Engineering. Advancing a "real" teaming of humans and production systems in basic and application-oriented research and preparing students as best as possible for this field of work is a shared motivation of both research institutions.

Because broad acceptance of innovative technical solutions ensures their value-adding utilization - and sustainable added value. Franziska Bocklisch: "In technical development as well as in operational implementation, the human with his expertise and needs should be at the forefront. Our approach is to adapt cyber-physical systems to human cognitive abilities - and not the other way around".

About the person:

Dr. habil Franziska Bocklisch has been leading the new group "Cognitive Teaming of Human and Cyber-Physical Production Systems" in the "Human in Production" department at Fraunhofer IWU since April 1, 2023. Its development is supported by the Fraunhofer funding program. “Attract" offers outstanding external scientists the opportunity to bring their ideas close to the market within the Fraunhofer Institute. At the Chair of Materials and Surface Engineering at TU Chemnitz, Franziska Bocklisch also leads the "Human-Cyber-Physical Systems" group, where she mainly conducts fundamental research on human-machine teaming.