ICAI: The Labs – AI for Autonomous Systems in the NL

In September, ICAI: The Labs is focused on AI for Autonomous Systems in the Netherlands. The EAISI AIMM lab and Delta lab each present their work and discuss challenges and developments made in this field.

The labs:

The EAISI AI-enabled Manufacturing and Maintenance (AIMM) Lab is a collaboration between Eindhoven University of Technology (TU/e), KMWE, Lely, Marel, and Nexperia. The lab’s goal is to improve decision-making in manufacturing and maintenance using artificial intelligence. AIMM Lab is based in Eindhoven.

The UvA and Bosch have agreed to extend their established research lab. Delta Lab 2 – the follow-up to the successful collaboration Delta Lab 1 – will focus on the use of artificial intelligence and machine learning for applications in computer vision, generative models and causal learning.

Program

12.00 (noon) Opening by YaSuei Cheng (ICAI)

12:05 Introduction of the EAISI AIMM lab by Geert-Jan van Houtum , followed by his presentation: “A Predictive Maintenance Concept for Geographically Dispersed Technical Systems”

12.25 Introduction of Delta Lab by Eric Nalisnick

12:30 Rajeev Verma presents “On the Calibration of Systems that Learn to Defer to Experts”

12.45 Discussion of what’s next in AI for Autonomous Systems

13:00 End

Abstracts

“A Predictive Maintenance Concept for Geographically Dispersed Technical Systems”

Thanks to IoT, it is nowadays possible to remotely monitor the health status of technical systems. This information can be used to come to a so-called predictive maintenance concept. Under such a concept, the aim is to replace a degrading component by a ready-for-use component just before a failure would occur. This can be done for components for which you can follow the degradation behavior or for which you can predict upcoming failures by some form of data analysis. For other components, you may only have information on the lifetime distribution and replacement decisions have to be taken based on the age of the component. A system has generally a mix of components: lifetimes are given for a first group of components, degradation processes for a second group of components, and data- based failure predictions for a third group of components. In addition, many systems are geographically dispersed and require an expensive visit of a service engineer for the execution of maintenance. Then maintenance actions for the various components have to be clustered in order to avoid high costs for engineer visits. We show how an appropriate predictive maintenance concept can be constructed for geographically dispersed technical systems. We will include a case at a manufacturer of agricultural high-tech equipment.

“On the Calibration of Systems that Learn to Defer to Experts”

Learning to defer (L2D) systems offer a promising solution to the problem of AI safety. For a given input, the system can defer the decision to a human if the human is more likely than the model to take the correct action. We study the calibration of L2D systems, investigating if the probabilities they output are sound. We find that Mozannar & Sontag’s (2020) multiclass framework is not calibrated with respect to expert correctness. Moreover, it is not even guaranteed to produce valid probabilities due to its parameterization being degenerate for this purpose. We propose an L2D system based on one-vs-all classifiers that is able to produce calibrated probabilities of expert correctness. Furthermore, our loss function is also a consistent surrogate for multiclass classification, like Mozannar & Sontag’s (2020). In addition to being calibrated, our model exhibits accuracy that is comparable to Mozannar & Sontag’s (2020) model (and often better) in tasks ranging from hate speech detection to galaxy classification to diagnosis of skin lesions.