Our research group in the EP-KI team develops solutions at the interface of business administration and Artificial Intelligence (AI). In the new blog post, they look at the reliability of AI systems. When can Machine Learning be trusted?
© freepik
In the last blog post, we opened up the topic of »Explainability of Artificial Intelligence« (AI), which we want to continue in this blog post. »AI systems should not only be the best possible. Sometimes they should say ‘I have no idea what I’m doing here, don’t trust me.’ That’s going to be really important«, says Stanford University professor Herb Lin.
Author: Manu Somasagar Kamalakar
Artificial Intelligence (AI) and Machine Learning (ML) are being used in almost every field as the world is moving towards digitalization. AI is presently not what's to come, it is already here and it is playing a significant role in our day-to-day life. However, the question is to what extent we can trust these systems.
If we are given an accurate model, why don't we just trust the model and ignore why it made a certain decision? – »One problem is that a single metric, such as classification accuracy, is an incomplete description of most real-world tasks.« [1]. The human mind is generally curious and wants to understand the reason for the decision by the model.
Since the beginning of AI, its central idea has been based on a mathematical simulation of the human learning process [2]. However, even after more than 60 years after its emergence, its answers do not satisfy us 100 percent. Explainability does not have the same relevance for all types of problems (this will play a bigger role in future blogposts). For example, an ordinary person is fine with not knowing why Amazon suggests her a specific movie, or why she always receives advertisements related to some specific topics. On the other hand, after receiving a very unfavorable diagnosis about her health, she would be very interested in knowing why. Closely related to learning is the human desire to find meaning in the world.
Let's now assume that we get sick suddenly. We will start to think about what we have done differently. After getting sick several times we will realize that illnesses always occur after visiting aunt Maria. The answer given by a black-box ML model would lead us to conclude that we should no longer visit Aunt Maria. The ML model will ignore the other times we also got sick because they were a minority. It only considers the fact that most of the times we got sick when we went to the aunt's house. Therefore, once we announce a visit Aunt Maria, the ML model will tell us »You're going to get sick«, because the model training depends on the data it sees. The human mind, full of curiosity and continuous learning, will recognize that Aunt Maria always prepares her special raspberry cake and that the last time we ate that cake in a restaurant we too became ill. As a consequence/Consequently, the human mind will know that »What makes me sick is the cake since I probably suffer from an allergy to raspberries. Nonetheless, I can continue visiting my aunt«. Our life and ML are full of examples like this.
Even simple ML methods can give an indication of why a prediction turned out the way it did. This is illustrated below with an example. Let us imagine that we are interested in a model that allows a prediction about the presence or non-presence of a disease. To do this, we use two pieces of information: weight and age.
A simple procedure for such a classification problem is logistic regression. In general, we would use it to model the probability of the presence of the disease. However, a typical application of logistic regression is also classification. In our example the probabilities are set to 0 or 1, depending on the presence of the disease (1) or not (0). Then, the logistic regression assigns probabilities between 0 and 1 to new observations. Next, to set the outcome to 0 (no disease) or 1 (disease) a threshold value is chosen. More precisely, if the probability is greater than the threshold, the disease is predicted to be present, otherwise it is not.
Let us now assume that we have used logistic regression to train a model that describes the relationship between the presence of the disease and the two characteristics (weight and age) – also called as features. This relationship is determined by parameters that are obtained during training. In our example, we obtain the values 0.1370 for weight and -0.0001 for age. This can be interpreted as follows for the weight as an example: If the weight increases by 1kg, the chance of developing the disease increases by a factor of 1.1503 [3].
In addition, a hypothesis test can be used to determine which characters have a relevant influence on the presence of the disease in the first place. In our example, this is weight. The fact that weight has a stronger influence than age is also clear from Figure 1. It can be seen that the weight is higher in people with disease than in people without disease, while the age distribution is similar for sick and healthy people.
Therefore, it turns out that even simple machine learning methods can tell us much more about the context we are modeling than giving just a number such as accuracy. However, it is important to emphasize that more complex models are harder to understand.
As announced, this will not be the last contribution on the explainability of artificial intelligence. In the future, there will be one or more posts with more depth. We are looking forward to your questions and topic requests.
[2] Definition: What is Artificial Intelligence (AI)?
[3] Scheffner A., Meyners M. und Langohr K. (2002): Logistische Regression. Arbeitsbericht Nr. 34, TU Dortmund.
Fraunhofer Institute for Industrial Mathematics ITWM