Hello. You have successfully accomplished the first week of our course, and we now go to the second one. Today, we will discuss decision-making based on model predictions, and we will discuss some interesting aspects of translating statistical quality metrics into financial metrics. In order to better understand your course roadmap, we remind you that on the first week, you discuss project valuation and basically financial terms which financial people use to come up with financial estimation. On the second week, we will cover model quality and its connection to the decision-making based on its predictions. Here we have some topics to discuss. The first one is simple threshold decisions. Well, when you have a model, you do not get the financial result in the business process automatically, you have to come up with a decision that you make based on the model prediction. After we will discuss benefit curve, which helps us to translate statistical ROC curve measures into financial effect. Then we will cover some connections between model quality and benefit curve and some different decision skims. After the second week, you will also have three weeks of this course where you will discuss estimating model risk discounts, A/B testing and unobservable one type error, which happens when we do not have representative data to make our estimates. Let's go with the week 2. The simple threshold decisions. What is that? Well, basically, you can consider a binary classification model, which as usual takes some X as input and has some probability of a target event on output. Well, the simplest example you can come up with is credit scoring. In this table, you can see some features or Xs and the final probability which model produces as the result of taking these Xs as input and producing the prediction. Everything is quite simple. When you have this probability of a target event, you can come up with some level a, that if probability is higher than this a, then some action is undertaken concerning this object or this observation or this client. We will denote this like Y with hat, and interval will be equal one and otherwise it will be equal null if no action is undertaken. For example, we can consider a equal 0.5 and if probability is higher than 0.5, then we reject the loan application. Another useful thing to discuss here is acceptance rate c. Well, it is due to the threshold level a, but this time it is a percentage of observations that satisfy this rule. It ranges from 0-1 or from zero percent to 100 percent. Basically in this small example, we have eight observations. You can see this eight rows in the table. We have threshold decision level 0.5, and we can calculate how many clients satisfy this decision rule. In our case, five out of eight satisfies. So five out of eight clients are going to be accepted and we will give a loan to them. The others will be rejected. This is as simple as it seems. If we have historical representative data, we can estimate model errors. Obviously, every model is wrong and mistakes the model makes are usually divided into two types. They are called the first and the second type. Actually they can be calculated given historical data for each threshold level a. See this table below, we have four situations, true positive, false negative, false positive, and true negative. In the first and the last case, we made the right decision. The first case, the client is the default one because target event equals 1 and we rejected his or her loan application. The second case is false negative because we provided a loan while the client was default client, so we lost our money. In the third case, we decided not to provide loan for this client while this client was actually credit worthy. We have some benefit foregone. We didn't get some interests rate from this client. The last case is also right decision because the client was creditworthy and we did not reject his or her loan application. It's quite easy. We can put these four types of situations into metrics. You have to be really familiar with that because we have some model predictions on the left and we have some observed target event in the columns. The client can be default or not default and model prediction may lead to rather accept or reject the loan. If we sum the total number of cases in the first column, we will have positive, which is denoted as P. It is the total number of default or bad clients. If we sum all the cases in the second column, we will get a negative number. It's the total number of creditworthy or good clients. I will remind you that the words positive and negative are not about the meaning of good or bad situation. Positive means that the model predicted the target event like one, and it is really one. In negative, the model predicted that the target event is zero and it is also zero. When we have calculated all those error types, we can come up with a financial result. How can we calculate this? Well, basically, we have two parts. The first is your income and the second is your loss. In case of credit scoring, the first part is called interest income, and the second is credit loss. Well, you have the formula on the slide which explains how the financial result is calculated. The first is number of accepted good clients. Basically we have a total number of creditworthy clients and we subtract the number of false positives from that. We multiply this by credit margin, which is the value that we get from the good client. After that, we'll have to subtract another term is credit loss. Basically it is the number of accepted bad clients, which we multiply by a loss given default, which means the amount that we are going to lose if we provide a loan to bad client. This formula gives us the financial results for our portfolio. Let's go to this slide and we have to say that LGD and margin are not universal error types values. In a most general case, they're called first type error cost and second type error costs. For credit scoring, they're just margin and LGD loss given default. As we can see from this formula, financial result is negative to false positive and false negative model errors, which is rather intuitive. Well, let's cover what we talked about in the first part. Financial result is negative to model prediction errors, which is quite expected and quite obvious. Two types of errors are false positive and false negative. They generally have different impact on financial result and we have to consider their costs.