In this lecture, we're going to talk about the the ML Surprise. By ML, or machine learning, what I mean is this process by which one computer writes a computer program to accomplish a task. The computer that is doing the writing figures out what the "best" program is by only looking at a set of examples. So, let's compare this to normal software engineering. Kind of this more typical approach, we have a human who analyzes the problem, writes a bunch of code, and then, this code becomes a program that can translate inputs to outputs. Maybe it's a calculator and it knows how to take two numbers and add them together. So, three and four produces seven. What happens in machine learning? Machine learning, we're kind of going to pull out the software engineer. And instead, we're going to use another computer that is only going to see many, many examples, many inputs paired with the desired output. And from these, that computer will figure out with the best "program" is to write. Now, obviously, this is not a technically correct fully mathematical academic definition of ML. That's fine. This is just going to give us the framework we need to have a conversation about ML in businesses for today's course. I want to talk about the broccoli surprise. It might seem like a tangent at first but it'll come back and it'll help us. So, when I was an undergrad, a new ice cream store opened up a couple doors down, and my friends and I decided to go check it out. We walk in, looks completely normal. They've got all the standard flavors. They've got mint, chocolate, peach. And then at the end, they've got this one that's the broccoli surprise. Being the great experimentalist I am, I have to try it. So, I asked for a sample. The lady gives me that tiny little scoop. It's white with green specks. It's sweet. It's creamy. It's richy. It's kind of just taste like vanilla. And so, I'm confused. There's no broccoli flavor here. So, I asked the lady, what's the surprise? And she says the surprise is there's no broccoli. With that in mind, I want to talk about the ML surprise, kind of leading the audience here. So, what I'm going to do is show you a couple of bar charts that portray how you would spend your effort in a variety of different tasks as you build a fully end-to-end ML system in your organization. We have things like defining the KPIs, what you should even be trying to accomplish, collecting the data, building the infrastructure, optimizing the ML algorithm itself, and then integrating with the rest of the preexisting systems at your organization. Now, very informally, but from many conversations I've had with new ML practitioners internally and with our external partners, I find most people really tend to focus just on optimizing the ML algorithm. They want to make sure they have the newest, coolest thing right out of the papers. They've tuned all the right hyperparameters. They have the right number of convolutional layers. A lot of very technical details about the ML. But when I look and I talk to practitioners inside Google that have had great success building these big systems, I find a very different story. In fact, what I find is that to optimize the ML algorithm takes a much smaller segment of effort than people expect. I've never found anyone who overestimated how hard it was going to be to get that data collection right in the first place. And we should really pay close attention to that data collection. And what I would say is infrastructure building, making sure that we can train our model many, many times and automatically and smoothly or making sure we can serve that model at scale to our end users. And in fact, these kind of more core, almost software tasks, end up really dominating how people spend their time and effort when they build these successful ML systems. And the final point is, once we get to ML, we have another advantage that everything about our users or operations are so well-measured that we can actually spend a little bit less time defining KPIs, maybe a little less organizational effort. Because it's no longer a theoretical approach. We're no longer relying on someone's intuition from a previous slide for some market research. We just measured everything we need to know about our users and this gives us great insights to figure out not what intermediate KPIs to use, which is how to get to the right, ultimate one, like customer lifetime value or net present value. So, why are we learning about ML if the surprise is ML is not so important? So, ML is great because the path we take to get to it is going to yield a lot of value all along the path. Maybe not every problem ends in ML but many will. And even those that don't will benefit from going down this journey.
