Hello, my name is Gwendolyn Stripling and I am a Machine Learning technical curriculum developer here at Google Cloud. Welcome to our Introduction to Feature Engineering course. In this video, I will provide an overview of feature engineering so that we can baseline some concepts. Feature engineering is the fourth course of the Machine Learning on Google Cloud Specialization. In this lesson, our objectives are to; Help you understand the definition, purpose, and process of feature engineering. Understand what makes a good feature, feature engineered numerical, categorical, geospatial, and temporal features, as well as create other types of features such as feature crosses and hashed and bucketize features. We're also going to look at how to improve the performance of a BQML and Keras model using feature engineering. Predictive models are constructed using supervised learning algorithms where classification or regression models are trained on historical data to predict feature outcomes. Feature engineering is a crucial step in the process of predictive modeling. It involves the transformation of a given feature with the objective of reducing the modeling error for a given target. The underlying representation of the data is crucial for the learning algorithm to work effectively. The training data used in machine learning can often be enhanced by extraction of features from the raw data collected. In the most cases, appropriate transformation of data is an essential prerequisite step before a model construction. Feature engineering can be defined as a process that attempts to create additional, relevant features from the existing raw features in the data and to increase the predictive power of the learning algorithm. It can also be defined as the process of combining domain knowledge, intuition, and data science skill sets to create features that make the models trained faster and provide more accurate predictions. Machine Learning models such as neural networks, accept a feature vector and provide a prediction. These models learn in a supervised fashion where a set of feature vectors with expected output is provided. From a practical perspective, many Machine Learning models must represent the feature has real numbered vectors because the feature values must be multiplied by the model weights. In some cases, the data is raw and must be transformed into feature vectors. Features, the columns of your data frame are key in existing Machine Learning models to learn. Better features results in faster training and more accurate predictions. Has the diagram shows feature columns are input into the model. Not has raw data, but has feature columns. Engineering new features from a provided feature set is a common practice. Such engineered features will either augment or replace portions of the existing feature vector. These engineered features are essentially calculated fields based on the values of the other features. As you will see later in the labs, feature vectors can be numerical, categorical, bucketize, crossed, and hashed. Engineering features is primarily a manual, time consuming task. The process involves brainstorming where you delve into the problem. What does it mean to delve into the problem? Well, if your goal is to solve a problem where you need to predict an outcome and get the best possible results from a predictive model, you need to determine if your source data can aid in this goal. In other words, how can you get the most out of your data for predictive modeling? This is the problem that the process and practice of feature engineering solves. You start by looking at a lot of data. If you work as an academic researcher, you may have access to a plethora of data sources from government data to industry data. If you work in a corporation, your data sources are the data your organization uses to manage the business. Well, for example, you may be looking at customer data, sales data, product data, inventory data, operational data, and people management data. This data could be in different formats. The customer data could be in a CSV file. The sales data could come from a JSON file. The operational data could be in an XML format. I think you get the picture. Depending on your problem and your domain you then need to leverage your domain specific engineered features. For example, if you're machine learning problem, is to predict seasonal sales, based on past customer purchases in a geographic location. Then you must either have domain specific knowledge or find domain-specific features to solve the problem. In this case, you may be looking for data that allows you to create features for customer purchases at a specific date and time, and in a certain geolocation or spatial region. Studying other feature engineering problems, is highly recommended. The example of seasonal sales, for example, lends itself to time, series, features and longitude and latitude features. You then must devise your features. This can be done in several ways. For example, in manual feature extraction, features are constructed manually. In this lesson, we use manual feature extraction, where we manually device features using Python and TensorFlow code libraries. Feature extraction is a process of dimensionality reduction by which an initial set of raw data is reduced to more manageable groups for processing. For example, principal component analysis, or PCA, is a way to reduce the number of variables or features in your dataset. As the name states, you are simply analyzing the principal components, your independent variables or features, to determine how well they predict your dependent variable or target, which is the final output you are trying to predict. In technical terms, you want to reduce the dimension of your feature space. By reducing the dimension of your feature space, you have fewer relationships between variables to consider and you are less likely to over-fit your model. In automatic feature extraction, the idea is to automatically learn, a set of features from potentially noisy raw data, that can be useful in supervised learning tasks, such as in computer vision. You can also device features using a combination of both automatic and manual methods. In this example here, the feature columns derived from this process can include, numerical, categorical, bucketized, crossed, embedding, and hashed feature columns. There's no well-defined basis for performing effective feature engineering. It involves domain knowledge, intuition, and most of all, a lengthy process of trial and error. The key learning is that different problems in the same domain, may need different features. It depends on you and your subject matter expertise, to determine which fields you want to start with for your hypothesis or your problem. Recall that feature engineering can be defined as the process of transforming raw data into features that are input into the final model. However, feature engineering in reality is an iterative process. For example, you can improve the accuracy of models by increasing the predictive power of learning algorithms through iteration. In other words, you create a baseline model with little to no feature engineering, as determined by your datatypes. Then add feature engineering to see how your model improves. Feature engineering types include, using indicator variables to isolate key information. For example, geolocation features for our in New York City taxi servers, where you'd isolate a certain area for your training dataset. Can also include highlighting interactions between two or more features, meaning that you can actually include the sum of two features. The difference between two features, the product of two features, and the quotient of two features. Other types include representing the same feature in a different way. For example, in numeric to categorical mappings, where you have grade, you can create categories and create a new feature grade with elementary school, middle school, and high school, as classes. Another example, is to group sparse classes, where you group similar classes and then group the remaining ones into a single other class. Another example, is to transform categorical features into dummy variables. Here are examples of the types of feature engineering transformations that you can perform on numeric, temporal, textural, and categorical features. You will see more in the upcoming videos in labs.
