If translated literally, a feature is best understood as a representative characteristic of observed numerical values in data.#
- Transforming data so that it can be used by machine learning algorithms
- Creating new features (representative values of the data) when making predictions with machine learning algorithms based on data
- [Author's opinion] Feature engineering is not a difficult concept. In fact, it is used a lot in everyday life. As a typical example, when evaluating whether a friend did well or poorly on an exam, we ask, “So, what’s your average score?” Why? Because we need a standard. The score that represents that person indicates their characteristics. The explanation above is just an example to help with understanding. That does not mean all data is limited to what is described above. Then, to generalize, feature engineering can be summed up as creating some value from data so that it can be used in a specific situation.
- Below is an excerpt from one part of the feature-processing process in example code for the Titanic dataset.
[Data Processing] Feature Generalization#
- Among the representative normalization functions used for generalization, logistic regression does not train properly when the feature ranges differ.
- One way to solve this problem is to adjust the ranges of the features as much as possible.
- A typical approach is to transform the feature mean to 0 and the standard deviation to 1.
- In addition to logistic regression, 'Random Forest' and 'LightGBM' fall into this category.
[Data Processing] Handling Missing Values#
- In general, missing (NA) values are handled as follows:
- Handle the missing values as they are.
- Fill in the missing values with representative values.
- Predict and fill in the missing values using other features.
- Create and use a new feature based on whether a value is missing.
