How to engineer features for machine learning

Question

Do you have some advice or reading how to engineer features for a machine learning task? Good input features are important even for a neural network. The chosen features will affect the needed number of hidden neurons and the needed number of training examples.

The following is an example problem, but I'm interested in feature engineering in general.

A motivation example: What would be a good input when looking at a puzzle (e.g., 15-puzzle or Sokoban)? Would it be possible to recognize which of the two states is closer to the goal?

Answer 1

Feature engineering is important for better accuracy of predictions in machine learning. 

Generally, feature engineering involves two components:

• Understanding the dataset: You should understand the properties of the task you're trying to solve and how they might interact with the strengths and limitations of the classifier you're using

• The main goal to predict by model: You need to explore more about the experimental work where you will be testing your expectations and find out what actually works and what doesn't.

The proper understanding of the problem is needed to work on features wisely. You can iterate the process many times for enhancement of the feature generation task.

Fitting Features to Your Classifier

Let’s say you're using SVM with a linear kernel. If you can find an interesting interaction between various attributes, that can measure and provide as an input to the classifier, then you need to manually construct features.

Notes Specifically on Puzzle Solving

To solve a problem with a complex state space, then you might want to use a reinforcement learning approach like Q-learning. This can help you to structure learning tasks that involve reaching some goal by a series of intermediate steps by the system.