How to train and predict using bag of words?

Question

I have a folder of images of a car from every angle. I want to use the bag of words approach to train the system in recognizing the car. Once the training is done, I want that if an image of that car is given it should be able to recognize it.

I have been trying to learn the BOW function in opencv in order to make this work and have come at a level where I do not know what to do now and some guidance would be appreciated.

Here is my code that I used to make the bag of words:

Ptr<FeatureDetector> features = FeatureDetector::create("SIFT");

    Ptr<DescriptorExtractor> descriptors = DescriptorExtractor::create("SIFT");

    Ptr<DescriptorMatcher> matcher = DescriptorMatcher::create("FlannBased");

    //defining terms for bowkmeans trainer

    TermCriteria tc(MAX_ITER + EPS, 10, 0.001);

    int dictionarySize = 1000;

    int retries = 1;

    int flags = KMEANS_PP_CENTERS;

    BOWKMeansTrainer bowTrainer(dictionarySize, tc, retries, flags);

    BOWImgDescriptorExtractor bowDE(descriptors, matcher);

    //training data now

    Mat features;

    Mat img = imread("c:\\1.jpg", 0);

    Mat img2 = imread("c:\\2.jpg", 0);

    vector<KeyPoint> keypoints, keypoints2;

    features->detect(img, keypoints);

    features->detect(img2,keypoints2);

    descriptor->compute(img, keypoints, features);

    Mat features2;

    descripto->compute(img2, keypoints2, features2);

    bowTrainer.add(features);

    bowTrainer.add(features2);

    Mat dictionary = bowTrainer.cluster();

    bowDE.setVocabulary(dictionary);

This is all based on the BOW documentation.

I think at this stage my system is trained. and the next step is predicting.

this is where I dont know what to do. If I use SVM or NormalBayesClassifier they both use the terms train and predict.

How do I predict and train after this? any guidance would be much appreciated. How do I connect the training of the classifier to my `bowDE`` function?

Answer 1

You can extract the actual bag of word descriptors. You can do this using the compute function from the BOWImgDescriptorExtractor. Something like

bowDE.compute(img, keypoints, bow_descriptor)

Using this function you create descriptors that you then gather into a matrix that serves as the input for the classifier functions. Maybe this tutorial can guide you a little bit.

The classification you usually need at least 2 classes. So you also need some images which do not contain cars to train a classifier.

Hope this answer helps you! Thus, Machine Learning Tutorial is considered one of the most important tools.