What's the best approach to recognize patterns in data, and what's the best way to learn more on the topic?

Question

A developer I am working with is developing a program that analyzes images of pavement to find cracks in the pavement. For every crack his program finds, it produces an entry in a file that tells me which pixels make up that particular crack. There are two problems with his software through:

1) It produces several false positives

2) If he finds a crack, he only finds small sections of it and denotes those sections as being separate cracks.

My job is to write software that will read this data, analyze it, and tell the difference between false-positives and actual cracks. I also need to determine how to group together all the small sections of a crack as one.

I have tried various ways of filtering the data to eliminate false-positives, and have been using neural networks to a limited degree of success to group cracks together. I understand there will be an error, but as of now, there is just too much error. Does anyone have any insight for a non-AI expert as to the best way to accomplish my task or learn more about it? What kinds of books should I read, or what kind of classes should I take?

EDIT My question is more about how to notice patterns in my coworker's data and identify those patterns as actual cracks. It's the higher-level logic that I'm concerned with, not so much the low-level logic.

EDIT In all actuality, it would take AT LEAST 20 sample images to give an accurate representation of the data I'm working with. It varies a lot. But I do have a sample here, here, and here. These images have already been processed by my coworker's process. The red, blue, and green data are what I have to classify (red stands for dark crack, blue stands for light crack, and green stands for a wide/sealed crack).

Answer 1

Your problem falls in the very broad category of image classification. It also sounds like you're dealing with a clustering problem (clustering is the task of grouping a set of objects in such a way that objects in the same group (cluster) are more similar (in some sense) to each other than to those in other groups (clusters)).

In your case, a clustering algorithm would attempt to repeatedly merge small cracks to form larger cracks, until some stopping criteria are met. The end result would be a smaller set of joined cracks. Of course, cracks are a little different than two-dimensional points -- part of the trick in getting a clustering algorithm to work here will be defining a useful distance metric between two cracks.

Some popular clustering algorithms include k-means clustering (demo) and hierarchical clustering. 

False positives can be tricky. If there aren't too many of them, you might be able to detect them by running a clustering algorithm with a limited number of clusters. Any cracks that don't appear in a cluster could be discarded. Alternatively, you may need to perform a pre-processing step that identifies and removes false positives (perhaps using methods others have suggested) before clustering will work.

Hope this helps!

If you want to make your career in Artificial Intelligence then go through this video:

Wish to gain an in-depth knowledge of AI? Check out our AI Course and gather more insights!