How to programmatically determine what language the content of a website is written in

Question

I would like to programmatically determine the language that the content of a website is written in.

The only thing that comes into my mind is to compare the content of the website with some set of words that are common to the particular language, and based on match percentage determine the language.

Are there any better and more robust ways to solve the problem?

Answer 1

Here is an implementation of Neural Network Tutorial with Language classifying example based on average frequencies of the letters http://fann.sourceforge.net/fann_en.pdf

If you only need language detection, you can practice on free web service:

http://detectlanguage.com

It can fit with Google Translate API request/response formats.