«

Suppose I need to classify 10000 web pages based on their languages.
What should I look for to determine the language of each web page? Any
advice is welcome.

On Thu, 10 Apr 2008, wrote:

> Suppose I need to classify 10000 web pages based on their languages.
> What should I look for to determine the language of each web page?

The "lang" attribute in HTML; the "xml:lang" attribute in XHTML.

In article <26414fbe-a0ef-48c9-af07->,
<> wrote:

>Suppose I need to classify 10000 web pages based on their languages.
>What should I look for to determine the language of each web page? Any
>advice is welcome.

Assuming you want to do this by inspection of the text (rather than
looking for xml:lang and the like), Google for language
identification. The first page lists several tools and a research
bibliography on the subject.

-- Richard
--
:wq