Lemmatisation
|
|
In computing, lemmatisation is the process of calculating the lemma for a given word. Since the process involves calculating the part of speech of a word in a sentence, it requires knowledge of the grammar of a language, and it can therefore be a great deal of work to implement a lemmatiser for a new language.
In many languages, words appear in several inflected forms. For example, in English, the verb 'to walk' may appear as 'walk', 'walked', 'walks', 'walking'. The base form, 'walk', that one might look up in a dictionary, is called the 'lexeme' for the word. The combination of the base form with the part of speech is often called the 'lemma' of the word.
Lemmatisation is closely related to stemming. The difference is that a stemmer operates on a single word without knowledge of the context, and therefore cannot discriminate between words which have different meanings depending on part of speech. However, stemmers are typically easier to implement, and run faster, and the reduced accuracy may not matter for some applications.
External links
- Lexed by Lionel Clément (http://www.lionel-clement.net/lexed/lexed-en.html)