The Brill tagger is a method for doing part-of-speech tagging. Part-of-speech tagging ( POS tagging or POST) also called grammatical tagging or word-category disambiguation, is the process of marking up the It was described by Eric Brill in his 1993 PhD thesis [1]. Dr Eric Brill is a Computer scientist specializing in Natural Language Processing. It can be summarized as an "error-driven transformation-based tagger". It is
- error-driven in the sense that it recourses to supervised learning
- transformation-based in the sense that a tag is assigned to each word and changed using a set of predefined rules. Supervised learning is a Machine learning technique for learning a function from training data Note: If the word is known, it first assigns the most frequent tag, or if the word is unknown, it naively assigns the tag "noun" to it. Applying over and over these rules, changing the incorrect tags, a quite high accuracy is achieved.
Algorithm
The algorithm goes as follows:
- Initialisation:
- Known words (in vocabulary): assigning the most frequent tag associated to a form of the word
- Unknown words (out of vocabulary) :
- Proper noun if capitalised and simple noun else (1992)
- Learning or guessing rules on the same basis as contextual rules (1994)
- Learning Phase
- Iteratively compute the error score of each candidate rule (difference between the number of errors before and after applying the rule)
- Select the best (higher score) rule.
- Add it to the rule set and apply it to the text.
- Repeat until no rule has a score above a given threshold (that is, until applying new rules leaves the text in the same state, which is then supposed to be the final state of the tagging).
Rules
Lexical rules are used for the initialisation, and contextual rules are used to correct the tags. A lexical rule is a form of syntactic rule used within many theories of natural language syntax
- Lexical rules: word → tag IF Condition (example: identification of suffixes like "-tion")
- Contextual rules: tag1 → tag2 IF Condition (example: "preceding/following tag is X", "preceding/following word is w")
Dapyx Software network: MP3 Explorer | Ebook Manager | Zenithic