Tokenise CJK text as n-grams. More...
Include dependency graph for cjk-tokenizer.h:
This graph shows which files directly or indirectly include this file:
Go to the source code of this file.
Classes | |
| class | CJKTokenIterator |
| Iterator returning unigrams and bigrams. More... | |
Namespaces | |
| CJK | |
Functions | |
| bool | CJK::is_cjk_enabled () |
| Should we use the CJK n-gram code? More... | |
| bool | CJK::codepoint_is_cjk (unsigned codepoint) |
| void | CJK::get_cjk (Xapian::Utf8Iterator &it) |
Detailed Description
Tokenise CJK text as n-grams.
Definition in file cjk-tokenizer.h.