Tokenise CJK text as n-grams. More...
#include <config.h>#include "cjk-tokenizer.h"#include "omassert.h"#include "xapian/unicode.h"#include <cstdlib>#include <string>
Include dependency graph for cjk-tokenizer.cc:
Go to the source code of this file.
Detailed Description
Tokenise CJK text as n-grams.
Definition in file cjk-tokenizer.cc.