xapian-core  1.4.19
cjk-tokenizer.cc File Reference

Tokenise CJK text as n-grams. More...

#include <config.h>
#include "cjk-tokenizer.h"
#include "omassert.h"
#include "xapian/unicode.h"
#include <cstdlib>
#include <string>
+ Include dependency graph for cjk-tokenizer.cc:

Go to the source code of this file.

Detailed Description

Tokenise CJK text as n-grams.

Definition in file cjk-tokenizer.cc.