xapian-core  1.4.21
Classes | Namespaces | Functions
cjk-tokenizer.h File Reference

Tokenise CJK text as n-grams. More...

#include "xapian/unicode.h"
#include <string>
+ Include dependency graph for cjk-tokenizer.h:
+ This graph shows which files directly or indirectly include this file:

Go to the source code of this file.


class  CJKTokenIterator
 Iterator returning unigrams and bigrams. More...




bool CJK::is_cjk_enabled ()
 Should we use the CJK n-gram code? More...
bool CJK::codepoint_is_cjk (unsigned codepoint)
void CJK::get_cjk (Xapian::Utf8Iterator &it)

Detailed Description

Tokenise CJK text as n-grams.

Definition in file cjk-tokenizer.h.