Handle text without explicit word breaks. More...

#include "xapian/unicode.h"
#include <string>

Include dependency graph for word-breaker.h:

This graph shows which files directly or indirectly include this file:

Classes
class	NgramIterator
	Iterator returning unigrams and bigrams. More...

Functions
bool	is_ngram_enabled ()
	Should we use the n-gram code? More...

bool	is_unbroken_script (unsigned codepoint)

bool	is_unbroken_wordchar (unsigned codepoint)

size_t	get_unbroken (Xapian::Utf8Iterator &it)

Detailed Description

Handle text without explicit word breaks.

Definition in file word-breaker.h.

Function Documentation

◆ get_unbroken()

size_t get_unbroken ( Xapian::Utf8Iterator & it )

Definition at line 142 of file word-breaker.cc.

References is_unbroken_wordchar().

Referenced by Xapian::break_words(), and Xapian::QueryParser::Internal::parse_term().

◆ is_ngram_enabled()

bool is_ngram_enabled ( )

Should we use the n-gram code?

The first time this is called it reads the environment variable XAPIAN_CJK_NGRAM and returns true if it is set to a non-empty value. Subsequent calls cache and return the same value.

Definition at line 43 of file word-breaker.cc.

References p.

Referenced by Xapian::TermGenerator::Internal::index_text(), Xapian::QueryParser::Internal::parse_query(), and Xapian::MSet::Internal::snippet().