Namespaces | |
| namespace | Internal |
Enumerations | |
| enum | category { UNASSIGNED, UPPERCASE_LETTER, LOWERCASE_LETTER, TITLECASE_LETTER, MODIFIER_LETTER, OTHER_LETTER, NON_SPACING_MARK, ENCLOSING_MARK, COMBINING_SPACING_MARK, DECIMAL_DIGIT_NUMBER, LETTER_NUMBER, OTHER_NUMBER, SPACE_SEPARATOR, LINE_SEPARATOR, PARAGRAPH_SEPARATOR, CONTROL, FORMAT, PRIVATE_USE, SURROGATE, CONNECTOR_PUNCTUATION, DASH_PUNCTUATION, OPEN_PUNCTUATION, CLOSE_PUNCTUATION, INITIAL_QUOTE_PUNCTUATION, FINAL_QUOTE_PUNCTUATION, OTHER_PUNCTUATION, MATH_SYMBOL, CURRENCY_SYMBOL, MODIFIER_SYMBOL, OTHER_SYMBOL } |
| Each Unicode character is in exactly one of these categories. More... | |
Functions | |
| unsigned | nonascii_to_utf8 (unsigned ch, char *buf) |
| Convert a single non-ASCII Unicode character to UTF-8. | |
| unsigned | to_utf8 (unsigned ch, char *buf) |
| Convert a single Unicode character to UTF-8. | |
| void | append_utf8 (std::string &s, unsigned ch) |
| Append the UTF-8 representation of a single Unicode character to a std::string. | |
| category | get_category (unsigned ch) |
| Return the category which a given Unicode character falls into. | |
| bool | is_wordchar (unsigned ch) |
| Test if a given Unicode character is "word character". | |
| bool | is_whitespace (unsigned ch) |
| Test if a given Unicode character is a whitespace character. | |
| bool | is_currency (unsigned ch) |
| Test if a given Unicode character is a currency symbol. | |
| unsigned | tolower (unsigned ch) |
| Convert a Unicode character to lowercase. | |
| unsigned | toupper (unsigned ch) |
| Convert a Unicode character to uppercase. | |
| std::string | tolower (const std::string &term) |
| Convert a UTF-8 std::string to lowercase. | |
| std::string | toupper (const std::string &term) |
| Convert a UTF-8 std::string to uppercase. | |
Each Unicode character is in exactly one of these categories.
| void Xapian::Unicode::append_utf8 | ( | std::string & | s, | |
| unsigned | ch | |||
| ) | [inline] |
Append the UTF-8 representation of a single Unicode character to a std::string.
Definition at line 283 of file unicode.h.
References to_utf8().
Referenced by Term::as_positional_cjk_term(), DEFINE_TESTCASE(), CJK::get_cjk(), Xapian::TermGenerator::Internal::index_text(), CJKTokenIterator::operator*(), CJKTokenIterator::operator++(), Xapian::QueryParser::Internal::parse_term(), tolower(), and toupper().
| category Xapian::Unicode::get_category | ( | unsigned | ch | ) | [inline] |
Return the category which a given Unicode character falls into.
Definition at line 289 of file unicode.h.
References Xapian::Unicode::Internal::get_category(), Xapian::Unicode::Internal::get_character_info(), and UNASSIGNED.
Referenced by is_currency(), Xapian::is_digit(), is_whitespace(), is_wordchar(), and Xapian::should_stem().
| bool Xapian::Unicode::is_currency | ( | unsigned | ch | ) | [inline] |
Test if a given Unicode character is a currency symbol.
Definition at line 324 of file unicode.h.
References CURRENCY_SYMBOL, and get_category().
Referenced by DEFINE_TESTCASE().
| bool Xapian::Unicode::is_whitespace | ( | unsigned | ch | ) | [inline] |
Test if a given Unicode character is a whitespace character.
Definition at line 314 of file unicode.h.
References CONTROL, get_category(), LINE_SEPARATOR, PARAGRAPH_SEPARATOR, and SPACE_SEPARATOR.
Referenced by DEFINE_TESTCASE(), and is_not_whitespace().
| bool Xapian::Unicode::is_wordchar | ( | unsigned | ch | ) | [inline] |
Test if a given Unicode character is "word character".
Definition at line 296 of file unicode.h.
References COMBINING_SPACING_MARK, CONNECTOR_PUNCTUATION, DECIMAL_DIGIT_NUMBER, ENCLOSING_MARK, get_category(), LETTER_NUMBER, LOWERCASE_LETTER, MODIFIER_LETTER, NON_SPACING_MARK, OTHER_LETTER, OTHER_NUMBER, TITLECASE_LETTER, and UPPERCASE_LETTER.
Referenced by Xapian::check_wordchar(), DEFINE_TESTCASE(), Xapian::TermGenerator::Internal::index_text(), is_not_wordchar(), and Xapian::QueryParser::Internal::parse_term().
| unsigned Xapian::Unicode::nonascii_to_utf8 | ( | unsigned | ch, | |
| char * | buf | |||
| ) |
Convert a single non-ASCII Unicode character to UTF-8.
This is intended mainly as a helper method for to_utf8().
| ch | The character (which must be > 128) to write to buf. | |
| buf | The buffer to write the character to - it must have space for (at least) 4 bytes. |
Definition at line 36 of file utf8itor.cc.
Referenced by to_utf8().
| unsigned Xapian::Unicode::to_utf8 | ( | unsigned | ch, | |
| char * | buf | |||
| ) | [inline] |
Convert a single Unicode character to UTF-8.
| ch | The character to write to buf. | |
| buf | The buffer to write the character to - it must have space for (at least) 4 bytes. |
Definition at line 272 of file unicode.h.
References nonascii_to_utf8().
Referenced by append_utf8().
| std::string Xapian::Unicode::tolower | ( | const std::string & | term | ) | [inline] |
Convert a UTF-8 std::string to lowercase.
Definition at line 348 of file unicode.h.
References append_utf8(), and tolower().
| unsigned Xapian::Unicode::tolower | ( | unsigned | ch | ) | [inline] |
Convert a Unicode character to lowercase.
Definition at line 329 of file unicode.h.
References Xapian::Unicode::Internal::get_case_type(), Xapian::Unicode::Internal::get_character_info(), and Xapian::Unicode::Internal::get_delta().
Referenced by Xapian::check_wordchar(), DEFINE_TESTCASE(), Xapian::TermGenerator::Internal::index_text(), main(), AuthorValueRangeProcessor::operator()(), Xapian::QueryParser::Internal::parse_term(), and tolower().
| std::string Xapian::Unicode::toupper | ( | const std::string & | term | ) | [inline] |
Convert a UTF-8 std::string to uppercase.
Definition at line 360 of file unicode.h.
References append_utf8(), and toupper().
| unsigned Xapian::Unicode::toupper | ( | unsigned | ch | ) | [inline] |
Convert a Unicode character to uppercase.
Definition at line 338 of file unicode.h.
References Xapian::Unicode::Internal::get_case_type(), Xapian::Unicode::Internal::get_character_info(), and Xapian::Unicode::Internal::get_delta().
Referenced by DEFINE_TESTCASE(), and toupper().