Xapian: Internal Source Documentation: Xapian::Utf8Iterator Class Reference

xapian-core 1.4.27

An iterator which returns Unicode character values from a UTF-8 encoded string. More...

#include <unicode.h>

Public Types

typedef std::input_iterator_tag	iterator_category
	We implement the semantics of an STL input_iterator. More...

typedef unsigned	value_type
	We implement the semantics of an STL input_iterator. More...

typedef size_t	difference_type
	We implement the semantics of an STL input_iterator. More...

typedef const unsigned *	pointer
	We implement the semantics of an STL input_iterator. More...

typedef const unsigned &	reference
	We implement the semantics of an STL input_iterator. More...

Public Member Functions
const char *	raw () const
	Return the raw const char* pointer for the current position. More...

size_t	left () const
	Return the number of bytes left in the iterator's buffer. More...

void	assign (const char *p_, size_t len)
	Assign a new string to the iterator. More...

void	assign (const std::string &s)
	Assign a new string to the iterator. More...

	Utf8Iterator (const char *p_)
	Create an iterator given a pointer to a null terminated string. More...

	Utf8Iterator (const char *p_, size_t len)
	Create an iterator given a pointer and a length. More...

	Utf8Iterator (const std::string &s)
	Create an iterator given a string. More...

	Utf8Iterator ()
	Create an iterator which is at the end of its iteration. More...

unsigned	operator* () const
	Get the current Unicode character value pointed to by the iterator. More...

Utf8Iterator	operator++ (int)
	Move forward to the next Unicode character. More...

Utf8Iterator &	operator++ ()
	Move forward to the next Unicode character. More...

bool	operator== (const Utf8Iterator &other) const
	Test two Utf8Iterators for equality. More...

bool	operator!= (const Utf8Iterator &other) const
	Test two Utf8Iterators for inequality. More...

Private Member Functions
bool	calculate_sequence_length () const

unsigned	get_char () const

	Utf8Iterator (const unsigned char p_, const unsigned char end_, unsigned seqlen_)

unsigned	strict_deref () const

Private Attributes
const unsigned char *	p

const unsigned char *	end

unsigned	seqlen

Detailed Description

An iterator which returns Unicode character values from a UTF-8 encoded string.

Definition at line 38 of file unicode.h.

Member Typedef Documentation

◆ difference_type

typedef size_t Xapian::Utf8Iterator::difference_type

We implement the semantics of an STL input_iterator.

Definition at line 206 of file unicode.h.

◆ iterator_category

typedef std::input_iterator_tag Xapian::Utf8Iterator::iterator_category

We implement the semantics of an STL input_iterator.

Definition at line 204 of file unicode.h.

◆ pointer

typedef const unsigned* Xapian::Utf8Iterator::pointer

We implement the semantics of an STL input_iterator.

Definition at line 207 of file unicode.h.

◆ reference

typedef const unsigned& Xapian::Utf8Iterator::reference

We implement the semantics of an STL input_iterator.

Definition at line 208 of file unicode.h.

◆ value_type

typedef unsigned Xapian::Utf8Iterator::value_type

We implement the semantics of an STL input_iterator.

Definition at line 205 of file unicode.h.

Constructor & Destructor Documentation

◆ Utf8Iterator() [1/5]

Xapian::Utf8Iterator::Utf8Iterator	(	const unsigned char *	p_,
		const unsigned char *	end_,
		unsigned	seqlen_
	)

inlineprivate

Definition at line 47 of file unicode.h.

◆ Utf8Iterator() [2/5]

Xapian::Utf8Iterator::Utf8Iterator ( const char * p_ )

explicit

Create an iterator given a pointer to a null terminated string.

The iterator will return characters from the start of the string when next called. The string is not copied into the iterator, so it must remain valid while the iteration is in progress.

Parameters

p_	A pointer to the start of the null terminated string to read.

Definition at line 67 of file utf8itor.cc.

◆ Utf8Iterator() [3/5]

Xapian::Utf8Iterator::Utf8Iterator	(	const char *	p_,
		size_t	len
	)

inline

Create an iterator given a pointer and a length.

The iterator will return characters from the start of the string when next called. The string is not copied into the iterator, so it must remain valid while the iteration is in progress.

Parameters

p_	A pointer to the start of the string to read.
len	The length of the string to read.

Definition at line 114 of file unicode.h.

◆ Utf8Iterator() [4/5]

Xapian::Utf8Iterator::Utf8Iterator ( const std::string & s )

inline

Create an iterator given a string.

The iterator will return characters from the start of the string when next called. The string is not copied into the iterator, so it must remain valid while the iteration is in progress.

Parameters

s	The string to read. Must not be modified while the iteration is in progress.

Definition at line 125 of file unicode.h.

◆ Utf8Iterator() [5/5]

Xapian::Utf8Iterator::Utf8Iterator ( )

inline

Create an iterator which is at the end of its iteration.

This can be compared to another iterator to check if the other iterator has reached its end.

Definition at line 132 of file unicode.h.

References XAPIAN_PURE_FUNCTION.

Member Function Documentation

◆ assign() [1/2]

void Xapian::Utf8Iterator::assign	(	const char *	p_,
		size_t	len
	)

inline

Assign a new string to the iterator.

The iterator will forget the string it was iterating through, and return characters from the start of the new string when next called. The string is not copied into the iterator, so it must remain valid while the iteration is in progress.

Parameters

p_	A pointer to the start of the string to read.
len	The length of the string to read.

Definition at line 72 of file unicode.h.

◆ assign() [2/2]

void Xapian::Utf8Iterator::assign ( const std::string & s )

inline

Assign a new string to the iterator.

The iterator will forget the string it was iterating through, and return characters from the start of the new string when next called. The string is not copied into the iterator, so it must remain valid while the iteration is in progress.

Parameters

s	The string to read. Must not be modified while the iteration is in progress.

Definition at line 92 of file unicode.h.

References assign().

Referenced by assign().

◆ calculate_sequence_length()

bool Xapian::Utf8Iterator::calculate_sequence_length ( ) const

private

Definition at line 73 of file utf8itor.cc.

References bad_cont().

◆ get_char()

unsigned Xapian::Utf8Iterator::get_char ( ) const

private

◆ left()

size_t Xapian::Utf8Iterator::left ( ) const

inline

Return the number of bytes left in the iterator's buffer.

Definition at line 59 of file unicode.h.

Referenced by Xapian::MSet::Internal::snippet().

◆ operator!=()

bool Xapian::Utf8Iterator::operator!= ( const Utf8Iterator & other ) const

inline

Test two Utf8Iterators for inequality.

Parameters

other The Utf8Iterator to compare this one with.

Returns: true iff the iterators do not point to the same position.

Definition at line 198 of file unicode.h.

References p.

◆ operator*()

unsigned Xapian::Utf8Iterator::operator* ( ) const

Get the current Unicode character value pointed to by the iterator.

If an invalid UTF-8 sequence is encountered, then the byte values comprising it are returned until valid UTF-8 or the end of the input is reached.

Returns unsigned(-1) if the iterator has reached the end of its buffer.

Definition at line 115 of file utf8itor.cc.

◆ operator++() [1/2]

Utf8Iterator Xapian::Utf8Iterator::operator++ ( int )

inline

Move forward to the next Unicode character.

Returns: An iterator pointing to the position before the move.

Definition at line 161 of file unicode.h.

◆ operator++() [2/2]

Utf8Iterator& Xapian::Utf8Iterator::operator++ ( )

inline

Move forward to the next Unicode character.

Returns: A reference to this object.

Definition at line 176 of file unicode.h.

◆ operator==()

bool Xapian::Utf8Iterator::operator== ( const Utf8Iterator & other ) const

inline

Test two Utf8Iterators for equality.

Parameters

other The Utf8Iterator to compare this one with.

Returns: true iff the iterators point to the same position.

Definition at line 189 of file unicode.h.

References p.

◆ raw()

const char* Xapian::Utf8Iterator::raw ( ) const

inline

Return the raw const char* pointer for the current position.

Definition at line 54 of file unicode.h.

Referenced by Xapian::SnipPipe::drain(), and Xapian::QueryParser::Internal::parse_term().

◆ strict_deref()

unsigned Xapian::Utf8Iterator::strict_deref ( ) const

private

Get the current Unicode character value pointed to by the iterator.

If an invalid UTF-8 sequence is encountered, then the byte values comprising it are returned with the top bit set (so the caller can differentiate these from the same values arising from valid UTF-8) until valid UTF-8 or the end of the input is reached.

Returns unsigned(-1) if the iterator has reached the end of its buffer.

Definition at line 128 of file utf8itor.cc.

Member Data Documentation

◆ end

const unsigned char* Xapian::Utf8Iterator::end

private

Definition at line 40 of file unicode.h.

◆ p

const unsigned char* Xapian::Utf8Iterator::p

private

Definition at line 39 of file unicode.h.

Referenced by operator!=(), and operator==().

◆ seqlen

unsigned Xapian::Utf8Iterator::seqlen

mutableprivate

Definition at line 41 of file unicode.h.

The documentation for this class was generated from the following files:

include/xapian/unicode.h
unicode/utf8itor.cc