backends/flint/flint_utils.h File Reference

#include "omassert.h"
#include <xapian/types.h>
#include <string>

Include dependency graph for flint_utils.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Typedefs

typedef unsigned char om_byte

typedef unsigned int om_uint32

typedef int om_int32

Functions

template<class T >

bool F_unpack_uint (const char **src, const char *src_end, T *resultptr)

FIXME: the pack and unpack int methods store in low-byte-first order

it might be easier to implement efficient specialisations with high-byte-first order.

template<class T >

string F_pack_uint (T value)

Generates a packed representation of an integer.

template<>

string F_pack_uint< bool > (bool value)

Generates a packed representation of a bool.

template<class T >

bool F_unpack_uint_last (const char **src, const char *src_end, T *resultptr)

Reads an unsigned integer from a string starting at a given position.

template<class T >

string F_pack_uint_last (T value)

Generates a packed representation of an integer.

template<class T >

string F_pack_uint_preserving_sort (T value)

Generate a packed representation of an integer, preserving sort order.

template<class T >

bool F_unpack_uint_preserving_sort (const char **src, const char *src_end, T *resultptr)

Unpack a unsigned integer, store in sort preserving order.

bool F_unpack_string (const char **src, const char *src_end, string &result)

string F_pack_string (const string &value)

string F_pack_string_preserving_sort (string value)

Pack a string into a representation which preserves sort order.

bool F_unpack_string_preserving_sort (const char **src, const char *src_end, string &result)

bool F_unpack_bool (const char **src, const char *src_end, bool *resultptr)

string F_pack_bool (bool value)

string flint_docid_to_key (Xapian::docid did)

Convert a document id to a key (suitable when the docid is the only component of the key).

Typedef Documentation

typedef unsigned char om_byte

Definition at line 34 of file flint_utils.h.

typedef int om_int32

Definition at line 36 of file flint_utils.h.

typedef unsigned int om_uint32

Definition at line 35 of file flint_utils.h.

Function Documentation

string F_pack_bool ( bool value ) [inline]

Definition at line 393 of file flint_utils.h.

Referenced by make_start_of_chunk(), and write_start_of_chunk().

string F_pack_string ( const string & value ) [inline]

Definition at line 324 of file flint_utils.h.

References F_pack_uint().

Referenced by FlintValueTable::encode_values(), and FlintTable_base::write_to_file().

string F_pack_string_preserving_sort ( string value ) [inline]

Pack a string into a representation which preserves sort order.

We do this by replacing zero bytes in the string with a zero byte followed by byte value 0xff, and then appending two zero bytes to the end.

Definition at line 335 of file flint_utils.h.

Referenced by FlintPostListTable::make_key(), FlintAllTermsList::next(), and FlintAllTermsList::skip_to().

template<class T >

string F_pack_uint ( T value ) [inline]

Generates a packed representation of an integer.

Parameters:

value

The integer to represent.

Returns:: A string containing the representation of the integer.

Definition at line 121 of file flint_utils.h.

References STATIC_ASSERT_UNSIGNED_TYPE.

Referenced by FlintPostlistChunkWriter::append(), FlintDatabaseReplicator::apply_changeset_from_conn(), FlintValueTable::encode_values(), F_pack_string(), FlintWritableDatabase::flush_postlist_changes(), FlintDatabase::get_revision_info(), make_start_of_chunk(), make_start_of_first_chunk(), make_wdf_and_length(), FlintCompact::merge_postlists(), FlintDatabase::send_whole_database(), FlintPositionListTable::set_positionlist(), FlintDatabase::set_revision_number(), FlintTermListTable::set_termlist(), FlintTable::write_changed_blocks(), FlintDatabase::write_changesets_to_fd(), write_start_of_chunk(), and FlintTable_base::write_to_file().

template<>

string F_pack_uint< bool > ( bool value ) [inline]

Generates a packed representation of a bool.

This is a specialisation of the template above.

Parameters:

value

The bool to represent.

Returns:: A string containing the representation of the bool.

template<class T >

string F_pack_uint_last ( T value ) [inline]

Generates a packed representation of an integer.

This encoding requires that we know the encoded length from out-of-band information (so is suitable when only one integer is encoded, or for the last integer encoded).

Parameters:

value

The integer to represent.

Returns:: A string containing the representation of the integer.

Definition at line 208 of file flint_utils.h.

References STATIC_ASSERT_UNSIGNED_TYPE.

Referenced by FlintWritableDatabase::flush_postlist_changes(), FlintSpellingTable::merge_changes(), FlintCompact::merge_postlists(), and FlintCompact::merge_spellings().

template<class T >

string F_pack_uint_preserving_sort ( T value ) [inline]

Generate a packed representation of an integer, preserving sort order.

This representation is less compact than the usual one, and has a limit of 256 bytes on the length of the integer. However, this is unlikely to ever be a problem.

Parameters:

value

The integer to represent.

Returns:: A string containing the representation of the integer.

Definition at line 233 of file flint_utils.h.

References STATIC_ASSERT_UNSIGNED_TYPE.

Referenced by flint_docid_to_key(), FlintPostListTable::make_key(), FlintPositionListTable::make_key(), FlintCompact::merge_docid_keyed(), FlintCompact::merge_postlists(), FlintPositionListTable::positionlist_count(), FlintPositionList::read_data(), and FlintAllDocsPostList::skip_to().

bool F_unpack_bool	(	const char **	src,
		const char *	src_end,
		bool *	resultptr
	)			`[inline]`

Definition at line 372 of file flint_utils.h.

Referenced by check_flint_table(), and read_start_of_chunk().

bool F_unpack_string	(	const char **	src,
		const char *	src_end,
		string &	result
	)			`[inline]`

Definition at line 303 of file flint_utils.h.

References F_unpack_uint().

Referenced by FlintDatabaseReplicator::apply_changeset_from_conn(), check_flint_table(), and FlintValueTable::unpack_entry().

bool F_unpack_string_preserving_sort	(	const char **	src,
		const char *	src_end,
		string &	result
	)			`[inline]`

Definition at line 347 of file flint_utils.h.

Referenced by check_flint_table(), copy_postlist(), get_tname_from_key(), FlintCompact::PostlistCursor::next(), FlintAllTermsList::next(), and FlintAllTermsList::skip_to().

template<class T >

bool F_unpack_uint	(	const char **	src,
		const char *	src_end,
		T *	resultptr
	)			`[inline]`

FIXME: the pack and unpack int methods store in low-byte-first order

it might be easier to implement efficient specialisations with high-byte-first order.

Reads an unsigned integer from a string starting at a given position.

Parameters:

	src	A pointer to a pointer to the data to read. The character pointer will be updated to point to the next character to read, or 0 if the method ran out of data. (It is only set to 0 in case of an error).
	src_end	A pointer to the byte after the end of the data to read the integer from.
	resultptr	A pointer to a place to store the result. If an error occurs, the value stored in this location is undefined. If this pointer is 0, the result is not stored, and the method simply skips over the result.

Returns:: True if an integer was successfully read. False if the read failed. Failure may either be due to the data running out (in which case *src will equal 0), or due to the value read overflowing the size of result (in which case *src will point to wherever the value ends, despite the overflow).

Definition at line 64 of file flint_utils.h.

References STATIC_ASSERT, and STATIC_ASSERT_UNSIGNED_TYPE.

Referenced by FlintDatabaseReplicator::apply_changeset_from_conn(), check_flint_table(), FlintDatabaseReplicator::check_revision_at_least(), FlintTable_base::do_unpack_uint(), F_unpack_string(), FlintTermList::FlintTermList(), FlintDatabase::get_changeset_revisions(), FlintTermListTable::get_doclength(), FlintAllDocsPostList::get_doclength(), FlintCompact::merge_postlists(), FlintTermList::next(), FlintCompact::PostlistCursor::next(), FlintPositionListTable::positionlist_count(), FlintDatabaseReplicator::process_changeset_chunk_base(), FlintDatabaseReplicator::process_changeset_chunk_blocks(), FlintTable_base::read(), FlintPositionList::read_data(), read_did_increase(), FlintDatabase::read_metainfo(), FlintPostList::read_number_of_entries(), read_start_of_chunk(), read_start_of_first_chunk(), read_wdf_and_length(), FlintValueTable::unpack_entry(), and FlintDatabase::write_changesets_to_fd().

template<class T >

bool F_unpack_uint_last	(	const char **	src,
		const char *	src_end,
		T *	resultptr
	)			`[inline]`

Reads an unsigned integer from a string starting at a given position.

This encoding requires that we know the encoded length from out-of-band information (so is suitable when only one integer is encoded, or for the last integer encoded).

Parameters:

	src	A pointer to a pointer to the data to read.
	src_end	A pointer to the byte after the end of the data to read the integer from.
	resultptr	A pointer to a place to store the result. If an error occurs, the value stored in this location is undefined. If this pointer is 0, the result is not stored, and the method simply skips over the result.

Returns:: True if an integer was successfully read. False if the read failed. Failure can hapen if the value read overflows the size of result.

Definition at line 173 of file flint_utils.h.

References STATIC_ASSERT, and STATIC_ASSERT_UNSIGNED_TYPE.

Referenced by FlintSpellingTable::add_word(), check_flint_table(), FlintSpellingWordsList::get_termfreq(), FlintSpellingTable::get_word_frequency(), FlintCompact::merge_postlists(), FlintCompact::merge_spellings(), FlintDatabase::read_metainfo(), and FlintSpellingTable::remove_word().

template<class T >

bool F_unpack_uint_preserving_sort	(	const char **	src,
		const char *	src_end,
		T *	resultptr
	)			`[inline]`

Unpack a unsigned integer, store in sort preserving order.

Parameters:

	src	A pointer to a pointer to the data to read. The character pointer will be updated to point to the next character to read, or 0 if the method ran out of data. (It is only set to 0 in case of an error).
	src_end	A pointer to the byte after the end of the data to read the integer from.
	resultptr	A pointer to a place to store the result. If an error occurs, the value stored in this location is undefined. If this pointer is 0, the result is not stored, and the method simply skips over the result.

Returns:: True if an integer was successfully read. False if the read failed. Failure may either be due to the data running out (in which case *src will equal 0), or due to the value read overflowing the size of result (in which case *src will point to wherever the value ends, despite the overflow).

Definition at line 269 of file flint_utils.h.

Referenced by check_flint_table(), copy_docid_keyed(), copy_position(), copy_postlist(), copy_termlist(), FlintPostlistChunkWriter::flush(), FlintPostListTable::get_chunk(), FlintCompact::merge_docid_keyed(), FlintPostList::move_to_chunk_containing(), FlintCompact::PostlistCursor::next(), FlintPostList::next_chunk(), and FlintAllDocsPostList::read_did_from_current_key().

string flint_docid_to_key ( Xapian::docid did ) [inline]

Convert a document id to a key (suitable when the docid is the only component of the key).

Definition at line 402 of file flint_utils.h.

References F_pack_uint_preserving_sort().

Referenced by FlintRecordTable::delete_record(), FlintTermListTable::delete_termlist(), FlintTermList::FlintTermList(), FlintTermListTable::get_doclength(), FlintRecordTable::get_record(), make_key(), FlintRecordTable::replace_record(), and FlintTermListTable::set_termlist().


Typedefs
typedef unsigned char	om_byte
typedef unsigned int	om_uint32
typedef int	om_int32
Functions
template<class T >
bool	F_unpack_uint (const char *src, const char src_end, T *resultptr)
	FIXME: the pack and unpack int methods store in low-byte-first order it might be easier to implement efficient specialisations with high-byte-first order.
template<class T >
string	F_pack_uint (T value)
	Generates a packed representation of an integer.
template<>
string	F_pack_uint< bool > (bool value)
	Generates a packed representation of a bool.
template<class T >
bool	F_unpack_uint_last (const char *src, const char src_end, T *resultptr)
	Reads an unsigned integer from a string starting at a given position.
template<class T >
string	F_pack_uint_last (T value)
	Generates a packed representation of an integer.
template<class T >
string	F_pack_uint_preserving_sort (T value)
	Generate a packed representation of an integer, preserving sort order.
template<class T >
bool	F_unpack_uint_preserving_sort (const char *src, const char src_end, T *resultptr)
	Unpack a unsigned integer, store in sort preserving order.
bool	F_unpack_string (const char *src, const char src_end, string &result)
string	F_pack_string (const string &value)
string	F_pack_string_preserving_sort (string value)
	Pack a string into a representation which preserves sort order.
bool	F_unpack_string_preserving_sort (const char *src, const char src_end, string &result)
bool	F_unpack_bool (const char *src, const char src_end, bool *resultptr)
string	F_pack_bool (bool value)
string	flint_docid_to_key (Xapian::docid did)
	Convert a document id to a key (suitable when the docid is the only component of the key).