A backend designed for efficient indexing and retrieval, using compressed posting lists and a btree storage scheme. More...

#include <chert_database.h>

Inheritance diagram for ChertDatabase:

Collaboration diagram for ChertDatabase:

Public Member Functions
	ChertDatabase (const string &db_dir_, int action=Xapian::DB_READONLY_, unsigned int block_size=0u)
	Create and open a chert database. More...

	~ChertDatabase ()

ChertCursor *	get_postlist_cursor () const
	Get a postlist table cursor (used by ChertValueList). More...

chert_revision_number_t	get_revision_number () const
	Get an object holding the revision number which the tables are opened at. More...

Xapian::doccount	get_doccount () const
	Virtual methods of Database::Internal. More...

Xapian::docid	get_lastdocid () const
	Return the last used document id of this (sub) database. More...

Xapian::totallength	get_total_length () const
	Return the total length of all documents in this database. More...

Xapian::termcount	get_doclength (Xapian::docid did) const
	Get the length of a given document. More...

Xapian::termcount	get_unique_terms (Xapian::docid did) const
	Get the number of unique term in document. More...

void	get_freqs (const string &term, Xapian::doccount termfreq_ptr, Xapian::termcount collfreq_ptr) const
	Returns frequencies for a term. More...

Xapian::doccount	get_value_freq (Xapian::valueno slot) const
	Return the frequency of a given value slot. More...

std::string	get_value_lower_bound (Xapian::valueno slot) const
	Get a lower bound on the values stored in the given value slot. More...

std::string	get_value_upper_bound (Xapian::valueno slot) const
	Get an upper bound on the values stored in the given value slot. More...

Xapian::termcount	get_doclength_lower_bound () const
	Get a lower bound on the length of a document in this DB. More...

Xapian::termcount	get_doclength_upper_bound () const
	Get an upper bound on the length of a document in this DB. More...

Xapian::termcount	get_wdf_upper_bound (const string &term) const
	Get an upper bound on the wdf of term term. More...

bool	term_exists (const string &tname) const
	Check whether a given term is in the database. More...

bool	has_positions () const
	Check whether this database contains any positional information. More...

LeafPostList *	open_post_list (const string &tname) const
	Open a posting list. More...

ValueList *	open_value_list (Xapian::valueno slot) const
	Open a value stream. More...

Xapian::Document::Internal *	open_document (Xapian::docid did, bool lazy) const
	Open a document. More...

PositionList *	open_position_list (Xapian::docid did, const string &term) const
	Open a position list for the given term in the given document. More...

TermList *	open_term_list (Xapian::docid did) const
	Open a term list. More...

TermList *	open_allterms (const string &prefix) const
	Open an allterms list. More...

TermList *	open_spelling_termlist (const string &word) const
	Create a termlist tree from trigrams of word. More...

TermList *	open_spelling_wordlist () const
	Return a termlist which returns the words which are spelling correction targets. More...

Xapian::doccount	get_spelling_frequency (const string &word) const
	Return the number of times word was added as a spelling. More...

TermList *	open_synonym_termlist (const string &term) const
	Open a termlist returning synonyms for a term. More...

TermList *	open_synonym_keylist (const string &prefix) const
	Open a termlist returning each term which has synonyms. More...

string	get_metadata (const string &key) const
	Get the metadata associated with a given key. More...

TermList *	open_metadata_keylist (const std::string &prefix) const
	Open a termlist returning each metadata key. More...

void	write_changesets_to_fd (int fd, const string &start_revision, bool need_whole_db, Xapian::ReplicationInfo *info)
	Write a set of changesets to a file descriptor. More...

string	get_revision_info () const
	Get a string describing the current revision of the database. More...

string	get_uuid () const
	Get a UUID for the database. More...

void	request_document (Xapian::docid) const
	Request and later collect a document from the database. More...

void	readahead_for_query (const Xapian::Query &query)

void	throw_termlist_table_close_exception () const

int	get_backend_info (string *path) const
	Get backend information about this database. More...

void	get_used_docid_range (Xapian::docid &first, Xapian::docid &last) const
	Find lowest and highest docids actually in use. More...

bool	locked () const
	Return true if the database is open for writing. More...

virtual bool	has_uncommitted_changes () const
	Return true if there are uncommitted changes. More...

Public Member Functions inherited from Xapian::Database::Internal
virtual	~Internal ()
	Destroy the database. More...

virtual void	keep_alive ()
	Send a keep-alive signal to a remote database, to stop it from timing out. More...

virtual void	add_spelling (const string &word, Xapian::termcount freqinc) const
	Add a word to the spelling dictionary. More...

virtual void	remove_spelling (const string &word, Xapian::termcount freqdec) const
	Remove a word from the spelling dictionary. More...

virtual void	add_synonym (const string &term, const string &synonym) const
	Add a synonym for a term. More...

virtual void	remove_synonym (const string &term, const string &synonym) const
	Remove a synonym for a term. More...

virtual void	clear_synonyms (const string &term) const
	Clear all synonyms for a term. More...

virtual void	set_metadata (const string &key, const string &value)
	Set the metadata associated with a given key. More...

virtual void	commit ()
	Commit pending modifications to the database. More...

void	begin_transaction (bool flushed)
	Begin a transaction. More...

void	commit_transaction ()
	Commit a transaction. More...

void	cancel_transaction ()
	Cancel a transaction. More...

virtual Xapian::docid	add_document (const Xapian::Document &document)
	Add a new document to the database. More...

virtual void	delete_document (Xapian::docid did)
	Delete a document in the database. More...

virtual void	delete_document (const string &unique_term)
	Delete any documents indexed by a term from the database. More...

virtual void	replace_document (Xapian::docid did, const Xapian::Document &document)
	Replace a given document in the database. More...

virtual Xapian::docid	replace_document (const string &unique_term, const Xapian::Document &document)
	Replace any documents matching a term. More...

virtual Xapian::Document::Internal *	collect_document (Xapian::docid did) const

virtual void	invalidate_doc_object (Xapian::Document::Internal *obj) const
	Notify the database that document is no longer valid. More...

Public Member Functions inherited from Xapian::Internal::intrusive_base
	intrusive_base ()
	Construct with no references. More...

Static Public Member Functions
static void	compact (Xapian::Compactor compactor, const char destdir, const std::vector< Xapian::Database::Internal * > &sources, const std::vector< Xapian::docid > &offset, size_t block_size, Xapian::Compactor::compaction_level compaction, unsigned flags, Xapian::docid last_docid)

Private Member Functions
bool	database_exists ()
	Return true if a database exists at the path specified for this database. More...

void	create_and_open_tables (unsigned int blocksize)
	Create new tables, and open them. More...

bool	open_tables_consistent ()
	Open all tables at most recent consistent revision. More...

void	get_database_write_lock (int flags, bool creating)
	Get a write lock on the database, or throw an Xapian::DatabaseLockError if failure. More...

void	open_tables (chert_revision_number_t revision)
	Open tables at specified revision number. More...

chert_revision_number_t	get_next_revision_number () const
	Get an object holding the next revision number which should be used in the tables. More...

void	set_revision_number (chert_revision_number_t new_revision)
	Set the revision number in the tables. More...

bool	reopen ()
	Re-open tables to recover from an overwritten condition, or just get most up-to-date version. More...

void	close ()
	Close all the tables permanently. More...

void	modifications_failed (chert_revision_number_t old_revision, chert_revision_number_t new_revision, const std::string &msg)
	Called if a modifications fail. More...

void	apply ()
	Apply any outstanding changes to the tables. More...

void	cancel ()
	Cancel any outstanding changes to the tables. More...

void	send_whole_database (RemoteConnection &conn, double end_time)
	Send a set of messages which transfer the whole database. More...

void	get_changeset_revisions (const string &path, chert_revision_number_t startrev, chert_revision_number_t endrev) const
	Get the revision stored in a changeset. More...

Private Attributes
std::string	db_dir
	Directory to store databases in. More...

bool	readonly
	Whether the database is readonly. More...

ChertVersion	version_file
	The file describing the Chert database. More...

ChertPostListTable	postlist_table
	Table storing posting lists. More...

ChertPositionListTable	position_table
	Table storing position lists. More...

ChertTermListTable	termlist_table
	Table storing term lists. More...

ChertValueManager	value_manager
	Value manager. More...

ChertSynonymTable	synonym_table
	Table storing synonym data. More...

ChertSpellingTable	spelling_table
	Table storing spelling correction data. More...

ChertRecordTable	record_table
	Table storing records. More...

FlintLock	lock
	Lock object. More...

unsigned int	max_changesets
	The maximum number of changesets which should be kept in the database. More...

ChertDatabaseStats	stats
	Database statistics. More...

Friends
class	ChertWritableDatabase

class	ChertTermList

class	ChertPostList

class	ChertAllTermsList

class	ChertAllDocsPostList

Additional Inherited Members
Public Attributes inherited from Xapian::Internal::intrusive_base
unsigned	_refs
	Reference count. More...

Protected Types inherited from Xapian::Database::Internal
enum	{ TRANSACTION_UNIMPLEMENTED = -1 , TRANSACTION_NONE = 0 , TRANSACTION_UNFLUSHED = 1 , TRANSACTION_FLUSHED = 2 }
	Transaction state. More...

Protected Member Functions inherited from Xapian::Database::Internal
bool	transaction_active () const

	Internal ()
	Create a database - called only by derived classes. More...

void	dtor_called ()
	Internal method to perform cleanup when a writable database is destroyed with uncommitted changes. More...

Protected Attributes inherited from Xapian::Database::Internal
enum Xapian::Database::Internal:: { ... }	transaction_state
	Transaction state. More...

Detailed Description

A backend designed for efficient indexing and retrieval, using compressed posting lists and a btree storage scheme.

Definition at line 59 of file chert_database.h.

Constructor & Destructor Documentation

◆ ChertDatabase()

ChertDatabase::ChertDatabase	(	const string &	db_dir_,
		int	action = `Xapian::DB_READONLY_`,
		unsigned int	block_size = `0u`
	)

Create and open a chert database.

Exceptions

Xapian::DatabaseCorruptError	is thrown if there is no consistent revision available.
Xapian::DatabaseOpeningError	thrown if database can't be opened.
Xapian::DatabaseVersionError	thrown if database is in an unsupported format. This implies that the database was created by an older or newer version of Xapian.

Parameters

dbdir	directory holding chert tables
block_size	Block size, in bytes, to use when creating tables. This is only important, and has the correct value, when the database is being created.

Definition at line 105 of file chert_database.cc.

References create_and_open_tables(), database_exists(), Xapian::DB_CREATE, Xapian::DB_CREATE_OR_OVERWRITE, db_dir, Xapian::DB_OPEN, get_database_write_lock(), ChertTable::get_latest_revision_number(), get_next_revision_number(), ChertTable::get_open_revision_number(), LOGCALL_CTOR, open_tables_consistent(), postlist_table, readonly, record_table, S_ISDIR, and set_revision_number().

◆ ~ChertDatabase()

ChertDatabase::~ChertDatabase ( )

Definition at line 176 of file chert_database.cc.

References LOGCALL_DTOR.

Member Function Documentation

◆ apply()

void ChertDatabase::apply ( )

private

Apply any outstanding changes to the tables.

If an error occurs during this operation, this will be signalled by an exception being thrown. In this case the contents of the tables on disk will be left in an unmodified state (though possibly with increased revision numbers), and the outstanding changes will be lost.

Definition at line 729 of file chert_database.cc.

References Xapian::Error::get_description(), get_next_revision_number(), get_revision_number(), ChertSpellingTable::is_modified(), ChertSynonymTable::is_modified(), ChertTable::is_modified(), ChertValueManager::is_modified(), LOGCALL_VOID, modifications_failed(), position_table, postlist_table, record_table, set_revision_number(), spelling_table, synonym_table, termlist_table, and value_manager.

Referenced by ChertWritableDatabase::apply().

◆ cancel()

void ChertDatabase::cancel ( )

privatevirtual

Cancel any outstanding changes to the tables.

Reimplemented from Xapian::Database::Internal.

Reimplemented in ChertWritableDatabase.

Definition at line 757 of file chert_database.cc.

References ChertSpellingTable::cancel(), ChertSynonymTable::cancel(), ChertTable::cancel(), ChertValueManager::cancel(), LOGCALL_VOID, position_table, postlist_table, record_table, spelling_table, synonym_table, termlist_table, and value_manager.

Referenced by ChertWritableDatabase::cancel(), and modifications_failed().

◆ close()

void ChertDatabase::close ( )

privatevirtual

Close all the tables permanently.

Implements Xapian::Database::Internal.

Reimplemented in ChertWritableDatabase.

Definition at line 499 of file chert_database.cc.

References ChertTable::close(), lock, LOGCALL_VOID, position_table, postlist_table, record_table, FlintLock::release(), spelling_table, synonym_table, and termlist_table.

Referenced by ChertWritableDatabase::close(), and modifications_failed().

◆ compact()

void ChertDatabase::compact	(	Xapian::Compactor *	compactor,
		const char *	destdir,
		const std::vector< Xapian::Database::Internal * > &	sources,
		const std::vector< Xapian::docid > &	offset,
		size_t	block_size,
		Xapian::Compactor::compaction_level	compaction,
		unsigned	flags,
		Xapian::docid	last_docid
	)

static

Definition at line 777 of file chert_compact.cc.

References Assert, CHERT_DEFAULT_BLOCK_SIZE, ChertTable::commit(), ChertTable::create_and_open(), Xapian::DBCOMPACT_MULTIPASS, Xapian::DBCOMPACT_SINGLE_FILE, DONT_COMPRESS, ChertTable::erase(), file_size(), ChertTable::flush_db(), Xapian::Compactor::FULLER, ChertTable::get_path(), has_uncommitted_changes(), FlintLock::lock(), ChertCompact::merge_docid_keyed(), ChertCompact::merge_postlists(), ChertCompact::merge_spellings(), ChertCompact::merge_synonyms(), ChertCompact::multimerge_postlists(), name, Glass::POSITION, position_table, Glass::POSTLIST, postlist_table, record_table, ChertTable::set_block_size(), ChertTable::set_full_compaction(), ChertTable::set_max_item_size(), Xapian::Compactor::set_status(), Glass::SPELLING, spelling_table, Xapian::Compactor::STANDARD, Xapian::Internal::str(), FlintLock::SUCCESS, SYNONYM, synonym_table, Glass::TERMLIST, termlist_table, and FlintLock::throw_databaselockerror().

Referenced by Xapian::Database::compact_().

◆ create_and_open_tables()

void ChertDatabase::create_and_open_tables ( unsigned int blocksize )

private

Create new tables, and open them.

Any existing tables will be removed first.

Definition at line 188 of file chert_database.cc.

References Assert, ChertVersion::create(), ChertLazyTable::create_and_open(), ChertTable::create_and_open(), ChertTermListTable::create_and_open(), database_exists(), ChertTable::get_open_revision_number(), LOGCALL_VOID, position_table, postlist_table, record_table, Xapian::revision(), spelling_table, stats, synonym_table, termlist_table, version_file, and ChertDatabaseStats::zero().

Referenced by ChertDatabase().

◆ database_exists()

bool ChertDatabase::database_exists ( )

private

Return true if a database exists at the path specified for this database.

Definition at line 182 of file chert_database.cc.

References ChertTable::exists(), LOGCALL, postlist_table, record_table, and RETURN.

Referenced by ChertDatabase(), create_and_open_tables(), and get_database_write_lock().

◆ get_backend_info()

int ChertDatabase::get_backend_info ( string * path ) const

inlinevirtual

Get backend information about this database.

Parameters

path	If non-NULL, and set the pointed to string to the file path of this database (or if to some string describing the database in a backend-specified format if "path" isn't a concept which make sense).

Returns: A constant indicating the backend type.

Implements Xapian::Database::Internal.

Definition at line 311 of file chert_database.h.

References BACKEND_CHERT, and db_dir.

◆ get_changeset_revisions()

void ChertDatabase::get_changeset_revisions	(	const string &	path,
		chert_revision_number_t *	startrev,
		chert_revision_number_t *	endrev
	)		const

private

Get the revision stored in a changeset.

Definition at line 338 of file chert_database.cc.

References CHANGES_MAGIC_STRING, CHANGES_VERSION, CONST_STRLEN, io_read(), O_CLOEXEC, posixy_open, REASONABLE_CHANGESET_SIZE, and unpack_uint().

Referenced by write_changesets_to_fd().

◆ get_database_write_lock()

void ChertDatabase::get_database_write_lock	(	int	flags,
		bool	creating
	)

private

Get a write lock on the database, or throw an Xapian::DatabaseLockError if failure.

Parameters

flags	Bit-wise or of zero or more Xapian::DB_* constants
creating	true if the database is in the process of being created - if false, will throw a DatabaseOpening error if the lock can't be acquired and the database doesn't exist.

Definition at line 512 of file chert_database.cc.

References database_exists(), db_dir, Xapian::DB_RETRY_LOCK, lock, FlintLock::lock(), LOGCALL_VOID, FlintLock::SUCCESS, FlintLock::throw_databaselockerror(), and FlintLock::UNKNOWN.

Referenced by ChertDatabase().

◆ get_doccount()

Xapian::doccount ChertDatabase::get_doccount ( ) const

virtual

Virtual methods of Database::Internal.

Implements Xapian::Database::Internal.

Definition at line 770 of file chert_database.cc.

References ChertRecordTable::get_doccount(), LOGCALL, record_table, and RETURN.

Referenced by ChertTermList::accumulate_stats(), check_db_dir(), open_post_list(), and ChertWritableDatabase::open_post_list().

◆ get_doclength()

Xapian::termcount ChertDatabase::get_doclength ( Xapian::docid did ) const

virtual

Get the length of a given document.

Document length, for the purposes of Xapian, is defined to be the number of instances of terms within a document. Expressed differently, the sum of the within document frequencies over all the terms in the document.

Parameters

did	The document id of the document whose length is being requested.

Implements Xapian::Database::Internal.

Reimplemented in ChertWritableDatabase.

Definition at line 791 of file chert_database.cc.

References Assert, ChertPostListTable::get_doclength(), LOGCALL, postlist_table, and RETURN.

Referenced by ChertModifiedPostList::get_doclength(), ChertPostList::get_doclength(), ChertWritableDatabase::get_doclength(), and open_document().

◆ get_doclength_lower_bound()

Xapian::termcount ChertDatabase::get_doclength_lower_bound ( ) const

virtual

Get a lower bound on the length of a document in this DB.

This bound does not include any zero-length documents.

Reimplemented from Xapian::Database::Internal.

Definition at line 847 of file chert_database.cc.

References ChertDatabaseStats::get_doclength_lower_bound(), and stats.

◆ get_doclength_upper_bound()

Xapian::termcount ChertDatabase::get_doclength_upper_bound ( ) const

virtual

Get an upper bound on the length of a document in this DB.

Reimplemented from Xapian::Database::Internal.

Definition at line 853 of file chert_database.cc.

References ChertDatabaseStats::get_doclength_upper_bound(), and stats.

◆ get_freqs()

void ChertDatabase::get_freqs	(	const string &	term,
		Xapian::doccount *	termfreq_ptr,
		Xapian::termcount *	collfreq_ptr
	)		const

virtual

Returns frequencies for a term.

Parameters

term	The term to get frequencies for
termfreq_ptr	Point to return number of docs indexed by term (or NULL not to return)
collfreq_ptr	Point to return number of occurrences of term in the database (or NULL not to return)

Implements Xapian::Database::Internal.

Reimplemented in ChertWritableDatabase.

Definition at line 816 of file chert_database.cc.

References Assert, ChertPostListTable::get_freqs(), LOGCALL_VOID, and postlist_table.

Referenced by ChertWritableDatabase::get_freqs(), ChertModifiedPostList::get_termfreq(), ChertTermList::get_termfreq(), and get_wdf_upper_bound().

◆ get_lastdocid()

Xapian::docid ChertDatabase::get_lastdocid ( ) const

virtual

Return the last used document id of this (sub) database.

Implements Xapian::Database::Internal.

Definition at line 777 of file chert_database.cc.

References ChertDatabaseStats::get_last_docid(), LOGCALL, RETURN, and stats.

Referenced by check_db_dir().

◆ get_metadata()

string ChertDatabase::get_metadata ( const string & key ) const

virtual

Get the metadata associated with a given key.

See Database::get_metadata() for more information.

Reimplemented from Xapian::Database::Internal.

Definition at line 990 of file chert_database.cc.

References ChertTable::get_exact_entry(), LOGCALL, postlist_table, and RETURN.

◆ get_next_revision_number()

chert_revision_number_t ChertDatabase::get_next_revision_number ( ) const

private

Get an object holding the next revision number which should be used in the tables.

Returns: the next revision number.

Definition at line 324 of file chert_database.cc.

References ChertTable::get_latest_revision_number(), LOGCALL, postlist_table, and RETURN.

Referenced by apply(), and ChertDatabase().

◆ get_postlist_cursor()

ChertCursor* ChertDatabase::get_postlist_cursor ( ) const

inline

Get a postlist table cursor (used by ChertValueList).

Definition at line 251 of file chert_database.h.

References ChertTable::cursor_get(), and postlist_table.

◆ get_revision_info()

string ChertDatabase::get_revision_info ( ) const

virtual

Get a string describing the current revision of the database.

Reimplemented from Xapian::Database::Internal.

Definition at line 1011 of file chert_database.cc.

References get_revision_number(), LOGCALL, pack_uint(), and RETURN.

◆ get_revision_number()

chert_revision_number_t ChertDatabase::get_revision_number ( ) const

Get an object holding the revision number which the tables are opened at.

Returns: the current revision number.

Definition at line 316 of file chert_database.cc.

References ChertTable::get_open_revision_number(), LOGCALL, postlist_table, and RETURN.

Referenced by apply(), check_db_dir(), get_revision_info(), send_whole_database(), set_revision_number(), and write_changesets_to_fd().

◆ get_spelling_frequency()

Xapian::doccount ChertDatabase::get_spelling_frequency ( const string & word ) const

virtual

Return the number of times word was added as a spelling.

Reimplemented from Xapian::Database::Internal.

Definition at line 969 of file chert_database.cc.

References ChertSpellingTable::get_word_frequency(), and spelling_table.

◆ get_total_length()

Xapian::totallength ChertDatabase::get_total_length ( ) const

virtual

Return the total length of all documents in this database.

Implements Xapian::Database::Internal.

Definition at line 784 of file chert_database.cc.

References ChertDatabaseStats::get_total_doclen(), LOGCALL, RETURN, and stats.

◆ get_unique_terms()

Xapian::termcount ChertDatabase::get_unique_terms ( Xapian::docid did ) const

virtual

Get the number of unique term in document.

Parameters

did	The document id of the document whose number of terms is being requested.

Implements Xapian::Database::Internal.

Reimplemented in ChertWritableDatabase.

Definition at line 800 of file chert_database.cc.

References Assert, ChertTermList::get_approx_size(), ChertPostListTable::get_doclength(), LOGCALL, postlist_table, and RETURN.

Referenced by ChertAllDocsModifiedPostList::get_unique_terms(), ChertModifiedPostList::get_unique_terms(), ChertPostList::get_unique_terms(), and ChertWritableDatabase::get_unique_terms().

◆ get_used_docid_range()

void ChertDatabase::get_used_docid_range	(	Xapian::docid &	first,
		Xapian::docid &	last
	)		const

virtual

Find lowest and highest docids actually in use.

Only used by compaction, so only needs to be implemented by backends which support compaction.

Reimplemented from Xapian::Database::Internal.

Definition at line 1037 of file chert_database.cc.

References ChertRecordTable::get_doccount(), ChertDatabaseStats::get_last_docid(), ChertPostListTable::get_used_docid_range(), postlist_table, record_table, and stats.

◆ get_uuid()

string ChertDatabase::get_uuid ( ) const

virtual

Get a UUID for the database.

The UUID will persist for the lifetime of the database.

Replicas (eg, made with the replication protocol, or by copying all the database files) will have the same UUID. However, copies (made with copydatabase, or xapian-compact) will have different UUIDs.

If the backend does not support UUIDs the empty string is returned.

Reimplemented from Xapian::Database::Internal.

Definition at line 1020 of file chert_database.cc.

References ChertVersion::get_uuid_string(), LOGCALL, RETURN, and version_file.

Referenced by send_whole_database(), and write_changesets_to_fd().

◆ get_value_freq()

Xapian::doccount ChertDatabase::get_value_freq ( Xapian::valueno slot ) const

virtual

Return the frequency of a given value slot.

This is the number of documents which have a (non-empty) value stored in the slot.

Parameters

slot	The value slot to examine.

Implements Xapian::Database::Internal.

Reimplemented in ChertWritableDatabase.

Definition at line 826 of file chert_database.cc.

References ChertValueManager::get_value_freq(), LOGCALL, RETURN, and value_manager.

Referenced by ChertWritableDatabase::get_value_freq().

◆ get_value_lower_bound()

std::string ChertDatabase::get_value_lower_bound ( Xapian::valueno slot ) const

virtual

Get a lower bound on the values stored in the given value slot.

If there are no values stored in the given value slot, this will return an empty string.

Parameters

slot	The value slot to examine.

Implements Xapian::Database::Internal.

Reimplemented in ChertWritableDatabase.

Definition at line 833 of file chert_database.cc.

References ChertValueManager::get_value_lower_bound(), LOGCALL, RETURN, and value_manager.

Referenced by ChertWritableDatabase::get_value_lower_bound().

◆ get_value_upper_bound()

std::string ChertDatabase::get_value_upper_bound ( Xapian::valueno slot ) const

virtual

Get an upper bound on the values stored in the given value slot.

If there are no values stored in the given value slot, this will return an empty string.

Parameters

slot	The value slot to examine.

Implements Xapian::Database::Internal.

Reimplemented in ChertWritableDatabase.

Definition at line 840 of file chert_database.cc.

References ChertValueManager::get_value_upper_bound(), LOGCALL, RETURN, and value_manager.

Referenced by ChertWritableDatabase::get_value_upper_bound().

◆ get_wdf_upper_bound()

Xapian::termcount ChertDatabase::get_wdf_upper_bound ( const string & term ) const

virtual

Get an upper bound on the wdf of term term.

Reimplemented from Xapian::Database::Internal.

Definition at line 859 of file chert_database.cc.

References get_freqs(), ChertDatabaseStats::get_wdf_upper_bound(), and stats.

◆ has_positions()

bool ChertDatabase::has_positions ( ) const

virtual

Check whether this database contains any positional information.

Implements Xapian::Database::Internal.

Definition at line 875 of file chert_database.cc.

References ChertTable::empty(), and position_table.

◆ has_uncommitted_changes()

bool ChertDatabase::has_uncommitted_changes ( ) const

virtual

Return true if there are uncommitted changes.

Reimplemented in ChertWritableDatabase.

Definition at line 1056 of file chert_database.cc.

Referenced by compact().

◆ locked()

bool ChertDatabase::locked ( ) const

virtual

Return true if the database is open for writing.

If this is a WritableDatabase, always returns true.

For a Database, test if there's a writer holding the lock (or if we can't test for a lock without taking it on the current platform, throw Xapian::UnimplementedError).

Reimplemented from Xapian::Database::Internal.

Definition at line 1050 of file chert_database.cc.

References lock, and FlintLock::test().

◆ modifications_failed()

void ChertDatabase::modifications_failed	(	chert_revision_number_t	old_revision,
		chert_revision_number_t	new_revision,
		const std::string &	msg
	)

private

Called if a modifications fail.

Parameters

msg	is a string description of the exception that was raised when the modifications failed.

Definition at line 701 of file chert_database.cc.

References cancel(), close(), Xapian::Error::get_msg(), open_tables(), and set_revision_number().

Referenced by apply().

◆ open_allterms()

TermList * ChertDatabase::open_allterms ( const string & prefix ) const

virtual

Open an allterms list.

This is a list of all the terms in the database

Parameters

prefix The prefix to restrict the terms to.

Returns: A pointer to the newly created allterms list. This object must be deleted by the caller after use.

Implements Xapian::Database::Internal.

Reimplemented in ChertWritableDatabase.

Definition at line 946 of file chert_database.cc.

References ChertAllTermsList, LOGCALL, and RETURN.

Referenced by ChertWritableDatabase::open_allterms().

◆ open_document()

Xapian::Document::Internal * ChertDatabase::open_document	(	Xapian::docid	did,
		bool	lazy
	)		const

virtual

Open a document.

This is used to access the values and data associated with a document. See class Xapian::Document::Internal for further details.

Parameters

did	The document id which is being requested.
lazy	No need to check that this document actually exists. Used when we already know that this document exists (only a hint - the backend may still check).

Returns: A pointer to the newly created document object. This object must be deleted by the caller after use.

Implements Xapian::Database::Internal.

Reimplemented in ChertWritableDatabase.

Definition at line 917 of file chert_database.cc.

References Assert, get_doclength(), LOGCALL, record_table, RETURN, and value_manager.

Referenced by ChertWritableDatabase::open_document().

◆ open_metadata_keylist()

TermList * ChertDatabase::open_metadata_keylist ( const std::string & prefix ) const

virtual

Open a termlist returning each metadata key.

Only metadata keys which are associated with a non-empty value will be returned.

Parameters

prefix If non-empty, only keys with this prefix are returned.

Reimplemented from Xapian::Database::Internal.

Definition at line 1001 of file chert_database.cc.

References ChertTable::cursor_get(), LOGCALL, postlist_table, and RETURN.

◆ open_position_list()

PositionList * ChertDatabase::open_position_list	(	Xapian::docid	did,
		const string &	tname
	)		const

virtual

Open a position list for the given term in the given document.

Parameters

did	The document id for which a position list is being requested.
tname	The term for which a position list is being requested.

Returns: A pointer to the newly created position list. This object must be deleted by the caller after use.

Implements Xapian::Database::Internal.

Definition at line 931 of file chert_database.cc.

References Assert, and position_table.

Referenced by ChertModifiedPostList::open_position_list(), and ChertModifiedPostList::read_position_list().

◆ open_post_list()

LeafPostList * ChertDatabase::open_post_list ( const string & tname ) const

virtual

Open a posting list.

Method defined by subclass to open a posting list. This is a list of all the documents which contain a given term.

Parameters

tname The term whose posting list is being requested.

Returns: A pointer to the newly created posting list. If the term doesn't exist, a LeafPostList object returning no documents is returned, which makes it easier to implement a search over multiple databases. This object must be deleted by the caller after use.

Implements Xapian::Database::Internal.

Reimplemented in ChertWritableDatabase.

Definition at line 881 of file chert_database.cc.

References ChertAllDocsPostList, ChertPostList, get_doccount(), ChertDatabaseStats::get_last_docid(), LOGCALL, RETURN, and stats.

◆ open_spelling_termlist()

TermList * ChertDatabase::open_spelling_termlist ( const string & word ) const

virtual

Create a termlist tree from trigrams of word.

You can assume word.size() > 1.

If there are no trigrams, returns NULL.

Reimplemented from Xapian::Database::Internal.

Definition at line 954 of file chert_database.cc.

References ChertSpellingTable::open_termlist(), and spelling_table.

◆ open_spelling_wordlist()

TermList * ChertDatabase::open_spelling_wordlist ( ) const

virtual

Return a termlist which returns the words which are spelling correction targets.

If there are no spelling correction targets, returns NULL.

Reimplemented from Xapian::Database::Internal.

Reimplemented in ChertWritableDatabase.

Definition at line 960 of file chert_database.cc.

References ChertTable::cursor_get(), and spelling_table.

Referenced by ChertWritableDatabase::open_spelling_wordlist().

◆ open_synonym_keylist()

TermList * ChertDatabase::open_synonym_keylist ( const string & prefix ) const

virtual

Open a termlist returning each term which has synonyms.

Parameters

prefix If non-empty, only terms with this prefix are returned.

Reimplemented from Xapian::Database::Internal.

Reimplemented in ChertWritableDatabase.

Definition at line 981 of file chert_database.cc.

References ChertTable::cursor_get(), and synonym_table.

Referenced by ChertWritableDatabase::open_synonym_keylist().

◆ open_synonym_termlist()

TermList * ChertDatabase::open_synonym_termlist ( const string & term ) const

virtual

Open a termlist returning synonyms for a term.

If term has no synonyms, returns NULL.

Reimplemented from Xapian::Database::Internal.

Definition at line 975 of file chert_database.cc.

References ChertSynonymTable::open_termlist(), and synonym_table.

◆ open_tables()

void ChertDatabase::open_tables ( chert_revision_number_t revision )

private

Open tables at specified revision number.

Exceptions

Xapian::InvalidArgumentError is thrown if the specified revision is not available.

Definition at line 293 of file chert_database.cc.

References ChertTable::get_block_size(), LOGCALL_VOID, ChertTable::open(), ChertPostListTable::open(), position_table, postlist_table, ChertVersion::read_and_check(), record_table, ChertValueManager::reset(), Xapian::revision(), ChertTable::set_block_size(), spelling_table, synonym_table, termlist_table, value_manager, and version_file.

Referenced by modifications_failed().

◆ open_tables_consistent()

bool ChertDatabase::open_tables_consistent ( )

private

Open all tables at most recent consistent revision.

Returns: true if the tables were reopened; false if we could tell they were already open at the latest revision.

Exceptions

Xapian::DatabaseCorruptError is thrown if there is no consistent revision available.

Definition at line 216 of file chert_database.cc.

References ChertTable::get_block_size(), ChertTable::get_open_revision_number(), LOGCALL, MAX_OPEN_RETRIES, ChertTable::open(), ChertPostListTable::open(), position_table, postlist_table, ChertDatabaseStats::read(), ChertVersion::read_and_check(), record_table, ChertValueManager::reset(), RETURN, Xapian::revision(), ChertTable::set_block_size(), spelling_table, stats, synonym_table, termlist_table, value_manager, and version_file.

Referenced by ChertDatabase(), and reopen().

◆ open_term_list()

TermList * ChertDatabase::open_term_list ( Xapian::docid did ) const

virtual

Open a term list.

This is a list of all the terms contained by a given document.

Parameters

did	The document id whose term list is being requested.

Returns: A pointer to the newly created term list. This object must be deleted by the caller after use.

Implements Xapian::Database::Internal.

Definition at line 906 of file chert_database.cc.

References Assert, ChertTermList, ChertTable::is_open(), LOGCALL, RETURN, termlist_table, and throw_termlist_table_close_exception().

◆ open_value_list()

ValueList * ChertDatabase::open_value_list ( Xapian::valueno slot ) const

virtual

Open a value stream.

This returns the value in a particular slot for each document.

Parameters

slot	The value slot.

Returns: Pointer to a new ValueList object which should be deleted by the caller once it is no longer needed.

Reimplemented from Xapian::Database::Internal.

Reimplemented in ChertWritableDatabase.

Definition at line 898 of file chert_database.cc.

References LOGCALL, and RETURN.

Referenced by ChertWritableDatabase::open_value_list().

◆ readahead_for_query()

void ChertDatabase::readahead_for_query ( const Xapian::Query & query )

virtual

Reimplemented from Xapian::Database::Internal.

Definition at line 480 of file chert_database.cc.

References Xapian::Query::get_unique_terms_begin(), ChertPostListTable::make_key(), postlist_table, query(), and ChertTable::readahead_key().

◆ reopen()

bool ChertDatabase::reopen ( )

privatevirtual

Re-open tables to recover from an overwritten condition, or just get most up-to-date version.

Reimplemented from Xapian::Database::Internal.

Definition at line 491 of file chert_database.cc.

References LOGCALL, open_tables_consistent(), readonly, and RETURN.

Referenced by write_changesets_to_fd().

◆ request_document()

void ChertDatabase::request_document ( Xapian::docid ) const

virtual

Request and later collect a document from the database.

Multiple documents can be requested with request_document(), and then collected with collect_document(). Allows the backend to optimise (e.g. the remote backend can start requests for all the documents so they fetch in parallel).

If a backend doesn't support this, request_document() can be a no-op and collect_document() the same as open_document().

Reimplemented from Xapian::Database::Internal.

Definition at line 474 of file chert_database.cc.

References ChertRecordTable::readahead_for_record(), and record_table.

◆ send_whole_database()

void ChertDatabase::send_whole_database	(	RemoteConnection &	conn,
		double	end_time
	)

private

Send a set of messages which transfer the whole database.

Definition at line 530 of file chert_database.cc.

References db_dir, encode_length(), RealTime::end_time(), get_revision_number(), get_uuid(), LOGCALL_VOID, O_CLOEXEC, pack_uint(), posixy_open, REPL_REPLY_DB_FILEDATA, REPL_REPLY_DB_FILENAME, REPL_REPLY_DB_HEADER, RemoteConnection::send_file(), and RemoteConnection::send_message().

Referenced by write_changesets_to_fd().

◆ set_revision_number()

void ChertDatabase::set_revision_number ( chert_revision_number_t new_revision )

private

Set the revision number in the tables.

This updates the disk tables so that the currently open revision becomes the specified revision number.

Parameters

new_revision The new revision number to store. This must be greater than the latest revision number (see get_latest_revision_number()), or undefined behaviour will result.

Definition at line 380 of file chert_database.cc.

References CHANGES_MAGIC_STRING, CHANGES_VERSION, ChertTable::commit(), create_changeset_file(), db_dir, ChertSpellingTable::flush_db(), ChertSynonymTable::flush_db(), ChertTable::flush_db(), get_revision_number(), io_unlink(), io_write(), LOGCALL_VOID, max_changesets, ChertValueManager::merge_changes(), pack_uint(), position_table, postlist_table, record_table, spelling_table, Xapian::Internal::str(), synonym_table, termlist_table, value_manager, and ChertTable::write_changed_blocks().

Referenced by apply(), ChertDatabase(), and modifications_failed().

◆ term_exists()

bool ChertDatabase::term_exists ( const string & tname ) const

virtual

Check whether a given term is in the database.

Parameters

tname The term whose presence is being checked.

Implements Xapian::Database::Internal.

Reimplemented in ChertWritableDatabase.

Definition at line 867 of file chert_database.cc.

References Assert, LOGCALL, postlist_table, RETURN, and ChertPostListTable::term_exists().

◆ throw_termlist_table_close_exception()

void ChertDatabase::throw_termlist_table_close_exception ( ) const

Definition at line 1027 of file chert_database.cc.

References ChertTable::is_open(), postlist_table, and ChertTable::throw_database_closed().

Referenced by ChertWritableDatabase::delete_document(), open_term_list(), and ChertWritableDatabase::replace_document().

◆ write_changesets_to_fd()

void ChertDatabase::write_changesets_to_fd	(	int	fd,
		const string &	start_revision,
		bool	need_whole_db,
		Xapian::ReplicationInfo *	info
	)

virtual

Write a set of changesets to a file descriptor.

This call may reopen the database, leaving it pointing to a more recent version of the database.

Reimplemented from Xapian::Database::Internal.

Definition at line 570 of file chert_database.cc.

References Xapian::ReplicationInfo::changed, Xapian::ReplicationInfo::changeset_count, db_dir, Xapian::ReplicationInfo::fullcopy_count, get_changeset_revisions(), get_revision_number(), get_uuid(), LOGCALL_VOID, MAX_DB_COPIES_PER_CONVERSATION, O_CLOEXEC, pack_uint(), posixy_open, reopen(), REPL_REPLY_CHANGESET, REPL_REPLY_DB_FOOTER, REPL_REPLY_END_OF_CHANGES, REPL_REPLY_FAIL, Xapian::revision(), RemoteConnection::send_file(), RemoteConnection::send_message(), send_whole_database(), Xapian::Internal::str(), and unpack_uint().

Friends And Related Function Documentation

◆ ChertAllDocsPostList

friend class ChertAllDocsPostList

friend

Definition at line 64 of file chert_database.h.

Referenced by open_post_list(), and ChertWritableDatabase::open_post_list().

◆ ChertAllTermsList

friend class ChertAllTermsList

friend

Definition at line 63 of file chert_database.h.

Referenced by open_allterms().

◆ ChertPostList

friend class ChertPostList

friend

Definition at line 62 of file chert_database.h.

Referenced by open_post_list(), and ChertWritableDatabase::open_post_list().

◆ ChertTermList

friend class ChertTermList

friend

Definition at line 61 of file chert_database.h.

Referenced by open_term_list().

◆ ChertWritableDatabase

friend class ChertWritableDatabase

friend

Definition at line 60 of file chert_database.h.

Member Data Documentation

◆ db_dir

std::string ChertDatabase::db_dir

private

Directory to store databases in.

Definition at line 68 of file chert_database.h.

Referenced by ChertDatabase(), get_backend_info(), get_database_write_lock(), send_whole_database(), set_revision_number(), and write_changesets_to_fd().

◆ lock

FlintLock ChertDatabase::lock

private

Lock object.

Definition at line 118 of file chert_database.h.

Referenced by close(), get_database_write_lock(), and locked().

◆ max_changesets

unsigned int ChertDatabase::max_changesets

private

The maximum number of changesets which should be kept in the database.

Definition at line 122 of file chert_database.h.

Referenced by set_revision_number().

◆ position_table

ChertPositionListTable ChertDatabase::position_table

private

◆ postlist_table

ChertPostListTable ChertDatabase::postlist_table

mutableprivate

Table storing posting lists.

Whenever an update is performed, this table is the first to be updated: therefore, its most recent revision number is the most recent anywhere in the database.

Definition at line 86 of file chert_database.h.

Referenced by apply(), cancel(), ChertWritableDatabase::cancel(), ChertDatabase(), close(), compact(), create_and_open_tables(), database_exists(), ChertWritableDatabase::flush_postlist_changes(), get_doclength(), get_freqs(), get_metadata(), get_next_revision_number(), get_postlist_cursor(), get_revision_number(), get_unique_terms(), get_used_docid_range(), ChertWritableDatabase::has_uncommitted_changes(), ChertAllTermsList::next(), open_metadata_keylist(), open_tables(), open_tables_consistent(), readahead_for_query(), ChertWritableDatabase::replace_document(), ChertWritableDatabase::set_metadata(), set_revision_number(), ChertAllTermsList::skip_to(), term_exists(), and throw_termlist_table_close_exception().

◆ readonly

bool ChertDatabase::readonly

private

Whether the database is readonly.

Definition at line 72 of file chert_database.h.

Referenced by ChertDatabase(), and reopen().

◆ record_table

ChertRecordTable ChertDatabase::record_table

private

Table storing records.

Whenever an update is performed, this table is the last to be updated: therefore, its most recent revision number is the most recent consistent revision available. If this table's most recent revision number is not available for all tables, there is no consistent revision available, and the database is corrupt.

Definition at line 115 of file chert_database.h.

Referenced by ChertWritableDatabase::add_document_(), apply(), cancel(), ChertDatabase(), close(), compact(), create_and_open_tables(), database_exists(), ChertWritableDatabase::delete_document(), get_doccount(), get_used_docid_range(), ChertWritableDatabase::has_uncommitted_changes(), open_document(), open_tables(), open_tables_consistent(), ChertWritableDatabase::replace_document(), request_document(), and set_revision_number().

◆ spelling_table

ChertSpellingTable ChertDatabase::spelling_table

mutableprivate

Table storing spelling correction data.

Definition at line 105 of file chert_database.h.

Referenced by ChertWritableDatabase::add_spelling(), apply(), cancel(), close(), compact(), create_and_open_tables(), get_spelling_frequency(), ChertWritableDatabase::has_uncommitted_changes(), open_spelling_termlist(), open_spelling_wordlist(), ChertWritableDatabase::open_spelling_wordlist(), open_tables(), open_tables_consistent(), ChertWritableDatabase::remove_spelling(), and set_revision_number().

◆ stats

ChertDatabaseStats ChertDatabase::stats

private

Database statistics.

Definition at line 125 of file chert_database.h.

Referenced by ChertWritableDatabase::add_document(), ChertWritableDatabase::add_document_(), ChertWritableDatabase::cancel(), create_and_open_tables(), ChertWritableDatabase::delete_document(), ChertWritableDatabase::flush_postlist_changes(), get_doclength_lower_bound(), get_doclength_upper_bound(), get_lastdocid(), get_total_length(), get_used_docid_range(), get_wdf_upper_bound(), open_post_list(), ChertWritableDatabase::open_post_list(), open_tables_consistent(), and ChertWritableDatabase::replace_document().

◆ synonym_table

ChertSynonymTable ChertDatabase::synonym_table

mutableprivate

Table storing synonym data.

Definition at line 101 of file chert_database.h.

Referenced by ChertWritableDatabase::add_synonym(), apply(), cancel(), ChertWritableDatabase::clear_synonyms(), close(), compact(), create_and_open_tables(), ChertWritableDatabase::has_uncommitted_changes(), open_synonym_keylist(), ChertWritableDatabase::open_synonym_keylist(), open_synonym_termlist(), open_tables(), open_tables_consistent(), ChertWritableDatabase::remove_synonym(), and set_revision_number().

◆ termlist_table

ChertTermListTable ChertDatabase::termlist_table

private

Table storing term lists.

Definition at line 94 of file chert_database.h.

Referenced by ChertWritableDatabase::add_document_(), apply(), cancel(), ChertTermList::ChertTermList(), close(), compact(), create_and_open_tables(), ChertWritableDatabase::delete_document(), ChertWritableDatabase::has_uncommitted_changes(), open_tables(), open_tables_consistent(), open_term_list(), ChertWritableDatabase::replace_document(), and set_revision_number().

◆ value_manager

ChertValueManager ChertDatabase::value_manager

mutableprivate

Value manager.

Definition at line 97 of file chert_database.h.

Referenced by ChertWritableDatabase::add_document_(), apply(), ChertWritableDatabase::apply(), cancel(), ChertWritableDatabase::delete_document(), get_value_freq(), get_value_lower_bound(), get_value_upper_bound(), ChertWritableDatabase::has_uncommitted_changes(), open_document(), open_tables(), open_tables_consistent(), ChertWritableDatabase::open_value_list(), ChertWritableDatabase::replace_document(), and set_revision_number().

◆ version_file

ChertVersion ChertDatabase::version_file

private

The file describing the Chert database.

This file has information about the format of the database which can't easily be stored in any of the individual tables.

Definition at line 78 of file chert_database.h.

Referenced by create_and_open_tables(), get_uuid(), open_tables(), and open_tables_consistent().

The documentation for this class was generated from the following files:

backends/chert/chert_database.h
backends/chert/chert_compact.cc
backends/chert/chert_database.cc

Public Member Functions

Static Public Member Functions

Private Member Functions

Private Attributes

Friends

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ ChertDatabase()

◆ ~ChertDatabase()

Member Function Documentation

◆ apply()

◆ cancel()

◆ close()

◆ compact()

◆ create_and_open_tables()

◆ database_exists()

◆ get_backend_info()

◆ get_changeset_revisions()

◆ get_database_write_lock()

◆ get_doccount()

◆ get_doclength()

◆ get_doclength_lower_bound()

◆ get_doclength_upper_bound()

◆ get_freqs()

◆ get_lastdocid()

◆ get_metadata()

◆ get_next_revision_number()

◆ get_postlist_cursor()

◆ get_revision_info()

◆ get_revision_number()

◆ get_spelling_frequency()

◆ get_total_length()

◆ get_unique_terms()

◆ get_used_docid_range()

◆ get_uuid()

◆ get_value_freq()

◆ get_value_lower_bound()

◆ get_value_upper_bound()

◆ get_wdf_upper_bound()

◆ has_positions()

◆ has_uncommitted_changes()

◆ locked()

◆ modifications_failed()

◆ open_allterms()

◆ open_document()

◆ open_metadata_keylist()

◆ open_position_list()

◆ open_post_list()

◆ open_spelling_termlist()

◆ open_spelling_wordlist()

◆ open_synonym_keylist()

◆ open_synonym_termlist()

◆ open_tables()

◆ open_tables_consistent()

◆ open_term_list()

◆ open_value_list()

◆ readahead_for_query()

◆ reopen()

◆ request_document()

◆ send_whole_database()

◆ set_revision_number()

◆ term_exists()

◆ throw_termlist_table_close_exception()

◆ write_changesets_to_fd()

Friends And Related Function Documentation

◆ ChertAllDocsPostList

◆ ChertAllTermsList

◆ ChertPostList

◆ ChertTermList

◆ ChertWritableDatabase

Member Data Documentation

◆ db_dir

◆ lock

◆ max_changesets

◆ position_table

◆ postlist_table

◆ readonly

◆ record_table

◆ spelling_table