xapian Package

xapian Package

Xapian is a highly adaptable toolkit which allows developers to easily add advanced indexing and search facilities to their own applications. It has built-in support for several families of weighting models and also supports a rich set of boolean query operators.

In addition to the doc strings provided by this python library, you may wish to look at the library’s overall documentation, either installed along with the bindings or online at <https://xapian.org/docs/bindings/python/>, as well as the library’s documentation, possibly installed with the library or with its development files, or again online at <https://xapian.org/docs/>.

exception xapian.AssertionError(*args)[source]

Bases: xapian.LogicError

AssertionError is thrown if a logical assertion inside Xapian fails.

In a debug build of Xapian, a failed assertion in the core library code will cause AssertionError to be thrown.

This represents a bug in Xapian (either an invariant, precondition, etc has been violated, or the assertion is incorrect!)

class xapian.BB2Weight(*args)[source]

Bases: xapian.Weight

This class implements the BB2 weighting scheme.

BB2 is a representative scheme of the Divergence from Randomness Framework by Gianni Amati.

It uses the Bose-Einstein probabilistic distribution (B) along with Stirling’s power approximation, the Bernoulli method to find the aftereffect of sampling (B) and the second wdf normalization proposed by Amati to normalize the wdf in the document to the length of the document (H2).

For more information about the DFR Framework and the BB2 scheme, please refer to : Gianni Amati and Cornelis Joost Van Rijsbergen Probabilistic models of information retrieval based on measuring the divergence from randomness ACM Transactions on Information Systems (TOIS) 20, (4), 2002, pp. 357-389.

thisown

The membership flag

class xapian.BM25PlusWeight(*args)[source]

Bases: xapian.Weight

Xapian::Weight subclass implementing the BM25+ probabilistic formula.

thisown

The membership flag

class xapian.BM25Weight(*args)[source]

Bases: xapian.Weight

Xapian::Weight subclass implementing the BM25 probabilistic formula.

thisown

The membership flag

class xapian.BoolWeight[source]

Bases: xapian.Weight

Class implementing a “boolean” weighting scheme.

This weighting scheme gives all documents zero weight.

thisown

The membership flag

class xapian.Compactor[source]

Bases: object

Compact a database, or merge and compact several.

FULL = 1
FULLER = 2
STANDARD = 0
add_source()

Add a source database.

void Xapian::Compactor::add_source(const std::string &srcdir)

Deprecated Use Database::compact(destdir[, compactor]) instead.

srcdir: The path to the source database to add.

compact()

Perform the actual compaction/merging operation.

void Xapian::Compactor::compact()

Deprecated Use Database::compact(destdir[, compactor]) instead.

resolve_duplicate_metadata()

Resolve multiple user metadata entries with the same key.

virtual std::string Xapian::Compactor::resolve_duplicate_metadata(const std::string &key, size_t num_tags, const std::string tags[])

When merging, if the same user metadata key is set in more than one input, then this method is called to allow this to be resolving in an appropriate way.

The default implementation just returns tags[0].

For multipass this will currently get called multiple times for the same key if there are duplicates to resolve in each pass, but this may change in the future.

Since 1.4.6, an implementation of this method can return an empty string to indicate that the appropriate result is to not set a value for this user metadata key in the output database. In older versions, you should not return an empty string.

key: The metadata key with duplicate entries.

num_tags: How many tags there are.

tags: An array of num_tags strings containing the tags to merge.

set_block_size()

Set the block size to use for tables in the output database.

void Xapian::Compactor::set_block_size(size_t block_size)

block_size: The block size to use. Valid block sizes are currently powers of two between 2048 and 65536, with the default being 8192, but the valid sizes and default may change in the future.

set_compaction_level()

Set the compaction level.

void Xapian::Compactor::set_compaction_level(compaction_level compaction)

compaction: Available values are: Xapian::Compactor::STANDARD - Don’t split items unnecessarily.

Xapian::Compactor::FULL - Split items whenever it saves space (the default).

Xapian::Compactor::FULLER - Allow oversize items to save more space (not recommended if you ever plan to update the compacted database).

set_destdir()

Set where to write the output.

void Xapian::Compactor::set_destdir(const std::string &destdir)

Deprecated Use Database::compact(destdir[, compactor]) instead.

destdir: Output path. This can be the same as an input if that input is a stub database (in which case the database(s) listed in the stub will be compacted to a new database and then the stub will be atomically updated to point to this new database).

set_multipass()

Set whether to merge postlists in multiple passes.

void Xapian::Compactor::set_multipass(bool multipass)

multipass: If true and merging more than 3 databases, merge the postlists in multiple passes, which is generally faster but requires more disk space for temporary files. By default we don’t do this.

set_renumber()

Set whether to preserve existing document id values.

void Xapian::Compactor::set_renumber(bool renumber)

renumber: The default is true, which means that document ids will be renumbered - currently by applying the same offset to all the document ids in a particular source database.

If false, then the document ids must be unique over all source databases. Currently the ranges of document ids in each source must not overlap either, though this restriction may be removed in the future.

set_status()

Update progress.

virtual void Xapian::Compactor::set_status(const std::string &table, const std::string &status)

Subclass this method if you want to get progress updates during compaction. This is called for each table first with empty status, And then one or more times with non-empty status.

The default implementation does nothing.

table: The table currently being compacted.

status: A status message.

thisown

The membership flag

class xapian.CoordWeight[source]

Bases: xapian.Weight

Xapian::Weight subclass implementing Coordinate Matching.

Each matching term score one point. See Managing Gigabytes, Second Edition p181.

init()

Allow the subclass to perform any initialisation it needs to.

void Xapian::CoordWeight::init(double factor_)

factor: Any scaling factor (e.g. from OP_SCALE_WEIGHT). If the Weight object is for the term-independent weight supplied by get_sumextra()/get_maxextra(), then init(0.0) is called (starting from Xapian 1.2.11 and 1.3.1 - earlier versions failed to call init() for such Weight objects).

thisown

The membership flag

class xapian.DLHWeight[source]

Bases: xapian.Weight

This class implements the DLH weighting scheme, which is a representative scheme of the Divergence from Randomness Framework by Gianni Amati.

This is a parameter free weighting scheme and it should be used with query expansion to obtain better results. It uses the HyperGeometric Probabilistic model and Laplace’s normalization to calculate the risk gain.

For more information about the DFR Framework and the DLH scheme, please refer to : a.) Gianni Amati and Cornelis Joost Van Rijsbergen Probabilistic models of information retrieval based on measuring the divergence from randomness ACM Transactions on Information Systems (TOIS) 20, (4), 2002, pp. 357-389. b.) FUB, IASI-CNR and University of Tor Vergata at TREC 2007 Blog Track. G. Amati and E. Ambrosi and M. Bianchi and C. Gaibisso and G. Gambosi. Proceedings of the 16th Text REtrieval Conference (TREC-2007), 2008.

thisown

The membership flag

class xapian.DPHWeight[source]

Bases: xapian.Weight

This class implements the DPH weighting scheme.

DPH is a representative scheme of the Divergence from Randomness Framework by Gianni Amati.

This is a parameter free weighting scheme and it should be used with query expansion to obtain better results. It uses the HyperGeometric Probabilistic model and Popper’s normalization to calculate the risk gain.

For more information about the DFR Framework and the DPH scheme, please refer to : a.) Gianni Amati and Cornelis Joost Van Rijsbergen Probabilistic models of information retrieval based on measuring the divergence from randomness ACM Transactions on Information Systems (TOIS) 20, (4), 2002, pp. 357-389. b.) FUB, IASI-CNR and University of Tor Vergata at TREC 2007 Blog Track. G. Amati and E. Ambrosi and M. Bianchi and C. Gaibisso and G. Gambosi. Proceedings of the 16th Text Retrieval Conference (TREC-2007), 2008.

thisown

The membership flag

class xapian.Database(*args)[source]

Bases: object

This class is used to access a database, or a group of databases.

For searching, this class is used in conjunction with an Enquire object.

InvalidArgumentError: will be thrown if an invalid argument is supplied, for example, an unknown database type.

DatabaseOpeningError: may be thrown if the database cannot be opened (for example, a required file cannot be found).

DatabaseVersionError: may be thrown if the database is in an unsupported format (for example, created by a newer version of Xapian which uses an incompatible format).

add_database()

Add an existing database (or group of databases) to those accessed by this object.

void Xapian::Database::add_database(const Database &database)

database: the database(s) to add.

allterms(prefix=None)

Get an iterator over all the terms in the database.

The iterator will return TermListItem objects, but these will not support access to wdf, or position information.

Access to term frequency information is only available until the iterator has moved on.

If prefix is supplied, only terms which start with that prefix will be returned.

static check()
close()

Close the database.

virtual void Xapian::Database::close()

This closes the database and closes all its file handles.

For a WritableDatabase, if a transaction is active it will be aborted, while if no transaction is active commit() will be implicitly called. Also the write lock is released.

Closing a database cannot be undone - in particular, calling reopen() after close() will not reopen it, but will instead throw a Xapian::DatabaseError exception.

Calling close() again on a database which has already been closed has no effect (and doesn’t raise an exception).

After close() has been called, calls to other methods of the database, and to methods of other objects associated with the database, will either:

behave exactly as they would have done if the database had not been closed (this can only happen if all the required data is cached)

raise a Xapian::DatabaseError exception indicating that the database is closed.

The reason for this behaviour is that otherwise we’d have to check that the database is still open on every method call on every object associated with a Database, when in many cases they are working on data which has already been loaded and so they are able to just behave correctly.

This method was added in Xapian 1.1.0.

compact()

Produce a compact version of this database.

void Xapian::Database::compact(int fd, unsigned flags, int block_size, Xapian::Compactor &compactor)

New 1.3.4. Various methods of the Compactor class were deprecated in 1.3.4.

The compactor functor allows handling progress output and specifying how user metadata is merged.

This variant writes a single-file database to the specified file descriptor. Only the glass backend supports such databases, so this form is only supported for this backend.

fd: File descriptor to write the compact version to. The descriptor needs to be readable and writable (open with O_RDWR) and seekable. The current file offset is used, allowing compacting to a single file database embedded within another file. Xapian takes ownership of the file descriptor and will close it before returning.

flags: Any of the following combined using bitwise-or (| in C++): Xapian::DBCOMPACT_NO_RENUMBER By default the document ids will be renumbered the output - currently by applying the same offset to all the document ids in a particular source database. If this flag is specified, then this renumbering doesn’t happen, but all the document ids must be unique over all source databases. Currently the ranges of document ids in each source must not overlap either, though this restriction may be removed in the future.

Xapian::DBCOMPACT_MULTIPASS If merging more than 3 databases, merge the postlists in multiple passes, which is generally faster but requires more disk space for temporary files.

Xapian::DBCOMPACT_SINGLE_FILE Produce a single-file database (only supported for glass currently) - this flag is implied in this form and need not be specified explicitly.

block_size: This specifies the block size (in bytes) for to use for the output. For glass, the block size must be a power of 2 between 2048 and 65536 (inclusive), and the default (also used if an invalid value is passed) is 8192 bytes.

compactor: Functor

get_avlength()

Get the average length of the documents in the database.

Xapian::doclength Xapian::Database::get_avlength() const

get_collection_freq()

Return the total number of occurrences of the given term.

Xapian::termcount Xapian::Database::get_collection_freq(const std::string &tname) const

This is the sum of the number of occurrences of the term in each document it indexes: i.e., the sum of the within document frequencies of the term.

tname: The term whose collection frequency is being requested.

get_doccount()

Get the number of documents in the database.

Xapian::doccount Xapian::Database::get_doccount() const

get_doclength()

Get the length of a document.

Xapian::termcount Xapian::Database::get_doclength(Xapian::docid did) const

get_doclength_lower_bound()

Get a lower bound on the length of a document in this DB.

Xapian::termcount Xapian::Database::get_doclength_lower_bound() const

This bound does not include any zero-length documents.

get_doclength_upper_bound()

Get an upper bound on the length of a document in this DB.

Xapian::termcount Xapian::Database::get_doclength_upper_bound() const

get_document()

Get a document from the database, given its document id.

Xapian::Document Xapian::Database::get_document(Xapian::docid did, unsigned flags) const

This method returns a Xapian::Document object which provides the information about a document.

did: The document id of the document to retrieve.

flags: Zero or more flags bitwise-or-ed together (currently only Xapian::DOC_ASSUME_VALID is supported).

A Xapian::Document object containing the document data

Xapian::DocNotFoundError: The document specified could not be found in the database.

Xapian::InvalidArgumentError: did was 0, which is not a valid document id.

get_lastdocid()

Get the highest document id which has been used in the database.

Xapian::docid Xapian::Database::get_lastdocid() const

get_metadata()

Get the user-specified metadata associated with a given key.

std::string Xapian::Database::get_metadata(const std::string &key) const

User-specified metadata allows you to store arbitrary information in the form of (key, value) pairs. See WritableDatabase::set_metadata() for more information.

When invoked on a Xapian::Database object representing multiple databases, currently only the metadata for the first is considered but this behaviour may change in the future.

If there is no piece of metadata associated with the specified key, an empty string is returned (this applies even for backends which don’t support metadata).

Empty keys are not valid, and specifying one will cause an exception.

key: The key of the metadata item to access.

The retrieved metadata item’s value.

Xapian::InvalidArgumentError: will be thrown if the key supplied is empty.

get_revision()

Get the revision of the database.

Xapian::rev Xapian::Database::get_revision() const

The revision is an unsigned integer which increases with each commit.

The database must have exactly one sub-database, which must be of type chert or glass. Otherwise an exception will be thrown.

Experimental - seehttps://xapian.org/docs/deprecation#experimental- features

get_spelling_suggestion()

Suggest a spelling correction.

std::string Xapian::Database::get_spelling_suggestion(const std::string &word, unsigned max_edit_distance=2) const

word: The potentially misspelled word.

max_edit_distance: Only consider words which are at most max_edit_distance edits from word. An edit is a character insertion, deletion, or the transposition of two adjacent characters (default is 2).

get_termfreq()

Get the number of documents in the database indexed by a given term.

Xapian::doccount Xapian::Database::get_termfreq(const std::string &tname) const

get_total_length()

Get the total length of all the documents in the database.

Xapian::totallength Xapian::Database::get_total_length() const

Added in Xapian 1.4.5.

get_unique_terms()

Get the number of unique terms in document.

Xapian::termcount Xapian::Database::get_unique_terms(Xapian::docid did) const

get_uuid()

Get a UUID for the database.

std::string Xapian::Database::get_uuid() const

The UUID will persist for the lifetime of the database.

Replicas (eg, made with the replication protocol, or by copying all the database files) will have the same UUID. However, copies (made with copydatabase, or xapian-compact) will have different UUIDs.

If the backend does not support UUIDs or this database has no subdatabases, the UUID will be empty.

If this database has multiple sub-databases, the UUID string will contain the UUIDs of all the sub-databases.

get_value_freq()

Return the frequency of a given value slot.

Xapian::doccount Xapian::Database::get_value_freq(Xapian::valueno slot) const

This is the number of documents which have a (non-empty) value stored in the slot.

slot: The value slot to examine.

get_value_lower_bound()

Get a lower bound on the values stored in the given value slot.

std::string Xapian::Database::get_value_lower_bound(Xapian::valueno slot) const

If there are no values stored in the given value slot, this will return an empty string.

slot: The value slot to examine.

get_value_upper_bound()

Get an upper bound on the values stored in the given value slot.

std::string Xapian::Database::get_value_upper_bound(Xapian::valueno slot) const

If there are no values stored in the given value slot, this will return an empty string.

slot: The value slot to examine.

get_wdf_upper_bound()

Get an upper bound on the wdf of term term.

Xapian::termcount Xapian::Database::get_wdf_upper_bound(const std::string &term) const

has_positions()

Does this database have any positional information?

bool Xapian::Database::has_positions() const

keep_alive()

Send a “keep-alive” to remote databases to stop them timing out.

void Xapian::Database::keep_alive()

Has no effect on non-remote databases.

locked()

Test if this database is currently locked for writing.

bool Xapian::Database::locked() const

If the underlying object is actually a WritableDatabase, always returns true.

Otherwise tests if there’s a writer holding the lock (or if we can’t test for a lock without taking it on the current platform, throw Xapian::UnimplementedError). If there’s an error while trying to test the lock, throws Xapian::DatabaseLockError.

For multi-databases, this tests each sub-database and returns true if any of them are locked.

metadata_keys(prefix='')

Get an iterator which returns all the metadata keys.

The iterator will return string objects.

If prefix is non-empty, only metadata keys with this prefix are returned.

positionlist(docid, tname)

Get an iterator over all the positions in a given document of a term.

The iterator will return integers, in ascending order.

postlist(tname)

Get an iterator over the postings which are indexed by a given term.

If tname is empty, an iterator over all the documents will be returned (this will contain one entry for each document, will always return a wdf of 1, and will not allow access to a position iterator).

reopen()

Re-open the database.

bool Xapian::Database::reopen()

This re-opens the database(s) to the latest available version(s). It can be used either to make sure the latest results are returned, or to recover from a Xapian::DatabaseModifiedError.

Calling reopen() on a database which has been closed (with close()) will always raise a Xapian::DatabaseError.

true if the database might have been reopened (if false is returned, the database definitely hasn’t been reopened, which applications may find useful when caching results, etc). In Xapian < 1.3.0, this method did not return a value.

spellings()

Get an iterator which returns all the spelling correction targets

The iterator will return TermListItem objects. Only the term frequency is available; wdf and positions are not meaningful.

synonym_keys(prefix='')

Get an iterator which returns all the terms which have synonyms.

The iterator will return string objects.

If prefix is non-empty, only terms with this prefix are returned.

synonyms(term)

Get an iterator which returns all the synonyms for a given term.

The term to return synonyms for is specified by the term parameter.

The iterator will return string objects.

term_exists()

Check if a given term exists in the database.

bool Xapian::Database::term_exists(const std::string &tname) const

tname: The term to test the existence of.

true if and only if the term exists in the database. This is the same as (get_termfreq(tname) != 0), but will often be more efficient.

termlist(docid)

Get an iterator over all the terms which index a given document ID.

The iterator will return TermListItem objects.

Access to term frequency and position information is only available until the iterator has moved on.

thisown

The membership flag

valuestream(slot)

Get an iterator over all the values stored in a slot in the database.

The iterator will return ValueStreamItem objects, in ascending order of document id.

exception xapian.DatabaseCorruptError(*args)[source]

Bases: xapian.DatabaseError

DatabaseCorruptError indicates database corruption was detected.

exception xapian.DatabaseCreateError(*args)[source]

Bases: xapian.DatabaseError

DatabaseCreateError indicates a failure to create a database.

exception xapian.DatabaseError(*args)[source]

Bases: xapian.RuntimeError

DatabaseError indicates some sort of database related error.

exception xapian.DatabaseLockError(*args)[source]

Bases: xapian.DatabaseError

DatabaseLockError indicates failure to lock a database.

exception xapian.DatabaseModifiedError(*args)[source]

Bases: xapian.DatabaseError

DatabaseModifiedError indicates a database was modified.

To recover after catching this error, you need to call Xapian::Database::reopen() on the Database and repeat the operation which failed.

exception xapian.DatabaseOpeningError(*args)[source]

Bases: xapian.DatabaseError

DatabaseOpeningError indicates failure to open a database.

exception xapian.DatabaseVersionError(*args)[source]

Bases: xapian.DatabaseOpeningError

DatabaseVersionError indicates that a database is in an unsupported format.

From time to time, new versions of Xapian will require the database format to be changed, to allow new information to be stored or new optimisations to be performed. Backwards compatibility will sometimes be maintained, so that new versions of Xapian can open old databases, but in some cases Xapian will be unable to open a database because it is in too old (or new) a format. This can be resolved either be upgrading or downgrading the version of Xapian in use, or by rebuilding the database from scratch with the current version of Xapian.

xapian.Database_check()
class xapian.DateRangeProcessor(*args)[source]

Bases: xapian.RangeProcessor

Handle a date range.

Begin and end must be dates in a recognised format.

thisown

The membership flag

class xapian.DateValueRangeProcessor(*args)[source]

Bases: xapian.StringValueRangeProcessor

Handle a date range.

Begin and end must be dates in a recognised format.

Deprecated Use Xapian::DateRangeProcessor instead (added in 1.3.6).

thisown

The membership flag

class xapian.DecreasingValueWeightPostingSource(slot_, range_start_=0, range_end_=0)[source]

Bases: xapian.ValueWeightPostingSource

Read weights from a value which is known to decrease as docid increases.

This posting source can be used, like ValueWeightPostingSource, to add a weight contribution to a query based on the values stored in a slot. The values in the slot must be serialised as by sortable_serialise().

However, this posting source is additionally given a range of document IDs, within which the weight is known to be decreasing. ie, for all documents with ids A and B within this range (including the endpoints), where A is less than B, the weight of A is less than or equal to the weight of B. This can allow the posting source to skip to the end of the range quickly if insufficient weight is left in the posting source for a particular source.

By default, the range is assumed to cover all document IDs.

The ordering property can be arranged at index time, or by sorting an indexed database to produce a new, sorted, database.

thisown

The membership flag

exception xapian.DocNotFoundError(*args)[source]

Bases: xapian.RuntimeError

Indicates an attempt to access a document not present in the database.

class xapian.Document[source]

Bases: object

A handle representing a document in a Xapian database.

The Document class fetches information from the database lazily. Usually this behaviour isn’t visible to users (except for the speed benefits), but if the document in the database is modified or deleted, then preexisting Document objects may return the old or new versions of data (or throw Xapian::DocNotFoundError in the case of deletion).

Since Database objects work on a snapshot of the database’s state, the situation above can only happen with a WritableDatabase object, or if you call Database::reopen() on a Database object.

We recommend you avoid designs where this behaviour is an issue, but if you need a way to make a non-lazy version of a Document object, you can do this like so:doc = Xapian::Document::unserialise(doc.serialise());

add_boolean_term()

Add a boolean filter term to the document.

void Xapian::Document::add_boolean_term(const std::string &term)

This method adds term to the document with wdf of 0 - this is generally what you want for a term used for boolean filtering as the wdf of such terms is ignored, and it doesn’t make sense for them to contribute to the document’s length.

If the specified term already indexes this document, this method has no effect.

It is exactly the same as add_term(term, 0).

This method was added in Xapian 1.0.18.

term: The term to add.

add_posting()

Add an occurrence of a term at a particular position.

void Xapian::Document::add_posting(const std::string &tname, Xapian::termpos tpos, Xapian::termcount wdfinc=1)

Multiple occurrences of the term at the same position are represented only once in the positional information, but do increase the wdf.

If the term is not already in the document, it will be added to it.

tname: The name of the term.

tpos: The position of the term.

wdfinc: The increment that will be applied to the wdf for this term.

add_term()

Add a term to the document, without positional information.

void Xapian::Document::add_term(const std::string &tname, Xapian::termcount wdfinc=1)

Any existing positional information for the term will be left unmodified.

tname: The name of the term.

wdfinc: The increment that will be applied to the wdf for this term (default: 1).

add_value()

Add a new value.

void Xapian::Document::add_value(Xapian::valueno slot, const std::string &value)

The new value will replace any existing value with the same number (or if the new value is empty, it will remove any existing value with the same number).

slot: The value slot to add the value in.

value: The value to set.

clear_terms()

Remove all terms (and postings) from the document.

void Xapian::Document::clear_terms()

clear_values()

Remove all values associated with the document.

void Xapian::Document::clear_values()

get_data()

Get data stored in the document.

std::string Xapian::Document::get_data() const

This is potentially a relatively expensive operation, and shouldn’t normally be used during the match (e.g. in a PostingSource or match decider functor. Put data for use by match deciders in a value instead.

get_docid()

Get the document id which is associated with this document (if any).

docid Xapian::Document::get_docid() const

NB If multiple databases are being searched together, then this will be the document id in the individual database, not the merged database!

If this document came from a database, return the document id in that database. Otherwise, return 0 (in Xapian 1.0.22/1.2.4 or later; prior to this the returned value was uninitialised).

get_value()

Get value by number.

std::string Xapian::Document::get_value(Xapian::valueno slot) const

Returns an empty string if no value with the given number is present in the document.

slot: The number of the value.

remove_posting()

Remove a posting of a term from the document.

void Xapian::Document::remove_posting(const std::string &tname, Xapian::termpos tpos, Xapian::termcount wdfdec=1)

Note that the term will still index the document even if all occurrences are removed. To remove a term from a document completely, use remove_term().

tname: The name of the term.

tpos: The position of the term.

wdfdec: The decrement that will be applied to the wdf when removing this posting. The wdf will not go below the value of 0.

Xapian::InvalidArgumentError: will be thrown if the term is not at the position specified in the position list for this term in this document.

Xapian::InvalidArgumentError: will be thrown if the term is not in the document

remove_postings()

Remove a range of postings for a term.

Xapian::termpos Xapian::Document::remove_postings(const std::string &term, Xapian::termpos term_pos_first, Xapian::termpos term_pos_last, Xapian::termcount wdf_dec=1)

Any instances of the term at positions >= term_pos_first and <= term_pos_last will be removed, and the wdf reduced by wdf_dec for each instance removed (the wdf will not ever go below zero though).

It’s OK if the term doesn’t occur in the range of positions specified (unlike remove_posting()). And if term_pos_first > term_pos_last, this method does nothing.

The number of postings removed.

Xapian::InvalidArgumentError: will be thrown if the term is not in the document

remove_term()

Remove a term and all postings associated with it.

void Xapian::Document::remove_term(const std::string &tname)

tname: The name of the term.

Xapian::InvalidArgumentError: will be thrown if the term is not in the document

remove_value()

Remove any value with the given number.

void Xapian::Document::remove_value(Xapian::valueno slot)

serialise()

Serialise document into a string.

std::string Xapian::Document::serialise() const

The document representation may change between Xapian releases: even between minor versions. However, it is guaranteed not to change if the remote database protocol has not changed between releases.

set_data()

Set data stored in the document.

void Xapian::Document::set_data(const std::string &data)

Xapian treats the data as an opaque blob. It may try to compress it, but other than that it will just store it and return it when requested.

data: The data to store.

termlist()

Get an iterator over all the terms in a document.

The iterator will return TermListItem objects.

Access to term frequency and position information is only available until the iterator has moved on.

Note that term frequency information is only meaningful for a document retrieved from a database. If term frequency information is requested for a document which was freshly created, an InvalidOperationError will be raised.

termlist_count()

The length of the termlist - i.e.

Xapian::termcount Xapian::Document::termlist_count() const

the number of different terms which index this document.

thisown

The membership flag

static unserialise()
values()

Get an iterator over all the values stored in a document.

The iterator will return ValueItem objects, in ascending order of value number.

values_count()

Count the values in this document.

Xapian::termcount Xapian::Document::values_count() const

class xapian.ESet[source]

Bases: object

Class representing a list of search results.

back()

Return iterator pointing to the last object in this ESet.

ESetIterator Xapian::ESet::back() const

empty()

Return true if this ESet object is empty.

bool Xapian::ESet::empty() const

get_ebound()

Return a bound on the full size of this ESet object.

Xapian::termcount Xapian::ESet::get_ebound() const

This is a bound on size() if get_eset() had been called with maxitems set high enough that all results were returned.

items
size()

Return number of items in this ESet object.

Xapian::doccount Xapian::ESet::size() const

thisown

The membership flag

class xapian.ESetItem(iter)[source]

Bases: object

An item returned from iteration of the ESet.

The item supports access to the following attributes:

  • term: The term corresponding to this ESet item.
  • weight: The weight corresponding to this ESet item.
term
weight
class xapian.ESetIter(eset)[source]

Bases: object

An iterator over the items in an ESet.

The iterator will return ESetItem objects.

next()[source]
class xapian.Enquire(database)[source]

Bases: object

This class provides an interface to the information retrieval system for the purpose of searching.

Databases are usually opened lazily, so exceptions may not be thrown where you would expect them to be. You should catch Xapian::Error exceptions when calling any method in Xapian::Enquire.

Xapian::InvalidArgumentError: will be thrown if an invalid argument is supplied, for example, an unknown database type.

ASCENDING = 1
DESCENDING = 0
DONT_CARE = 2
INCLUDE_QUERY_TERMS = 1
USE_EXACT_TERMFREQ = 2
add_matchspy(decider)[source]

Add a matchspy.

void Xapian::Enquire::add_matchspy(MatchSpy *spy)

This matchspy will be called with some of the documents which match the query, during the match process. Exactly which of the matching documents are passed to it depends on exactly when certain optimisations occur during the match process, but it can be controlled to some extent by setting the checkatleast parameter to get_mset().

In particular, if there are enough matching documents, at least the number specified by checkatleast will be passed to the matchspy. This means that you can force the matchspy to be shown all matching documents by setting checkatleast to the number of documents in the database.

spy: The MatchSpy subclass to add. The caller must ensure that this remains valid while the Enquire object remains active, or until clear_matchspies() is called.

clear_matchspies()[source]

Remove all the matchspies.

void Xapian::Enquire::clear_matchspies()

get_eset()

Get the expand set for the given rset.

ESet Xapian::Enquire::get_eset(Xapian::termcount maxitems, const RSet &rset, int flags, double k, const Xapian::ExpandDecider *edecider=NULL, double min_wt=0.0) const

maxitems: the maximum number of items to return.

rset: the relevance set to use when performing the expand operation.

flags: zero or more of these values |-ed together: Xapian::Enquire::INCLUDE_QUERY_TERMS query terms may be returned from expand

Xapian::Enquire::USE_EXACT_TERMFREQ for multi dbs, calculate the exact termfreq; otherwise an approximation is used which can greatly improve efficiency, but still returns good results.

k: the parameter k in the query expansion algorithm (default is 1.0)

edecider: a decision functor to use to decide whether a given term should be put in the ESet

min_wt: the minimum weight for included terms

An ESet object containing the results of the expand.

Xapian::InvalidArgumentError: See class documentation.

get_mset()

Get (a portion of) the match set for the current query.

MSet Xapian::Enquire::get_mset(Xapian::doccount first, Xapian::doccount maxitems, const RSet *omrset, const MatchDecider *mdecider=0) const

first: the first item in the result set to return. A value of zero corresponds to the first item returned being that with the highest score. A value of 10 corresponds to the first 10 items being ignored, and the returned items starting at the eleventh.

maxitems: the maximum number of items to return. If you want all matches, then you can pass the result of calling get_doccount() on the Database object (though if you are doing this so you can filter results, you are likely to get much better performance by using Xapian’s match-time filtering features instead). You can pass 0 for maxitems which will give you an empty MSet with valid statistics (such as get_matches_estimated()) calculated without looking at any postings, which is very quick, but means the estimates may be more approximate and the bounds may be much looser.

omrset: the relevance set to use when performing the query.

mdecider: a decision functor to use to decide whether a given document should be put in the MSet.

A Xapian::MSet object containing the results of the query.

Xapian::InvalidArgumentError: See class documentation.

get_query()[source]

Get the current query.

const Xapian::Query& Xapian::Enquire::get_query() const

If called before set_query(), this will return a default initialised Query object.

matching_terms(which)

Get an iterator over the terms which match a given match set item.

The match set item to consider is specified by the which parameter, which may be a document ID, or an MSetItem object.

The iterator will return string objects.

set_collapse_key()

Set the collapse key to use for queries.

void Xapian::Enquire::set_collapse_key(Xapian::valueno collapse_key, Xapian::doccount collapse_max=1)

collapse_key: value number to collapse on - at most one MSet entry with each particular value will be returned (default is Xapian::BAD_VALUENO which means no collapsing).

collapse_max: Max number of items with the same key to leave after collapsing (default 1).

The MSet returned by get_mset() will have only the “best” (at most) collapse_max entries with each particular value of collapse_key (“best” being highest ranked - i.e. highest weight or highest sorting key).

An example use might be to create a value for each document containing an MD5 hash of the document contents. Then duplicate documents from different sources can be eliminated at search time by collapsing with collapse_max = 1 (it’s better to eliminate duplicates at index time, but this may not be always be possible - for example the search may be over more than one Xapian database).

Another use is to group matches in a particular category (e.g. you might collapse a mailing list search on the Subject: so that there’s only one result per discussion thread). In this case you can use get_collapse_count() to give the user some idea how many other results there are. And if you index the Subject: as a boolean term as well as putting it in a value, you can offer a link to a non-collapsed search restricted to that thread using a boolean filter.

set_cutoff()

Set the percentage and/or weight cutoffs.

void Xapian::Enquire::set_cutoff(int percent_cutoff, double weight_cutoff=0)

percent_cutoff: Minimum percentage score for returned documents. If a document has a lower percentage score than this, it will not appear in the MSet. If your intention is to return only matches which contain all the terms in the query, then it’s more efficient to use Xapian::Query::OP_AND instead of Xapian::Query::OP_OR in the query than to use set_cutoff(100). (default 0 => no percentage cut-off).

weight_cutoff: Minimum weight for a document to be returned. If a document has a lower score that this, it will not appear in the MSet. It is usually only possible to choose an appropriate weight for cutoff based on the results of a previous run of the same query; this is thus mainly useful for alerting operations. The other potential use is with a user specified weighting scheme. (default 0 => no weight cut-off).

set_docid_order()

Set sort order for document IDs.

void Xapian::Enquire::set_docid_order(docid_order order)

This order only has an effect on documents which would otherwise have equal rank. When ordering by relevance without a sort key, this means documents with equal weight. For a boolean match with no sort key, this means all documents. And if a sort key is used, this means documents with the same sort key (and also equal weight if ordering on relevance before or after the sort key).

order: This can be: Xapian::Enquire::ASCENDING docids sort in ascending order (default)

Xapian::Enquire::DESCENDING docids sort in descending order

Xapian::Enquire::DONT_CARE docids sort in whatever order is most efficient for the backend

Note: If you add documents in strict date order, then a boolean search - i.e. set_weighting_scheme(Xapian::BoolWeight()) - with set_docid_order(Xapian::Enquire::DESCENDING) is an efficient way to perform “sort by date, newest first”, and with set_docid_order(Xapian::Enquire::ASCENDING) a very efficient way to perform “sort by date, oldest first”.

set_expansion_scheme()

Set the weighting scheme to use for expansion.

void Xapian::Enquire::set_expansion_scheme(const std::string &eweightname_, double expand_k_=1.0) const

If you don’t call this method, the default is as if you’d used:

get_expansion_scheme(“trad”);

eweightname_: A string in lowercase specifying the name of the scheme to be used. The following schemes are currently available: “bo1” : The Bo1 scheme for query expansion. “trad” : The TradWeight scheme for query expansion.

expand_k_: The parameter required for TradWeight query expansion. A default value of 1.0 is used if none is specified.

set_query(query, qlen=0)[source]

Set the query to run.

void Xapian::Enquire::set_query(const Xapian::Query &query, Xapian::termcount qlen=0)

query: the new query to run.

qlen: the query length to use in weight calculations - by default the sum of the wqf of all terms is used.

set_sort_by_key(sorter, reverse)[source]

Set the sorting to be by key generated from values only.

void Xapian::Enquire::set_sort_by_key(Xapian::KeyMaker *sorter, bool reverse)

sorter: The functor to use for generating keys.

reverse: If true, reverses the sort order.

set_sort_by_key_then_relevance(sorter, reverse)[source]

Set the sorting to be by keys generated from values, then by relevance for documents with identical keys.

void Xapian::Enquire::set_sort_by_key_then_relevance(Xapian::KeyMaker *sorter, bool reverse)

sorter: The functor to use for generating keys.

reverse: If true, reverses the sort order.

set_sort_by_relevance()

Set the sorting to be by relevance only.

void Xapian::Enquire::set_sort_by_relevance()

This is the default.

set_sort_by_relevance_then_key(sorter, reverse)[source]

Set the sorting to be by relevance, then by keys generated from values.

void Xapian::Enquire::set_sort_by_relevance_then_key(Xapian::KeyMaker *sorter, bool reverse)

Note that with the default BM25 weighting scheme parameters, non- identical documents will rarely have the same weight, so this setting will give very similar results to set_sort_by_relevance(). It becomes more useful with particular BM25 parameter settings (e.g. BM25Weight(1,0,1,0,0)) or custom weighting schemes.

sorter: The functor to use for generating keys.

reverse: If true, reverses the sort order of the generated keys. Beware that in 1.2.16 and earlier, the sense of this parameter was incorrectly inverted and inconsistent with the other set_sort_by_... methods. This was fixed in 1.2.17, so make that version a minimum requirement if this detail matters to your application.

set_sort_by_relevance_then_value()

Set the sorting to be by relevance then value.

void Xapian::Enquire::set_sort_by_relevance_then_value(Xapian::valueno sort_key, bool reverse)

Note that sorting by values uses a string comparison, so to use this to sort by a numeric value you’ll need to store the numeric values in a manner which sorts appropriately. For example, you could use Xapian::sortable_serialise() (which works for floating point numbers as well as integers), or store numbers padded with leading zeros or spaces, or with the number of digits prepended.

Note that with the default BM25 weighting scheme parameters, non- identical documents will rarely have the same weight, so this setting will give very similar results to set_sort_by_relevance(). It becomes more useful with particular BM25 parameter settings (e.g. BM25Weight(1,0,1,0,0)) or custom weighting schemes.

sort_key: value number to sort on.

reverse: If true, reverses the sort order of sort_key. Beware that in 1.2.16 and earlier, the sense of this parameter was incorrectly inverted and inconsistent with the other set_sort_by_... methods. This was fixed in 1.2.17, so make that version a minimum requirement if this detail matters to your application.

set_sort_by_value()

Set the sorting to be by value only.

void Xapian::Enquire::set_sort_by_value(Xapian::valueno sort_key, bool reverse)

Note that sorting by values uses a string comparison, so to use this to sort by a numeric value you’ll need to store the numeric values in a manner which sorts appropriately. For example, you could use Xapian::sortable_serialise() (which works for floating point numbers as well as integers), or store numbers padded with leading zeros or spaces, or with the number of digits prepended.

sort_key: value number to sort on.

reverse: If true, reverses the sort order.

set_sort_by_value_then_relevance()

Set the sorting to be by value, then by relevance for documents with the same value.

void Xapian::Enquire::set_sort_by_value_then_relevance(Xapian::valueno sort_key, bool reverse)

Note that sorting by values uses a string comparison, so to use this to sort by a numeric value you’ll need to store the numeric values in a manner which sorts appropriately. For example, you could use Xapian::sortable_serialise() (which works for floating point numbers as well as integers), or store numbers padded with leading zeros or spaces, or with the number of digits prepended.

sort_key: value number to sort on.

reverse: If true, reverses the sort order.

set_time_limit()

Set a time limit for the match.

void Xapian::Enquire::set_time_limit(double time_limit)

Matches with check_at_least set high can take a long time in some cases. You can set a time limit on this, after which check_at_least will be turned off.

time_limit: time in seconds after which to disable check_at_least (default: 0.0 which means no time limit)

Limitations:

This feature is currently supported on platforms which support POSIX interval timers. Interaction with the remote backend when using multiple databases may have bugs. There’s not currently a way to force the match to end after a certain time.

set_weighting_scheme()

Set the weighting scheme to use for queries.

void Xapian::Enquire::set_weighting_scheme(const Weight &weight_)

weight_: the new weighting scheme. If no weighting scheme is specified, the default is BM25 with the default parameters.

thisown

The membership flag

exception xapian.Error(*args, **kwargs)[source]

Bases: exceptions.Exception

All exceptions thrown by Xapian are subclasses of Xapian::Error.

This class can not be instantiated directly - instead a subclass should be used.

get_context()

Optional context information.

const std::string& Xapian::Error::get_context() const

This context is intended for use by Xapian::ErrorHandler (for example so it can know which remote server is unreliable and report the problem and remove that server from those being searched). But it’s typically a plain-text string, and so also fit for human consumption.

get_error_string()

Returns any system error string associated with this exception.

const char* Xapian::Error::get_error_string() const

The system error string may come from errno, h_errno (on UNIX), or GetLastError() (on MS Windows). If there is no associated system error string, NULL is returned.

get_msg()

Message giving details of the error, intended for human consumption.

const std::string& Xapian::Error::get_msg() const

get_type()

The type of this error (e.g. “DocNotFoundError”.)

const char* Xapian::Error::get_type() const

class xapian.ExpandDecider[source]

Bases: object

Virtual base class for expand decider functor.

release()

Start reference counting this object.

const ExpandDecider* Xapian::ExpandDecider::release() const

You can hand ownership of a dynamically allocated ExpandDecider object to Xapian by calling release() and then passing the object to a Xapian method. Xapian will arrange to delete the object once it is no longer required.

thisown

The membership flag

class xapian.ExpandDeciderAnd(first_, second_)[source]

Bases: xapian.ExpandDecider

ExpandDecider subclass which rejects terms using two ExpandDeciders.

Terms are only accepted if they are accepted by both of the specified ExpandDecider objects.

thisown

The membership flag

class xapian.ExpandDeciderFilterPrefix(prefix_)[source]

Bases: xapian.ExpandDecider

ExpandDecider subclass which restrict terms to a particular prefix.

ExpandDeciderFilterPrefix provides an easy way to choose terms with a particular prefix when generating an ESet.

thisown

The membership flag

exception xapian.FeatureUnavailableError(*args)[source]

Bases: xapian.RuntimeError

Indicates an attempt to use a feature which is unavailable.

Typically a feature is unavailable because it wasn’t compiled in, or because it requires other software or facilities which aren’t available.

class xapian.FieldProcessor[source]

Bases: object

Base class for field processors.

release()

Start reference counting this object.

const FieldProcessor* Xapian::FieldProcessor::release() const

You can hand ownership of a dynamically allocated FieldProcessor object to Xapian by calling release() and then passing the object to a Xapian method. Xapian will arrange to delete the object once it is no longer required.

thisown

The membership flag

class xapian.FixedWeightPostingSource(wt)[source]

Bases: xapian.PostingSource

A posting source which returns a fixed weight for all documents.

This returns entries for all documents in the given database, with a fixed weight (specified by a parameter to the constructor).

thisown

The membership flag

class xapian.GreatCircleMetric(*args)[source]

Bases: xapian.LatLongMetric

Calculate the great-circle distance between two coordinates on a sphere.

Experimental - seehttps://xapian.org/docs/deprecation#experimental- features

This uses the haversine formula to calculate the distance. Note that this formula is subject to inaccuracy due to numerical errors for coordinates on the opposite side of the sphere.

Seehttp://en.wikipedia.org/wiki/Haversine_formula

thisown

The membership flag

class xapian.IfB2Weight(*args)[source]

Bases: xapian.Weight

This class implements the IfB2 weighting scheme.

IfB2 is a representative scheme of the Divergence from Randomness Framework by Gianni Amati.

It uses the Inverse term frequency model (If), the Bernoulli method to find the aftereffect of sampling (B) and the second wdf normalization proposed by Amati to normalize the wdf in the document to the length of the document (H2).

For more information about the DFR Framework and the IfB2 scheme, please refer to: Gianni Amati and Cornelis Joost Van Rijsbergen Probabilistic models of information retrieval based on measuring the divergence from randomness ACM Transactions on Information Systems (TOIS) 20, (4), 2002, pp. 357-389.

thisown

The membership flag

class xapian.InL2Weight(*args)[source]

Bases: xapian.Weight

This class implements the InL2 weighting scheme.

InL2 is a representative scheme of the Divergence from Randomness Framework by Gianni Amati.

This weighting scheme is useful for tasks that require early precision.

It uses the Inverse document frequency model (In), the Laplace method to find the aftereffect of sampling (L) and the second wdf normalization proposed by Amati to normalize the wdf in the document to the length of the document (H2).

For more information about the DFR Framework and the InL2 scheme, please refer to: Gianni Amati and Cornelis Joost Van Rijsbergen Probabilistic models of information retrieval based on measuring the divergence from randomness ACM Transactions on Information Systems (TOIS) 20, (4), 2002, pp. 357-389.

thisown

The membership flag

class xapian.IneB2Weight(*args)[source]

Bases: xapian.Weight

This class implements the IneB2 weighting scheme.

IneB2 is a representative scheme of the Divergence from Randomness Framework by Gianni Amati.

It uses the Inverse expected document frequency model (Ine), the Bernoulli method to find the aftereffect of sampling (B) and the second wdf normalization proposed by Amati to normalize the wdf in the document to the length of the document (H2).

For more information about the DFR Framework and the IneB2 scheme, please refer to: Gianni Amati and Cornelis Joost Van Rijsbergen Probabilistic models of information retrieval based on measuring the divergence from randomness ACM Transactions on Information Systems (TOIS) 20, (4), 2002, pp. 357-389.

thisown

The membership flag

exception xapian.InternalError(*args)[source]

Bases: xapian.RuntimeError

InternalError indicates a runtime problem of some sort.

exception xapian.InvalidArgumentError(*args)[source]

Bases: xapian.LogicError

InvalidArgumentError indicates an invalid parameter value was passed to the API.

exception xapian.InvalidOperationError(*args)[source]

Bases: xapian.LogicError

InvalidOperationError indicates the API was used in an invalid way.

class xapian.KeyMaker[source]

Bases: object

Virtual base class for key making functors.

release()

Start reference counting this object.

const KeyMaker* Xapian::KeyMaker::release() const

You can hand ownership of a dynamically allocated KeyMaker object to Xapian by calling release() and then passing the object to a Xapian method. Xapian will arrange to delete the object once it is no longer required.

thisown

The membership flag

class xapian.LMWeight(*args)[source]

Bases: xapian.Weight

Xapian::Weight subclass implementing the Language Model formula.

This class implements the “Language Model” Weighting scheme, as described by the early papers on LM by Bruce Croft.

LM works by comparing the query to a Language Model of the document. The language model itself is parameter-free, though LMWeight takes parameters which specify the smoothing used.

thisown

The membership flag

class xapian.LatLongCoord(*args)[source]

Bases: object

A latitude-longitude coordinate.

Experimental - seehttps://xapian.org/docs/deprecation#experimental- features

Note that latitude-longitude coordinates are only precisely meaningful if the datum used to define them is specified. This class ignores this issue - it is up to the caller to ensure that the datum used for each coordinate in a system is consistent.

latitude
longitude
serialise()

Return a serialised representation of the coordinate.

std::string Xapian::LatLongCoord::serialise() const

thisown

The membership flag

unserialise()

Unserialise a buffer and set this object to its coordinate.

void Xapian::LatLongCoord::unserialise(const char **ptr, const char *end)

The buffer may contain further data after that for the coordinate.

ptr: A pointer to the start of the string. This will be updated to point to the end of the data representing the coordinate.

end: A pointer to the end of the string.

Xapian::SerialisationError: if the string does not start with a valid serialised latitude- longitude pair.

class xapian.LatLongCoords(*args)[source]

Bases: object

A sequence of latitude-longitude coordinates.

Experimental - seehttps://xapian.org/docs/deprecation#experimental- features

append()

Append a coordinate to the end of the sequence.

void Xapian::LatLongCoords::append(const LatLongCoord &coord)

begin()

Get a begin iterator for the coordinates.

LatLongCoordsIterator Xapian::LatLongCoords::begin() const

empty()

Return true if and only if there are no coordinates in the container.

bool Xapian::LatLongCoords::empty() const

end()

Get an end iterator for the coordinates.

LatLongCoordsIterator Xapian::LatLongCoords::end() const

serialise()

Return a serialised form of the coordinate list.

std::string Xapian::LatLongCoords::serialise() const

size()

Get the number of coordinates in the container.

size_t Xapian::LatLongCoords::size() const

thisown

The membership flag

unserialise()

Unserialise a string and set this object to the coordinates in it.

void Xapian::LatLongCoords::unserialise(const std::string &serialised)

serialised: the string to unserialise the coordinates from.

Xapian::SerialisationError: if the string does not contain a valid serialised latitude-longitude pair, or contains junk at the end of it.

class xapian.LatLongCoordsIter(start, end)[source]

Bases: object

An iterator over all the coordinates in a LatLongCoords object.

The iterator returns LatLongCoord objects.

next()[source]
class xapian.LatLongDistanceKeyMaker(*args)[source]

Bases: xapian.KeyMaker

KeyMaker subclass which sorts by distance from a latitude/longitude.

Experimental - seehttps://xapian.org/docs/deprecation#experimental- features

Results are ordered by the distance from a fixed point, or list of points, calculated according to the metric supplied. If multiple points are supplied (either in the constructor, or in the coordinates stored in a document), the closest pointwise distance is used.

If a document contains no coordinate stored in the specified slot, a special value for the distance will be used. This defaults to a large number, so that such results get a low rank, but may be specified by a constructor parameter.

thisown

The membership flag

class xapian.LatLongDistancePostingSource(*args)[source]

Bases: xapian.ValuePostingSource

Posting source which returns a weight based on geospatial distance.

Experimental - seehttps://xapian.org/docs/deprecation#experimental- features

Results are weighted by the distance from a fixed point, or list of points, calculated according to the metric supplied. If multiple points are supplied (either in the constructor, or in the coordinates stored in a document), the closest pointwise distance is used.

Documents further away than a specified maximum range (or with no location stored in the specified slot) will not be returned.

The weight returned is computed from the distance using the formula:

k1 * pow(distance + k1, -k2)

(Where k1 and k2 are (strictly) positive, floating point constants, which default to 1000 and 1, respectively. Distance is measured in metres, so this means that something at the centre gets a weight of 1.0, something 1km away gets a weight of 0.5, and something 3km away gets a weight of 0.25, etc)

thisown

The membership flag

class xapian.LatLongMetric[source]

Bases: object

Base class for calculating distances between two lat/long coordinates.

Experimental - seehttps://xapian.org/docs/deprecation#experimental- features

name()

Return the full name of the metric.

virtual std::string Xapian::LatLongMetric::name() const =0

This is used when serialising and unserialising metrics; for example, for performing remote searches.

If the subclass is in a C++ namespace, the namespace should be included in the name, using ”::” as a separator. For example, for a LatLongMetric subclass called “FooLatLongMetric” in the “Xapian” namespace the result of this call should be “Xapian::FooLatLongMetric”.

pointwise_distance()

Return the distance between two coordinates, in metres.

virtual double Xapian::LatLongMetric::pointwise_distance(const LatLongCoord &a, const LatLongCoord &b) const =0

thisown

The membership flag

exception xapian.LogicError(*args, **kwargs)[source]

Bases: xapian.Error

The base class for exceptions indicating errors in the program logic.

A subclass of LogicError will be thrown if Xapian detects a violation of a class invariant or a logical precondition or postcondition, etc.

class xapian.MSet[source]

Bases: object

Class representing a list of search results.

SNIPPET_BACKGROUND_MODEL = 1
SNIPPET_EMPTY_WITHOUT_MATCH = 4
SNIPPET_EXHAUSTIVE = 2
back()

Return iterator pointing to the last object in this MSet.

MSetIterator Xapian::MSet::back() const

convert_to_percent()

Convert the weight of the current iterator position to a percentage.

int Xapian::MSet::convert_to_percent(const MSetIterator &it) const

The matching document with the highest weight will get 100% if it matches all the weighted query terms, and proportionally less if it only matches some, and other weights are scaled by the same factor.

Documents with a non-zero score will always score at least 1%.

Note that these generally aren’t percentages of anything meaningful (unless you use a custom weighting formula where they are!)

empty()

Return true if this MSet object is empty.

bool Xapian::MSet::empty() const

fetch()

Prefetch hint the whole MSet.

void Xapian::MSet::fetch() const

For a remote database, this may start a pipelined fetch of the requested documents from the remote server.

For a disk-based database, this may send prefetch hints to the operating system such that the disk blocks the requested documents are stored in are more likely to be in the cache when we come to actually read them.

get_docid()
get_document()
get_document_percentage()
get_firstitem()

Rank of first item in this MSet.

Xapian::doccount Xapian::MSet::get_firstitem() const

This is the parameter first passed to Xapian::Enquire::get_mset().

get_hit(index)

Get an item from the MSet.

The supplied index is relative to the start of the MSet, not the absolute rank of the item.

Returns an MSetItem.

get_matches_estimated()

Estimate of the total number of matching documents.

Xapian::doccount Xapian::MSet::get_matches_estimated() const

get_matches_lower_bound()

Lower bound on the total number of matching documents.

Xapian::doccount Xapian::MSet::get_matches_lower_bound() const

get_matches_upper_bound()

Upper bound on the total number of matching documents.

Xapian::doccount Xapian::MSet::get_matches_upper_bound() const

get_max_attained()

The maximum weight attained by any document.

double Xapian::MSet::get_max_attained() const

get_max_possible()

The maximum possible weight any document could achieve.

double Xapian::MSet::get_max_possible() const

get_termfreq()

Get the termfreq of a term.

Xapian::doccount Xapian::MSet::get_termfreq(const std::string &term) const

The number of documents term occurs in.

get_termweight()

Get the term weight of a term.

double Xapian::MSet::get_termweight(const std::string &term) const

The maximum weight that term could have contributed to a document.

get_uncollapsed_matches_estimated()

Estimate of the total number of matching documents before collapsing.

Xapian::doccount Xapian::MSet::get_uncollapsed_matches_estimated() const

Conceptually the same as get_matches_estimated() for the same query without any collapse part (though the actual value may differ).

get_uncollapsed_matches_lower_bound()

Lower bound on the total number of matching documents before collapsing.

Xapian::doccount Xapian::MSet::get_uncollapsed_matches_lower_bound() const

Conceptually the same as get_matches_lower_bound() for the same query without any collapse part (though the actual value may differ).

get_uncollapsed_matches_upper_bound()

Upper bound on the total number of matching documents before collapsing.

Xapian::doccount Xapian::MSet::get_uncollapsed_matches_upper_bound() const

Conceptually the same as get_matches_upper_bound() for the same query without any collapse part (though the actual value may differ).

items
size()

Return number of items in this MSet object.

Xapian::doccount Xapian::MSet::size() const

snippet()

Generate a snippet.

std::string Xapian::MSet::snippet(const std::string &text, size_t length=500, const Xapian::Stem &stemmer=Xapian::Stem(), unsigned flags=SNIPPET_BACKGROUND_MODEL|SNIPPET_EXHAUSTIVE, const std::string &hi_start=”<b>”, const std::string &hi_end=”</b>”, const std::string &omit=”...”) const

This method selects a continuous run of words from text, based mainly on where the query matches (currently terms, exact phrases and wildcards are taken into account). If flag SNIPPET_BACKGROUND_MODEL is used (which it is by default) then the selection algorithm also considers the non-query terms in the text with the aim of showing a context which provides more useful information.

The size of the text selected can be controlled by the length parameter, which specifies a number of bytes of text to aim to select. However slightly more text may be selected. Also the size of any escaping, highlighting or omission markers is not considered.

The returned text is escaped to make it suitable for use in HTML (though beware that in upstream releases 1.4.5 and earlier this escaping was sometimes incomplete), and matches with the query will be highlighted using hi_start and hi_end.

If the snippet seems to start or end mid-sentence, then omit is prepended or append (respectively) to indicate this.

The same stemming algorithm which was used to build the query should be specified in stemmer.

And flags contains flags controlling behaviour.

Added in 1.3.5.

thisown

The membership flag

class xapian.MSetItem(iter, mset)[source]

Bases: object

An item returned from iteration of the MSet.

The item supports access to the following attributes and properties:

  • docid: The Xapian document ID corresponding to this MSet item.
  • weight: The weight corresponding to this MSet item.
  • rank: The rank of this MSet item. The rank is the position in the total set of matching documents of this item. The highest document is given a rank of 0. If the MSet did not start at the highest matching document, because a non-zero ‘start’ parameter was supplied to get_mset(), the first document in the MSet will have a rank greater than 0 (in fact, it will be equal to the value of ‘start’ supplied to get_mset()).
  • percent: The percentage score assigned to this MSet item.
  • document: The document for this MSet item. This can be used to access the document data, or any other information stored in the document (such as term lists). It is lazily evaluated.
  • collapse_key: The value of the key which was used for collapsing.
  • collapse_count: An estimate of the number of documents that have been collapsed into this one.

The collapse count estimate will always be less than or equal to the actual number of other documents satisfying the match criteria with the same collapse key as this document. If may be 0 even though there are other documents with the same collapse key which satisfying the match criteria. However if this method returns non-zero, there definitely are other such documents. So this method may be used to inform the user that there are “at least N other matches in this group”, or to control whether to offer a “show other documents in this group” feature (but note that it may not offer it in every case where it would show other documents).

collapse_count
collapse_key
docid
document

The document object corresponding to this MSet item.

percent
rank
weight
class xapian.MSetIter(mset)[source]

Bases: object

An iterator over the items in an MSet.

The iterator will return MSetItem objects, which will be evaluated lazily where appropriate.

next()[source]
class xapian.MatchDecider[source]

Bases: object

Base class for matcher decision functor.

thisown

The membership flag

class xapian.MatchSpy[source]

Bases: object

Abstract base class for match spies.

The subclasses will generally accumulate information seen during the match, to calculate aggregate functions, or other profiles of the matching documents.

merge_results()

Unserialise some results, and merge them into this matchspy.

virtual void Xapian::MatchSpy::merge_results(const std::string &serialised)

The order in which results are merged should not be significant, since this order is not specified (and will vary depending on the speed of the search in each sub-database).

If you don’t want to support the remote backend in your match spy, you can use the default implementation which simply throws Xapian::UnimplementedError.

serialised: A string containing the serialised results.

name()

Return the name of this match spy.

virtual std::string Xapian::MatchSpy::name() const

This name is used by the remote backend. It is passed with the serialised parameters to the remote server so that it knows which class to create.

Return the full namespace-qualified name of your class here - if your class is called MyApp::FooMatchSpy, return “MyApp::FooMatchSpy” from this method.

If you don’t want to support the remote backend in your match spy, you can use the default implementation which simply throws Xapian::UnimplementedError.

release()

Start reference counting this object.

const MatchSpy* Xapian::MatchSpy::release() const

You can hand ownership of a dynamically allocated MatchSpy object to Xapian by calling release() and then passing the object to a Xapian method. Xapian will arrange to delete the object once it is no longer required.

thisown

The membership flag

class xapian.MultiValueKeyMaker[source]

Bases: xapian.KeyMaker

KeyMaker subclass which combines several values.

When the result is used for sorting, results are ordered by the first value. In the event of a tie, the second is used. If this is the same for both, the third is used, and so on. If reverse is true for a value, then the sort order for that value is reversed.

When used for collapsing, the documents will only be considered equal if all the values specified match. If none of the specified values are set then the generated key will be empty, so such documents won’t be collapsed (which is consistent with the behaviour in the “collapse on a value” case). If you’d prefer that documents with none of the keys set are collapsed together, then you can set reverse for at least one of the values. Other than this, it isn’t useful to set reverse for collapsing.

add_value()

Add a value slot to the list to build a key from.

void Xapian::MultiValueKeyMaker::add_value(Xapian::valueno slot, bool reverse=false, const std::string &defvalue=std::string())

slot: The value slot to add

reverse: Adjust values from this slot to reverse their sort order (default: false)

defvalue: Value to use for documents which don’t have a value set in this slot (default: empty). This can be used to make such documents sort after all others by passing get_value_upper_bound(slot) + “x” this is guaranteed to be greater than any value in this slot.

thisown

The membership flag

exception xapian.NetworkError(*args)[source]

Bases: xapian.RuntimeError

Indicates a problem communicating with a remote database.

exception xapian.NetworkTimeoutError(*args)[source]

Bases: xapian.NetworkError

Indicates a timeout expired while communicating with a remote database.

class xapian.NumberRangeProcessor(*args)[source]

Bases: xapian.RangeProcessor

Handle a number range.

This class must be used on values which have been encoded using Xapian::sortable_serialise() which turns numbers into strings which will sort in the same order as the numbers (the same values can be used to implement a numeric sort).

thisown

The membership flag

class xapian.NumberValueRangeProcessor(*args)[source]

Bases: xapian.StringValueRangeProcessor

Handle a number range.

This class must be used on values which have been encoded using Xapian::sortable_serialise() which turns numbers into strings which will sort in the same order as the numbers (the same values can be used to implement a numeric sort).

Deprecated Use Xapian::NumberRangeProcessor instead (added in 1.3.6).

thisown

The membership flag

class xapian.PL2PlusWeight(*args)[source]

Bases: xapian.Weight

Xapian::Weight subclass implementing the PL2+ probabilistic formula.

thisown

The membership flag

class xapian.PL2Weight(*args)[source]

Bases: xapian.Weight

This class implements the PL2 weighting scheme.

PL2 is a representative scheme of the Divergence from Randomness Framework by Gianni Amati.

This weighting scheme is useful for tasks that require early precision.

It uses the Poisson approximation of the Binomial Probabilistic distribution (P) along with Stirling’s approximation for the factorial value, the Laplace method to find the aftereffect of sampling (L) and the second wdf normalization proposed by Amati to normalize the wdf in the document to the length of the document (H2).

For more information about the DFR Framework and the PL2 scheme, please refer to : Gianni Amati and Cornelis Joost Van Rijsbergen Probabilistic models of information retrieval based on measuring the divergence from randomness ACM Transactions on Information Systems (TOIS) 20, (4), 2002, pp. 357-389.

thisown

The membership flag

class xapian.PositionIter(start=0, end=0)[source]

Bases: object

An iterator over a position list.

The iterator will return integers, in ascending order.

next()[source]
class xapian.PostingItem(iter)[source]

Bases: object

An item returned from iteration of a posting list.

The item supports access to the following attributes and properties:

  • docid: The document ID corresponding to this PostingItem.
  • doclength: The length of the document corresponding to this PostingItem.
  • wdf: The within document frequency of the term which the posting list is for in the document corresponding to this PostingItem.
  • positer: An iterator over the positions which the term corresponing to this posting list occurs at in the document corresponding to this PostingItem. This is only available until the iterator which returned this item next moves.
docid
doclength
positer

A position iterator for the current posting (if meaningful).

The iterator will return integers representing the positions that the term occurs at.

This will raise a InvalidOperationError exception if the iterator this item came from doesn’t support position lists, or if the iterator has moved on since the item was returned from it.

wdf
class xapian.PostingIter(start, end, has_positions=False)[source]

Bases: object

An iterator over a posting list.

The iterator will return PostingItem objects, which will be evaluated lazily where appropriate.

next()[source]
skip_to(docid)[source]

Skip the iterator forward.

The iterator is advanced to the first document with a document ID which is greater than or equal to the supplied document ID.

If there are no such items, this will raise StopIteration.

This returns the item which the iterator is moved to. The subsequent item will be returned the next time that next() is called (unless skip_to() is called again first).

class xapian.PostingSource[source]

Bases: object

Base class which provides an “external” source of postings.

at_end()

Return true if the current position is past the last entry in this list.

virtual bool Xapian::PostingSource::at_end() const =0

At least one of next(), skip_to() or check() will be called before this method is first called.

check()

Check if the specified docid occurs.

virtual bool Xapian::PostingSource::check(Xapian::docid did, double min_wt)

The caller is required to ensure that the specified document id did actually exists in the database. If it does, it must move to that document id, and return true. If it does not, it may either:

return true, having moved to a definite position (including “at_end”), which must be the same position as skip_to() would have moved to.

or

return false, having moved to an “indeterminate” position, such that a subsequent call to next() or skip_to() will move to the next matching position after did.

Generally, this method should act like skip_to() and return true if that can be done at little extra cost.

Otherwise it should simply check if a particular docid is present, returning true if it is, and false if it isn’t.

The default implementation calls skip_to() and always returns true.

Xapian will always call init() on a PostingSource before calling this for the first time.

Note: in the case of a multi-database search, the docid specified is the docid in the single subdatabase relevant to this posting source. See the init() method for details.

did: The document id to check.

min_wt: The minimum weight contribution that is needed (this is just a hint which subclasses may ignore).

get_docid()

Return the current docid.

virtual Xapian::docid Xapian::PostingSource::get_docid() const =0

This method may assume that it will only be called when there is a “current document”. See get_weight() for details.

Note: in the case of a multi-database search, the returned docid should be in the single subdatabase relevant to this posting source. See the init() method for details.

get_maxweight()

Return the currently set upper bound on what get_weight() can return.

double Xapian::PostingSource::get_maxweight() const

get_termfreq_est()

An estimate of the number of documents this object can return.

virtual Xapian::doccount Xapian::PostingSource::get_termfreq_est() const =0

It must always be true that:

get_termfreq_min() <= get_termfreq_est() <= get_termfreq_max()

Xapian will always call init() on a PostingSource before calling this for the first time.

get_termfreq_max()

An upper bound on the number of documents this object can return.

virtual Xapian::doccount Xapian::PostingSource::get_termfreq_max() const =0

Xapian will always call init() on a PostingSource before calling this for the first time.

get_termfreq_min()

A lower bound on the number of documents this object can return.

virtual Xapian::doccount Xapian::PostingSource::get_termfreq_min() const =0

Xapian will always call init() on a PostingSource before calling this for the first time.

get_weight()

Return the weight contribution for the current document.

virtual double Xapian::PostingSource::get_weight() const

This default implementation always returns 0, for convenience when implementing “weight-less” PostingSource subclasses.

This method may assume that it will only be called when there is a “current document”. In detail: Xapian will always call init() on a PostingSource before calling this for the first time. It will also only call this if the PostingSource reports that it is pointing to a valid document (ie, it will not call it before calling at least one of next(), skip_to() or check(), and will ensure that the PostingSource is not at the end by calling at_end()).

init()

Set this PostingSource to the start of the list of postings.

virtual void Xapian::PostingSource::init(const Database &db)=0

This is called automatically by the matcher prior to each query being processed.

If a PostingSource is used for multiple searches, init() will therefore be called multiple times, and must handle this by using the database passed in the most recent call.

db: The database which the PostingSource should iterate through.

Note: in the case of a multi-database search, a separate PostingSource will be used for each database (the separate PostingSources will be obtained using clone()), and each PostingSource will be passed one of the sub-databases as the db parameter here. The db parameter will therefore always refer to a single database. All docids passed to, or returned from, the PostingSource refer to docids in that single database, rather than in the multi- database.

name()

Name of the posting source class.

virtual std::string Xapian::PostingSource::name() const

This is used when serialising and unserialising posting sources; for example, for performing remote searches.

If the subclass is in a C++ namespace, the namespace should be included in the name, using ”::” as a separator. For example, for a PostingSource subclass called “FooPostingSource” in the “Xapian” namespace the result of this call should be “Xapian::FooPostingSource”.

This should only be implemented if serialise() and unserialise() are also implemented. The default implementation returns an empty string.

If this returns an empty string, Xapian will assume that serialise() and unserialise() are not implemented.

next()

Advance the current position to the next matching document.

virtual void Xapian::PostingSource::next(double min_wt)=0

The PostingSource starts before the first entry in the list, so next(), skip_to() or check() must be called before any methods which need the context of the current position.

Xapian will always call init() on a PostingSource before calling this for the first time.

min_wt: The minimum weight contribution that is needed (this is just a hint which subclasses may ignore).

release()

Start reference counting this object.

const PostingSource* Xapian::PostingSource::release() const

You can hand ownership of a dynamically allocated PostingSource object to Xapian by calling release() and then passing the object to a Xapian method. Xapian will arrange to delete the object once it is no longer required.

set_maxweight()

Specify an upper bound on what get_weight() will return from now on.

void Xapian::PostingSource::set_maxweight(double max_weight)

This upper bound is used by the matcher to perform various optimisations, so if you can return a good bound, then matches will generally run faster.

This method should be called after calling init(), and may be called during iteration if the upper bound drops. It is probably only useful to call from subclasses (it was actually a “protected” method prior to Xapian 1.3.4, but that makes it tricky to wrap for other languages).

It is valid for the posting source to have returned a higher value from get_weight() earlier in the iteration, but the posting source must not return a higher value from get_weight() than the currently set upper bound, and the upper bound must not be increased (until init() has been called).

If you don’t call this method, the upper bound will default to 0, for convenience when implementing “weight-less” PostingSource subclasses.

max_weight: The upper bound to set.

skip_to()

Advance to the specified docid.

virtual void Xapian::PostingSource::skip_to(Xapian::docid did, double min_wt)

If the specified docid isn’t in the list, position ourselves on the first document after it (or at_end() if no greater docids are present).

If the current position is already the specified docid, this method will leave the position unmodified.

If the specified docid is earlier than the current position, the behaviour is unspecified. A sensible behaviour would be to leave the current position unmodified, but it is also reasonable to move to the specified docid.

The default implementation calls next() repeatedly, which works but skip_to() can often be implemented much more efficiently.

Xapian will always call init() on a PostingSource before calling this for the first time.

Note: in the case of a multi-database search, the docid specified is the docid in the single subdatabase relevant to this posting source. See the init() method for details.

did: The document id to advance to.

min_wt: The minimum weight contribution that is needed (this is just a hint which subclasses may ignore).

thisown

The membership flag

class xapian.Query(*args)[source]

Bases: object

Class representing a query.

MatchAll = <xapian.Query; proxy of <Swig Object of type 'Xapian::Query *'> >
MatchNothing = <xapian.Query; proxy of <Swig Object of type 'Xapian::Query *'> >
OP_AND = 0
OP_AND_MAYBE = 4
OP_AND_NOT = 2
OP_ELITE_SET = 10
OP_FILTER = 5
OP_INVALID = 99
OP_MAX = 14
OP_NEAR = 6
OP_OR = 1
OP_PHRASE = 7
OP_SCALE_WEIGHT = 9
OP_SYNONYM = 13
OP_VALUE_GE = 11
OP_VALUE_LE = 12
OP_VALUE_RANGE = 8
OP_WILDCARD = 15
OP_XOR = 3
WILDCARD_LIMIT_ERROR = 0
WILDCARD_LIMIT_FIRST = 1
WILDCARD_LIMIT_MOST_FREQUENT = 2
empty()

Check if this query is Xapian::Query::MatchNothing.

bool Xapian::Query::empty() const

get_length()

Return the length of this query object.

Xapian::termcount Xapian::Query::get_length() const

get_num_subqueries()

Get the number of subqueries of the top level query.

size_t Xapian::Query::get_num_subqueries() const

get_subquery()

Read a top level subquery.

const Query Xapian::Query::get_subquery(size_t n) const

n: Return the n-th subquery (starting from 0) - only valid when 0 <= n < get_num_subqueries().

get_type()

Get the type of the top level of the query.

op Xapian::Query::get_type() const

get_unique_terms_begin()

Begin iterator for unique terms in the query object.

const TermIterator Xapian::Query::get_unique_terms_begin() const

Terms are sorted and terms with the same name removed from the list.

If you want the terms in ascending query position order, see get_terms_begin().

serialise()

Serialise this object into a string.

std::string Xapian::Query::serialise() const

thisown

The membership flag

static unserialise()
class xapian.QueryParser[source]

Bases: object

Build a Xapian::Query object from a user query string.

FLAG_AUTO_MULTIWORD_SYNONYMS = 1024
FLAG_AUTO_SYNONYMS = 512
FLAG_BOOLEAN = 1
FLAG_BOOLEAN_ANY_CASE = 8
FLAG_CJK_NGRAM = 2048
FLAG_DEFAULT = 7
FLAG_LOVEHATE = 4
FLAG_PARTIAL = 64
FLAG_PHRASE = 2
FLAG_PURE_NOT = 32
FLAG_SPELLING_CORRECTION = 128
FLAG_SYNONYM = 256
FLAG_WILDCARD = 16
STEM_ALL = 2
STEM_ALL_Z = 3
STEM_NONE = 0
STEM_SOME = 1
STEM_SOME_FULL_POS = 4
add_boolean_prefix(s, proc, exclusive=True)[source]

Register a FieldProcessor for a boolean prefix.

void Xapian::QueryParser::add_boolean_prefix(const std::string &field, Xapian::FieldProcessor *proc, bool exclusive)

This is an older version of this method - use the version with the grouping parameter in preference to this one.

add_prefix(s, proc)[source]

Register a FieldProcessor.

void Xapian::QueryParser::add_prefix(const std::string &field, Xapian::FieldProcessor *proc)

add_rangeprocessor(rproc)[source]

Register a RangeProcessor.

void Xapian::QueryParser::add_rangeprocessor(Xapian::RangeProcessor *range_proc, const std::string *grouping=NULL)

add_valuerangeprocessor(vrproc)[source]

Register a ValueRangeProcessor.

void Xapian::QueryParser::add_valuerangeprocessor(Xapian::ValueRangeProcessor *vrproc)

This method is provided for API compatibility with Xapian 1.2.x and is deprecated - use add_rangeprocessor() with a RangeProcessor instead.

Compatibility shim.

get_corrected_query_string()

Get the spelling-corrected query string.

std::string Xapian::QueryParser::get_corrected_query_string() const

This will only be set if FLAG_SPELLING_CORRECTION is specified when QueryParser::parse_query() was last called.

If there were no corrections, an empty string is returned.

get_default_op()

Get the current default operator.

Query::op Xapian::QueryParser::get_default_op() const

parse_query()

Parse a query.

Query Xapian::QueryParser::parse_query(const std::string &query_string, unsigned flags=FLAG_DEFAULT, const std::string &default_prefix=std::string())

query_string: A free-text query as entered by a user

flags: Zero or more QueryParser::feature_flag specifying what features the QueryParser should support. Combine multiple values with bitwise-or (|) (default FLAG_DEFAULT).

default_prefix: The default term prefix to use (default none). For example, you can pass “A” when parsing an “Author” field.

If: the query string can’t be parsed, then Xapian::QueryParserError is thrown. You can get an English error message to report to the user by catching it and calling get_msg() on the caught exception. The current possible values (in case you want to translate them) are:

Unknown range operation

parse error

Syntax: <expression> AND <expression>

Syntax: <expression> AND NOT <expression>

Syntax: <expression> NOT <expression>

Syntax: <expression> OR <expression>

Syntax: <expression> XOR <expression>

set_database()

Specify the database being searched.

void Xapian::QueryParser::set_database(const Database &db)

db: The database to use for spelling correction (FLAG_SPELLING_CORRECTION), and synonyms (FLAG_SYNONYM, FLAG_AUTO_SYNONYMS, and FLAG_AUTO_MULTIWORD_SYNONYMS).

set_default_op()

Set the default operator.

void Xapian::QueryParser::set_default_op(Query::op default_op)

default_op: The operator to use to combine non-filter query items when no explicit operator is used.

So for example, ‘weather forecast’ is parsed as if it were ‘weather OR forecast’ by default.

The most useful values for this are OP_OR (the default) and OP_AND. OP_NEAR, OP_PHRASE, OP_ELITE_SET, OP_SYNONYM and OP_MAX are also permitted. Passing other values will result in InvalidArgumentError being thrown.

set_max_expansion()

Specify the maximum expansion of a wildcard and/or partial term.

void Xapian::QueryParser::set_max_expansion(Xapian::termcount max_expansion, int max_type=Xapian::Query::WILDCARD_LIMIT_ERROR, unsigned flags=FLAG_WILDCARD|FLAG_PARTIAL)

Note: you must also set FLAG_WILDCARD and/or FLAG_PARTIAL in the flags parameter to parse_query() for this setting to have anything to affect.

If you don’t call this method, the default settings are no limit on wildcard expansion, and partial terms expanding to the most frequent 100 terms - i.e. as if you’d called:

set_max_expansion(0); set_max_expansion(100, Xapian::Query::WILDCARD_LIMIT_MOST_FREQUENT, Xapian::QueryParser::FLAG_PARTIAL);

max_expansion: The maximum number of terms each wildcard in the query can expand to, or 0 for no limit (which is the default).

max_type: Xapian::Query::WILDCARD_LIMIT_ERROR, Xapian::Query::WILDCARD_LIMIT_FIRST or Xapian::Query::WILDCARD_LIMIT_MOST_FREQUENT (default: Xapian::Query::WILDCARD_LIMIT_ERROR).

flags: What to set the limit for (default: FLAG_WILDCARD|FLAG_PARTIAL, setting the limit for both wildcards and partial terms).

1.3.3

set_max_wildcard_expansion()

Specify the maximum expansion of a wildcard.

void Xapian::QueryParser::set_max_wildcard_expansion(Xapian::termcount)

If any wildcard expands to more than max_expansion terms, an exception will be thrown.

This method is provided for API compatibility with Xapian 1.2.x and is deprecated - replace it with:

set_max_wildcard_expansion(max_expansion, Xapian::Query::WILDCARD_LIMIT_ERROR, Xapian::QueryParser::FLAG_WILDCARD);

set_stemmer()

Set the stemmer.

void Xapian::QueryParser::set_stemmer(const Xapian::Stem &stemmer)

This sets the stemming algorithm which will be used by the query parser. The stemming algorithm will be used according to the stemming strategy set by set_stemming_strategy(). As of 1.3.1, this defaults to STEM_SOME, but in earlier versions the default was STEM_NONE. If you want to work with older versions, you should explicitly set a stemming strategy as well as setting a stemmer, otherwise your stemmer won’t actually be used.

stemmer: The Xapian::Stem object to set.

set_stemming_strategy()

Set the stemming strategy.

void Xapian::QueryParser::set_stemming_strategy(stem_strategy strategy)

This controls how the query parser will apply the stemming algorithm. Note that the stemming algorithm is only applied to words in free-text fields - boolean filter terms are never stemmed.

strategy: The strategy to use - possible values are: STEM_NONE: Don’t perform any stemming. (default in Xapian <= 1.3.0)

STEM_SOME: Stem all terms except for those which start with a capital letter, or are followed by certain characters (currently: (/@<>=*[{” ), or are used with operators which need positional information. Stemmed terms are prefixed with ‘Z’. (default in Xapian >= 1.3.1)

STEM_SOME_FULL_POS: Like STEM_SOME but also stems terms used with operators which need positional information. Added in Xapian 1.4.8.

STEM_ALL: Stem all terms (note: no ‘Z’ prefix is added).

STEM_ALL_Z: Stem all terms (note: ‘Z’ prefix is added). (new in Xapian 1.2.11 and 1.3.1)

set_stopper(stopper)[source]

Set the stopper.

void Xapian::QueryParser::set_stopper(const Stopper *stop=NULL)

stop: The Stopper object to set (default NULL, which means no stopwords).

stoplist()

Get an iterator over all the stopped terms from the previous query.

This returns an iterator over all the terms which were omitted from the previously parsed query due to being considered to be stopwords. Each instance of a word omitted from the query is represented in the returned list, in the order in which the

The iterator will return string objects.

thisown

The membership flag

unstemlist(tname)

Get an iterator over all the unstemmed forms of a stemmed term.

This returns an iterator which returns all the unstemmed words which were stemmed to the stemmed form specified by tname when parsing the previous query. Each instance of a word which stems to tname is returned by the iterator in the order in which the words appeared in the query - an individual unstemmed word may thus occur multiple times.

The iterator will return string objects.

exception xapian.QueryParserError(*args)[source]

Bases: xapian.RuntimeError

Indicates a query string can’t be parsed.

class xapian.RSet[source]

Bases: object

A relevance set (R-Set).

This is the set of documents which are marked as relevant, for use in modifying the term weights, and in performing query expansion.

add_document()

Add a document to the relevance set.

void Xapian::RSet::add_document(const Xapian::MSetIterator &i)

contains()

Test if a given document in the relevance set.

bool Xapian::RSet::contains(const Xapian::MSetIterator &i) const

empty()

Test if this R-Set is empty.

bool Xapian::RSet::empty() const

remove_document()

Remove a document from the relevance set.

void Xapian::RSet::remove_document(const Xapian::MSetIterator &i)

size()

The number of documents in this R-Set.

Xapian::doccount Xapian::RSet::size() const

thisown

The membership flag

exception xapian.RangeError(*args)[source]

Bases: xapian.RuntimeError

RangeError indicates an attempt to access outside the bounds of a container.

class xapian.RangeProcessor(*args)[source]

Bases: object

Base class for range processors.

check_range()

Check prefix/suffix on range.

Xapian::Query Xapian::RangeProcessor::check_range(const std::string &b, const std::string &e)

If they match, remove the prefix/suffix and then call operator()() to try to handle the range.

release()

Start reference counting this object.

const RangeProcessor* Xapian::RangeProcessor::release() const

You can hand ownership of a dynamically allocated RangeProcessor object to Xapian by calling release() and then passing the object to a Xapian method. Xapian will arrange to delete the object once it is no longer required.

thisown

The membership flag

class xapian.Registry[source]

Bases: object

Registry for user subclasses.

This class provides a way for the remote server to look up user subclasses when unserialising.

get_lat_long_metric()

Get a lat-long metric given a name.

const Xapian::LatLongMetric* Xapian::Registry::get_lat_long_metric(const std::string &name) const

The returned metric is owned by the registry object.

Returns NULL if the metric could not be found.

get_match_spy()

Get a match spy given a name.

const Xapian::MatchSpy* Xapian::Registry::get_match_spy(const std::string &name) const

name: The name of the match spy to find.

An object with the requested name, or NULL if the match spy could not be found. The returned object is owned by the registry and so must not be deleted by the caller.

get_posting_source()

Get a posting source given a name.

const Xapian::PostingSource* Xapian::Registry::get_posting_source(const std::string &name) const

name: The name of the posting source to find.

An object with the requested name, or NULL if the posting source could not be found. The returned object is owned by the registry and so must not be deleted by the caller.

get_weighting_scheme()

Get the weighting scheme given a name.

const Xapian::Weight* Xapian::Registry::get_weighting_scheme(const std::string &name) const

name: The name of the weighting scheme to find.

An object with the requested name, or NULL if the weighting scheme could not be found. The returned object is owned by the registry and so must not be deleted by the caller.

register_lat_long_metric()

Register a user-defined lat-long metric class.

void Xapian::Registry::register_lat_long_metric(const Xapian::LatLongMetric &metric)

register_match_spy()

Register a user-defined match spy class.

void Xapian::Registry::register_match_spy(const Xapian::MatchSpy &spy)

spy: The match spy to register.

register_posting_source()

Register a user-defined posting source class.

void Xapian::Registry::register_posting_source(const Xapian::PostingSource &source)

source: The posting source to register.

register_weighting_scheme()

Register a weighting scheme.

void Xapian::Registry::register_weighting_scheme(const Xapian::Weight &wt)

wt: The weighting scheme to register.

thisown

The membership flag

exception xapian.RuntimeError(*args, **kwargs)[source]

Bases: xapian.Error

The base class for exceptions indicating errors only detectable at runtime.

A subclass of RuntimeError will be thrown if Xapian detects an error which is exception derived from RuntimeError is thrown when an error is caused by problems with the data or environment rather than a programming mistake.

exception xapian.SerialisationError(*args)[source]

Bases: xapian.RuntimeError

Indicates an error in the std::string serialisation of an object.

class xapian.SimpleStopper(*args)[source]

Bases: xapian.Stopper

Simple implementation of Stopper class - this will suit most users.

add()

Add a single stop word.

void Xapian::SimpleStopper::add(const std::string &word)

thisown

The membership flag

class xapian.Stem(*args)[source]

Bases: object

Class representing a stemming algorithm.

static get_available_languages()
thisown

The membership flag

class xapian.StemImplementation[source]

Bases: object

Class representing a stemming algorithm implementation.

thisown

The membership flag

class xapian.Stopper[source]

Bases: object

Base class for stop-word decision functor.

release()

Start reference counting this object.

const Stopper* Xapian::Stopper::release() const

You can hand ownership of a dynamically allocated Stopper object to Xapian by calling release() and then passing the object to a Xapian method. Xapian will arrange to delete the object once it is no longer required.

thisown

The membership flag

class xapian.StringValueRangeProcessor(*args)[source]

Bases: xapian.ValueRangeProcessor

Handle a string range.

The end points can be any strings.

Deprecated Use Xapian::RangeProcessor instead (added in 1.3.6).

thisown

The membership flag

class xapian.TermGenerator[source]

Bases: object

Parses a piece of text and generate terms.

This module takes a piece of text and parses it to produce words which are then used to generate suitable terms for indexing. The terms generated are suitable for use with Query objects produced by the QueryParser class.

FLAG_CJK_NGRAM = 2048
FLAG_SPELLING = 128
STEM_ALL = 2
STEM_ALL_Z = 3
STEM_NONE = 0
STEM_SOME = 1
STEM_SOME_FULL_POS = 4
STOP_ALL = 1
STOP_NONE = 0
STOP_STEMMED = 2
get_document()

Get the current document.

const Xapian::Document& Xapian::TermGenerator::get_document() const

get_termpos()

Get the current term position.

Xapian::termpos Xapian::TermGenerator::get_termpos() const

increase_termpos()

Increase the term position used by index_text.

void Xapian::TermGenerator::increase_termpos(Xapian::termpos delta=100)

This can be used between indexing text from different fields or other places to prevent phrase searches from spanning between them (e.g. between the title and body text, or between two chapters in a book).

delta: Amount to increase the term position by (default: 100).

index_text()

Index some text in a std::string.

void Xapian::TermGenerator::index_text(const std::string &text, Xapian::termcount wdf_inc=1, const std::string &prefix=std::string())

text: The text to index.

wdf_inc: The wdf increment (default 1).

prefix: The term prefix to use (default is no prefix).

index_text_without_positions()

Index some text in a std::string without positional information.

void Xapian::TermGenerator::index_text_without_positions(const std::string &text, Xapian::termcount wdf_inc=1, const std::string &prefix=std::string())

Just like index_text, but no positional information is generated. This means that the database will be significantly smaller, but that phrase searching and NEAR won’t be supported.

text: The text to index.

wdf_inc: The wdf increment (default 1).

prefix: The term prefix to use (default is no prefix).

set_database()

Set the database to index spelling data to.

void Xapian::TermGenerator::set_database(const Xapian::WritableDatabase &db)

set_document()

Set the current document.

void Xapian::TermGenerator::set_document(const Xapian::Document &doc)

set_flags()

Set flags.

flags Xapian::TermGenerator::set_flags(flags toggle, flags mask=flags(0))

The new value of flags is: (flags & mask) ^ toggle

To just set the flags, pass the new flags in toggle and the default value for mask.

toggle: Flags to XOR.

mask: Flags to AND with first.

The old flags setting.

set_max_word_length()

Set the maximum length word to index.

void Xapian::TermGenerator::set_max_word_length(unsigned max_word_length)

The limit is on the length of a word prior to stemming and prior to adding any term prefix.

The backends mostly impose a limit on the length of terms (often of about 240 bytes), but it’s generally useful to have a lower limit to help prevent the index being bloated by useless junk terms from trying to indexing things like binary data, uuencoded data, ASCII art, etc.

This method was new in Xapian 1.3.1.

max_word_length: The maximum length word to index, in bytes in UTF-8 representation. Default is 64.

set_stemmer()

Set the Xapian::Stem object to be used for generating stemmed terms.

void Xapian::TermGenerator::set_stemmer(const Xapian::Stem &stemmer)

set_stemming_strategy()

Set the stemming strategy.

void Xapian::TermGenerator::set_stemming_strategy(stem_strategy strategy)

This method controls how the stemming algorithm is applied. It was new in Xapian 1.3.1.

strategy: The strategy to use - possible values are: STEM_NONE: Don’t perform any stemming - only unstemmed terms are generated.

STEM_SOME: Generate both stemmed (with a “Z” prefix) and unstemmed terms. No positional information is stored for unstemmed terms. This is the default strategy.

STEM_SOME_FULL_POS: Like STEM_SOME but positional information is stored for both stemmed and unstemmed terms. Added in Xapian 1.4.8.

STEM_ALL: Generate only stemmed terms (but without a “Z” prefix).

STEM_ALL_Z: Generate only stemmed terms (with a “Z” prefix).

set_stopper(stopper)[source]

Set the Xapian::Stopper object to be used for identifying stopwords.

void Xapian::TermGenerator::set_stopper(const Xapian::Stopper *stop=NULL)

Stemmed forms of stopwords aren’t indexed, but unstemmed forms still are so that searches for phrases including stop words still work.

stop: The Stopper object to set (default NULL, which means no stopwords).

set_stopper_strategy()

Set the stopper strategy.

void Xapian::TermGenerator::set_stopper_strategy(stop_strategy strategy)

The method controls how the stopper is used. It was added in Xapian 1.4.1.

You need to also call set_stopper() for this to have any effect.

strategy: The strategy to use - possible values are: STOP_NONE: Don’t use the stopper.

STOP_ALL: If a word is identified as a stop word, skip it completely.

STOP_STEMMED: If a word is identified as a stop word, index its unstemmed form but skip the stem. Unstemmed forms are indexed with positional information by default, so this allows searches for phrases containing stopwords to be supported. (This is the default mode).

set_termpos()

Set the current term position.

void Xapian::TermGenerator::set_termpos(Xapian::termpos termpos)

termpos: The new term position to set.

thisown

The membership flag

class xapian.TermIter(start, end, has_termfreq=0, has_wdf=0, has_positions=0, return_strings=False)[source]

Bases: object

An iterator over a term list.

The iterator will return TermListItem objects, which will be evaluated lazily where appropriate.

EAGER = 2
INVALID = 0
LAZY = 1
next()[source]
skip_to(term)[source]

Skip the iterator forward.

The iterator is advanced to the first term at or after the current position which is greater than or equal to the supplied term.

If there are no such items, this will raise StopIteration.

This returns the item which the iterator is moved to. The subsequent item will be returned the next time that next() is called (unless skip_to() is called again first).

class xapian.TermListItem(iter, term)[source]

Bases: object

An item returned from iteration of a term list.

The item supports access to the following attributes and properties:

  • term: The term corresponding to this TermListItem.
  • wdf: The within document frequency of this term.
  • termfreq: The number of documents in the collection which are indexed by the term
  • positer: An iterator over the positions which the term appears at in the document. This is only available until the iterator which returned this item next moves.
positer

A position iterator for the current term (if meaningful).

The iterator will return integers representing the positions that the term occurs at.

This will raise a InvalidOperationError exception if the iterator this item came from doesn’t support position lists, or if the iterator has moved on since the item was returned from it.

term
termfreq

The term frequency of the current term (if meaningful).

This is the number of documents in the collection which are indexed by the term.

This will raise a InvalidOperationError exception if the iterator this item came from doesn’t support term frequencies.

wdf

The within-document-frequency of the current term (if meaningful).

This will raise a InvalidOperationError exception if the iterator this item came from doesn’t support within-document-frequencies.

class xapian.TfIdfWeight(*args)[source]

Bases: xapian.Weight

Xapian::Weight subclass implementing the tf-idf weighting scheme.

thisown

The membership flag

class xapian.TradWeight(k=1.0)[source]

Bases: xapian.Weight

Xapian::Weight subclass implementing the traditional probabilistic formula.

This class implements the “traditional” Probabilistic Weighting scheme, as described by the early papers on Probabilistic Retrieval. BM25 generally gives better results.

TradWeight(k) is equivalent to BM25Weight(k, 0, 0, 1, 0), except that the latter returns weights (k+1) times larger.

thisown

The membership flag

exception xapian.UnimplementedError(*args)[source]

Bases: xapian.LogicError

UnimplementedError indicates an attempt to use an unimplemented feature.

class xapian.ValueCountMatchSpy(*args)[source]

Bases: xapian.MatchSpy

Class for counting the frequencies of values in the matching documents.

get_total()

Return the total number of documents tallied.

size_t Xapian::ValueCountMatchSpy::get_total() const

thisown

The membership flag

top_values(maxvalues)

Get an iterator over the most frequent values for the slot.

Values will be returned in descending order of frequency. Values with the same frequency will be returned in ascending alphabetical order.

The iterator will return TermListItem objects: the value can be accessed as the term property, and the frequency can be accessed as the termfreq property.

values()

Get an iterator over all the values in the slot.

Values will be returned in ascending alphabetical order.

The iterator will return TermListItem objects: the value can be accessed as the term property, and the frequency can be accessed as the termfreq property.

class xapian.ValueItem(num, value)[source]

Bases: object

An item returned from iteration of the values in a document.

The item supports access to the following attributes:

  • num: The number of the value.
  • value: The contents of the value.
num
value
class xapian.ValueIter(start, end)[source]

Bases: object

An iterator over all the values stored in a document.

The iterator will return ValueItem objects, in ascending order of value number.

next()[source]
class xapian.ValueMapPostingSource(slot_)[source]

Bases: xapian.ValuePostingSource

A posting source which looks up weights in a map using values as the key.

This allows will return entries for all documents in the given database which have a value in the slot specified. The values will be mapped to the corresponding weight in the weight map. If there is no mapping for a particular value, the default weight will be returned (which itself defaults to 0.0).

add_mapping()

Add a mapping.

void Xapian::ValueMapPostingSource::add_mapping(const std::string &key, double wt)

key: The key looked up from the value slot.

wt: The weight to give this key.

clear_mappings()

Clear all mappings.

void Xapian::ValueMapPostingSource::clear_mappings()

set_default_weight()

Set a default weight for document values not in the map.

void Xapian::ValueMapPostingSource::set_default_weight(double wt)

wt: The weight to set as the default.

thisown

The membership flag

class xapian.ValuePostingSource(slot_)[source]

Bases: xapian.PostingSource

A posting source which generates weights from a value slot.

This is a base class for classes which generate weights using values stored in the specified slot. For example, ValueWeightPostingSource uses sortable_unserialise to convert values directly to weights.

The upper bound on the weight returned is set to DBL_MAX. Subclasses should call set_maxweight() in their init() methods after calling ValuePostingSource::init() if they know a tighter bound on the weight.

at_end()

Return true if the current position is past the last entry in this list.

bool Xapian::ValuePostingSource::at_end() const

At least one of next(), skip_to() or check() will be called before this method is first called.

check()

Check if the specified docid occurs.

bool Xapian::ValuePostingSource::check(Xapian::docid min_docid, double min_wt)

The caller is required to ensure that the specified document id did actually exists in the database. If it does, it must move to that document id, and return true. If it does not, it may either:

return true, having moved to a definite position (including “at_end”), which must be the same position as skip_to() would have moved to.

or

return false, having moved to an “indeterminate” position, such that a subsequent call to next() or skip_to() will move to the next matching position after did.

Generally, this method should act like skip_to() and return true if that can be done at little extra cost.

Otherwise it should simply check if a particular docid is present, returning true if it is, and false if it isn’t.

The default implementation calls skip_to() and always returns true.

Xapian will always call init() on a PostingSource before calling this for the first time.

Note: in the case of a multi-database search, the docid specified is the docid in the single subdatabase relevant to this posting source. See the init() method for details.

did: The document id to check.

min_wt: The minimum weight contribution that is needed (this is just a hint which subclasses may ignore).

done()

End the iteration.

void Xapian::ValuePostingSource::done()

Calls to at_end() will return true after calling this method.

Added in 1.2.23 and 1.3.5.

get_database()

The database we’re reading values from.

Xapian::Database Xapian::ValuePostingSource::get_database() const

Added in 1.2.23 and 1.3.5.

get_docid()

Return the current docid.

Xapian::docid Xapian::ValuePostingSource::get_docid() const

This method may assume that it will only be called when there is a “current document”. See get_weight() for details.

Note: in the case of a multi-database search, the returned docid should be in the single subdatabase relevant to this posting source. See the init() method for details.

get_slot()

The slot we’re reading values from.

Xapian::valueno Xapian::ValuePostingSource::get_slot() const

Added in 1.2.23 and 1.3.5.

get_started()

Flag indicating if we’ve started (true if we have).

bool Xapian::ValuePostingSource::get_started() const

Added in 1.2.23 and 1.3.5.

get_termfreq_est()

An estimate of the number of documents this object can return.

Xapian::doccount Xapian::ValuePostingSource::get_termfreq_est() const

It must always be true that:

get_termfreq_min() <= get_termfreq_est() <= get_termfreq_max()

Xapian will always call init() on a PostingSource before calling this for the first time.

get_termfreq_max()

An upper bound on the number of documents this object can return.

Xapian::doccount Xapian::ValuePostingSource::get_termfreq_max() const

Xapian will always call init() on a PostingSource before calling this for the first time.

get_termfreq_min()

A lower bound on the number of documents this object can return.

Xapian::doccount Xapian::ValuePostingSource::get_termfreq_min() const

Xapian will always call init() on a PostingSource before calling this for the first time.

get_value()

Read current value.

std::string Xapian::ValuePostingSource::get_value() const

Added in 1.2.23 and 1.3.5.

init()

Set this PostingSource to the start of the list of postings.

void Xapian::ValuePostingSource::init(const Database &db_)

This is called automatically by the matcher prior to each query being processed.

If a PostingSource is used for multiple searches, init() will therefore be called multiple times, and must handle this by using the database passed in the most recent call.

db: The database which the PostingSource should iterate through.

Note: in the case of a multi-database search, a separate PostingSource will be used for each database (the separate PostingSources will be obtained using clone()), and each PostingSource will be passed one of the sub-databases as the db parameter here. The db parameter will therefore always refer to a single database. All docids passed to, or returned from, the PostingSource refer to docids in that single database, rather than in the multi- database.

next()

Advance the current position to the next matching document.

void Xapian::ValuePostingSource::next(double min_wt)

The PostingSource starts before the first entry in the list, so next(), skip_to() or check() must be called before any methods which need the context of the current position.

Xapian will always call init() on a PostingSource before calling this for the first time.

min_wt: The minimum weight contribution that is needed (this is just a hint which subclasses may ignore).

set_termfreq_est()

An estimate of the term frequency.

void Xapian::ValuePostingSource::set_termfreq_est(Xapian::doccount termfreq_est_)

Subclasses should set this if they are overriding the next(), skip_to() or check() methods.

Added in 1.2.23 and 1.3.5.

set_termfreq_max()

An upper bound on the term frequency.

void Xapian::ValuePostingSource::set_termfreq_max(Xapian::doccount termfreq_max_)

Subclasses should set this if they are overriding the next(), skip_to() or check() methods.

Added in 1.2.23 and 1.3.5.

set_termfreq_min()

Set a lower bound on the term frequency.

void Xapian::ValuePostingSource::set_termfreq_min(Xapian::doccount termfreq_min_)

Subclasses should set this if they are overriding the next(), skip_to() or check() methods to return fewer documents.

Added in 1.2.23 and 1.3.5.

skip_to()

Advance to the specified docid.

void Xapian::ValuePostingSource::skip_to(Xapian::docid min_docid, double min_wt)

If the specified docid isn’t in the list, position ourselves on the first document after it (or at_end() if no greater docids are present).

If the current position is already the specified docid, this method will leave the position unmodified.

If the specified docid is earlier than the current position, the behaviour is unspecified. A sensible behaviour would be to leave the current position unmodified, but it is also reasonable to move to the specified docid.

The default implementation calls next() repeatedly, which works but skip_to() can often be implemented much more efficiently.

Xapian will always call init() on a PostingSource before calling this for the first time.

Note: in the case of a multi-database search, the docid specified is the docid in the single subdatabase relevant to this posting source. See the init() method for details.

did: The document id to advance to.

min_wt: The minimum weight contribution that is needed (this is just a hint which subclasses may ignore).

thisown

The membership flag

class xapian.ValueRangeProcessor[source]

Bases: object

Base class for value range processors.

release()

Start reference counting this object.

const ValueRangeProcessor* Xapian::ValueRangeProcessor::release() const

You can hand ownership of a dynamically allocated ValueRangeProcessor object to Xapian by calling release() and then passing the object to a Xapian method. Xapian will arrange to delete the object once it is no longer required.

thisown

The membership flag

class xapian.ValueSetMatchDecider(slot, inclusive_)[source]

Bases: xapian.MatchDecider

MatchDecider filtering results based on whether document values are in a user- defined set.

add_value()

Add a value to the test set.

void Xapian::ValueSetMatchDecider::add_value(const std::string &value)

value: The value to add to the test set.

remove_value()

Remove a value from the test set.

void Xapian::ValueSetMatchDecider::remove_value(const std::string &value)

value: The value to remove from the test set.

thisown

The membership flag

class xapian.ValueStreamItem(docid, value)[source]

Bases: object

An item returned from iteration of the values in a document.

The item supports access to the following attributes:

  • docid: The docid for the item.
  • value: The contents of the value.
docid
value
class xapian.ValueStreamIter(start, end)[source]

Bases: object

An iterator over all the values stored in a document.

The iterator will return ValueStreamItem objects, in ascending order of value number.

next()[source]
skip_to(docid)[source]

Skip the iterator forward.

The iterator is advanced to the first document with a document ID which is greater than or equal to the supplied document ID.

If there are no such items, this will raise StopIteration.

This returns the item which the iterator is moved to. The subsequent item will be returned the next time that next() is called (unless skip_to() is called again first).

class xapian.ValueWeightPostingSource(slot_)[source]

Bases: xapian.ValuePostingSource

A posting source which reads weights from a value slot.

This returns entries for all documents in the given database which have a non empty values in the specified slot. It returns a weight calculated by applying sortable_unserialise to the value stored in the slot (so the values stored should probably have been calculated by applying sortable_serialise to a floating point number at index time).

The upper bound on the weight returned is set using the upper bound on the values in the specified slot, or DBL_MAX if value bounds aren’t supported by the current backend.

For efficiency, this posting source doesn’t check that the stored values are valid in any way, so it will never raise an exception due to invalid stored values. In particular, it doesn’t ensure that the unserialised values are positive, which is a requirement for weights. The behaviour if the slot contains values which unserialise to negative values is undefined.

get_weight()

Return the weight contribution for the current document.

double Xapian::ValueWeightPostingSource::get_weight() const

This default implementation always returns 0, for convenience when implementing “weight-less” PostingSource subclasses.

This method may assume that it will only be called when there is a “current document”. In detail: Xapian will always call init() on a PostingSource before calling this for the first time. It will also only call this if the PostingSource reports that it is pointing to a valid document (ie, it will not call it before calling at least one of next(), skip_to() or check(), and will ensure that the PostingSource is not at the end by calling at_end()).

init()

Set this PostingSource to the start of the list of postings.

void Xapian::ValueWeightPostingSource::init(const Database &db_)

This is called automatically by the matcher prior to each query being processed.

If a PostingSource is used for multiple searches, init() will therefore be called multiple times, and must handle this by using the database passed in the most recent call.

db: The database which the PostingSource should iterate through.

Note: in the case of a multi-database search, a separate PostingSource will be used for each database (the separate PostingSources will be obtained using clone()), and each PostingSource will be passed one of the sub-databases as the db parameter here. The db parameter will therefore always refer to a single database. All docids passed to, or returned from, the PostingSource refer to docids in that single database, rather than in the multi- database.

name()

Name of the posting source class.

std::string Xapian::ValueWeightPostingSource::name() const

This is used when serialising and unserialising posting sources; for example, for performing remote searches.

If the subclass is in a C++ namespace, the namespace should be included in the name, using ”::” as a separator. For example, for a PostingSource subclass called “FooPostingSource” in the “Xapian” namespace the result of this call should be “Xapian::FooPostingSource”.

This should only be implemented if serialise() and unserialise() are also implemented. The default implementation returns an empty string.

If this returns an empty string, Xapian will assume that serialise() and unserialise() are not implemented.

thisown

The membership flag

class xapian.Weight(*args, **kwargs)[source]

Bases: object

Abstract base class for weighting schemes.

ABSOLUTE_DISCOUNT_SMOOTHING = 3
DIRICHLET_PLUS_SMOOTHING = 5
DIRICHLET_SMOOTHING = 2
JELINEK_MERCER_SMOOTHING = 4
TWO_STAGE_SMOOTHING = 1
get_maxextra()

Return an upper bound on what get_sumextra() can return for any document.

virtual double Xapian::Weight::get_maxextra() const =0

This information is used by the matcher to perform various optimisations, so strive to make the bound as tight as possible.

get_maxpart()

Return an upper bound on what get_sumpart() can return for any document.

virtual double Xapian::Weight::get_maxpart() const =0

This information is used by the matcher to perform various optimisations, so strive to make the bound as tight as possible.

get_sumextra()

Calculate the term-independent weight component for a document.

virtual double Xapian::Weight::get_sumextra(Xapian::termcount doclen, Xapian::termcount uniqterms) const =0

The parameter gives information about the document which may be used in the calculations:

doclen: The document’s length (unnormalised).

uniqterms: The number of unique terms in the document.

get_sumpart()

Calculate the weight contribution for this object’s term to a document.

virtual double Xapian::Weight::get_sumpart(Xapian::termcount wdf, Xapian::termcount doclen, Xapian::termcount uniqterms) const =0

The parameters give information about the document which may be used in the calculations:

wdf: The within document frequency of the term in the document.

doclen: The document’s length (unnormalised).

uniqterms: Number of unique terms in the document (used for absolute smoothing).

name()

Return the name of this weighting scheme.

virtual std::string Xapian::Weight::name() const

This name is used by the remote backend. It is passed along with the serialised parameters to the remote server so that it knows which class to create.

Return the full namespace-qualified name of your class here - if your class is called FooWeight, return “FooWeight” from this method ( Xapian::BM25Weight returns “Xapian::BM25Weight” here).

If you don’t want to support the remote backend, you can use the default implementation which simply returns an empty string.

thisown

The membership flag

exception xapian.WildcardError(*args)[source]

Bases: xapian.RuntimeError

WildcardError indicates an error expanding a wildcarded query.

class xapian.WritableDatabase(*args)[source]

Bases: xapian.Database

This class provides read/write access to a database.

add_document()

Add a new document to the database.

Xapian::docid Xapian::WritableDatabase::add_document(const Xapian::Document &document)

This method adds the specified document to the database, returning a newly allocated document ID. Automatically allocated document IDs come from a per-database monotonically increasing counter, so IDs from deleted documents won’t be reused.

If you want to specify the document ID to be used, you should call replace_document() instead.

Note that changes to the database won’t be immediately committed to disk; see commit() for more details.

As with all database modification operations, the effect is atomic: the document will either be fully added, or the document fails to be added and an exception is thrown (possibly at a later time when commit() is called or the database is closed).

document: The new document to be added.

The document ID of the newly added document.

Xapian::DatabaseError: will be thrown if a problem occurs while writing to the database.

Xapian::DatabaseCorruptError: will be thrown if the database is in a corrupt state.

add_spelling()

Add a word to the spelling dictionary.

void Xapian::WritableDatabase::add_spelling(const std::string &word, Xapian::termcount freqinc=1) const

If the word is already present, its frequency is increased.

word: The word to add.

freqinc: How much to increase its frequency by (default 1).

add_synonym()

Add a synonym for a term.

void Xapian::WritableDatabase::add_synonym(const std::string &term, const std::string &synonym) const

term: The term to add a synonym for.

synonym: The synonym to add. If this is already a synonym for term, then no action is taken.

begin_transaction()

Begin a transaction.

void Xapian::WritableDatabase::begin_transaction(bool flushed=true)

In Xapian a transaction is a group of modifications to the database which are linked such that either all will be applied simultaneously or none will be applied at all. Even in the case of a power failure, this characteristic should be preserved (as long as the filesystem isn’t corrupted, etc).

A transaction is started with begin_transaction() and can either be committed by calling commit_transaction() or aborted by calling cancel_transaction().

By default, a transaction implicitly calls commit() before and after so that the modifications stand and fall without affecting modifications before or after.

The downside of these implicit calls to commit() is that small transactions can harm indexing performance in the same way that explicitly calling commit() frequently can.

If you’re applying atomic groups of changes and only wish to ensure that each group is either applied or not applied, then you can prevent the automatic commit() before and after the transaction by starting the transaction with begin_transaction(false). However, if cancel_transaction is called (or if commit_transaction isn’t called before the WritableDatabase object is destroyed) then any changes which were pending before the transaction began will also be discarded.

Transactions aren’t currently supported by the InMemory backend.

flushed: Is this a flushed transaction? By default transactions are “flushed”, which means that committing a transaction will ensure those changes are permanently written to the database. By contrast, unflushed transactions only ensure that changes within the transaction are either all applied or all aren’t.

Xapian::UnimplementedError: will be thrown if transactions are not available for this database type.

Xapian::InvalidOperationError: will be thrown if this is called at an invalid time, such as when a transaction is already in progress.

cancel_transaction()

Abort the transaction currently in progress, discarding the pending modifications made to the database.

void Xapian::WritableDatabase::cancel_transaction()

If an error occurs in this method, an exception will be thrown, but the transaction will be cancelled anyway.

Xapian::DatabaseError: will be thrown if a problem occurs while modifying the database.

Xapian::DatabaseCorruptError: will be thrown if the database is in a corrupt state.

Xapian::InvalidOperationError: will be thrown if a transaction is not currently in progress.

Xapian::UnimplementedError: will be thrown if transactions are not available for this database type.

clear_synonyms()

Remove all synonyms for a term.

void Xapian::WritableDatabase::clear_synonyms(const std::string &term) const

term: The term to remove all synonyms for. If the term has no synonyms, no action is taken.

commit()

Commit any pending modifications made to the database.

void Xapian::WritableDatabase::commit()

For efficiency reasons, when performing multiple updates to a database it is best (indeed, almost essential) to make as many modifications as memory will permit in a single pass through the database. To ensure this, Xapian batches up modifications.

This method may be called at any time to commit any pending modifications to the database.

If any of the modifications fail, an exception will be thrown and the database will be left in a state in which each separate addition, replacement or deletion operation has either been fully performed or not performed at all: it is then up to the application to work out which operations need to be repeated.

It’s not valid to call commit() within a transaction.

Beware of calling commit() too frequently: this will make indexing take much longer.

Note that commit() need not be called explicitly: it will be called automatically when the database is closed, or when a sufficient number of modifications have been made. By default, this is every 10000 documents added, deleted, or modified. This value is rather conservative, and if you have a machine with plenty of memory, you can improve indexing throughput dramatically by setting XAPIAN_FLUSH_THRESHOLD in the environment to a larger value.

This method was new in Xapian 1.1.0 - in earlier versions it was called flush().

Xapian::DatabaseError: will be thrown if a problem occurs while modifying the database.

Xapian::DatabaseCorruptError: will be thrown if the database is in a corrupt state.

commit_transaction()

Complete the transaction currently in progress.

void Xapian::WritableDatabase::commit_transaction()

If this method completes successfully and this is a flushed transaction, all the database modifications made during the transaction will have been committed to the database.

If an error occurs, an exception will be thrown, and none of the modifications made to the database during the transaction will have been applied to the database.

In all cases the transaction will no longer be in progress.

Xapian::DatabaseError: will be thrown if a problem occurs while modifying the database.

Xapian::DatabaseCorruptError: will be thrown if the database is in a corrupt state.

Xapian::InvalidOperationError: will be thrown if a transaction is not currently in progress.

Xapian::UnimplementedError: will be thrown if transactions are not available for this database type.

delete_document()

Delete any documents indexed by a term from the database.

void Xapian::WritableDatabase::delete_document(const std::string &unique_term)

This method removes any documents indexed by the specified term from the database.

A major use is for convenience when UIDs from another system are mapped to terms in Xapian, although this method has other uses (for example, you could add a “deletion date” term to documents at index time and use this method to delete all documents due for deletion on a particular date).

unique_term: The term to remove references to.

Xapian::DatabaseError: will be thrown if a problem occurs while writing to the database.

Xapian::DatabaseCorruptError: will be thrown if the database is in a corrupt state.

flush()

Pre-1.1.0 name for commit().

void Xapian::WritableDatabase::flush()

Use commit() instead.

remove_spelling()

Remove a word from the spelling dictionary.

void Xapian::WritableDatabase::remove_spelling(const std::string &word, Xapian::termcount freqdec=1) const

The word’s frequency is decreased, and if would become zero or less then the word is removed completely.

word: The word to remove.

freqdec: How much to decrease its frequency by (default 1).

remove_synonym()

Remove a synonym for a term.

void Xapian::WritableDatabase::remove_synonym(const std::string &term, const std::string &synonym) const

term: The term to remove a synonym for.

synonym: The synonym to remove. If this isn’t currently a synonym for term, then no action is taken.

replace_document()

Replace any documents matching a term.

Xapian::docid Xapian::WritableDatabase::replace_document(const std::string &unique_term, const Xapian::Document &document)

This method replaces any documents indexed by the specified term with the specified document. If any documents are indexed by the term, the lowest document ID will be used for the document, otherwise a new document ID will be generated as for add_document.

One common use is to allow UIDs from another system to easily be mapped to terms in Xapian. Note that this method doesn’t automatically add unique_term as a term, so you’ll need to call document.add_term(unique_term) first when using replace_document() in this way.

Note that changes to the database won’t be immediately committed to disk; see commit() for more details.

As with all database modification operations, the effect is atomic: the document(s) will either be fully replaced, or the document(s) fail to be replaced and an exception is thrown (possibly at a later time when commit() is called or the database is closed).

unique_term: The “unique” term.

document: The new document.

The document ID that document was given.

Xapian::DatabaseError: will be thrown if a problem occurs while writing to the database.

Xapian::DatabaseCorruptError: will be thrown if the database is in a corrupt state.

set_metadata()

Set the user-specified metadata associated with a given key.

void Xapian::WritableDatabase::set_metadata(const std::string &key, const std::string &metadata)

This method sets the metadata value associated with a given key. If there is already a metadata value stored in the database with the same key, the old value is replaced. If you want to delete an existing item of metadata, just set its value to the empty string.

User-specified metadata allows you to store arbitrary information in the form of (key, value) pairs.

There’s no hard limit on the number of metadata items, or the size of the metadata values. Metadata keys have a limited length, which depend on the backend. We recommend limiting them to 200 bytes. Empty keys are not valid, and specifying one will cause an exception.

Metadata modifications are committed to disk in the same way as modifications to the documents in the database are: i.e., modifications are atomic, and won’t be committed to disk immediately (see commit() for more details). This allows metadata to be used to link databases with versioned external resources by storing the appropriate version number in a metadata item.

You can also use the metadata to store arbitrary extra information associated with terms, documents, or postings by encoding the termname and/or document id into the metadata key.

key: The key of the metadata item to set.

metadata: The value of the metadata item to set.

Xapian::DatabaseError: will be thrown if a problem occurs while writing to the database.

Xapian::DatabaseCorruptError: will be thrown if the database is in a corrupt state.

Xapian::InvalidArgumentError: will be thrown if the key supplied is empty.

Xapian::UnimplementedError: will be thrown if the database backend in use doesn’t support user- specified metadata.

thisown

The membership flag

xapian.chert_open(*args)[source]

Construct a Database object for update access to a Chert database.

WritableDatabase Xapian::Chert::open(const std::string &dir, int action, int block_size=0)

dir: pathname of the directory containing the database.

action: determines handling of existing/non-existing database: Xapian::DB_CREATE fail if database already exist, otherwise create new database.

Xapian::DB_CREATE_OR_OPEN open existing database, or create new database if none exists.

Xapian::DB_CREATE_OR_OVERWRITE overwrite existing database, or create new database if none exists.

Xapian::DB_OPEN open existing database, failing if none exists.

block_size: the Btree blocksize to use (in bytes), which must be a power of two between 2048 and 65536 (inclusive). The default (also used if an invalid value if passed) is 8192 bytes. This parameter is ignored when opening an existing database.

xapian.inmemory_open()[source]

Construct a WritableDatabase object for a new, empty InMemory database.

WritableDatabase Xapian::InMemory::open()

Only a writable InMemory database can be created, since a read-only one would always remain empty.

xapian.major_version()
xapian.metres_to_miles()
xapian.miles_to_metres()
xapian.minor_version()
xapian.new_instancemethod

alias of instancemethod

xapian.open_stub(*args)[source]

Construct a WritableDatabase object for a stub database file.

WritableDatabase Xapian::Auto::open_stub(const std::string &file, int action)

The stub database file must contain serialised parameters for exactly one database.

file: pathname of the stub database file.

action: determines handling of existing/non-existing database: Xapian::DB_CREATE fail if database already exist, otherwise create new database.

Xapian::DB_CREATE_OR_OPEN open existing database, or create new database if none exists.

Xapian::DB_CREATE_OR_OVERWRITE overwrite existing database, or create new database if none exists.

Xapian::DB_OPEN open existing database, failing if none exists.

xapian.remote_open(*args)[source]

Construct a Database object for read-only access to a remote database accessed via a program.

Database Xapian::Remote::open(const std::string &program, const std::string &args, useconds_t timeout=10000)

Access to the remote database is done by running an external program and communicating with it on stdin/stdout.

program: the external program to run.

args: space-separated list of arguments to pass to program.

timeout: timeout in milliseconds. If this timeout is exceeded for any individual operation on the remote database then Xapian::NetworkTimeoutError is thrown. A timeout of 0 means don’t timeout. (Default is 10000ms, which is 10 seconds).

xapian.remote_open_writable(*args)[source]

Construct a WritableDatabase object for update access to a remote database accessed via a program.

WritableDatabase Xapian::Remote::open_writable(const std::string &program, const std::string &args, useconds_t timeout=0, int flags=0)

Access to the remote database is done by running an external program and communicating with it on stdin/stdout.

program: the external program to run.

args: space-separated list of arguments to pass to program.

timeout: timeout in milliseconds. If this timeout is exceeded for any individual operation on the remote database then Xapian::NetworkTimeoutError is thrown. (Default is 0, which means don’t timeout).

flags: Xapian::DB_RETRY_LOCK or 0.

xapian.revision()
xapian.sortable_serialise()
xapian.sortable_unserialise()
xapian.version_string()
xapian.weakref_proxy()

proxy(object[, callback]) – create a proxy object that weakly references ‘object’. ‘callback’, if given, is called with a reference to the proxy when ‘object’ is about to be finalized.