|
xapian-core
2.0.0
|
Sharded database backend. More...
#include <multi_database.h>
Inheritance diagram for MultiDatabase:
Collaboration diagram for MultiDatabase:Public Member Functions | |
| MultiDatabase (size_type reserve_size, bool read_only) | |
| size_type | size () const |
| void | reserve (size_type new_size) |
| void | push_back (Xapian::Database::Internal *shard) |
| bool | reopen () |
| Reopen the database to the latest available revision. More... | |
| void | close () |
| Close the database. More... | |
| PostList * | open_post_list (std::string_view term) const |
| Return a PostList suitable for use in a PostingIterator. More... | |
| LeafPostList * | open_leaf_post_list (std::string_view term, bool need_read_pos) const |
| Create a LeafPostList for use during a match. More... | |
| TermList * | open_term_list (Xapian::docid did) const |
| TermList * | open_term_list_direct (Xapian::docid did) const |
| Like open_term_list() but without MultiTermList wrapper. More... | |
| TermList * | open_allterms (std::string_view prefix) const |
| bool | has_positions () const |
| Check whether this database contains any positional information. More... | |
| PositionList * | open_position_list (Xapian::docid did, std::string_view term) const |
| Xapian::doccount | get_doccount () const |
| Xapian::docid | get_lastdocid () const |
| Return the last used document id of this (sub) database. More... | |
| Xapian::totallength | get_total_length () const |
| Return the total length of all documents in this database. More... | |
| void | get_freqs (std::string_view term, Xapian::doccount *tf_ptr, Xapian::termcount *cf_ptr) const |
| Returns frequencies for a term. More... | |
| Xapian::doccount | get_value_freq (Xapian::valueno slot) const |
| Return the frequency of a given value slot. More... | |
| std::string | get_value_lower_bound (Xapian::valueno slot) const |
| Get a lower bound on the values stored in the given value slot. More... | |
| std::string | get_value_upper_bound (Xapian::valueno slot) const |
| Get an upper bound on the values stored in the given value slot. More... | |
| Xapian::termcount | get_doclength_lower_bound () const |
| Get a lower bound on the length of a document in this DB. More... | |
| Xapian::termcount | get_doclength_upper_bound () const |
| Get an upper bound on the length of a document in this DB. More... | |
| Xapian::termcount | get_wdf_upper_bound (std::string_view term) const |
| Get an upper bound on the wdf of term term. More... | |
| Xapian::termcount | get_unique_terms_lower_bound () const |
| Get a lower bound on the unique terms size of a document in this DB. More... | |
| Xapian::termcount | get_unique_terms_upper_bound () const |
| Get an upper bound on the unique terms size of a document in this DB. More... | |
| ValueList * | open_value_list (Xapian::valueno slot) const |
| Open a value stream. More... | |
| Xapian::termcount | get_doclength (Xapian::docid did) const |
| Xapian::termcount | get_unique_terms (Xapian::docid did) const |
| Get the number of unique terms in document. More... | |
| Xapian::termcount | get_wdfdocmax (Xapian::docid did) const |
| Get the max wdf in document. More... | |
| Xapian::Document::Internal * | open_document (Xapian::docid did, bool lazy) const |
| Open a handle on a document. More... | |
| bool | term_exists (std::string_view term) const |
| void | keep_alive () |
| TermList * | open_spelling_termlist (std::string_view word) const |
| Create a termlist tree from trigrams of word. More... | |
| TermList * | open_spelling_wordlist () const |
| Return a termlist which returns the words which are spelling correction targets. More... | |
| Xapian::doccount | get_spelling_frequency (std::string_view word) const |
| Return the number of times word was added as a spelling. More... | |
| TermList * | open_synonym_termlist (std::string_view term) const |
| Open a termlist returning synonyms for a term. More... | |
| TermList * | open_synonym_keylist (std::string_view prefix) const |
| Open a termlist returning each term which has synonyms. More... | |
| std::string | get_metadata (std::string_view key) const |
| Get the metadata associated with a given key. More... | |
| TermList * | open_metadata_keylist (std::string_view prefix) const |
| Open a termlist returning each metadata key. More... | |
| std::string | get_uuid () const |
| Get a UUID for the database. More... | |
| bool | locked () const |
| Return true if the database is open for writing. More... | |
| void | write_changesets_to_fd (int fd, std::string_view start_revision, bool need_whole_db, Xapian::ReplicationInfo *info) |
| Write a set of changesets to a file descriptor. More... | |
| void | invalidate_doc_object (Xapian::Document::Internal *obj) const |
| Notify the database that document is no longer valid. More... | |
| Xapian::rev | get_revision () const |
| Get revision number of database (if meaningful). More... | |
| int | get_backend_info (std::string *path) const |
| Get backend information about this database. More... | |
| void | commit () |
| Commit pending modifications to the database. More... | |
| void | cancel () |
| Cancel pending modifications to the database. More... | |
| void | begin_transaction (bool flushed) |
| Begin transaction. More... | |
| void | end_transaction (bool do_commit) |
| End transaction. More... | |
| Xapian::docid | add_document (const Xapian::Document &doc) |
| void | delete_document (Xapian::docid did) |
| void | delete_document (std::string_view term) |
| Delete any documents indexed by a term from the database. More... | |
| void | replace_document (Xapian::docid did, const Xapian::Document &doc) |
| Xapian::docid | replace_document (std::string_view term, const Xapian::Document &doc) |
| Replace any documents matching a term. More... | |
| void | request_document (Xapian::docid did) const |
| Request a document. More... | |
| void | add_spelling (std::string_view word, Xapian::termcount freqinc) const |
| Add a word to the spelling dictionary. More... | |
| Xapian::termcount | remove_spelling (std::string_view word, Xapian::termcount freqdec) const |
| Remove a word from the spelling dictionary. More... | |
| void | add_synonym (std::string_view term, std::string_view synonym) const |
| Add a synonym for a term. More... | |
| void | remove_synonym (std::string_view term, std::string_view synonym) const |
| Remove a synonym for a term. More... | |
| void | clear_synonyms (std::string_view term) const |
| Clear all synonyms for a term. More... | |
| void | set_metadata (std::string_view key, std::string_view value) |
| Set the metadata associated with a given key. More... | |
| std::string | reconstruct_text (Xapian::docid did, size_t length, std::string_view prefix, Xapian::termpos start_pos, Xapian::termpos end_pos) const |
| std::string | get_description () const |
| Return a string describing this object. More... | |
Public Member Functions inherited from Xapian::Database::Internal | |
| virtual | ~Internal () |
| We have virtual methods and want to be able to delete derived classes using a pointer to the base class, so we need a virtual destructor. More... | |
| virtual void | readahead_for_query (const Query &query) const |
| virtual void | get_used_docid_range (docid &first, docid &last) const |
| Find lowest and highest docids actually in use. More... | |
| virtual Internal * | update_lock (int flags) |
| Lock a read-only database for writing or unlock a writable database. More... | |
Public Member Functions inherited from Xapian::Internal::intrusive_base | |
| intrusive_base () | |
| Construct with no references. More... | |
Private Attributes | |
| Xapian::SmallVectorI< Xapian::Database::Internal > | shards |
Friends | |
| class | Matcher |
| class | PostListTree |
| class | ValueStreamDocument |
| class | Xapian::Database |
Additional Inherited Members | |
Public Types inherited from Xapian::Database::Internal | |
| typedef Xapian::doccount | size_type |
Public Attributes inherited from Xapian::Internal::intrusive_base | |
| unsigned | _refs |
| Reference count. More... | |
Protected Types inherited from Xapian::Database::Internal | |
| enum | transaction_state { TRANSACTION_READONLY = -2 , TRANSACTION_UNIMPLEMENTED = -1 , TRANSACTION_NONE = 0 , TRANSACTION_UNFLUSHED = 1 , TRANSACTION_FLUSHED = 2 } |
| Transaction state enum. More... | |
Protected Member Functions inherited from Xapian::Database::Internal | |
| Internal (transaction_state transaction_support) | |
| Only constructable as a base class for derived classes. More... | |
| bool | is_read_only () const |
| Test if this shard is read-only. More... | |
| bool | transaction_active () const |
| Test if a transaction is currently active. More... | |
| void | dtor_called () |
| Helper to process uncommitted changes when a writable db is destroyed. More... | |
Protected Attributes inherited from Xapian::Database::Internal | |
| transaction_state | state |
| Current transaction state. More... | |
Sharded database backend.
Definition at line 44 of file multi_database.h.
|
inlineexplicit |
Definition at line 53 of file multi_database.h.
|
virtual |
Reimplemented from Xapian::Database::Internal.
Definition at line 627 of file multi_database.cc.
References rare, shard_docid(), shard_number(), and UNSIGNED_OVERFLOW_OK.
|
virtual |
Add a word to the spelling dictionary.
If the word is already present, its frequency is increased.
| word | The word to add. |
| freqinc | How much to increase its frequency by. |
Reimplemented from Xapian::Database::Internal.
Definition at line 713 of file multi_database.cc.
|
virtual |
Add a synonym for a term.
If synonym is already a synonym for term, then no action is taken.
Reimplemented from Xapian::Database::Internal.
Definition at line 732 of file multi_database.cc.
References term.
|
virtual |
Begin transaction.
Reimplemented from Xapian::Database::Internal.
Definition at line 611 of file multi_database.cc.
|
virtual |
Cancel pending modifications to the database.
Reimplemented from Xapian::Database::Internal.
Definition at line 603 of file multi_database.cc.
|
virtual |
Clear all synonyms for a term.
If term has no synonyms, no action is taken.
Reimplemented from Xapian::Database::Internal.
Definition at line 748 of file multi_database.cc.
References term.
|
virtual |
Close the database.
Implements Xapian::Database::Internal.
Definition at line 57 of file multi_database.cc.
|
virtual |
Commit pending modifications to the database.
Reimplemented from Xapian::Database::Internal.
Definition at line 595 of file multi_database.cc.
|
virtual |
Delete any documents indexed by a term from the database.
Reimplemented from Xapian::Database::Internal.
Definition at line 653 of file multi_database.cc.
References term.
|
virtual |
Reimplemented from Xapian::Database::Internal.
Definition at line 645 of file multi_database.cc.
References shard_docid(), and shard_number().
|
virtual |
End transaction.
| do_commit | If true, commits the transaction; if false, cancels the transaction. |
Reimplemented from Xapian::Database::Internal.
Definition at line 619 of file multi_database.cc.
|
virtual |
Get backend information about this database.
| path | If non-NULL, and set the pointed to string to the file path of this database (or to some string describing the database in a backend-specified format if "path" isn't a concept which makes sense). |
Implements Xapian::Database::Internal.
Definition at line 587 of file multi_database.cc.
References Assert, and BACKEND_UNKNOWN.
|
virtual |
Return a string describing this object.
Implements Xapian::Database::Internal.
Definition at line 778 of file multi_database.cc.
|
virtual |
Implements Xapian::Database::Internal.
Definition at line 149 of file multi_database.cc.
|
virtual |
Implements Xapian::Database::Internal.
Definition at line 360 of file multi_database.cc.
References Assert, shard_docid(), and shard_number().
|
virtual |
Get a lower bound on the length of a document in this DB.
This bound does not include any zero-length documents.
Implements Xapian::Database::Internal.
Definition at line 269 of file multi_database.cc.
References negate_unsigned().
|
virtual |
Get an upper bound on the length of a document in this DB.
Implements Xapian::Database::Internal.
Definition at line 288 of file multi_database.cc.
|
virtual |
Returns frequencies for a term.
| term | The term to get frequencies for |
| termfreq_ptr | Point to return number of docs indexed by term (or NULL not to return) |
| collfreq_ptr | Point to return number of occurrences of term in the database (or NULL not to return) |
Implements Xapian::Database::Internal.
Definition at line 192 of file multi_database.cc.
|
virtual |
Return the last used document id of this (sub) database.
Implements Xapian::Database::Internal.
Definition at line 162 of file multi_database.cc.
References unshard().
|
virtual |
Get the metadata associated with a given key.
See Database::get_metadata() for more information.
Reimplemented from Xapian::Database::Internal.
Definition at line 523 of file multi_database.cc.
|
virtual |
Get revision number of database (if meaningful).
Reimplemented from Xapian::Database::Internal.
Definition at line 573 of file multi_database.cc.
|
virtual |
Return the number of times word was added as a spelling.
Reimplemented from Xapian::Database::Internal.
Definition at line 466 of file multi_database.cc.
|
virtual |
Return the total length of all documents in this database.
Implements Xapian::Database::Internal.
Definition at line 179 of file multi_database.cc.
|
virtual |
Get the number of unique terms in document.
| did | The document id of the document to return this value for. |
Implements Xapian::Database::Internal.
Definition at line 371 of file multi_database.cc.
References Assert, shard_docid(), and shard_number().
|
virtual |
Get a lower bound on the unique terms size of a document in this DB.
Reimplemented from Xapian::Database::Internal.
Definition at line 310 of file multi_database.cc.
References negate_unsigned().
|
virtual |
Get an upper bound on the unique terms size of a document in this DB.
Reimplemented from Xapian::Database::Internal.
Definition at line 330 of file multi_database.cc.
|
virtual |
Get a UUID for the database.
The UUID will persist for the lifetime of the database.
Replicas (eg, made with the replication protocol, or by copying all the database files) will have the same UUID. However, copies (made with copydatabase, or xapian-compact) will have different UUIDs.
If the backend does not support UUIDs the empty string is returned.
Reimplemented from Xapian::Database::Internal.
Definition at line 535 of file multi_database.cc.
|
virtual |
Return the frequency of a given value slot.
This is the number of documents which have a (non-empty) value stored in the slot.
| slot | The value slot to examine. |
Implements Xapian::Database::Internal.
Definition at line 230 of file multi_database.cc.
|
virtual |
Get a lower bound on the values stored in the given value slot.
If there are no values stored in the given value slot, this will return an empty string.
| slot | The value slot to examine. |
Implements Xapian::Database::Internal.
Definition at line 243 of file multi_database.cc.
|
virtual |
Get an upper bound on the values stored in the given value slot.
If there are no values stored in the given value slot, this will return an empty string.
| slot | The value slot to examine. |
Implements Xapian::Database::Internal.
Definition at line 257 of file multi_database.cc.
|
virtual |
Get an upper bound on the wdf of term term.
Implements Xapian::Database::Internal.
Definition at line 298 of file multi_database.cc.
|
virtual |
Get the max wdf in document.
| did | The document id of the document to return this value for. |
Implements Xapian::Database::Internal.
Definition at line 382 of file multi_database.cc.
References Assert, shard_docid(), and shard_number().
|
virtual |
Check whether this database contains any positional information.
Implements Xapian::Database::Internal.
Definition at line 129 of file multi_database.cc.
|
virtual |
Notify the database that document is no longer valid.
This is used to invalidate references to a document kept by a database for doing lazy updates. If we moved to using a weak_ptr instead we wouldn't need a special method for this, but it would involve a fair bit of reorganising of other parts of the code.
Reimplemented from Xapian::Database::Internal.
Definition at line 580 of file multi_database.cc.
References Assert.
|
virtual |
Reimplemented from Xapian::Database::Internal.
Definition at line 414 of file multi_database.cc.
|
virtual |
Return true if the database is open for writing.
If this is a WritableDatabase, always returns true.
For a Database, test if there's a writer holding the lock (or if we can't test for a lock without taking it on the current platform, throw Xapian::UnimplementedError).
Reimplemented from Xapian::Database::Internal.
Definition at line 552 of file multi_database.cc.
|
virtual |
Implements Xapian::Database::Internal.
Definition at line 110 of file multi_database.cc.
|
virtual |
Open a handle on a document.
The returned handle provides access to document data and document values.
| did | The document id to open. |
| lazy | If true, there's no need to check that this document actually exists (only a hint - the backend may still check). Used to avoid unnecessary work when we already know that the requested document exists. |
Implements Xapian::Database::Internal.
Definition at line 393 of file multi_database.cc.
References Assert, shard_docid(), and shard_number().
|
virtual |
Create a LeafPostList for use during a match.
| term | The term to open a postlist for, or the empty string to create an all-docs postlist. |
| need_read_pos | Does the postlist need to support read_position_list()? Note that open_position_list() may still be called even if need_read_pos is false. |
Implements Xapian::Database::Internal.
Definition at line 84 of file multi_database.cc.
References Assert.
|
virtual |
Open a termlist returning each metadata key.
Only metadata keys which are associated with a non-empty value will be returned.
| prefix | If non-empty, only keys with this prefix are returned. |
Reimplemented from Xapian::Database::Internal.
Definition at line 529 of file multi_database.cc.
|
virtual |
Implements Xapian::Database::Internal.
Definition at line 140 of file multi_database.cc.
References shard_docid(), shard_number(), and term.
|
virtual |
Return a PostList suitable for use in a PostingIterator.
Implements Xapian::Database::Internal.
Definition at line 65 of file multi_database.cc.
References term.
|
virtual |
Create a termlist tree from trigrams of word.
You can assume word.size() > 1.
If there are no trigrams, returns NULL.
Reimplemented from Xapian::Database::Internal.
Definition at line 422 of file multi_database.cc.
References make_termlist_merger().
|
virtual |
Return a termlist which returns the words which are spelling correction targets.
If there are no spelling correction targets, returns NULL.
Reimplemented from Xapian::Database::Internal.
Definition at line 444 of file multi_database.cc.
|
virtual |
Open a termlist returning each term which has synonyms.
| prefix | If non-empty, only terms with this prefix are returned. |
Reimplemented from Xapian::Database::Internal.
Definition at line 501 of file multi_database.cc.
References make_termlist_merger().
|
virtual |
Open a termlist returning synonyms for a term.
If term has no synonyms, returns NULL.
Reimplemented from Xapian::Database::Internal.
Definition at line 479 of file multi_database.cc.
References make_termlist_merger(), and term.
|
virtual |
Implements Xapian::Database::Internal.
Definition at line 92 of file multi_database.cc.
References open_term_list_direct().
|
virtual |
Like open_term_list() but without MultiTermList wrapper.
MultiDatabase::open_term_list() wraps the returns TermList in a MultiTermList, but we don't want that for query expansion.
Implements Xapian::Database::Internal.
Definition at line 98 of file multi_database.cc.
References shard_docid(), Xapian::TermIterator::Internal::shard_index, and shard_number().
Referenced by open_term_list().
|
virtual |
Open a value stream.
This returns the value in a particular slot for each document.
| slot | The value slot. |
Reimplemented from Xapian::Database::Internal.
Definition at line 340 of file multi_database.cc.
|
inline |
Definition at line 63 of file multi_database.h.
References Xapian::SmallVectorI< TI >::push_back(), and shards.
Referenced by Xapian::Database::add_database_(), and Xapian::Database::compact_().
|
virtual |
Reimplemented from Xapian::Database::Internal.
Definition at line 762 of file multi_database.cc.
References Assert, shard_docid(), and shard_number().
|
virtual |
Remove a word from the spelling dictionary.
The word's frequency is decreased, and if would become zero or less then the word is removed completely.
| word | The word to remove. |
| freqdec | How much to decrease its frequency by. |
Reimplemented from Xapian::Database::Internal.
Definition at line 720 of file multi_database.cc.
|
virtual |
Remove a synonym for a term.
If synonym isn't a synonym for term, then no action is taken.
Reimplemented from Xapian::Database::Internal.
Definition at line 739 of file multi_database.cc.
References term.
|
virtual |
Reopen the database to the latest available revision.
Database backends which don't support simultaneous update and reading probably don't need to do anything here.
Reimplemented from Xapian::Database::Internal.
Definition at line 45 of file multi_database.cc.
|
virtual |
Replace any documents matching a term.
Reimplemented from Xapian::Database::Internal.
Definition at line 669 of file multi_database.cc.
References rare, shard_docid(), shard_number(), term, and UNSIGNED_OVERFLOW_OK.
|
virtual |
Reimplemented from Xapian::Database::Internal.
Definition at line 661 of file multi_database.cc.
References shard_docid(), and shard_number().
|
virtual |
Request a document.
This tells the database that we're going to want a particular document soon. It's just a hint which the backend may ignore, but for glass it issues a preread hint on the file with the document data in, and for the remote backend it might cause the document to be fetched asynchronously (this isn't currently implemented though).
It can be called for multiple documents in turn, and a common usage pattern would be to iterate over an MSet and request the documents, then iterate over it again to actually get and display them.
The default implementation is a no-op.
Reimplemented from Xapian::Database::Internal.
Definition at line 702 of file multi_database.cc.
References Assert, shard_docid(), and shard_number().
|
inline |
Definition at line 61 of file multi_database.h.
References Xapian::SmallVector_::reserve(), and shards.
Referenced by Xapian::Database::add_database_().
|
virtual |
Set the metadata associated with a given key.
See WritableDatabase::set_metadata() for more information.
Reimplemented from Xapian::Database::Internal.
Definition at line 756 of file multi_database.cc.
|
virtual |
Reimplemented from Xapian::Database::Internal.
Definition at line 39 of file multi_database.cc.
|
virtual |
Implements Xapian::Database::Internal.
Definition at line 404 of file multi_database.cc.
References term.
|
virtual |
Write a set of changesets to a file descriptor.
This call may reopen the database, leaving it pointing to a more recent version of the database.
Reimplemented from Xapian::Database::Internal.
Definition at line 563 of file multi_database.cc.
|
friend |
Definition at line 45 of file multi_database.h.
|
friend |
Definition at line 46 of file multi_database.h.
|
friend |
Definition at line 47 of file multi_database.h.
|
friend |
Definition at line 48 of file multi_database.h.
|
private |
Definition at line 50 of file multi_database.h.
Referenced by ValueStreamDocument::new_shard(), push_back(), and reserve().