xapian-core  2.0.0
Classes | Public Types | Public Member Functions | Private Types | Private Member Functions | Private Attributes | Friends | List of all members
Xapian::MSet Class Reference

Class representing a list of search results. More...

#include <mset.h>

+ Collaboration diagram for Xapian::MSet:

Classes

class  Internal
 Xapian::MSet internals. More...
 

Public Types

enum  {
  SNIPPET_BACKGROUND_MODEL = 1 , SNIPPET_EXHAUSTIVE = 2 , SNIPPET_EMPTY_WITHOUT_MATCH = 4 , SNIPPET_NGRAMS = 2048 ,
  SNIPPET_CJK_NGRAM = SNIPPET_NGRAMS , SNIPPET_WORD_BREAKS = 4096
}
 

Public Member Functions

 MSet (const MSet &o)
 Copying is allowed. More...
 
MSetoperator= (const MSet &o)
 Copying is allowed. More...
 
 MSet (MSet &&o)
 Move constructor. More...
 
MSetoperator= (MSet &&o)
 Move assignment operator. More...
 
 MSet ()
 Default constructor. More...
 
 ~MSet ()
 Destructor. More...
 
template<typename Iterator >
void replace_weights (Iterator first, Iterator last)
 Assigns new weights and updates MSet. More...
 
void sort_by_relevance ()
 Sorts the list of documents in MSet according to their weights. More...
 
int convert_to_percent (double weight) const
 Convert a weight to a percentage. More...
 
int convert_to_percent (const MSetIterator &it) const
 Convert the weight of the current iterator position to a percentage. More...
 
Xapian::doccount get_termfreq (std::string_view term) const
 Get the termfreq of a term. More...
 
double get_termweight (std::string_view term) const
 Get the term weight of a term. More...
 
Xapian::doccount get_firstitem () const
 Rank of first item in this MSet. More...
 
Xapian::doccount get_matches_lower_bound () const
 Lower bound on the total number of matching documents. More...
 
Xapian::doccount get_matches_estimated () const
 Estimate of the total number of matching documents. More...
 
Xapian::doccount get_matches_upper_bound () const
 Upper bound on the total number of matching documents. More...
 
Xapian::doccount get_uncollapsed_matches_lower_bound () const
 Lower bound on the total number of matching documents before collapsing. More...
 
Xapian::doccount get_uncollapsed_matches_estimated () const
 Estimate of the total number of matching documents before collapsing. More...
 
Xapian::doccount get_uncollapsed_matches_upper_bound () const
 Upper bound on the total number of matching documents before collapsing. More...
 
double get_max_attained () const
 The maximum weight attained by any document. More...
 
double get_max_possible () const
 The maximum possible weight any document could achieve. More...
 
std::string snippet (std::string_view text, size_t length=500, const Xapian::Stem &stemmer=Xapian::Stem(), unsigned flags=SNIPPET_BACKGROUND_MODEL|SNIPPET_EXHAUSTIVE, std::string_view hi_start="<b>", std::string_view hi_end="</b>", std::string_view omit="...") const
 Generate a snippet. More...
 
void fetch (const MSetIterator &begin, const MSetIterator &end) const
 Prefetch hint a range of items. More...
 
void fetch (const MSetIterator &item) const
 Prefetch hint a single MSet item. More...
 
void fetch () const
 Prefetch hint the whole MSet. More...
 
Xapian::doccount size () const
 Return number of items in this MSet object. More...
 
bool empty () const
 Return true if this MSet object is empty. More...
 
void swap (MSet &o)
 Efficiently swap this MSet object with another. More...
 
MSetIterator begin () const
 Return iterator pointing to the first item in this MSet. More...
 
MSetIterator end () const
 Return iterator pointing to just after the last item in this MSet. More...
 
MSetIterator operator[] (Xapian::doccount i) const
 Return iterator pointing to the i-th object in this MSet. More...
 
MSetIterator back () const
 Return iterator pointing to the last object in this MSet. More...
 
std::string get_description () const
 Return a string describing this object. More...
 

Private Types

typedef Xapian::MSetIterator value_type
 
typedef Xapian::doccount size_type
 
typedef Xapian::doccount_diff difference_type
 
typedef Xapian::MSetIterator iterator
 
typedef Xapian::MSetIterator const_iterator
 
typedef value_typepointer
 
typedef const value_typeconst_pointer
 
typedef value_type reference
 
typedef const value_type const_reference
 

Private Member Functions

void fetch_ (Xapian::doccount first, Xapian::doccount last) const
 
void set_item_weight (Xapian::doccount i, double wt)
 Update the weight corresponding to the document indexed at position i with wt. More...
 
 MSet (Internal *internal_)
 
Xapian::doccount max_size () const
 

Private Attributes

Xapian::Internal::intrusive_ptr_nonnull< Internalinternal
 

Friends

class MSetIterator
 

Detailed Description

Class representing a list of search results.

Definition at line 46 of file mset.h.

Member Typedef Documentation

◆ const_iterator

Definition at line 508 of file mset.h.

◆ const_pointer

typedef const value_type* Xapian::MSet::const_pointer
private

Definition at line 512 of file mset.h.

◆ const_reference

Definition at line 516 of file mset.h.

◆ difference_type

Definition at line 504 of file mset.h.

◆ iterator

Definition at line 506 of file mset.h.

◆ pointer

Definition at line 510 of file mset.h.

◆ reference

Definition at line 514 of file mset.h.

◆ size_type

Definition at line 502 of file mset.h.

◆ value_type

MSet is what the C++ STL calls a container.

The following typedefs allow the class to be used in templates in the same way the standard containers can be.

These are deliberately hidden from the Doxygen-generated docs, as the machinery here isn't interesting to API users. They just need to know that Xapian container classes are compatible with the STL.

See "The C++ Programming Language", 3rd ed. section 16.3.1:

Definition at line 500 of file mset.h.

Member Enumeration Documentation

◆ anonymous enum

anonymous enum
Enumerator
SNIPPET_BACKGROUND_MODEL 

Model the relevancy of non-query terms in MSet::snippet().

    Non-query terms will be assigned a small weight, and the snippet
    will tend to prefer snippets which contain a more interesting
    background (where the query term content is equivalent).
SNIPPET_EXHAUSTIVE 

Exhaustively evaluate candidate snippets in MSet::snippet().

    Without this flag, snippet generation will stop once it thinks
    it has found a "good enough" snippet, which will generally reduce
    the time taken to generate a snippet.
SNIPPET_EMPTY_WITHOUT_MATCH 

Return the empty string if no term got matched.

    If enabled, snippet() returns an empty string if not a single match
    was found in text. If not enabled, snippet() returns a (sub)string
    of text without any highlighted terms.
SNIPPET_NGRAMS 

Generate n-grams for scripts without explicit word breaks.

    Text in other scripts is split into words as normal.

    Enable this option to highlight search results for queries parsed
    with the QueryParser::FLAG_NGRAMS flag.

    The TermGenerator::FLAG_NGRAMS flag needs to have been used at
    index time.

    This mode can also be enabled by setting environment variable
    XAPIAN_CJK_NGRAM to a non-empty value (but doing so was deprecated
    in 1.4.11).

    In 1.4.x this feature was specific to CJK (Chinese, Japanese and
    Korean), but in 2.0.0 it's been extended to other languages.  To
    reflect this change the new and preferred name is SNIPPET_NGRAMS,
    which was added as an alias for forward compatibility in Xapian
    1.4.23.  Use SNIPPET_CJK_NGRAM instead if you aim to support Xapian
    &lt; 1.4.23.

    @since Added in Xapian 1.4.23.
SNIPPET_CJK_NGRAM 

Generate n-grams for scripts without explicit word breaks.

    Old name - use SNIPPET_NGRAMS instead unless you aim to support
    Xapian &lt; 1.4.23.

    @since Added in Xapian 1.4.11.
SNIPPET_WORD_BREAKS 

Find word breaks for text in scripts without explicit word breaks.

    Enable this option to highlight search results for queries parsed
    with the QueryParser::FLAG_WORD_BREAKS flag.  Spans of text
    written in such scripts are split into words using ICU (which uses
    heuristics and/or dictionaries to do so).  Text in other scripts is
    split into words as normal.

    The TermGenerator::FLAG_WORD_BREAKS flag needs to have been used at
    index time.

    @since Added in Xapian 2.0.0.

Definition at line 317 of file mset.h.

Constructor & Destructor Documentation

◆ MSet() [1/4]

Xapian::MSet::MSet ( const MSet o)
default

Copying is allowed.

The internals are reference counted, so copying is cheap.

◆ MSet() [2/4]

Xapian::MSet::MSet ( MSet &&  o)
default

Move constructor.

◆ MSet() [3/4]

Xapian::MSet::MSet ( )

Default constructor.

Creates an empty MSet, mostly useful as a placeholder.

Definition at line 59 of file mset.cc.

◆ MSet() [4/4]

Xapian::MSet::MSet ( Internal internal_)
explicitprivate

Wrap an existing Internal.

Definition at line 61 of file mset.cc.

◆ ~MSet()

Xapian::MSet::~MSet ( )

Destructor.

Definition at line 63 of file mset.cc.

Member Function Documentation

◆ back()

MSetIterator Xapian::MSet::back ( ) const
inline

Return iterator pointing to the last object in this MSet.

Definition at line 803 of file mset.h.

References MSetIterator.

Referenced by DEFINE_TESTCASE().

◆ begin()

MSetIterator Xapian::MSet::begin ( ) const
inline

Return iterator pointing to the first item in this MSet.

Definition at line 786 of file mset.h.

References MSetIterator, and size().

Referenced by DEFINE_TESTCASE(), main(), print_mset_percentages(), print_mset_weights(), and test_mset_order_equal().

◆ convert_to_percent() [1/2]

int Xapian::MSet::convert_to_percent ( const MSetIterator it) const
inline

Convert the weight of the current iterator position to a percentage.

If the weighting scheme gives everything zero weight (like Xapian::BoolWeight does) then all results will score 100%.

Otherwise the percentage is calculated as a linear scaling of the relevance weight, with the scale factor determined by the matching document with the highest weight. This result scores 100% if it matches all the weighted query terms, and proportionally less if it only matches some.

The returned percentage is an integer. If the calculated percentage before rounding is non-zero but less than 1% it is rounded up to 1% so that a result scoring 0% means it has zero weight.

Similarly, percentages over 99% but less than 100% are always rounded down, so a result scoring 100% means it matches all weighted query terms.

Note that these generally aren't percentages of anything meaningful (unless you use a custom weighting formula where they are!) but like the weights they are based on, higher values should indicate more relevant results.

Definition at line 808 of file mset.h.

References convert_to_percent(), and Xapian::MSetIterator::get_weight().

◆ convert_to_percent() [2/2]

int Xapian::MSet::convert_to_percent ( double  weight) const

Convert a weight to a percentage.

If the weighting scheme gives everything zero weight (like Xapian::BoolWeight does) then all results will score 100%.

Otherwise the percentage is calculated as a linear scaling of the relevance weight, with the scale factor determined by the matching document with the highest weight. This result scores 100% if it matches all the weighted query terms, and proportionally less if it only matches some.

The returned percentage is an integer. If the calculated percentage before rounding is non-zero but less than 1% it is rounded up to 1% so that a result scoring 0% means it has zero weight.

Similarly, percentages over 99% but less than 100% are always rounded down, so a result scoring 100% means it matches all weighted query terms.

Note that these generally aren't percentages of anything meaningful (unless you use a custom weighting formula where they are!) but like the weights they are based on, higher values should indicate more relevant results.

Definition at line 275 of file mset.cc.

Referenced by convert_to_percent(), DEFINE_TESTCASE(), Xapian::MSetIterator::get_percent(), and print_mset_percentages().

◆ empty()

bool Xapian::MSet::empty ( ) const
inline

Return true if this MSet object is empty.

Definition at line 467 of file mset.h.

Referenced by DEFINE_TESTCASE(), Matcher::get_mset(), and operator==().

◆ end()

MSetIterator Xapian::MSet::end ( ) const
inline

Return iterator pointing to just after the last item in this MSet.

Definition at line 791 of file mset.h.

References MSetIterator.

Referenced by DEFINE_TESTCASE(), main(), print_mset_percentages(), print_mset_weights(), and test_mset_order_equal().

◆ fetch() [1/3]

void Xapian::MSet::fetch ( ) const
inline

Prefetch hint the whole MSet.

For a remote database, this may start a pipelined fetch of the requested documents from the remote server.

For a disk-based database, this may send prefetch hints to the operating system such that the disk blocks the requested documents are stored in are more likely to be in the cache when we come to actually read them.

Definition at line 461 of file mset.h.

◆ fetch() [2/3]

void Xapian::MSet::fetch ( const MSetIterator begin,
const MSetIterator end 
) const
inline

Prefetch hint a range of items.

For a remote database, this may start a pipelined fetch of the requested documents from the remote server.

For a disk-based database, this may send prefetch hints to the operating system such that the disk blocks the requested documents are stored in are more likely to be in the cache when we come to actually read them.

Definition at line 774 of file mset.h.

References fetch_(), and Xapian::MSetIterator::off_from_end.

Referenced by DEFINE_TESTCASE().

◆ fetch() [3/3]

void Xapian::MSet::fetch ( const MSetIterator item) const
inline

Prefetch hint a single MSet item.

For a remote database, this may start a pipelined fetch of the requested documents from the remote server.

For a disk-based database, this may send prefetch hints to the operating system such that the disk blocks the requested documents are stored in are more likely to be in the cache when we come to actually read them.

Definition at line 780 of file mset.h.

References fetch_(), and Xapian::MSetIterator::off_from_end.

◆ fetch_()

void Xapian::MSet::fetch_ ( Xapian::doccount  first,
Xapian::doccount  last 
) const
private

Definition at line 66 of file mset.cc.

Referenced by fetch().

◆ get_description()

std::string Xapian::MSet::get_description ( ) const

Return a string describing this object.

Definition at line 394 of file mset.cc.

Referenced by DEFINE_TESTCASE().

◆ get_firstitem()

Xapian::doccount Xapian::MSet::get_firstitem ( ) const

Rank of first item in this MSet.

This is the parameter first passed to Xapian::Enquire::get_mset().

Definition at line 312 of file mset.cc.

Referenced by DEFINE_TESTCASE(), and Xapian::MSetIterator::get_rank().

◆ get_matches_estimated()

Xapian::doccount Xapian::MSet::get_matches_estimated ( ) const

◆ get_matches_lower_bound()

Xapian::doccount Xapian::MSet::get_matches_lower_bound ( ) const

Lower bound on the total number of matching documents.

Definition at line 318 of file mset.cc.

Referenced by DEFINE_TESTCASE(), main(), operator==(), and PerfTestLogger::search_end().

◆ get_matches_upper_bound()

Xapian::doccount Xapian::MSet::get_matches_upper_bound ( ) const

Upper bound on the total number of matching documents.

Definition at line 334 of file mset.cc.

Referenced by DEFINE_TESTCASE(), main(), operator==(), and PerfTestLogger::search_end().

◆ get_max_attained()

double Xapian::MSet::get_max_attained ( ) const

The maximum weight attained by any document.

Definition at line 362 of file mset.cc.

Referenced by DEFINE_TESTCASE().

◆ get_max_possible()

double Xapian::MSet::get_max_possible ( ) const

The maximum possible weight any document could achieve.

Definition at line 368 of file mset.cc.

Referenced by DEFINE_TESTCASE(), and operator==().

◆ get_termfreq()

Xapian::doccount Xapian::MSet::get_termfreq ( std::string_view  term) const

Get the termfreq of a term.

Returns
The number of documents which term occurs in. This considers all documents in the database being searched, so gives the same answer as db.get_termfreq(term) (but is more efficient for query terms as it returns a value cached during the search.)

Since 2.0.0, this method returns 0 if called on an MSet which is not associated with a database (which is consistent with Database::get_termfreq() returning 0 when called on a Database with no sub-databases); in earlier versions, Xapian::InvalidOperationError was thrown in this case.

Definition at line 281 of file mset.cc.

References Xapian::MSet::Internal::enquire, internal, rare, Xapian::MSet::Internal::stats, term, and usual.

Referenced by DEFINE_TESTCASE(), and main().

◆ get_termweight()

double Xapian::MSet::get_termweight ( std::string_view  term) const

Get the term weight of a term.

Returns
The maximum weight that term could have contributed to a document.

Since 2.0.0, this method returns 0.0 if called on an MSet which is not associated with a database, or with a term which wasn't present in the query (since in both cases the term contributes no weight to any matching documents); in earlier versions, Xapian::InvalidOperationError was thrown for the first case, and Xapian::InvalidArgumentError for the second.

Definition at line 300 of file mset.cc.

References internal, Xapian::MSet::Internal::stats, term, and usual.

Referenced by DEFINE_TESTCASE().

◆ get_uncollapsed_matches_estimated()

Xapian::doccount Xapian::MSet::get_uncollapsed_matches_estimated ( ) const

Estimate of the total number of matching documents before collapsing.

Conceptually the same as get_matches_estimated() for the same query without any collapse part (though the actual value may differ).

Definition at line 346 of file mset.cc.

References internal, round_estimate(), Xapian::MSet::Internal::uncollapsed_estimated, Xapian::MSet::Internal::uncollapsed_lower_bound, and Xapian::MSet::Internal::uncollapsed_upper_bound.

Referenced by DEFINE_TESTCASE().

◆ get_uncollapsed_matches_lower_bound()

Xapian::doccount Xapian::MSet::get_uncollapsed_matches_lower_bound ( ) const

Lower bound on the total number of matching documents before collapsing.

Conceptually the same as get_matches_lower_bound() for the same query without any collapse part (though the actual value may differ).

Definition at line 340 of file mset.cc.

Referenced by DEFINE_TESTCASE().

◆ get_uncollapsed_matches_upper_bound()

Xapian::doccount Xapian::MSet::get_uncollapsed_matches_upper_bound ( ) const

Upper bound on the total number of matching documents before collapsing.

Conceptually the same as get_matches_upper_bound() for the same query without any collapse part (though the actual value may differ).

Definition at line 356 of file mset.cc.

Referenced by DEFINE_TESTCASE().

◆ max_size()

Xapian::doccount Xapian::MSet::max_size ( ) const
inlineprivate

MSet is what the C++ STL calls a container.

The following methods allow the class to be used in templates in the same way the standard containers can be.

These are deliberately hidden from the Doxygen-generated docs, as the machinery here isn't interesting to API users. They just need to know that Xapian container classes are compatible with the STL.

Definition at line 530 of file mset.h.

◆ operator=() [1/2]

MSet & Xapian::MSet::operator= ( const MSet o)
default

Copying is allowed.

The internals are reference counted, so assignment is cheap.

◆ operator=() [2/2]

MSet & Xapian::MSet::operator= ( MSet &&  o)
default

Move assignment operator.

◆ operator[]()

MSetIterator Xapian::MSet::operator[] ( Xapian::doccount  i) const
inline

Return iterator pointing to the i-th object in this MSet.

Definition at line 798 of file mset.h.

References MSetIterator, and size().

◆ replace_weights()

template<typename Iterator >
void Xapian::MSet::replace_weights ( Iterator  first,
Iterator  last 
)
inline

Assigns new weights and updates MSet.

Dereferencing the Iterator should return a double.

The weights returned by the iterator are assigned to elements of the MSet in rank order.

Parameters
beginBegin iterator.
endEnd iterator.
Exceptions
Xapian::InvalidArgumentis thrown if the total number of elements in the input doesn't match the total number of documents in MSet.
Since
Added in Xapian 2.0.0.

Definition at line 130 of file mset.h.

Referenced by DEFINE_TESTCASE().

◆ set_item_weight()

void Xapian::MSet::set_item_weight ( Xapian::doccount  i,
double  wt 
)
private

Update the weight corresponding to the document indexed at position i with wt.

The MSet's max_possible and max_attained are also updated.

This method must be called to update the weight of every document in the MSet for i = 0 to mset.size() - 1 in ascending order to avoid miscalculation of max_attained and max_possible.

Parameters
iMSet index to update
wtnew weight to assign to the document at index i
Since
Added in Xapian 2.0.0.

Definition at line 72 of file mset.cc.

◆ size()

Xapian::doccount Xapian::MSet::size ( ) const

◆ snippet()

std::string Xapian::MSet::snippet ( std::string_view  text,
size_t  length = 500,
const Xapian::Stem stemmer = Xapian::Stem(),
unsigned  flags = SNIPPET_BACKGROUND_MODEL|SNIPPET_EXHAUSTIVE,
std::string_view  hi_start = "<b>",
std::string_view  hi_end = "</b>",
std::string_view  omit = "..." 
) const

Generate a snippet.

This method selects a continuous run of words from text, based mainly on where the query matches (currently terms, exact phrases and wildcards are taken into account). If flag SNIPPET_BACKGROUND_MODEL is used (which it is by default) then the selection algorithm also considers the non-query terms in the text with the aim of showing a context which provides more useful information.

The size of the text selected can be controlled by the length parameter, which specifies a number of bytes of text to aim to select. However slightly more text may be selected. Also the size of any escaping, highlighting or omission markers is not considered.

The returned text is escaped to make it suitable for use in HTML (though beware that in upstream releases 1.4.5 and earlier this escaping was sometimes incomplete), and matches with the query will be highlighted using hi_start and hi_end.

If the snippet seems to start or end mid-sentence, then omit is prepended or append (respectively) to indicate this.

The same stemming algorithm which was used to build the query should be specified in stemmer.

And flags contains flags controlling behaviour.

Since
Added in 1.3.5.

Definition at line 380 of file mset.cc.

References stemmer.

Referenced by DEFINE_TESTCASE().

◆ sort_by_relevance()

void Xapian::MSet::sort_by_relevance ( )

Sorts the list of documents in MSet according to their weights.

Use after calling MSet::replace_weights.

This invalidates any MSetIterator objects active on this MSet.

Since
Added in Xapian 2.0.0.

Definition at line 268 of file mset.cc.

References get_msetcmp_function(), internal, Xapian::MSet::Internal::items, Xapian::Enquire::Internal::REL, and Heap::sort().

Referenced by DEFINE_TESTCASE().

◆ swap()

void Xapian::MSet::swap ( MSet o)
inline

Efficiently swap this MSet object with another.

Definition at line 470 of file mset.h.

References internal.

Friends And Related Function Documentation

◆ MSetIterator

friend class MSetIterator
friend

Definition at line 47 of file mset.h.

Referenced by back(), begin(), end(), and operator[]().

Member Data Documentation

◆ internal

Xapian::Internal::intrusive_ptr_nonnull<Internal> Xapian::MSet::internal
private

The documentation for this class was generated from the following files: