This class implements the BB2 weighting scheme. More...

#include <weight.h>

Inheritance diagram for Xapian::BB2Weight:

Collaboration diagram for Xapian::BB2Weight:

Public Member Functions
	BB2Weight (double c)
	Construct a BB2Weight. More...

	BB2Weight ()

std::string	name () const
	Return the name of this weighting scheme. More...

std::string	serialise () const
	Return this object's parameters serialised as a single string. More...

BB2Weight *	unserialise (const std::string &serialised) const
	Unserialise parameters. More...

double	get_sumpart (Xapian::termcount wdf, Xapian::termcount doclen, Xapian::termcount uniqterms) const
	Calculate the weight contribution for this object's term to a document. More...

double	get_maxpart () const
	Return an upper bound on what get_sumpart() can return for any document. More...

double	get_sumextra (Xapian::termcount doclen, Xapian::termcount uniqterms) const
	Calculate the term-independent weight component for a document. More...

double	get_maxextra () const
	Return an upper bound on what get_sumextra() can return for any document. More...

Public Member Functions inherited from Xapian::Weight
	Weight ()
	Default constructor, needed by subclass constructors. More...

virtual	~Weight ()
	Virtual destructor, because we have virtual methods. More...

Private Member Functions
BB2Weight *	clone () const
	Clone this object. More...

void	init (double factor)
	Allow the subclass to perform any initialisation it needs to. More...

Private Attributes
double	param_c
	The wdf normalization parameter in the formula. More...

double	upper_bound
	The upper bound on the weight. More...

double	c_product_avlen
	The constant values to be used in get_sumpart(). More...

double	B_constant

double	wt

double	stirling_constant_1

double	stirling_constant_2

Additional Inherited Members
Public Types inherited from Xapian::Weight
enum	type_smoothing { TWO_STAGE_SMOOTHING = 1, DIRICHLET_SMOOTHING = 2, ABSOLUTE_DISCOUNT_SMOOTHING = 3, JELINEK_MERCER_SMOOTHING = 4, DIRICHLET_PLUS_SMOOTHING = 5 }
	Type of smoothing to use with the Language Model Weighting scheme. More...

Protected Types inherited from Xapian::Weight
enum	stat_flags { COLLECTION_SIZE = 1, RSET_SIZE = 2, AVERAGE_LENGTH = 4, TERMFREQ = 8, RELTERMFREQ = 16, QUERY_LENGTH = 32, WQF = 64, WDF = 128, DOC_LENGTH = 256, DOC_LENGTH_MIN = 512, DOC_LENGTH_MAX = 1024, WDF_MAX = 2048, COLLECTION_FREQ = 4096, UNIQUE_TERMS = 8192, TOTAL_LENGTH = COLLECTION_SIZE \| AVERAGE_LENGTH }
	Stats which the weighting scheme can use (see need_stat()). More...

Protected Member Functions inherited from Xapian::Weight
void	need_stat (stat_flags flag)
	Tell Xapian that your subclass will want a particular statistic. More...

	Weight (const Weight &)
	Don't allow copying. More...

Xapian::doccount	get_collection_size () const
	The number of documents in the collection. More...

Xapian::doccount	get_rset_size () const
	The number of documents marked as relevant. More...

Xapian::doclength	get_average_length () const
	The average length of a document in the collection. More...

Xapian::doccount	get_termfreq () const
	The number of documents which this term indexes. More...

Xapian::doccount	get_reltermfreq () const
	The number of relevant documents which this term indexes. More...

Xapian::termcount	get_collection_freq () const
	The collection frequency of the term. More...

Xapian::termcount	get_query_length () const
	The length of the query. More...

Xapian::termcount	get_wqf () const
	The within-query-frequency of this term. More...

Xapian::termcount	get_doclength_upper_bound () const
	An upper bound on the maximum length of any document in the shard. More...

Xapian::termcount	get_doclength_lower_bound () const
	A lower bound on the minimum length of any document in the shard. More...

Xapian::termcount	get_wdf_upper_bound () const
	An upper bound on the wdf of this term in the shard. More...

Xapian::totallength	get_total_length () const
	Total length of all documents in the collection. More...

Detailed Description

This class implements the BB2 weighting scheme.

BB2 is a representative scheme of the Divergence from Randomness Framework by Gianni Amati.

It uses the Bose-Einstein probabilistic distribution (B) along with Stirling's power approximation, the Bernoulli method to find the aftereffect of sampling (B) and the second wdf normalization proposed by Amati to normalize the wdf in the document to the length of the document (H2).

For more information about the DFR Framework and the BB2 scheme, please refer to : Gianni Amati and Cornelis Joost Van Rijsbergen Probabilistic models of information retrieval based on measuring the divergence from randomness ACM Transactions on Information Systems (TOIS) 20, (4), 2002, pp. 357-389.

Definition at line 1060 of file weight.h.

Constructor & Destructor Documentation

◆ BB2Weight() [1/2]

Xapian::BB2Weight::BB2Weight ( double c )

explicit

Construct a BB2Weight.

Parameters

c A strictly positive parameter controlling the extent of the normalization of the wdf to the document length. A default value of 1 is suitable for longer queries but it may need to be changed for shorter queries. For more information, please refer to Gianni Amati's PHD thesis titled Probabilistic Models for Information Retrieval based on Divergence from Randomness.

Definition at line 40 of file bb2weight.cc.

References Xapian::Weight::AVERAGE_LENGTH, Xapian::Weight::COLLECTION_FREQ, Xapian::Weight::COLLECTION_SIZE, Xapian::Weight::DOC_LENGTH, Xapian::Weight::DOC_LENGTH_MAX, Xapian::Weight::DOC_LENGTH_MIN, Xapian::Weight::need_stat(), param_c, Xapian::Weight::TERMFREQ, Xapian::Weight::WDF, Xapian::Weight::WDF_MAX, and Xapian::Weight::WQF.

◆ BB2Weight() [2/2]

Xapian::BB2Weight::BB2Weight ( )

inline

Definition at line 1091 of file weight.h.

References name.

Referenced by clone(), and unserialise().

Member Function Documentation

◆ clone()

BB2Weight * Xapian::BB2Weight::clone ( ) const

privatevirtual

Clone this object.

This method allocates and returns a copy of the object it is called on.

If your subclass is called FooWeight and has parameters a and b, then you would implement FooWeight::clone() like so:

FooWeight * FooWeight::clone() const { return new FooWeight(a, b); }

Note that the returned object will be deallocated by Xapian after use with "delete". If you want to handle the deletion in a special way (for example when wrapping the Xapian API for use from another language) then you can define a static operator delete method in your subclass as shown here: https://trac.xapian.org/ticket/554#comment:1

Implements Xapian::Weight.

Definition at line 57 of file bb2weight.cc.

References BB2Weight(), and param_c.

◆ get_maxextra()

double Xapian::BB2Weight::get_maxextra ( ) const

virtual

Return an upper bound on what get_sumextra() can return for any document.

This information is used by the matcher to perform various optimisations, so strive to make the bound as tight as possible.

Implements Xapian::Weight.

Definition at line 187 of file bb2weight.cc.

◆ get_maxpart()

double Xapian::BB2Weight::get_maxpart ( ) const

virtual

Return an upper bound on what get_sumpart() can return for any document.

This information is used by the matcher to perform various optimisations, so strive to make the bound as tight as possible.

Implements Xapian::Weight.

Definition at line 175 of file bb2weight.cc.

References upper_bound.

◆ get_sumextra()

double Xapian::BB2Weight::get_sumextra	(	Xapian::termcount	doclen,
		Xapian::termcount	uniqterms
	)		const

virtual

Calculate the term-independent weight component for a document.

The parameter gives information about the document which may be used in the calculations:

Parameters

doclen	The document's length (unnormalised).
uniqterms	The number of unique terms in the document.

Implements Xapian::Weight.

Definition at line 181 of file bb2weight.cc.

◆ get_sumpart()

double Xapian::BB2Weight::get_sumpart	(	Xapian::termcount	wdf,
		Xapian::termcount	doclen,
		Xapian::termcount	uniqterms
	)		const

virtual

Calculate the weight contribution for this object's term to a document.

The parameters give information about the document which may be used in the calculations:

Parameters

wdf	The within document frequency of the term in the document.
doclen	The document's length (unnormalised).
uniqterms	Number of unique terms in the document (used for absolute smoothing).

Implements Xapian::Weight.

Definition at line 143 of file bb2weight.cc.

References B_constant, c_product_avlen, Xapian::Weight::get_collection_freq(), Xapian::Weight::get_collection_size(), log2(), rare, stirling_constant_1, stirling_constant_2, Xapian::stirling_value(), and wt.

◆ init()

void Xapian::BB2Weight::init ( double factor )

privatevirtual

Allow the subclass to perform any initialisation it needs to.

Parameters

factor Any scaling factor (e.g. from OP_SCALE_WEIGHT). If the Weight object is for the term-independent weight supplied by get_sumextra()/get_maxextra(), then init(0.0) is called (starting from Xapian 1.2.11 and 1.3.1 - earlier versions failed to call init() for such Weight objects).

Implements Xapian::Weight.

Definition at line 63 of file bb2weight.cc.

References B_constant, c_product_avlen, Xapian::Weight::get_average_length(), Xapian::Weight::get_collection_freq(), Xapian::Weight::get_collection_size(), Xapian::Weight::get_doclength_lower_bound(), Xapian::Weight::get_doclength_upper_bound(), Xapian::Weight::get_termfreq(), Xapian::Weight::get_wdf_upper_bound(), Xapian::Weight::get_wqf(), log2(), param_c, rare, stirling_constant_1, stirling_constant_2, Xapian::stirling_value(), upper_bound, and wt.

◆ name()

string Xapian::BB2Weight::name ( ) const

virtual

Return the name of this weighting scheme.

This name is used by the remote backend. It is passed along with the serialised parameters to the remote server so that it knows which class to create.

Return the full namespace-qualified name of your class here - if your class is called FooWeight, return "FooWeight" from this method (Xapian::BM25Weight returns "Xapian::BM25Weight" here).

If you don't want to support the remote backend, you can use the default implementation which simply returns an empty string.

Reimplemented from Xapian::Weight.

Definition at line 120 of file bb2weight.cc.

◆ serialise()

string Xapian::BB2Weight::serialise ( ) const

virtual

Return this object's parameters serialised as a single string.

If you don't want to support the remote backend, you can use the default implementation which simply throws Xapian::UnimplementedError.

Reimplemented from Xapian::Weight.

Definition at line 126 of file bb2weight.cc.

References param_c, and serialise_double().

◆ unserialise()

BB2Weight * Xapian::BB2Weight::unserialise ( const std::string & serialised ) const

virtual

Unserialise parameters.

This method unserialises parameters serialised by the serialise() method and allocates and returns a new object initialised with them.

If you don't want to support the remote backend, you can use the default implementation which simply throws Xapian::UnimplementedError.

Note that the returned object will be deallocated by Xapian after use with "delete". If you want to handle the deletion in a special way (for example when wrapping the Xapian API for use from another language) then you can define a static operator delete method in your subclass as shown here: https://trac.xapian.org/ticket/554#comment:1

Parameters

serialised A string containing the serialised parameters.

Reimplemented from Xapian::Weight.

Definition at line 132 of file bb2weight.cc.

References BB2Weight(), rare, and unserialise_double().

Member Data Documentation

◆ B_constant

double Xapian::BB2Weight::B_constant

private

Definition at line 1069 of file weight.h.

Referenced by get_sumpart(), and init().

◆ c_product_avlen

double Xapian::BB2Weight::c_product_avlen

private

The constant values to be used in get_sumpart().

Definition at line 1068 of file weight.h.

Referenced by get_sumpart(), and init().

◆ param_c

double Xapian::BB2Weight::param_c

private

The wdf normalization parameter in the formula.

Definition at line 1062 of file weight.h.

Referenced by BB2Weight(), clone(), init(), and serialise().

◆ stirling_constant_1

double Xapian::BB2Weight::stirling_constant_1

private

Definition at line 1071 of file weight.h.

Referenced by get_sumpart(), and init().

◆ stirling_constant_2

double Xapian::BB2Weight::stirling_constant_2

private

Definition at line 1072 of file weight.h.

Referenced by get_sumpart(), and init().

◆ upper_bound

double Xapian::BB2Weight::upper_bound

private

The upper bound on the weight.

Definition at line 1065 of file weight.h.

Referenced by get_maxpart(), and init().

◆ wt

double Xapian::BB2Weight::wt

private

Definition at line 1070 of file weight.h.

Referenced by get_sumpart(), and init().

The documentation for this class was generated from the following files:

include/xapian/weight.h
weight/bb2weight.cc

Public Member Functions

Private Member Functions

Private Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ BB2Weight() [1/2]

◆ BB2Weight() [2/2]

Member Function Documentation

◆ clone()

◆ get_maxextra()

◆ get_maxpart()

◆ get_sumextra()

◆ get_sumpart()

◆ init()

◆ name()

◆ serialise()

◆ unserialise()

Member Data Documentation

◆ B_constant

◆ c_product_avlen

◆ param_c

◆ stirling_constant_1

◆ stirling_constant_2

◆ upper_bound

◆ wt