xapian-core  1.4.18
Classes | Public Types | Public Member Functions | Private Attributes | List of all members
Xapian::QueryParser Class Reference

Build a Xapian::Query object from a user query string. More...

#include <queryparser.h>

+ Collaboration diagram for Xapian::QueryParser:

Classes

class  Internal
 

Public Types

enum  feature_flag {
  FLAG_BOOLEAN = 1, FLAG_PHRASE = 2, FLAG_LOVEHATE = 4, FLAG_BOOLEAN_ANY_CASE = 8,
  FLAG_WILDCARD = 16, FLAG_PURE_NOT = 32, FLAG_PARTIAL = 64, FLAG_SPELLING_CORRECTION = 128,
  FLAG_SYNONYM = 256, FLAG_AUTO_SYNONYMS = 512, FLAG_AUTO_MULTIWORD_SYNONYMS = 1024, FLAG_CJK_NGRAM = 2048,
  FLAG_ACCUMULATE = 65536, FLAG_DEFAULT = FLAG_PHRASE|FLAG_BOOLEAN|FLAG_LOVEHATE
}
 Enum of feature flags. More...
 
enum  stem_strategy {
  STEM_NONE, STEM_SOME, STEM_ALL, STEM_ALL_Z,
  STEM_SOME_FULL_POS
}
 Stemming strategies, for use with set_stemming_strategy(). More...
 

Public Member Functions

 QueryParser (const QueryParser &o)
 Copy constructor. More...
 
QueryParseroperator= (const QueryParser &o)
 Assignment. More...
 
 QueryParser ()
 Default constructor. More...
 
 ~QueryParser ()
 Destructor. More...
 
void set_stemmer (const Xapian::Stem &stemmer)
 Set the stemmer. More...
 
void set_stemming_strategy (stem_strategy strategy)
 Set the stemming strategy. More...
 
void set_stopper (const Stopper *stop=NULL)
 Set the stopper. More...
 
void set_default_op (Query::op default_op)
 Set the default operator. More...
 
Query::op get_default_op () const
 Get the current default operator. More...
 
void set_database (const Database &db)
 Specify the database being searched. More...
 
void set_max_expansion (Xapian::termcount max_expansion, int max_type=Xapian::Query::WILDCARD_LIMIT_ERROR, unsigned flags=FLAG_WILDCARD|FLAG_PARTIAL)
 Specify the maximum expansion of a wildcard and/or partial term. More...
 
void set_max_wildcard_expansion (Xapian::termcount)
 Specify the maximum expansion of a wildcard. More...
 
Query parse_query (const std::string &query_string, unsigned flags=FLAG_DEFAULT, const std::string &default_prefix=std::string())
 Parse a query. More...
 
void add_prefix (const std::string &field, const std::string &prefix)
 Add a free-text field term prefix. More...
 
void add_prefix (const std::string &field, Xapian::FieldProcessor *proc)
 Register a FieldProcessor. More...
 
void add_boolean_prefix (const std::string &field, const std::string &prefix, const std::string *grouping=NULL)
 Add a boolean term prefix allowing the user to restrict a search with a boolean filter specified in the free text query. More...
 
void add_boolean_prefix (const std::string &field, const std::string &prefix, bool exclusive)
 Add a boolean term prefix allowing the user to restrict a search with a boolean filter specified in the free text query. More...
 
void add_boolean_prefix (const std::string &field, Xapian::FieldProcessor *proc, const std::string *grouping=NULL)
 Register a FieldProcessor for a boolean prefix. More...
 
void add_boolean_prefix (const std::string &field, Xapian::FieldProcessor *proc, bool exclusive)
 Register a FieldProcessor for a boolean prefix. More...
 
TermIterator stoplist_begin () const
 Begin iterator over terms omitted from the query as stopwords. More...
 
TermIterator stoplist_end () const
 End iterator over terms omitted from the query as stopwords. More...
 
TermIterator unstem_begin (const std::string &term) const
 Begin iterator over unstemmed forms of the given stemmed query term. More...
 
TermIterator unstem_end (const std::string &) const
 End iterator over unstemmed forms of the given stemmed query term. More...
 
void add_rangeprocessor (Xapian::RangeProcessor *range_proc, const std::string *grouping=NULL)
 Register a RangeProcessor. More...
 
void add_valuerangeprocessor (Xapian::ValueRangeProcessor *vrproc)
 Register a ValueRangeProcessor. More...
 
std::string get_corrected_query_string () const
 Get the spelling-corrected query string. More...
 
std::string get_description () const
 Return a string describing this object. More...
 

Private Attributes

Xapian::Internal::intrusive_ptr< Internalinternal
 

Detailed Description

Build a Xapian::Query object from a user query string.

Definition at line 770 of file queryparser.h.

Member Enumeration Documentation

◆ feature_flag

Enum of feature flags.

Enumerator
FLAG_BOOLEAN 

Support AND, OR, etc and bracketed subexpressions.

FLAG_PHRASE 

Support quoted phrases.

FLAG_LOVEHATE 

Support + and -.

FLAG_BOOLEAN_ANY_CASE 

Support AND, OR, etc even if they aren't in ALLCAPS.

FLAG_WILDCARD 

Support wildcards.

At present only right truncation (e.g. Xap*) is supported.

Currently you can't use wildcards with boolean filter prefixes, or in a phrase (either an explicitly quoted one, or one implicitly generated by hyphens or other punctuation).

In Xapian 1.2.x, you needed to tell the QueryParser object which database to expand wildcards from by calling set_database(). In Xapian 1.3.3, OP_WILDCARD was added and wildcards are now expanded when Enquire::get_mset() is called, with the expansion using the database being searched.

FLAG_PURE_NOT 

Allow queries such as 'NOT apples'.

These require the use of a list of all documents in the database which is potentially expensive, so this feature isn't enabled by default.

FLAG_PARTIAL 

Enable partial matching.

Partial matching causes the parser to treat the query as a "partially entered" search. This will automatically treat the final word as a wildcarded match, unless it is followed by whitespace, to produce more stable results from interactive searches.

Currently FLAG_PARTIAL doesn't do anything if the final word in the query has a boolean filter prefix, or if it is in a phrase (either an explicitly quoted one, or one implicitly generated by hyphens or other punctuation). It also doesn't do anything if if the final word is part of a value range.

In Xapian 1.2.x, you needed to tell the QueryParser object which database to expand wildcards from by calling set_database(). In Xapian 1.3.3, OP_WILDCARD was added and wildcards are now expanded when Enquire::get_mset() is called, with the expansion using the database being searched.

FLAG_SPELLING_CORRECTION 

Enable spelling correction.

For each word in the query which doesn't exist as a term in the database, Database::get_spelling_suggestion() will be called and if a suggestion is returned, a corrected version of the query string will be built up which can be read using QueryParser::get_corrected_query_string(). The query returned is based on the uncorrected query string however - if you want a parsed query based on the corrected query string, you must call QueryParser::parse_query() again.

NB: You must also call set_database() for this to work.

FLAG_SYNONYM 

Enable synonym operator '~'.

NB: You must also call set_database() for this to work.

FLAG_AUTO_SYNONYMS 

Enable automatic use of synonyms for single terms.

NB: You must also call set_database() for this to work.

FLAG_AUTO_MULTIWORD_SYNONYMS 

Enable automatic use of synonyms for single terms and groups of terms.

NB: You must also call set_database() for this to work.

FLAG_CJK_NGRAM 

Enable generation of n-grams from CJK text.

With this enabled, spans of CJK characters are split into unigrams and bigrams, with the unigrams carrying positional information. Non-CJK characters are split into words as normal.

The corresponding option needs to have been used at index time.

Flag added in Xapian 1.3.4 and 1.2.22. This mode can be enabled in 1.2.8 and later by setting environment variable XAPIAN_CJK_NGRAM to a non-empty value (but doing so was deprecated in 1.4.11).

FLAG_ACCUMULATE 

Accumulate unstem and stoplist results.

By default, the unstem and stoplist data is reset by a call to parse_query(), which makes sense if you use the same QueryParser object to parse a series of independent queries.

If you're using the same QueryParser object to parse several fields on the same query form, you may want to have the unstem and stoplist data combined for all of them, in which case you can use this flag to prevent this data from being reset.

Since
Added in Xapian 1.4.18.
FLAG_DEFAULT 

The default flags.

Used if you don't explicitly pass any to parse_query(). The default flags are FLAG_PHRASE|FLAG_BOOLEAN|FLAG_LOVEHATE.

Added in Xapian 1.0.11.

Definition at line 778 of file queryparser.h.

◆ stem_strategy

Stemming strategies, for use with set_stemming_strategy().

Enumerator
STEM_NONE 
STEM_SOME 
STEM_ALL 
STEM_ALL_Z 
STEM_SOME_FULL_POS 

Definition at line 906 of file queryparser.h.

Constructor & Destructor Documentation

◆ QueryParser() [1/2]

QueryParser::QueryParser ( const QueryParser o)

Copy constructor.

Definition at line 65 of file queryparser.cc.

◆ QueryParser() [2/2]

QueryParser::QueryParser ( )

Default constructor.

Definition at line 79 of file queryparser.cc.

Referenced by operator=().

◆ ~QueryParser()

QueryParser::~QueryParser ( )

Destructor.

Definition at line 81 of file queryparser.cc.

Member Function Documentation

◆ add_boolean_prefix() [1/4]

void QueryParser::add_boolean_prefix ( const std::string &  field,
const std::string &  prefix,
const std::string *  grouping = NULL 
)

Add a boolean term prefix allowing the user to restrict a search with a boolean filter specified in the free text query.

For example:

qp.add_boolean_prefix("site", "H");

This allows the user to restrict a search with site:xapian.org which will be converted to Hxapian.org combined with any weighted query with Xapian::Query::OP_FILTER.

If multiple boolean filters are specified in a query for the same prefix, they will be combined with the Xapian::Query::OP_OR operator. Then, if there are boolean filters for different prefixes, they will be combined with the Xapian::Query::OP_AND operator.

Multiple fields can be mapped to the same prefix (so for example you can make site: and domain: aliases for each other). Instances of fields with different aliases but the same prefix will still be combined with the OR operator.

For example, if "site" and "domain" map to "H", but author maps to "A", a search for "site:foo domain:bar author:Fred" will map to "(Hfoo OR Hbar) AND Afred".

As of 1.0.4, you can call this method multiple times with the same value of field to allow a single field to be mapped to multiple prefixes. Multiple terms being generated for such a field, and combined with Xapian::Query::OP_OR.

Calling this method with an empty string for field will cause a Xapian::InvalidArgumentError.

If you call add_prefix() and add_boolean_prefix() for the same value of field, a Xapian::InvalidOperationError exception will be thrown.

In 1.0.3 and earlier, subsequent calls to this method with the same value of field had no effect.

Parameters
fieldThe user visible field name
prefixThe term prefix to map this to
groupingControls how multiple filters are combined - filters with the same grouping value are combined with OP_OR, then the resulting queries are combined with OP_AND. If NULL, then field is used for grouping. If an empty string, then a unique grouping is created for each filter (this is sometimes useful when each document can have multiple terms with this prefix). [default: NULL]

Definition at line 197 of file queryparser.cc.

References Assert.

Referenced by DEFINE_TESTCASE(), and main().

◆ add_boolean_prefix() [2/4]

void Xapian::QueryParser::add_boolean_prefix ( const std::string &  field,
const std::string &  prefix,
bool  exclusive 
)
inline

Add a boolean term prefix allowing the user to restrict a search with a boolean filter specified in the free text query.

This is an older version of this method - use the version with the grouping parameter in preference to this one.

Parameters
fieldThe user visible field name
prefixThe term prefix to map this to
exclusiveControls how multiple filters are combined. If true then prefix is used as the grouping value, so terms with the same prefix are combined with OP_OR, then the resulting queries are combined with OP_AND. If false, then a unique grouping is created for each filter (this is sometimes useful when each document can have multiple terms with this prefix).

Definition at line 1191 of file queryparser.h.

◆ add_boolean_prefix() [3/4]

void QueryParser::add_boolean_prefix ( const std::string &  field,
Xapian::FieldProcessor proc,
const std::string *  grouping = NULL 
)

Register a FieldProcessor for a boolean prefix.

Definition at line 205 of file queryparser.cc.

References Assert.

◆ add_boolean_prefix() [4/4]

void Xapian::QueryParser::add_boolean_prefix ( const std::string &  field,
Xapian::FieldProcessor proc,
bool  exclusive 
)
inline

Register a FieldProcessor for a boolean prefix.

This is an older version of this method - use the version with the grouping parameter in preference to this one.

Definition at line 1211 of file queryparser.h.

◆ add_prefix() [1/2]

void QueryParser::add_prefix ( const std::string &  field,
const std::string &  prefix 
)

Add a free-text field term prefix.

For example:

qp.add_prefix("author", "A");

This allows the user to search for author:Orwell which will be converted to a search for the term "Aorwell".

Multiple fields can be mapped to the same prefix. For example, you can make title: and subject: aliases for each other.

As of 1.0.4, you can call this method multiple times with the same value of field to allow a single field to be mapped to multiple prefixes. Multiple terms being generated for such a field, and combined with Xapian::Query::OP_OR.

If any prefixes are specified for the empty field name (i.e. you call this method with an empty string as the first parameter) these prefixes will be used for terms without a field specifier. If you do this and also specify the default_prefix parameter to parse_query(), then the default_prefix parameter will override.

If the prefix parameter is empty, then "field:word" will produce the term "word" (and this can be one of several prefixes for a particular field, or for terms without a field specifier).

If you call add_prefix() and add_boolean_prefix() for the same value of field, a Xapian::InvalidOperationError exception will be thrown.

In 1.0.3 and earlier, subsequent calls to this method with the same value of field had no effect.

Parameters
fieldThe user visible field name
prefixThe term prefix to map this to

Definition at line 183 of file queryparser.cc.

References Assert.

Referenced by DEFINE_TESTCASE(), and main().

◆ add_prefix() [2/2]

void QueryParser::add_prefix ( const std::string &  field,
Xapian::FieldProcessor proc 
)

Register a FieldProcessor.

Definition at line 190 of file queryparser.cc.

References Assert.

◆ add_rangeprocessor()

void QueryParser::add_rangeprocessor ( Xapian::RangeProcessor range_proc,
const std::string *  grouping = NULL 
)

Register a RangeProcessor.

Definition at line 234 of file queryparser.cc.

References Assert.

Referenced by DEFINE_TESTCASE().

◆ add_valuerangeprocessor()

void Xapian::QueryParser::add_valuerangeprocessor ( Xapian::ValueRangeProcessor vrproc)
inline

Register a ValueRangeProcessor.

This method is provided for API compatibility with Xapian 1.2.x and is deprecated - use add_rangeprocessor() with a RangeProcessor instead.

Compatibility shim.

Definition at line 1246 of file queryparser.h.

References Xapian::BAD_VALUENO, Xapian::Query::OP_INVALID, and Xapian::RangeProcessor::operator()().

Referenced by DEFINE_TESTCASE().

◆ get_corrected_query_string()

string QueryParser::get_corrected_query_string ( ) const

Get the spelling-corrected query string.

This will only be set if FLAG_SPELLING_CORRECTION is specified when QueryParser::parse_query() was last called.

If there were no corrections, an empty string is returned.

Definition at line 242 of file queryparser.cc.

Referenced by DEFINE_TESTCASE(), and main().

◆ get_default_op()

Query::op QueryParser::get_default_op ( ) const

Get the current default operator.

Definition at line 135 of file queryparser.cc.

Referenced by DEFINE_TESTCASE().

◆ get_description()

string QueryParser::get_description ( ) const

Return a string describing this object.

Definition at line 248 of file queryparser.cc.

◆ operator=()

QueryParser & QueryParser::operator= ( const QueryParser o)
default

Assignment.

Definition at line 68 of file queryparser.cc.

References internal, and QueryParser().

◆ parse_query()

Query QueryParser::parse_query ( const std::string &  query_string,
unsigned  flags = FLAG_DEFAULT,
const std::string &  default_prefix = std::string() 
)

Parse a query.

Parameters
query_stringA free-text query as entered by a user
flagsZero or more QueryParser::feature_flag specifying what features the QueryParser should support. Combine multiple values with bitwise-or (|) (default FLAG_DEFAULT).
default_prefixThe default term prefix to use (default none). For example, you can pass "A" when parsing an "Author" field.
Exceptions
Ifthe query string can't be parsed, then Xapian::QueryParserError is thrown. You can get an English error message to report to the user by catching it and calling get_msg() on the caught exception. The current possible values (in case you want to translate them) are:
  • Unknown range operation
  • parse error
  • Syntax: <expression> AND <expression>
  • Syntax: <expression> AND NOT <expression>
  • Syntax: <expression> NOT <expression>
  • Syntax: <expression> OR <expression>
  • Syntax: <expression> XOR <expression>

Definition at line 161 of file queryparser.cc.

References FLAG_ACCUMULATE, FLAG_CJK_NGRAM, and internal.

Referenced by DEFINE_TESTCASE(), main(), test_qp_flag_wildcard3_helper(), and time_query_parse().

◆ set_database()

void QueryParser::set_database ( const Database db)

Specify the database being searched.

Parameters
dbThe database to use for spelling correction (FLAG_SPELLING_CORRECTION), and synonyms (FLAG_SYNONYM, FLAG_AUTO_SYNONYMS, and FLAG_AUTO_MULTIWORD_SYNONYMS).

Definition at line 141 of file queryparser.cc.

Referenced by DEFINE_TESTCASE(), main(), test_qp_flag_wildcard3_helper(), and time_query_parse().

◆ set_default_op()

void QueryParser::set_default_op ( Query::op  default_op)

Set the default operator.

Parameters
default_opThe operator to use to combine non-filter query items when no explicit operator is used.

So for example, 'weather forecast' is parsed as if it were 'weather OR forecast' by default.

The most useful values for this are OP_OR (the default) and OP_AND. OP_NEAR, OP_PHRASE, OP_ELITE_SET, OP_SYNONYM and OP_MAX are also permitted. Passing other values will result in InvalidArgumentError being thrown.

Definition at line 102 of file queryparser.cc.

References Xapian::Query::OP_AND, Xapian::Query::OP_ELITE_SET, Xapian::Query::OP_MAX, Xapian::Query::OP_NEAR, Xapian::Query::OP_OR, Xapian::Query::OP_PHRASE, and Xapian::Query::OP_SYNONYM.

Referenced by DEFINE_TESTCASE(), and main().

◆ set_max_expansion()

void QueryParser::set_max_expansion ( Xapian::termcount  max_expansion,
int  max_type = Xapian::Query::WILDCARD_LIMIT_ERROR,
unsigned  flags = FLAG_WILDCARD|FLAG_PARTIAL 
)

Specify the maximum expansion of a wildcard and/or partial term.

Note: you must also set FLAG_WILDCARD and/or FLAG_PARTIAL in the flags parameter to parse_query() for this setting to have anything to affect.

If you don't call this method, the default settings are no limit on wildcard expansion, and partial terms expanding to the most frequent 100 terms - i.e. as if you'd called:

set_max_expansion(0); set_max_expansion(100, Xapian::Query::WILDCARD_LIMIT_MOST_FREQUENT, Xapian::QueryParser::FLAG_PARTIAL);

Parameters
max_expansionThe maximum number of terms each wildcard in the query can expand to, or 0 for no limit (which is the default).
max_typeXapian::Query::WILDCARD_LIMIT_ERROR, Xapian::Query::WILDCARD_LIMIT_FIRST or Xapian::Query::WILDCARD_LIMIT_MOST_FREQUENT (default: Xapian::Query::WILDCARD_LIMIT_ERROR).
flagsWhat to set the limit for (default: FLAG_WILDCARD|FLAG_PARTIAL, setting the limit for both wildcards and partial terms).
Since
1.3.3

Definition at line 146 of file queryparser.cc.

References FLAG_PARTIAL, and FLAG_WILDCARD.

Referenced by test_qp_flag_wildcard3_helper().

◆ set_max_wildcard_expansion()

void Xapian::QueryParser::set_max_wildcard_expansion ( Xapian::termcount  max_expansion)
inline

Specify the maximum expansion of a wildcard.

If any wildcard expands to more than max_expansion terms, an exception will be thrown.

This method is provided for API compatibility with Xapian 1.2.x and is deprecated - replace it with:

set_max_wildcard_expansion(max_expansion, Xapian::Query::WILDCARD_LIMIT_ERROR, Xapian::QueryParser::FLAG_WILDCARD);

Definition at line 1283 of file queryparser.h.

References Xapian::sortable_serialise_(), Xapian::Query::WILDCARD_LIMIT_ERROR, and XAPIAN_VISIBILITY_DEFAULT.

◆ set_stemmer()

void QueryParser::set_stemmer ( const Xapian::Stem stemmer)

Set the stemmer.

This sets the stemming algorithm which will be used by the query parser. The stemming algorithm will be used according to the stemming strategy set by set_stemming_strategy(). As of 1.3.1, this defaults to STEM_SOME, but in earlier versions the default was STEM_NONE. If you want to work with older versions, you should explicitly set a stemming strategy as well as setting a stemmer, otherwise your stemmer won't actually be used.

Parameters
stemmerThe Xapian::Stem object to set.

Definition at line 84 of file queryparser.cc.

References stemmer.

Referenced by DEFINE_TESTCASE(), and main().

◆ set_stemming_strategy()

void QueryParser::set_stemming_strategy ( stem_strategy  strategy)

Set the stemming strategy.

This controls how the query parser will apply the stemming algorithm. Note that the stemming algorithm is only applied to words in free-text fields - boolean filter terms are never stemmed.

Parameters
strategyThe strategy to use - possible values are:
  • STEM_NONE: Don't perform any stemming. (default in Xapian <= 1.3.0)
  • STEM_SOME: Stem all terms except for those which start with a capital letter, or are followed by certain characters (currently: (/@<>=*[{" ), or are used with operators which need positional information. Stemmed terms are prefixed with 'Z'. (default in Xapian >= 1.3.1)
  • STEM_SOME_FULL_POS: Like STEM_SOME but also stems terms used with operators which need positional information. Added in Xapian 1.4.8.
  • STEM_ALL: Stem all terms (note: no 'Z' prefix is added).
  • STEM_ALL_Z: Stem all terms (note: 'Z' prefix is added). (new in Xapian 1.2.11 and 1.3.1)

Definition at line 90 of file queryparser.cc.

Referenced by DEFINE_TESTCASE(), and main().

◆ set_stopper()

void QueryParser::set_stopper ( const Stopper stop = NULL)

Set the stopper.

Parameters
stopThe Stopper object to set (default NULL, which means no stopwords).

Definition at line 96 of file queryparser.cc.

Referenced by DEFINE_TESTCASE(), and main().

◆ stoplist_begin()

TermIterator QueryParser::stoplist_begin ( ) const

Begin iterator over terms omitted from the query as stopwords.

Definition at line 214 of file queryparser.cc.

Referenced by DEFINE_TESTCASE().

◆ stoplist_end()

TermIterator Xapian::QueryParser::stoplist_end ( ) const
inline

End iterator over terms omitted from the query as stopwords.

Definition at line 1225 of file queryparser.h.

Referenced by DEFINE_TESTCASE().

◆ unstem_begin()

TermIterator QueryParser::unstem_begin ( const std::string &  term) const

Begin iterator over unstemmed forms of the given stemmed query term.

Definition at line 221 of file queryparser.cc.

References Xapian::operator*().

Referenced by DEFINE_TESTCASE().

◆ unstem_end()

TermIterator Xapian::QueryParser::unstem_end ( const std::string &  ) const
inline

End iterator over unstemmed forms of the given stemmed query term.

Definition at line 1233 of file queryparser.h.

Referenced by DEFINE_TESTCASE().

Member Data Documentation

◆ internal

Xapian::Internal::intrusive_ptr<Internal> Xapian::QueryParser::internal
private

Reference counted internals.

Definition at line 773 of file queryparser.h.

Referenced by operator=(), and parse_query().


The documentation for this class was generated from the following files: