ChertTable Class Reference

Class managing a Btree table in a Chert database. More...

#include <chert_table.h>

Inheritance diagram for ChertTable:

Inheritance graph
[legend]
Collaboration diagram for ChertTable:

Collaboration graph
[legend]

List of all members.

Public Member Functions

 ChertTable (const char *tablename_, const std::string &path_, bool readonly_, int compress_strategy_=DONT_COMPRESS, bool lazy=false)
 Create a new Btree object.
 ~ChertTable ()
 Close the Btree.
void close (bool permanent=false)
 Close the Btree.
bool exists () const
 Determine whether the btree exists on disk.
void open ()
 Open the btree at the latest revision.
bool open (chert_revision_number_t revision_)
 Open the btree at a given revision.
bool is_open () const
 Return true if this table is open.
void flush_db ()
 Flush any outstanding changes to the DB file of the table.
void commit (chert_revision_number_t revision, int changes_fd=-1, const std::string *changes_tail=NULL)
 Commit any outstanding changes to the table.
void write_changed_blocks (int changes_fd)
 Append the list of blocks changed to a changeset file.
void cancel ()
 Cancel any outstanding changes.
bool get_exact_entry (const std::string &key, std::string &tag) const
 Read an entry from the table, if and only if it is exactly that being asked for.
bool key_exists (const std::string &key) const
 Check if a key exists in the Btree.
bool read_tag (Cursor *C_, std::string *tag, bool keep_compressed) const
 Read the tag value for the key pointed to by cursor C_.
void add (const std::string &key, std::string tag, bool already_compressed=false)
 Add a key/tag pair to the table, replacing any existing pair with the same key.
bool del (const std::string &key)
 Delete an entry from the table.
void erase ()
 Erase this table from disk.
void set_block_size (unsigned int block_size_)
 Set the block size.
unsigned int get_block_size () const
 Get the block size.
void create_and_open (unsigned int blocksize)
 Create a new empty btree structure on disk and open it at the initial revision.
void set_full_compaction (bool parity)
chert_revision_number_t get_latest_revision_number () const
 Get the latest revision number stored in this table.
chert_revision_number_t get_open_revision_number () const
 Get the revision number at which this table is currently open.
chert_tablesize_t get_entry_count () const
 Return a count of the number of entries in the table.
bool empty () const
 Return true if there are no entries in the table.
ChertCursorcursor_get () const
 Get a cursor for reading from the table.
bool is_modified () const
 Determine whether the object contains uncommitted modifications.
void set_max_item_size (size_t block_capacity)
 Set the maximum item size given the block capacity.
 XAPIAN_NORETURN (static void throw_database_closed())
 Throw an exception indicating that the database is closed.

Protected Member Functions

bool do_open_to_read (bool revision_supplied, chert_revision_number_t revision_)
 Perform the opening operation to read.
bool do_open_to_write (bool revision_supplied, chert_revision_number_t revision_, bool create_db=false)
 Perform the opening operation to write.
bool basic_open (bool revision_supplied, chert_revision_number_t revision)
bool find (Cursor *) const
 find(C_) searches for the key of B->kt in the B-tree.
int delete_kt ()
void read_block (uint4 n, byte *p) const
 read_block(n, p) reads block n of the DB file to address p.
void write_block (uint4 n, const byte *p) const
 write_block(n, p) writes block n in the DB file from address p.
 XAPIAN_NORETURN (void set_overwritten() const)
void block_to_cursor (Cursor *C_, int j, uint4 n) const
void alter ()
 Btree::alter(); is called when the B-tree is to be altered.
void compact (byte *p)
 compact(p) compact the block at p by shuffling all the items up to the end.
void enter_key (int j, Key prevkey, Key newkey)
 enter_key(j, prevkey, newkey) is called after a block split.
int mid_point (byte *p)
 mid_point(p) finds the directory entry in c that determines the approximate mid point of the data in the block at p.
void add_item_to_block (byte *p, Item_wr kt, int c)
 add_item_to_block(p, kt_, c) adds item kt_ to the block at p.
void add_item (Item_wr kt, int j)
 ChertTable::add_item(kt_, j) adds item kt_ to the block at cursor level C[j].
void delete_item (int j, bool repeatedly)
 ChertTable::delete_item(j, repeatedly) is (almost) the converse of add_item.
int add_kt (bool found)
 add_kt(found) adds the item (key-tag pair) at B->kt into the B-tree, using cursor C.
void read_root ()
void split_root (uint4 split_n)
 Btree needs to gain a new level to insert more items: so split root block and construct a new one.
void form_key (const std::string &key) const
char other_base_letter () const
void lazy_alloc_deflate_zstream () const
 Allocate the zstream for deflating, if not already allocated.
void lazy_alloc_inflate_zstream () const
 Allocate the zstream for inflating, if not already allocated.
bool prev (Cursor *C_, int j) const
bool next (Cursor *C_, int j) const
bool prev_default (Cursor *C_, int j) const
bool next_default (Cursor *C_, int j) const
bool prev_for_sequential (Cursor *C_, int dummy) const
bool next_for_sequential (Cursor *C_, int dummy) const

Static Protected Member Functions

static int find_in_block (const byte *p, Key key, bool leaf, int c)
 find_in_block(p, key, leaf, c) searches for the key in the block at p.
static uint4 block_given_by (const byte *p, int c)
 block_given_by(p, c) finds the item at block address p, directory offset c, and returns its tag value as an integer.

Protected Attributes

const char * tablename
 The name of the table (used when writing changesets).
chert_revision_number_t revision_number
 revision number of the opened B-tree.
chert_tablesize_t item_count
 keeps a count of the number of items in the B-tree.
unsigned int block_size
 block size of the B tree in bytes
chert_revision_number_t latest_revision_number
 Revision number of the other base, or zero if there is only one base file.
bool both_bases
 set to true if baseA and baseB both exist as valid bases.
char base_letter
 the value 'A' or 'B' of the current base
bool faked_root_block
 true if the root block is faked (not written to disk).
bool sequential
 true iff the data has been written in a single write in sequential order.
int handle
 File descriptor of the table.
int level
 number of levels, counting from 0
uint4 root
 the root block of the B-tree
Item_wr kt
 buffer of size block_size for making up key-tag items
bytebuffer
 buffer of size block_size for reforming blocks
ChertTable_base base
 For writing back as file baseA or baseB.
std::string name
 The path name of the B tree.
int seq_count
 count of the number of successive instances of purely sequential addition, starting at SEQ_START_POINT (neg) and going up to zero.
uint4 changed_n
 the last block to be changed by an addition
int changed_c
 directory offset corresponding to last block to be changed by an addition
size_t max_item_size
 maximum size of an item (key-tag pair)
bool Btree_modified
 Set to true the first time the B-tree is modified.
bool full_compaction
 set to true when full compaction is to be achieved
bool writable
 Set to true when the database is opened to write.
bool cursor_created_since_last_modification
 Flag for tracking when cursors need to rebuild.
unsigned long cursor_version
 Version count for tracking when cursors need to rebuild.
Cursor C [BTREE_CURSOR_LEVELS]
bytesplit_p
 Buffer used when splitting a block.
int compress_strategy
 DONT_COMPRESS or Z_DEFAULT_STRATEGY, Z_FILTERED, Z_HUFFMAN_ONLY, Z_RLE.
z_stream * deflate_zstream
 Zlib state object for deflating.
z_stream * inflate_zstream
 Zlib state object for inflating.
bool lazy
 If true, don't create the table until it's needed.

Private Member Functions

 ChertTable (const ChertTable &)
 Copying not allowed.
ChertTableoperator= (const ChertTable &)
 Assignment not allowed.
bool really_empty () const
 Return true if there are no entries in the table.

Friends

class ChertCursor


Detailed Description

Class managing a Btree table in a Chert database.

A table is a store holding a set of key/tag pairs.

A key is used to access a block of data in a chert table.

Keys are of limited length.

Keys may not be empty (each Btree has a special empty key for internal use).

A tag is a piece of data associated with a given key. The contents of the tag are opaque to the Btree.

Tags may be of arbitrary length (the Btree imposes a very large limit). Note though that they will be loaded into memory in their entirety, so should not be permitted to grow without bound in normal usage.

Tags which are null strings _are_ valid, and are different from a tag simply not being in the table.

Definition at line 289 of file chert_table.h.


Constructor & Destructor Documentation

ChertTable::ChertTable ( const ChertTable  )  [private]

Copying not allowed.

ChertTable::ChertTable ( const char *  tablename_,
const std::string &  path_,
bool  readonly_,
int  compress_strategy_ = DONT_COMPRESS,
bool  lazy = false 
)

Create a new Btree object.

This does not create the table on disk - the create_and_open() method must be called to create the table on disk.

This also does not open the table - either the create_and_open() or open() methods must be called before use is made of the table.

Parameters:
tablename_ The name of the table (used in changesets).
path_ Path at which the table is stored.
readonly_ whether to open the table for read only access.
compress_strategy_ DONT_COMPRESS, Z_DEFAULT_STRATEGY, Z_FILTERED, Z_HUFFMAN_ONLY, or Z_RLE.
lazy If true, don't create the table until it's needed.

Definition at line 1552 of file chert_table.cc.

References LOGCALL_CTOR.

ChertTable::~ChertTable (  ) 

Close the Btree.

Any outstanding changes (ie, changes made without commit() having subsequently been called) will be lost.

Definition at line 1741 of file chert_table.cc.

References close(), deflate_zstream, inflate_zstream, and LOGCALL_DTOR.


Member Function Documentation

void ChertTable::add ( const std::string &  key,
std::string  tag,
bool  already_compressed = false 
)

Add a key/tag pair to the table, replacing any existing pair with the same key.

If an error occurs during the operation, an exception will be thrown.

If key is empty, then the null item is replaced.

e.g. btree.add("TODAY", "Mon 9 Oct 2000");

Parameters:
key The key to store in the table.
tag The tag to store in the table.
already_compressed true if tag is already compressed, for example because it is being opaquely copied (default: false).

Definition at line 1019 of file chert_table.cc.

References add_kt(), Assert, block_size, Btree_modified, BYTE_PAIR_RANGE, C, C2, CompileTimeAssert, COMPRESS_MIN, compress_strategy, create_and_open(), cursor_created_since_last_modification, cursor_version, D2, deflate_zstream, delete_kt(), DONT_COMPRESS, find(), form_key(), full_compaction, handle, I2, item_count, K1, Item_base< T >::key(), kt, lazy_alloc_deflate_zstream(), Key::length(), LOGCALL_VOID, max_item_size, Cursor::p, Item_wr::set_component_of(), Item_wr::set_components_of(), Item_wr::set_tag(), TOTAL_FREE, and writable.

Referenced by copy_docid_keyed(), copy_position(), copy_postlist(), copy_termlist(), copy_unchanged(), Chert::PostlistChunkWriter::flush(), ChertValueManager::merge_changes(), ChertSynonymTable::merge_changes(), ChertSpellingTable::merge_changes(), ChertPostListTable::merge_changes(), ChertCompact::merge_docid_keyed(), ChertCompact::merge_postlists(), ChertCompact::merge_spellings(), ChertCompact::merge_synonyms(), ChertRecordTable::replace_record(), ChertWritableDatabase::set_metadata(), ChertPositionListTable::set_positionlist(), ChertTermListTable::set_termlist(), ChertValueManager::set_value_stats(), ChertDatabaseStats::write(), and ValueUpdater::write_tag().

void ChertTable::add_item ( Item_wr  kt_,
int  j 
) [protected]

ChertTable::add_item(kt_, j) adds item kt_ to the block at cursor level C[j].

If there is not enough room the block splits and the item is then added to the appropriate half.

Definition at line 718 of file chert_table.cc.

References add_item_to_block(), Assert, AssertRel, base, block_size, Cursor::c, C, changed_c, changed_n, compact(), D2, DIR_END, DIR_START, enter_key(), level, LOGCALL_VOID, MAX_FREE, mid_point(), Cursor::n, ChertTable_base::next_free_block(), Cursor::p, seq_count, SET_DIR_END, Item_base< T >::size(), split_p, split_root(), TOTAL_FREE, writable, and write_block().

Referenced by add_kt(), enter_key(), and split_root().

void ChertTable::add_item_to_block ( byte p,
Item_wr  kt_,
int  c 
) [protected]

add_item_to_block(p, kt_, c) adds item kt_ to the block at p.

c is the offset in the directory that needs to be expanded to accommodate the new entry for the item. We know before this is called that there is enough contiguous room for the item in the block, so it's just a matter of shuffling up any directory entries after where we're inserting and copying in the item.

Definition at line 683 of file chert_table.cc.

References Assert, AssertRel, D2, DIR_END, Item_base< T >::get_address(), LOGCALL_VOID, MAX_FREE, SET_DIR_END, SET_MAX_FREE, SET_TOTAL_FREE, setD, Item_base< T >::size(), TOTAL_FREE, and writable.

Referenced by add_item().

int ChertTable::add_kt ( bool  found  )  [protected]

add_kt(found) adds the item (key-tag pair) at B->kt into the B-tree, using cursor C.

found == find() is handed over as a parameter from Btree::add. Btree::alter() prepares for the alteration to the B-tree. Then there are a number of cases to consider:

If an item with the same key is in the B-tree (found is true), the new kt replaces it.

If then kt is smaller, or the same size as, the item it replaces, kt is put in the same place as the item it replaces, and the TOTAL_FREE measure is reduced.

If kt is larger than the item it replaces it is put in the MAX_FREE space if there is room, and the directory entry and space counts are adjusted accordingly.

  • But if there is not room we do it the long way: the old item is deleted with delete_item and kt is added in with add_item.

If the key of kt is not in the B-tree (found is false), the new kt is added in with add_item.

Definition at line 889 of file chert_table.cc.

References add_item(), alter(), Assert, Cursor::c, C, changed_c, changed_n, Item_base< T >::components_of(), D2, delete_item(), DIR_END, Item_base< T >::get_address(), kt, LOGCALL, MAX_FREE, Cursor::p, RETURN, seq_count, SEQ_START_POINT, sequential, SET_MAX_FREE, SET_TOTAL_FREE, setD, Item_base< T >::size(), TOTAL_FREE, and writable.

Referenced by add().

void ChertTable::alter (  )  [protected]

Btree::alter(); is called when the B-tree is to be altered.

It causes new blocks to be forced for the current set of blocks in the cursor.

The point is that if a block at level 0 is to be altered it may get a new number. Then the pointer to this block from level 1 will need changing. So the block at level 1 needs altering and may get a new block number. Then the pointer to this block from level 2 will need changing ... and so on back to the root.

The clever bit here is spotting the cases when we can make an early exit from this process. If C[j].rewrite is true, C[j+k].rewrite will be true for k = 1,2 ... We have been through all this before, and there is no need to do it again. If C[j].n was free at the start of the transaction, we can copy it back to the same place without violating the integrity of the B-tree. We don't then need a new n and can return. The corresponding C[j].rewrite may be true or false in that case.

Definition at line 413 of file chert_table.cc.

References Assert, base, ChertTable_base::block_free_at_start(), C, ChertTable_base::free_block(), latest_revision_number, level, LOGCALL_VOID, Cursor::n, ChertTable_base::next_free_block(), Cursor::p, REVISION, Cursor::rewrite, SET_REVISION, and writable.

Referenced by add_kt(), and delete_kt().

bool ChertTable::basic_open ( bool  revision_supplied,
chert_revision_number_t  revision 
) [protected]

static uint4 ChertTable::block_given_by ( const byte p,
int  c 
) [static, protected]

block_given_by(p, c) finds the item at block address p, directory offset c, and returns its tag value as an integer.

Referenced by ChertTableCheck::block_check(), find(), next_default(), and prev_default().

void ChertTable::block_to_cursor ( Cursor C_,
int  j,
uint4  n 
) const [protected]

void ChertTable::cancel (  ) 

void ChertTable::close ( bool  permanent = false  ) 

Close the Btree.

This closes and frees any of the btree structures which have been created and opened.

Parameters:
permanent If true, the Btree will not reopen on demand.

Definition at line 1760 of file chert_table.cc.

References buffer, C, Item_base< T >::get_address(), handle, kt, level, LOGCALL_VOID, Cursor::p, and split_p.

Referenced by basic_open(), ChertDatabase::close(), commit(), create_and_open(), do_open_to_read(), do_open_to_write(), erase(), open(), and ~ChertTable().

void ChertTable::commit ( chert_revision_number_t  revision,
int  changes_fd = -1,
const std::string *  changes_tail = NULL 
)

Commit any outstanding changes to the table.

Commit changes made by calling add() and del() to the Btree.

If an error occurs during the operation, this will be signalled by an exception. In case of error, changes made will not be committed to the Btree - they will be discarded.

Parameters:
new_revision The new revision number to store. This must be greater than the latest revision number (see get_latest_revision_number()), or an exception will be thrown.
changes_fd The file descriptor to write changes to. Defaults to -1, meaning no changes will be written.

Definition at line 1813 of file chert_table.cc.

References Assert, base, base_letter, BLK_UNUSED, both_bases, BTREE_CURSOR_LEVELS, Btree_modified, Cursor::c, C, changed_c, changed_n, ChertTable_base::clear_bit_map(), close(), ChertTable_base::commit(), DIR_START, faked_root_block, handle, io_sync(), item_count, latest_revision_number, level, LOGCALL_VOID, msvc_posix_rename(), Cursor::n, other_base_letter(), read_root(), revision_number, Cursor::rewrite, root, seq_count, SEQ_START_POINT, sequential, ChertTable_base::set_have_fakeroot(), ChertTable_base::set_item_count(), ChertTable_base::set_level(), ChertTable_base::set_revision(), ChertTable_base::set_root(), ChertTable_base::set_sequential(), tablename, unlink(), writable, and ChertTable_base::write_to_file().

Referenced by compact_chert(), main(), ChertCompact::multimerge_postlists(), and ChertDatabase::set_revision_number().

void ChertTable::compact ( byte p  )  [protected]

compact(p) compact the block at p by shuffling all the items up to the end.

MAX_FREE(p) is then maximized, and is equal to TOTAL_FREE(p).

Definition at line 522 of file chert_table.cc.

References Assert, block_size, buffer, D2, DIR_END, DIR_START, Item_base< T >::get_address(), LOGCALL_VOID, SET_MAX_FREE, SET_TOTAL_FREE, setD, Item_base< T >::size(), and writable.

Referenced by add_item(), and split_root().

void ChertTable::create_and_open ( unsigned int  blocksize  ) 

Create a new empty btree structure on disk and open it at the initial revision.

The table must be writable - it doesn't make sense to create a table that is read-only!

The block size must be less than 64K, where K = 1024. It is unwise to use a small block size (less than 1024 perhaps), so we enforce a minimum block size of 2K.

Example:

Btree btree("X-"); btree.create_and_open(8192); // Files will be X-DB, X-baseA (and X-baseB).

Parameters:
blocksize - Size of blocks to use.
Exceptions:
Xapian::DatabaseCreateError if the table can't be created.
Xapian::InvalidArgumentError if the requested blocksize is unsuitable.

Reimplemented in ChertLazyTable, and ChertTermListTable.

Definition at line 1708 of file chert_table.cc.

References Assert, close(), do_open_to_write(), handle, io_unlink(), LOGCALL_VOID, revision_number, ChertTable_base::set_block_size(), set_block_size(), ChertTable_base::set_have_fakeroot(), ChertTable_base::set_revision(), ChertTable_base::set_sequential(), writable, and ChertTable_base::write_to_file().

Referenced by add(), compact_chert(), ChertDatabase::create_and_open_tables(), main(), and ChertCompact::multimerge_postlists().

ChertCursor * ChertTable::cursor_get (  )  const

bool ChertTable::del ( const std::string &  key  ) 

Delete an entry from the table.

The entry will be removed from the table, if it exists. If it does not exist, no action will be taken. The item with an empty key can't be removed, and false is returned.

If an error occurs during the operation, this will be signalled by an exception.

e.g. bool deleted = btree.del("TODAY")

Parameters:
key The key to remove from the table.
Returns:
true if an entry was removed; false if it did not exist.

Definition at line 1141 of file chert_table.cc.

References Assert, Btree_modified, CHERT_BTREE_MAX_KEY_LEN, cursor_created_since_last_modification, cursor_version, delete_kt(), form_key(), handle, item_count, kt, LOGCALL, RETURN, Item_wr::set_component_of(), and writable.

Referenced by ChertPositionListTable::delete_positionlist(), ChertRecordTable::delete_record(), ChertTermListTable::delete_termlist(), Chert::PostlistChunkWriter::flush(), ChertValueManager::merge_changes(), ChertSynonymTable::merge_changes(), ChertSpellingTable::merge_changes(), ChertPostListTable::merge_changes(), ChertWritableDatabase::set_metadata(), ChertValueManager::set_value_stats(), and ValueUpdater::write_tag().

void ChertTable::delete_item ( int  j,
bool  repeatedly 
) [protected]

ChertTable::delete_item(j, repeatedly) is (almost) the converse of add_item.

If repeatedly is true, the process repeats at the next level when a block has been completely emptied, freeing the block and taking out the pointer to it. Emptied root blocks are also removed, which reduces the number of levels in the B-tree.

Definition at line 816 of file chert_table.cc.

References Assert, base, BLK_UNUSED, block_to_cursor(), Cursor::c, C, D2, DIR_END, DIR_START, ChertTable_base::free_block(), level, LOGCALL_VOID, MAX_FREE, Cursor::n, Cursor::p, Cursor::rewrite, SET_DIR_END, SET_MAX_FREE, SET_TOTAL_FREE, TOTAL_FREE, and writable.

Referenced by add_kt(), and delete_kt().

int ChertTable::delete_kt (  )  [protected]

Definition at line 955 of file chert_table.cc.

References alter(), Assert, C, delete_item(), find(), LOGCALL, RETURN, seq_count, SEQ_START_POINT, sequential, and writable.

Referenced by add(), and del().

bool ChertTable::do_open_to_read ( bool  revision_supplied,
chert_revision_number_t  revision_ 
) [protected]

Perform the opening operation to read.

Return true iff the open succeeded.

Definition at line 2002 of file chert_table.cc.

References basic_open(), BLK_UNUSED, block_size, C, close(), handle, lazy, level, LOGCALL, Cursor::n, O_BINARY, open(), Cursor::p, read_root(), RETURN, and revision_number.

Referenced by open().

bool ChertTable::do_open_to_write ( bool  revision_supplied,
chert_revision_number_t  revision_,
bool  create_db = false 
) [protected]

Perform the opening operation to write.

Return true iff the open succeeded.

Definition at line 1491 of file chert_table.cc.

References basic_open(), BLK_UNUSED, block_size, buffer, C, changed_c, changed_n, close(), DIR_START, handle, lazy, level, LOGCALL, Cursor::n, O_BINARY, open(), Cursor::p, read_root(), RETURN, revision_number, seq_count, SEQ_START_POINT, split_p, writable, and zeroed_new().

Referenced by create_and_open(), and open().

bool ChertTable::empty (  )  const [inline]

void ChertTable::enter_key ( int  j,
Key  prevkey,
Key  newkey 
) [protected]

enter_key(j, prevkey, newkey) is called after a block split.

It enters in the block at level C[j] a separating key for the block at level C[j - 1]. The key itself is newkey. prevkey is the preceding key, and at level 1 newkey can be trimmed down to the first point of difference to prevkey for entry in C[j].

This code looks longer than it really is. If j exceeds the number of B-tree levels the root block has split and we have to construct a new one, but this is a rare event.

The key is constructed in b, with block number C[j - 1].n as tag, and this is added in with add_item. add_item may itself cause a block split, with a further call to enter_key. Hence the recursion.

Definition at line 591 of file chert_table.cc.

References add_item(), Assert, AssertRel, Cursor::c, C, C2, D2, find_in_block(), Key::get_address(), getint4(), I2, K1, Item_base< T >::key(), Key::length(), LOGCALL_VOID, Cursor::n, Cursor::p, Cursor::rewrite, Item_wr::set_key_and_block(), SET_TOTAL_FREE, TOTAL_FREE, and writable.

Referenced by add_item().

void ChertTable::erase (  ) 

Erase this table from disk.

Definition at line 1685 of file chert_table.cc.

References close(), io_unlink(), and LOGCALL_VOID.

Referenced by compact_chert(), ChertLazyTable::create_and_open(), and main().

bool ChertTable::exists (  )  const

Determine whether the btree exists on disk.

Definition at line 1678 of file chert_table.cc.

References file_exists(), and LOGCALL.

Referenced by ChertDatabase::database_exists().

bool ChertTable::find ( Cursor C_  )  const [protected]

find(C_) searches for the key of B->kt in the B-tree.

Result is true if found, false otherwise. When false, the B_tree cursor is positioned at the last key in the B-tree <= the search key. Goes to first (null) item in B-tree when key length == 0.

Definition at line 488 of file chert_table.cc.

References block_given_by(), block_to_cursor(), DIR_START, find_in_block(), Item_base< T >::key(), kt, level, LOGCALL, Key::p, and RETURN.

Referenced by add(), delete_kt(), ChertCursor::find_entry(), ChertCursor::find_entry_ge(), get_exact_entry(), and key_exists().

int ChertTable::find_in_block ( const byte p,
Key  key,
bool  leaf,
int  c 
) [static, protected]

find_in_block(p, key, leaf, c) searches for the key in the block at p.

leaf is true for a data block, and false for an index block (when the first key is dummy and never needs to be tested). What we get is the directory entry to the last key <= the key being searched for.

The lookup is by binary chop, with i and j set to the left and right ends of the search area. In sequential addition, c will often be the answer, so we test the keys round c and move i and j towards c if possible.

Definition at line 458 of file chert_table.cc.

References D2, DIR_END, DIR_START, Key::get_address(), LOGCALL_STATIC, and RETURN.

Referenced by enter_key(), and find().

void ChertTable::flush_db (  ) 

Flush any outstanding changes to the DB file of the table.

This must be called before commit, to ensure that the DB file is ready to be switched to a new version by the commit.

Reimplemented in ChertSpellingTable, and ChertSynonymTable.

Definition at line 1790 of file chert_table.cc.

References Assert, Btree_modified, C, faked_root_block, handle, level, LOGCALL_VOID, writable, and write_block().

Referenced by compact_chert(), main(), ChertCompact::multimerge_postlists(), and ChertDatabase::set_revision_number().

void ChertTable::form_key ( const std::string &  key  )  const [protected]

unsigned int ChertTable::get_block_size (  )  const [inline]

Get the block size.

Definition at line 499 of file chert_table.h.

Referenced by ChertDatabase::open_tables(), and ChertDatabase::open_tables_consistent().

chert_tablesize_t ChertTable::get_entry_count (  )  const [inline]

Return a count of the number of entries in the table.

The count does not include the ever-present item with null key.

Use empty() if you only want to know if the table is empty or not.

Returns:
The number of entries in the table.

Definition at line 563 of file chert_table.h.

Referenced by ChertRecordTable::get_doccount().

bool ChertTable::get_exact_entry ( const std::string &  key,
std::string &  tag 
) const

Read an entry from the table, if and only if it is exactly that being asked for.

If the key is found in the table, then the tag is copied to tag. If the key is not found tag is left unchanged.

The result is true iff the specified key is found in the Btree.

Parameters:
key The key to look for in the table.
tag A tag object to fill with the value if found.
Returns:
true if key is found in table, false if key is not found in table.

Definition at line 1177 of file chert_table.cc.

References Assert, C, CHERT_BTREE_MAX_KEY_LEN, find(), form_key(), handle, LOGCALL, read_tag(), and RETURN.

Referenced by ChertSynonymTable::add_synonym(), ChertSpellingTable::add_word(), ChertValueManager::delete_document(), Chert::PostlistChunkWriter::flush(), ChertValueManager::get_all_values(), ChertPostListTable::get_collection_freq(), ChertDatabase::get_metadata(), ChertRecordTable::get_record(), ChertPostListTable::get_termfreq(), ChertValueManager::get_value_stats(), ChertSpellingTable::get_word_frequency(), ChertSpellingTable::merge_changes(), ChertPostListTable::merge_changes(), ChertSynonymTable::open_termlist(), ChertSpellingTable::open_termlist(), ChertPositionListTable::positionlist_count(), ChertDatabaseStats::read(), ChertPositionList::read_data(), ChertSynonymTable::remove_synonym(), ChertSpellingTable::remove_word(), and ChertPositionListTable::set_positionlist().

chert_revision_number_t ChertTable::get_latest_revision_number (  )  const [inline]

Get the latest revision number stored in this table.

This gives the higher of the revision numbers held in the base files of the B-tree, or just the revision number if there's only one base file.

It is possible that there are other, older, revisions of this table available, and indeed that the revision currently open is one of these older revisions.

Definition at line 538 of file chert_table.h.

Referenced by ChertDatabase::ChertDatabase(), and ChertDatabase::get_next_revision_number().

chert_revision_number_t ChertTable::get_open_revision_number (  )  const [inline]

Get the revision number at which this table is currently open.

It is possible that there are other, more recent or older revisions available.

Returns:
the current revision number.

Definition at line 550 of file chert_table.h.

Referenced by ChertDatabaseReplicator::apply_changeset_from_conn(), ChertDatabase::ChertDatabase(), ChertDatabase::create_and_open_tables(), ChertDatabase::get_revision_number(), and ChertDatabase::open_tables_consistent().

bool ChertTable::is_modified (  )  const [inline]

Determine whether the object contains uncommitted modifications.

Returns:
true if there have been modifications since the last the last call to commit().

Reimplemented in ChertSpellingTable, and ChertSynonymTable.

Definition at line 593 of file chert_table.h.

Referenced by ChertDatabase::apply().

bool ChertTable::is_open (  )  const [inline]

bool ChertTable::key_exists ( const std::string &  key  )  const

Check if a key exists in the Btree.

This is just like get_exact_entry() except it doesn't read the tag value so is more efficient if you only want to check that the key exists.

Parameters:
key The key to look for in the table.
Returns:
true if key is found in table, false if key is not found in table.

Definition at line 1200 of file chert_table.cc.

References Assert, C, CHERT_BTREE_MAX_KEY_LEN, find(), form_key(), LOGCALL, and RETURN.

Referenced by ChertPostListTable::merge_changes(), and ChertPostListTable::term_exists().

void ChertTable::lazy_alloc_deflate_zstream (  )  const [protected]

Allocate the zstream for deflating, if not already allocated.

Definition at line 1603 of file chert_table.cc.

References compress_strategy, deflate_zstream, rare, Xapian::Internal::str(), and usual.

Referenced by add().

void ChertTable::lazy_alloc_inflate_zstream (  )  const [protected]

Allocate the zstream for inflating, if not already allocated.

Definition at line 1641 of file chert_table.cc.

References inflate_zstream, rare, Xapian::Internal::str(), and usual.

Referenced by read_tag().

int ChertTable::mid_point ( byte p  )  [protected]

mid_point(p) finds the directory entry in c that determines the approximate mid point of the data in the block at p.

Definition at line 653 of file chert_table.cc.

References Assert, block_size, D2, DIR_END, DIR_START, LOGCALL, RETURN, and TOTAL_FREE.

Referenced by add_item().

bool ChertTable::next ( Cursor C_,
int  j 
) const [inline, protected]

bool ChertTable::next_default ( Cursor C_,
int  j 
) const [protected]

bool ChertTable::next_for_sequential ( Cursor C_,
int  dummy 
) const [protected]

bool ChertTable::open ( chert_revision_number_t  revision_  ) 

Open the btree at a given revision.

Like Btree::open, but try to open at the given revision number and fail if that isn't possible.

Parameters:
revision_ - revision number to open.
Returns:
true if table is successfully opened at desired revision; false if table cannot be opened at desired revision (but table is otherwise consistent).
Exceptions:
Xapian::DatabaseCorruptError will be thrown if the table is in a corrupt state.
Xapian::DatabaseOpeningError will be thrown if the table cannot be opened (but is not corrupt - eg, permission problems, not present, etc).

Reimplemented in ChertPostListTable.

Definition at line 2065 of file chert_table.cc.

References AssertEq, close(), do_open_to_read(), do_open_to_write(), LOGCALL, LOGLINE, RETURN, revision_number, and writable.

void ChertTable::open (  ) 

ChertTable& ChertTable::operator= ( const ChertTable  )  [private]

Assignment not allowed.

char ChertTable::other_base_letter (  )  const [inline, protected]

Definition at line 644 of file chert_table.h.

Referenced by commit(), and write_block().

bool ChertTable::prev ( Cursor C_,
int  j 
) const [inline, protected]

Definition at line 749 of file chert_table.h.

Referenced by ChertCursor::find_entry(), and ChertCursor::prev().

bool ChertTable::prev_default ( Cursor C_,
int  j 
) const [protected]

bool ChertTable::prev_for_sequential ( Cursor C_,
int  dummy 
) const [protected]

void ChertTable::read_block ( uint4  n,
byte p 
) const [protected]

read_block(n, p) reads block n of the DB file to address p.

Definition at line 189 of file chert_table.cc.

References Assert, base, block_size, ChertTable_base::get_bit_map_size(), handle, io_read(), LOGCALL_VOID, and Xapian::Internal::str().

Referenced by block_to_cursor(), next_for_sequential(), prev_for_sequential(), and write_changed_blocks().

void ChertTable::read_root (  )  [protected]

bool ChertTable::read_tag ( Cursor C_,
std::string *  tag,
bool  keep_compressed 
) const

Read the tag value for the key pointed to by cursor C_.

Parameters:
keep_compressed Don't uncompress the tag - e.g. useful if it's just being opaquely copied.
Returns:
true if current_tag holds compressed data (always false if keep_compressed was false).

Definition at line 1213 of file chert_table.cc.

References Item_base< T >::append_chunk(), C2, Item_base< T >::components_of(), Item_base< T >::get_compressed(), I2, inflate_zstream, K1, lazy_alloc_inflate_zstream(), LOGCALL, LOGLINE, max_item_size, next(), RETURN, setint4(), and Xapian::Internal::str().

Referenced by get_exact_entry(), and ChertCursor::read_tag().

bool ChertTable::really_empty (  )  const [private]

Return true if there are no entries in the table.

Definition at line 1589 of file chert_table.cc.

References ChertCursor::find_entry(), handle, and ChertCursor::next().

void ChertTable::set_block_size ( unsigned int  block_size_  ) 

Set the block size.

It's only safe to do this before the table is created.

Definition at line 1696 of file chert_table.cc.

References block_size, BYTE_PAIR_RANGE, CHERT_DEFAULT_BLOCK_SIZE, and LOGCALL_VOID.

Referenced by compact_chert(), create_and_open(), ChertLazyTable::create_and_open(), main(), ChertDatabase::open_tables(), and ChertDatabase::open_tables_consistent().

void ChertTable::set_full_compaction ( bool  parity  ) 

Definition at line 1298 of file chert_table.cc.

References Assert, full_compaction, LOGCALL_VOID, seq_count, and writable.

Referenced by compact_chert(), and main().

void ChertTable::set_max_item_size ( size_t  block_capacity  )  [inline]

Set the maximum item size given the block capacity.

At least this many items of maximum size must fit into a block. The default is BLOCK_CAPACITY (which is currently 4).

Definition at line 600 of file chert_table.h.

References BLOCK_CAPACITY, D2, and DIR_START.

Referenced by basic_open(), and compact_chert().

void ChertTable::split_root ( uint4  split_n  )  [protected]

Btree needs to gain a new level to insert more items: so split root block and construct a new one.

Definition at line 546 of file chert_table.cc.

References add_item(), base, block_size, BTREE_CURSOR_LEVELS, Cursor::c, C, compact(), DIR_START, Item_wr::form_null_key(), latest_revision_number, level, LOGCALL_VOID, Cursor::n, ChertTable_base::next_free_block(), Cursor::p, Cursor::rewrite, SET_DIR_END, SET_LEVEL, SET_REVISION, STRINGIZE, and zeroed_new().

Referenced by add_item().

void ChertTable::write_block ( uint4  n,
const byte p 
) const [protected]

write_block(n, p) writes block n in the DB file from address p.

When writing we check to see if the DB file has already been modified. If not (so this is the first write) the old base is deleted. This prevents the possibility of it being opened subsequently as an invalid base.

Definition at line 241 of file chert_table.cc.

References Assert, AssertEqParanoid, AssertParanoid, base, ChertTable_base::block_free_at_start(), block_size, both_bases, ChertTable_base::get_bit_map_size(), handle, io_unlink(), io_write(), latest_revision_number, LOGCALL_VOID, other_base_letter(), REVISION, revision_number, and writable.

Referenced by add_item(), block_to_cursor(), and flush_db().

void ChertTable::write_changed_blocks ( int  changes_fd  ) 

Append the list of blocks changed to a changeset file.

Parameters:
changes_fd The file descriptor to write changes to.

Definition at line 1911 of file chert_table.cc.

References Assert, base, block_size, ChertTable_base::calculate_last_block(), faked_root_block, ChertTable_base::find_changed_block(), handle, io_write(), LOGCALL_VOID, pack_string(), pack_uint(), read_block(), and tablename.

Referenced by ChertDatabase::set_revision_number().

ChertTable::XAPIAN_NORETURN ( void set_overwritten()  const  )  [protected]

ChertTable::XAPIAN_NORETURN ( static void   throw_database_closed()  ) 

Throw an exception indicating that the database is closed.


Friends And Related Function Documentation

friend class ChertCursor [friend]

Definition at line 290 of file chert_table.h.

Referenced by cursor_get().


Member Data Documentation

char ChertTable::base_letter [protected]

the value 'A' or 'B' of the current base

Definition at line 678 of file chert_table.h.

Referenced by basic_open(), cancel(), ChertTableCheck::check(), and commit().

unsigned int ChertTable::block_size [protected]

bool ChertTable::both_bases [mutable, protected]

set to true if baseA and baseB both exist as valid bases.

The unused base is deleted as soon as a write to the Btree takes place.

Definition at line 675 of file chert_table.h.

Referenced by basic_open(), commit(), and write_block().

bool ChertTable::Btree_modified [mutable, protected]

Set to true the first time the B-tree is modified.

Definition at line 734 of file chert_table.h.

Referenced by add(), cancel(), commit(), del(), and flush_db().

byte* ChertTable::buffer [protected]

buffer of size block_size for reforming blocks

Definition at line 710 of file chert_table.h.

Referenced by close(), compact(), and do_open_to_write().

Cursor ChertTable::C[BTREE_CURSOR_LEVELS] [mutable, protected]

int ChertTable::changed_c [protected]

directory offset corresponding to last block to be changed by an addition

Definition at line 728 of file chert_table.h.

Referenced by add_item(), add_kt(), cancel(), commit(), and do_open_to_write().

uint4 ChertTable::changed_n [protected]

the last block to be changed by an addition

Definition at line 724 of file chert_table.h.

Referenced by add_item(), add_kt(), cancel(), commit(), and do_open_to_write().

DONT_COMPRESS or Z_DEFAULT_STRATEGY, Z_FILTERED, Z_HUFFMAN_ONLY, Z_RLE.

Definition at line 785 of file chert_table.h.

Referenced by add(), and lazy_alloc_deflate_zstream().

Flag for tracking when cursors need to rebuild.

Definition at line 743 of file chert_table.h.

Referenced by add(), ChertCursor::ChertCursor(), and del().

unsigned long ChertTable::cursor_version [protected]

Version count for tracking when cursors need to rebuild.

Definition at line 746 of file chert_table.h.

Referenced by add(), del(), ChertCursor::find_entry(), ChertCursor::find_entry_ge(), ChertCursor::next(), ChertCursor::prev(), and ChertCursor::rebuild().

z_stream* ChertTable::deflate_zstream [mutable, protected]

Zlib state object for deflating.

Definition at line 788 of file chert_table.h.

Referenced by add(), lazy_alloc_deflate_zstream(), and ~ChertTable().

bool ChertTable::faked_root_block [protected]

true if the root block is faked (not written to disk).

false otherwise. This is true when the btree hasn't been modified yet.

Definition at line 684 of file chert_table.h.

Referenced by basic_open(), cancel(), ChertTableCheck::check(), commit(), flush_db(), read_root(), and write_changed_blocks().

bool ChertTable::full_compaction [protected]

set to true when full compaction is to be achieved

Definition at line 737 of file chert_table.h.

Referenced by add(), and set_full_compaction().

int ChertTable::handle [protected]

File descriptor of the table.

If the table is lazily created and doesn't yet exist, this will be -1.

If close() has been called, this will be -2.

Definition at line 698 of file chert_table.h.

Referenced by add(), basic_open(), cancel(), close(), commit(), create_and_open(), cursor_get(), del(), do_open_to_read(), do_open_to_write(), flush_db(), get_exact_entry(), read_block(), really_empty(), write_block(), and write_changed_blocks().

z_stream* ChertTable::inflate_zstream [mutable, protected]

Zlib state object for inflating.

Definition at line 791 of file chert_table.h.

Referenced by lazy_alloc_inflate_zstream(), read_tag(), and ~ChertTable().

keeps a count of the number of items in the B-tree.

Definition at line 661 of file chert_table.h.

Referenced by add(), basic_open(), cancel(), ChertTableCheck::check(), commit(), and del().

Item_wr ChertTable::kt [mutable, protected]

buffer of size block_size for making up key-tag items

Definition at line 707 of file chert_table.h.

Referenced by add(), add_kt(), basic_open(), close(), del(), find(), and form_key().

Revision number of the other base, or zero if there is only one base file.

Definition at line 669 of file chert_table.h.

Referenced by alter(), basic_open(), cancel(), commit(), next_for_sequential(), prev_for_sequential(), read_root(), split_root(), and write_block().

bool ChertTable::lazy [protected]

If true, don't create the table until it's needed.

Definition at line 794 of file chert_table.h.

Referenced by compact_chert(), do_open_to_read(), and do_open_to_write().

int ChertTable::level [protected]

size_t ChertTable::max_item_size [protected]

maximum size of an item (key-tag pair)

Definition at line 731 of file chert_table.h.

Referenced by add(), and read_tag().

std::string ChertTable::name [protected]

The path name of the B tree.

Definition at line 716 of file chert_table.h.

uint4 ChertTable::root [protected]

the root block of the B-tree

Definition at line 704 of file chert_table.h.

Referenced by basic_open(), cancel(), commit(), and read_root().

int ChertTable::seq_count [protected]

count of the number of successive instances of purely sequential addition, starting at SEQ_START_POINT (neg) and going up to zero.

Definition at line 721 of file chert_table.h.

Referenced by add_item(), add_kt(), cancel(), commit(), delete_kt(), do_open_to_write(), and set_full_compaction().

bool ChertTable::sequential [protected]

true iff the data has been written in a single write in sequential order.

Definition at line 689 of file chert_table.h.

Referenced by add_kt(), basic_open(), cancel(), commit(), and delete_kt().

byte* ChertTable::split_p [protected]

Buffer used when splitting a block.

This buffer holds the split off part of the block. It's only used when updating (in ChertTable::add_item().

Definition at line 781 of file chert_table.h.

Referenced by add_item(), close(), and do_open_to_write().

const char* ChertTable::tablename [protected]

The name of the table (used when writing changesets).

Definition at line 649 of file chert_table.h.

Referenced by commit(), and write_changed_blocks().

bool ChertTable::writable [protected]


The documentation for this class was generated from the following files:

Documentation for Xapian (version 1.2.13).
Generated on 9 Jan 2013 by Doxygen 1.5.9.