a generic test suite engine More...

#include "noreturn.h"
#include "output.h"
#include "stringutils.h"
#include <iomanip>
#include <map>
#include <sstream>
#include <string>
#include <vector>
#include <cfloat>
#include "testmacros.h"

Include dependency graph for testsuite.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
class	TestFail
	Class which is thrown when a test case fails. More...

class	TestSkip
	Class which is thrown when a test case is to be skipped. More...

struct	test_desc
	Structure holding a description of a test. More...

class	test_driver
	The test driver. This class takes care of running the tests. More...

struct	test_driver::result
	A structure used to report the summary of tests passed and failed. More...

Macros
#define	UNITTEST_CHECK_EXCEPTION

#define	THROW_TEST_(EXCEPTION, MSG)
	Helper macro. More...

#define	FAIL_TEST(MSG) THROW_TEST_(TestFail, MSG)
	Fail the current testcase with message MSG. More...

#define	SKIP_TEST(MSG) THROW_TEST_(TestSkip, MSG)
	Skip the current testcase with message MSG. More...

#define	TEST_AND_EXPLAIN(a, b)
	Test a condition, and display the test with an extra explanation if the condition fails. More...

#define	TEST(a) TEST_AND_EXPLAIN(a, "")
	Test a condition, without an additional explanation for failure. More...

#define	TEST_EQUAL(a, b)
	Test for equality of two things. More...

#define	TEST_STRINGS_EQUAL(a, b)
	Test for equality of two strings. More...

#define	TEST_EQUAL_DOUBLE(a, b)
	Test two doubles for near equality. More...

#define	TEST_NOT_EQUAL_DOUBLE(a, b)
	Test two doubles for non-near-equality. More...

#define	TEST_NOT_EQUAL(a, b)
	Test for non-equality of two things. More...

#define	DEFINE_TESTCASE(S, COND) void test_##S()

Functions
bool	TEST_EQUAL_DOUBLE_ (double a, double b)
	Helper function for TEST_EQUAL_DOUBLE macro. More...

void	XFAIL (const char *msg)
	Mark a testcase as expected to fail. More...

Variables
int	verbose
	The global verbose flag. More...

const char *	expected_failure
	Set to a string explanation for testcases expected to fail. More...

const char *	expected_exception
	The exception type we were expecting in TEST_EXCEPTION. More...

std::ostringstream	tout
	The output stream. More...

Detailed Description

a generic test suite engine

Definition in file testsuite.h.

Macro Definition Documentation

◆ DEFINE_TESTCASE

#define DEFINE_TESTCASE	(	S,
		COND
	)	void test_##S()

Definition at line 309 of file testsuite.h.

◆ FAIL_TEST

#define FAIL_TEST ( MSG ) THROW_TEST_(TestFail, MSG)

Fail the current testcase with message MSG.

MSG is written to an std::ostream and so can contain <<.

Definition at line 68 of file testsuite.h.

Referenced by apply_changeset(), DEFINE_TESTCASE(), do_read(), do_write(), get_changeset(), CPUTimer::get_current_cputime(), EstimatePS::get_docid(), get_file_size(), MyDontAskWeightPostingSource::get_weight(), ChangeMaxweightPostingSource::get_weight(), make_sparse_db(), make_tg_db(), mktmpdir(), EstimatePS::next(), NeverUseMeKeyMaker::operator()(), EstimatePS::skip_to(), test_stemdict(), test_stemjunk(), test_stemrandom(), test_stringcomp1(), and truncated_copy().

◆ SKIP_TEST

#define SKIP_TEST ( MSG ) THROW_TEST_(TestSkip, MSG)

Skip the current testcase with message MSG.

MSG is written to an std::ostream and so can contain <<.

Definition at line 74 of file testsuite.h.

Referenced by DEFINE_TESTCASE(), CPUTimer::get_current_cputime(), qp_scale1_helper(), skip_test_for_backend(), skip_test_unless_backend(), test_ioblock1(), test_scalability(), and test_stemdict().

◆ TEST

#define TEST ( a ) TEST_AND_EXPLAIN(a, "")

Test a condition, without an additional explanation for failure.

Definition at line 275 of file testsuite.h.

Referenced by check_double_serialisation(), check_msets_contain_same_docs(), dbcheck(), DEFINE_TESTCASE(), parsesigned_helper(), parseunsigned_helper(), test_addoverflows1(), test_autoptr1(), test_chartype1(), test_closefrom1(), test_ioblock1(), test_muloverflows1(), test_pack_uint_preserving_sort1(), test_pack_uint_preserving_sort2(), test_serialiseerror1(), test_serialiselength2(), test_suboverflows1(), and test_uuid1().

◆ TEST_AND_EXPLAIN

#define TEST_AND_EXPLAIN	(	a,
		b
	)

Value:

do {\
        bool test_and_explain_fail_ = !(a);\
	UNITTEST_CHECK_EXCEPTION\
	if (test_and_explain_fail_)\
            FAIL_TEST(STRINGIZE(a) << '\n' << b << '\n');\
    } while (0)

Test a condition, and display the test with an extra explanation if the condition fails.

NB: wrapped in do { ... } while (0) so a trailing ';' works correctly.

Definition at line 267 of file testsuite.h.

Referenced by DEFINE_TESTCASE(), mset_expect_order_(), mset_range_is_same(), mset_range_is_same_weights(), test_mset_order_equal(), test_refcnt1(), test_refcnt2(), and test_sortableserialise1().

◆ TEST_EQUAL

#define TEST_EQUAL	(	a,
		b
	)

Value:

TEST_AND_EXPLAIN(((a) == (b)), \
        "Expected '" STRINGIZE(a) "' and '" STRINGIZE(b) "' to be equal:" \
        " were " << (a) << " and " << (b))

Test for equality of two things.

Definition at line 278 of file testsuite.h.

Referenced by apply_changeset(), check_double_serialisation(), check_equal_dbs(), check_msets_contain_same_docs(), check_sparse_uid_terms(), check_vals(), dbcheck(), DEFINE_TESTCASE(), DEFINE_TESTCASE_(), get_changeset(), CheckStatsWeight::get_sumpart(), parsesigned_helper(), parseunsigned_helper(), closedb1_iterators::perform(), replicate_with_brokenness(), test_addoverflows1(), test_autoptr1(), test_chartype1(), test_closefrom1(), test_emptyterm1_helper(), test_emptyterm2_helper(), test_exception1(), test_ioblock1(), test_log2(), test_movesupport1(), test_muloverflows1(), test_pack_uint_preserving_sort1(), test_pack_uint_preserving_sort2(), test_refcnt1(), test_serialiselength1(), test_serialiselength2(), test_sortableserialise1(), test_stemdict(), test_strbool1(), test_suboverflows1(), test_temporarydtor1(), test_tostring1(), test_uuid1(), and tostring_helper().

◆ TEST_EQUAL_DOUBLE

#define TEST_EQUAL_DOUBLE	(	a,
		b
	)

Value:

TEST_AND_EXPLAIN(TEST_EQUAL_DOUBLE_((a), (b)), \
        "Expected '" STRINGIZE(a) "' and '" STRINGIZE(b) "' to be (nearly) equal:" \
        " were " << setprecision(DBL_DIG) << (a) << " and " << (b) << ")" << setprecision(6))

Test two doubles for near equality.

Definition at line 295 of file testsuite.h.

Referenced by dbcheck(), and DEFINE_TESTCASE().

◆ TEST_NOT_EQUAL

#define TEST_NOT_EQUAL	(	a,
		b
	)

Value:

TEST_AND_EXPLAIN(((a) != (b)), \
        "Expected '" STRINGIZE(a) "' and '" STRINGIZE(b) "' not to be equal:" \
        " were " << (a) << " and " << (b))

Test for non-equality of two things.

Definition at line 305 of file testsuite.h.

Referenced by check_vals(), DEFINE_TESTCASE(), closedb1_iterators::perform(), and test_uuid1().

◆ TEST_NOT_EQUAL_DOUBLE

#define TEST_NOT_EQUAL_DOUBLE	(	a,
		b
	)

Value:

TEST_AND_EXPLAIN(!TEST_EQUAL_DOUBLE_((a), (b)), \
        "Expected '" STRINGIZE(a) "' and '" STRINGIZE(b) "' not to be (nearly) equal:" \
        " were " << setprecision(DBL_DIG) << (a) << " and " << (b) << ")" << setprecision(6))

Test two doubles for non-near-equality.

Definition at line 300 of file testsuite.h.

Referenced by DEFINE_TESTCASE().

◆ TEST_STRINGS_EQUAL

#define TEST_STRINGS_EQUAL	(	a,
		b
	)

Value:

TEST_AND_EXPLAIN(((a) == (b)), \
        "Expected " STRINGIZE(a) " and " STRINGIZE(b) " to be equal, were:\n\"" \
        << (a) << "\"\n\"" << (b) << '"')

Test for equality of two strings.

If they aren't equal, show each on a separate line so the difference can be seen clearly.

Definition at line 287 of file testsuite.h.

Referenced by DEFINE_TESTCASE(), and test_serialiseerror1().

◆ THROW_TEST_

#define THROW_TEST_	(	EXCEPTION,
		MSG
	)

Value:

do { \
        if (verbose) { \
            tout << __FILE__ ":" STRINGIZE(__LINE__) ": " << MSG << '\n'; \
        } \
        throw EXCEPTION(); \
    } while (0)

Helper macro.

Definition at line 56 of file testsuite.h.

◆ UNITTEST_CHECK_EXCEPTION

#define UNITTEST_CHECK_EXCEPTION

Definition at line 31 of file testsuite.h.

Function Documentation

◆ TEST_EQUAL_DOUBLE_()

bool TEST_EQUAL_DOUBLE_	(	double	a,
		double	b
	)

Helper function for TEST_EQUAL_DOUBLE macro.

Definition at line 907 of file testsuite.cc.

Referenced by mset_range_is_same(), and mset_range_is_same_weights().

◆ XFAIL()

void XFAIL ( const char * msg )

inline

Mark a testcase as expected to fail.

Parameters

msg	An static string explaining why the testcase is expected to fail. Must not be NULL.

This is intended to be used temporarily to mark tests for known bugs before the bugs are fixed. If the test fails, the result will be shown as "XFAIL" and this won't cause the test run to fail. However, if a test marked in this way actually passed, the result will be shown as "XPASS" and the test run will fail. (So XFAIL is explicitly not suitable for marking "flaky" testcases - please fix flaky testcases rather than trying to find a way to mark them as flaky!)

This macro should be used inside the testcase code. It can be used inside a conditional if the testcase is only expected to fail in certain situations (for example, only for some backends) - it only has an effect if it is actually executed.

Definition at line 332 of file testsuite.h.

References expected_failure.

Referenced by DEFINE_TESTCASE(), and XFAIL_FOR_BACKEND().

Variable Documentation

◆ expected_exception

const char* expected_exception

The exception type we were expecting in TEST_EXCEPTION.

Definition at line 98 of file testsuite.cc.

Referenced by test_driver::runtest().

◆ expected_failure

const char* expected_failure

Set to a string explanation for testcases expected to fail.

Definition at line 100 of file testsuite.cc.

Referenced by test_driver::runtest(), and XFAIL().

◆ tout

std::ostringstream tout

The output stream.

Data written to this stream will only appear when a test fails.

The output stream.

Definition at line 103 of file testsuite.cc.

Referenced by check_vals(), DEFINE_TESTCASE(), CheckStatsWeight::get_sumpart(), initrand(), make_sparse_db(), mset_range_is_same(), mset_range_is_same_weights(), parsesigned_helper(), parseunsigned_helper(), print_mset_percentages(), print_mset_weights(), qp_scale1_helper(), replicate_with_brokenness(), test_driver::runtest(), test_autoptr::test_autoptr(), test_driver::test_driver(), test_refcnt::test_refcnt(), test_scalability(), test_serialiselength1(), test_sortableserialise1(), test_stemdict(), test_stemjunk(), test_stemrandom(), tostring_helper(), test_driver::write_and_clear_tout(), test_autoptr::~test_autoptr(), and test_refcnt::~test_refcnt().

◆ verbose

int verbose

The global verbose flag.

Definition at line 78 of file testsuite.cc.

Referenced by test_driver::do_run_tests(), test_driver::parse_command_line(), and test_driver::runtest().

Classes

Macros

Functions

Variables

Detailed Description

Macro Definition Documentation

◆ DEFINE_TESTCASE

◆ FAIL_TEST

◆ SKIP_TEST

◆ TEST

◆ TEST_AND_EXPLAIN

◆ TEST_EQUAL

◆ TEST_EQUAL_DOUBLE

◆ TEST_NOT_EQUAL

◆ TEST_NOT_EQUAL_DOUBLE

◆ TEST_STRINGS_EQUAL

◆ THROW_TEST_

◆ UNITTEST_CHECK_EXCEPTION

Function Documentation

◆ TEST_EQUAL_DOUBLE_()

◆ XFAIL()

Variable Documentation

◆ expected_exception

◆ expected_failure

◆ tout

◆ verbose