diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/Readme.md | 19 | ||||
| -rw-r--r-- | docs/assertions.md | 82 | ||||
| -rw-r--r-- | docs/build-systems.md | 86 | ||||
| -rw-r--r-- | docs/command-line.md | 220 | ||||
| -rw-r--r-- | docs/configuration.md | 73 | ||||
| -rw-r--r-- | docs/contributing.md | 23 | ||||
| -rw-r--r-- | docs/logging.md | 52 | ||||
| -rw-r--r-- | docs/own-main.md | 68 | ||||
| -rw-r--r-- | docs/slow-compiles.md | 22 | ||||
| -rw-r--r-- | docs/test-cases-and-sections.md | 86 | ||||
| -rw-r--r-- | docs/test-fixtures.md | 32 | ||||
| -rw-r--r-- | docs/tostring.md | 70 | ||||
| -rw-r--r-- | docs/tutorial.md | 249 | ||||
| -rw-r--r-- | docs/why-catch.md | 42 |
14 files changed, 1124 insertions, 0 deletions
diff --git a/docs/Readme.md b/docs/Readme.md new file mode 100644 index 0000000..6621fb8 --- /dev/null +++ b/docs/Readme.md @@ -0,0 +1,19 @@ +These are the currently documented areas of the framework. There is more to come. + +Before looking at this material be sure to read the [tutorial](tutorial.md) + +* [Assertion macros](assertions.md) +* [Logging macros](logging.md) +* [Test cases and sections](test-cases-and-sections.md) +* [Test fixtures](test-fixtures.md) +* [Command line](command-line.md) +* [Build systems](build-systems.md) +* [Supplying your own main()](own-main.md) +* [Configuration](configuration.md) +* [String Conversions](tostring.md) +* [Why are my tests slow to compile?](slow-compiles.md) + +Other + +* [Why Catch?](why-catch.md) +* [Contributing](contributing.md) diff --git a/docs/assertions.md b/docs/assertions.md new file mode 100644 index 0000000..82bb96a --- /dev/null +++ b/docs/assertions.md @@ -0,0 +1,82 @@ +# Assertion Macros + +Most test frameworks have a large collection of assertion macros to capture all possible conditional forms (```_EQUALS```, ```_NOTEQUALS```, ```_GREATER_THAN``` etc). + +Catch is different. Because it decomposes natural C-style conditional expressions most of these forms are reduced to one or two that you will use all the time. That said there are a rich set of auxilliary macros as well. We'll describe all of these here. + +Most of these macros come in two forms: + +## Natural Expressions + +The ```REQUIRE``` family of macros tests an expression and aborts the test case if it fails. +The ```CHECK``` family are equivalent but execution continues in the same test case even if the assertion fails. This is useful if you have a series of essentially orthogonal assertions and it is useful to see all the results rather than stopping at the first failure. + +* **REQUIRE(** _expression_ **)** and +* **CHECK(** _expression_ **)** + +Evaluates the expression and records the result. If an exception is thrown it is caught, reported, and counted as a failure. These are the macros you will use most of the time + +Examples: +``` +CHECK( str == "string value" ); +CHECK( thisReturnsTrue() ); +REQUIRE( i == 42 ); +``` + +* **REQUIRE_FALSE(** _expression_ **)** and +* **CHECK_FALSE(** _expression_ **)** + +Evaluates the expression and records the _logical NOT_ of the result. If an exception is thrown it is caught, reported, and counted as a failure. +(these forms exist as a workaround for the fact that ! prefixed expressions cannot be decomposed). + +Example: +``` +REQUIRE_FALSE( thisReturnsFalse() ); +``` + +### Floating point comparisons + +When comparing floating point numbers - especially if at least one of them has been computed - great care must be taken to allow for rounding errors and inexact representations. + +Catch provides a way to perform tolerant comparisons of floating point values through use of a wrapper class called ```Approx```. ```Approx``` can be used on either side of a comparison expression. It overloads the comparisons operators to take a tolerance into account. Here's a simple example: + +``` +REQUIRE( performComputation() == Approx( 2.1 ) ); +``` + +By default a small epsilon value is used that covers many simple cases of rounding errors. When this is insufficent the epsilon value (the amount within which a difference either way is ignored) can be specified by calling the ```epsilon()``` method on the ```Approx``` instance. e.g.: + +``` +REQUIRE( 22/7 == Approx( 3.141 ).epsilon( 0.01 ) ); +``` + +When dealing with very large or very small numbers it can be useful to specify a scale, which can be achieved by calling the ```scale()``` method on the ```Approx``` instance. + +## Exceptions + +* **REQUIRE_THROWS(** _expression_ **)** and +* **CHECK_THROWS(** _expression_ **)** + +Expects that an exception (of any type) is be thrown during evaluation of the expression. + +* **REQUIRE_THROWS_AS(** _expression_, _exception type_ **)** and +* **CHECK_THROWS_AS(** _expression_, _exception type_ **)** + +Expects that an exception of the _specified type_ is thrown during evaluation of the expression. + +* **REQUIRE_NOTHROW(** _expression_ **)** and +* **CHECK_NOTHROW(** _expression_ **)** + +Expects that no exception is thrown during evaluation of the expression. + +## Matcher expressions + +To support Matchers a slightly different form is used. Matchers will be more fully documented elsewhere. *Note that Matchers are still at early stage development and are subject to change.* + +* **REQUIRE_THAT(** _lhs_, _matcher call_ **)** and +* **CHECK_THAT(** _lhs_, _matcher call_ **)** + + +--- + +[Home](Readme.md)
\ No newline at end of file diff --git a/docs/build-systems.md b/docs/build-systems.md new file mode 100644 index 0000000..b001dc1 --- /dev/null +++ b/docs/build-systems.md @@ -0,0 +1,86 @@ +# Integration with build systems + +Build Systems may refer to low-level tools, like CMake, or larger systems that run on servers, like Jenkins or TeamCity. This page will talk about both. + +# Continuous Integration systems + +Probably the most important aspect to using Catch with a build server is the use of different reporters. Catch comes bundled with three reporters that should cover the majority of build servers out there - although adding more for better integration with some is always a possibility (as has been done with TeamCity). + +Two of these reporters are built in (XML and JUnit) and the third (TeamCity) is included as a separate header. It's possible that the other two may be split out in the future too - as that would make the core of Catch smaller for those that don't need them. + +## XML Reporter +```-r xml``` + +The XML Reporter writes in an XML format that is specific to Catch. + +The advantage of this format is that it corresponds well to the way Catch works (especially the more unusual features, such as nested sections) and is a fully streaming format - that is it writes output as it goes, without having to store up all its results before it can start writing. + +The disadvantage is that, being specific to Catch, no existing build servers understand the format natively. It can be used as input to an XSLT transformation that could covert it to, say, HTML - although this loses the streaming advantage, of course. + +## JUnit Reporter +```-r junit``` + +The JUnit Reporter writes in an XML format that mimics the JUnit ANT schema. + +The advantage of this format is that the JUnit Ant schema is widely understood by most build servers and so can usually be consumed with no additional work. + +The disadvantage is that this schema was designed to correspond to how JUnit works - and there is a significant mismatch with how Catch works. Additionally the format is not streamable (because opening elements hold counts of failed and passing tests as attributes) - so the whole test run must complete before it can be written. + +## TeamCity Reporter +```-r teamcity``` + +The TeamCity Reporter writes TeamCity service messages to stdout. In order to be able to use this reporter an additional header must also be included. + +```catch_reporter_teamcity.hpp``` can be found in the ```include\reporters``` directory. It should be included in the same file that ```#define```s ```CATCH_CONFIG_MAIN``` or ```CATCH_CONFIG_RUNNER```. The ```#include``` should be placed after ```#include```ing Catch itself. + +e.g.: + +``` +#define CATCH_CONFIG_MAIN +#include "catch.hpp" +#include "catch_reporter_teamcity.hpp" +``` + +Being specific to TeamCity this is the best reporter to use with it - but it is completely unsuitable for any other purpose. It is a streaming format (it writes as it goes) - although test results don't appear in the TeamCity interface until the completion of a suite (usually the whole test run). + +# Low-level tools + +## CMake + +You can use the following CMake script to automatically fetch Catch from github and configure it as an external project: + +```CMake +cmake_minimum_required(VERSION 2.8.8) +project(catch_builder CXX) +include(ExternalProject) +find_package(Git REQUIRED) + +ExternalProject_Add( + catch + PREFIX ${CMAKE_BINARY_DIR}/catch + GIT_REPOSITORY https://github.com/philsquared/Catch.git + TIMEOUT 10 + UPDATE_COMMAND ${GIT_EXECUTABLE} pull + CONFIGURE_COMMAND "" + BUILD_COMMAND "" + INSTALL_COMMAND "" + LOG_DOWNLOAD ON + ) + +# Expose required variable (CATCH_INCLUDE_DIR) to parent scope +ExternalProject_Get_Property(catch source_dir) +set(CATCH_INCLUDE_DIR ${source_dir}/include CACHE INTERNAL "Path to include folder for Catch") +``` + +If you put it in, e.g., `${PROJECT_SRC_DIR}/${EXT_PROJECTS_DIR}/catch/`, you can use it in your project by adding the following to your root CMake file: + +```CMake +# Includes Catch in the project: +add_subdirectory(${EXT_PROJECTS_DIR}/catch) +include_directories(${CATCH_INCLUDE_DIR} ${COMMON_INCLUDES}) +enable_testing(true) # Enables unit-testing. +``` + +--- + +[Home](Readme.md)
\ No newline at end of file diff --git a/docs/command-line.md b/docs/command-line.md new file mode 100644 index 0000000..d1d4bd4 --- /dev/null +++ b/docs/command-line.md @@ -0,0 +1,220 @@ +Catch works quite nicely without any command line options at all - but for those times when you want greater control the following options are available. +Click one of the followings links to take you straight to that option - or scroll on to browse the available options. + +<a href="#specifying-which-tests-to-run"> ` <test-spec> ...`</a><br /> +<a href="#usage"> ` -h, -?, --help`</a><br /> +<a href="#listing-available-tests-tags-or-reporters"> ` -l, --list-tests`</a><br /> +<a href="#listing-available-tests-tags-or-reporters"> ` -t, --list-tags`</a><br /> +<a href="#showing-results-for-successful-tests"> ` -s, --success`</a><br /> +<a href="#breaking-into-the-debugger"> ` -b, --break`</a><br /> +<a href="#eliding-assertions-expected-to-throw"> ` -e, --nothrow`</a><br /> +<a href="#invisibles"> ` -i, --invisibles`</a><br /> +<a href="#sending-output-to-a-file"> ` -o, --out`</a><br /> +<a href="#choosing-a-reporter-to-use"> ` -r, --reporter`</a><br /> +<a href="#naming-a-test-run"> ` -n, --name`</a><br /> +<a href="#aborting-after-a-certain-number-of-failures"> ` -a, --abort`</a><br /> +<a href="#aborting-after-a-certain-number-of-failures"> ` -x, --abortx`</a><br /> +<a href="#warnings"> ` -w, --warn`</a><br /> +<a href="#reporting-timings"> ` -d, --durations`</a><br /> +<a href="#input-file"> ` -f, --input-file`</a><br /> + +</br> + +<a href="#list-test-names-only"> ` --list-test-names-only`</a><br /> +<a href="#listing-available-tests-tags-or-reporters"> ` --list-reporters`</a><br /> +<a href="#order"> ` --order`</a><br /> +<a href="#rng-seed"> ` --rng-seed`</a><br /> + +</br> + + + +<a id="specifying-which-tests-to-run"></a> +## Specifying which tests to run + +<pre><test-spec> ...</pre> + +Test cases, wildcarded test cases, tags and tag expressions are all passed directly as arguments. Tags are distinguished by being enclosed in square brackets. + +If no test specs are supplied then all test cases, except "hidden" tests, are run. +A test is hidden by giving it any tag starting with (or just) a period (```.```) - or, in the deprecated case, tagged ```[hide]``` or given name starting with `'./'`. To specify hidden tests from the command line ```[.]``` or ```[hide]``` can be used *regardless of how they were declared*. + +Specs must be enclosed in quotes if they contain spaces. If they do not contain spaces the quotes are optional. + +Wildcards consist of the `*` character at the beginning and/or end of test case names and can substitute for any number of any characters (including none). + +Test specs are case insensitive. + +If a spec is prefixed with `exclude:` or the `~` character then the pattern matches an exclusion. This means that tests matching the pattern are excluded from the set - even if a prior inclusion spec included them. Subsequent inclusion specs will take precendence, however. +Inclusions and exclusions are evaluated in left-to-right order. + +Test case examples: + +<pre>thisTestOnly Matches the test case called, 'thisTestOnly' +"this test only" Matches the test case called, 'this test only' +these* Matches all cases starting with 'these' +exclude:notThis Matches all tests except, 'notThis' +~notThis Matches all tests except, 'notThis' +~*private* Matches all tests except those that contain 'private' +a* ~ab* abc Matches all tests that start with 'a', except those that + start with 'ab', except 'abc', which is included +</pre> + +Names within square brackets are interpreted as tags. +A series of tags form an AND expression wheras a comma-separated sequence forms an OR expression. e.g.: + +<pre>[one][two],[three]</pre> +This matches all tests tagged `[one]` and `[two]`, as well as all tests tagged `[three]` + + +<a id="choosing-a-reporter-to-use"></a> +## Choosing a reporter to use + +<pre>-r, --reporter <reporter></pre> + +A reporter is an object that formats and structures the output of running tests, and potentially summarises the results. By default a console reporter is used that writes, IDE friendly, textual output. Catch comes bundled with some alternative reporters, but more can be added in client code.<br /> +The bundled reporters are: + +<pre>-r console +-r compact +-r xml +-r junit +</pre> + +The JUnit reporter is an xml format that follows the structure of the JUnit XML Report ANT task, as consumed by a number of third-party tools, including Continuous Integration servers such as Hudson. If not otherwise needed, the standard XML reporter is preferred as this is a streaming reporter, whereas the Junit reporter needs to hold all its results until the end so it can write the overall results into attributes of the root node. + +<a id="breaking-into-the-debugger"></a> +## Breaking into the debugger +<pre>-b, --break</pre> + +In some IDEs (currently XCode and Visual Studio) it is possible for Catch to break into the debugger on a test failure. This can be very helpful during debug sessions - especially when there is more than one path through a particular test. +In addition to the command line option, ensure you have built your code with the DEBUG preprocessor symbol + +<a id="showing-results-for-successful-tests"></a> +## Showing results for successful tests +<pre>-s, --success</pre> + +Usually you only want to see reporting for failed tests. Sometimes it's useful to see *all* the output (especially when you don't trust that that test you just added worked first time!). +To see successful, as well as failing, test results just pass this option. Note that each reporter may treat this option differently. The Junit reporter, for example, logs all results regardless. + +<a id="aborting-after-a-certain-number-of-failures"></a> +## Aborting after a certain number of failures +<pre>-a, --abort +-x, --abortx [<failure threshold>] +</pre> + +If a ```REQUIRE``` assertion fails the test case aborts, but subsequent test cases are still run. +If a ```CHECK``` assertion fails even the current test case is not aborted. + +Sometimes this results in a flood of failure messages and you'd rather just see the first few. Specifying ```-a``` or ```--abort``` on its own will abort the whole test run on the first failed assertion of any kind. Use ```-x``` or ```--abortx``` followed by a number to abort after that number of assertion failures. + +<a id="listing-available-tests-tags-or-reporters"></a> +## Listing available tests, tags or reporters +<pre>-l, --list-tests +-t, --list-tags +--list-reporters +</pre> + +```-l``` or ```--list-tests`` will list all registered tests, along with any tags. +If one or more test-specs have been supplied too then only the matching tests will be listed. + +```-t``` or ```--list-tags``` lists all available tags, along with the number of test cases they match. Again, supplying test specs limits the tags that match. + +```--list-reporters``` lists the available reporters. + +<a id="sending-output-to-a-file"></a> +## Sending output to a file +<pre>-o, --out <filename> +</pre> + +Use this option to send all output to a file. By default output is sent to stdout (note that uses of stdout and stderr *from within test cases* are redirected and included in the report - so even stderr will effectively end up on stdout). + +<a id="naming-a-test-run"></a> +## Naming a test run +<pre>-n, --name <name for test run></pre> + +If a name is supplied it will be used by the reporter to provide an overall name for the test run. This can be useful if you are sending to a file, for example, and need to distinguish different test runs - either from different Catch executables or runs of the same executable with different options. If not supplied the name is defaulted to the name of the executable. + +<a id="eliding-assertions-expected-to-throw"></a> +## Eliding assertions expected to throw +<pre>-e, --nothrow</pre> + +Skips all assertions that test that an exception is thrown, e.g. ```REQUIRE_THROWS```. + +These can be a nuisance in certain debugging environments that may break when exceptions are thrown (while this is usually optional for handled exceptions, it can be useful to have enabled if you are trying to track down something unexpected). + +Sometimes exceptions are expected outside of one of the assertions that tests for them (perhaps thrown and caught within the code-under-test). The whole test case can be skipped when using ```-e``` by marking it with the ```[!throws]``` tag. + +When running with this option any throw checking assertions are skipped so as not to contribute additional noise. Be careful if this affects the behaviour of subsequent tests. + +<a id="invisibles"></a> +## Make whitespace visible +<pre>-i, --invisibles</pre> + +If a string comparison fails due to differences in whitespace - especially leading or trailing whitespace - it can be hard to see what's going on. +This option transforms tabs and newline characters into ```\t``` and ```\n``` respectively when printing. + +<a id="warnings"></a> +## Warnings +<pre>-w, --warn <warning name></pre> + +Enables reporting of warnings (only one, at time of this writing). If a warning is issued it fails the test. + +The ony available warning, presently, is ```NoAssertions```. This warning fails a test case, or (leaf) section if no assertions (```REQUIRE```/ ```CHECK``` etc) are encountered. + +<a id="reporting-timings"></a> +## Reporting timings +<pre>-d, --durations <yes/no></pre> + +When set to ```yes``` Catch will report the duration of each test case, in milliseconds. Note that it does this regardless of whether a test case passes or fails. Note, also, the certain reporters (e.g. Junit) always report test case durations regardless of this option being set or not. + +<a id="input-file"></a> +## Load test names to run from a file +<pre>-f, --input-file <filename></pre> + +Provide the name of a file that contains a list of test case names - one per line. Blank lines are skipped and anything after the comment character, ```#```, is ignored. + +A useful way to generate an initial instance of this file is to use the <a href="#list-test-names-only">list-test-names-only</a> option. This can then be manually curated to specify a specific subset of tests - or in a specific order. + +<a id="list-test-names-only"></a> +## Just test names +<pre>--list-test-names-only</pre> + +This option lists all available tests in a non-indented form, one on each line. This makes it ideal for saving to a file and feeding back into the <a href="#input-file">```-f``` or ```--input-file```</a> option. + + +<a id="order"></a> +## Specify the order test cases are run +<pre>--order <decl|lex|rand></pre> + +Test cases are ordered one of three ways: + + +### decl +Declaration order. The order the tests were originally declared in. Note that ordering between files is not guaranteed and is implementation dependent. + +### lex +Lexicographically sorted. Tests are sorted, alpha-numerically, by name. + +### rand +Randomly sorted. Test names are sorted using ```std::random_shuffle()```. By default the random number generator is seeded with 0 - and so the order is repeatable. To control the random seed see <a href="#rng-seed">rng-seed</a>. + +<a id="rng-seed"></a> +## Specify a seed for the Random Number Generator +<pre>--rng-seed <'time'|number></pre> + +Sets a seed for the random number generator using ```std::srand()```. +If a number is provided this is used directly as the seed so the random pattern is repeatable. +Alternatively if the keyword ```time``` is provided then the result of calling ```std::time(0)``` is used and so the pattern becomes unpredictable. + +In either case the actual value for the seed is printed as part of Catch's output so if an issue is discovered that is sensitive to test ordering the ordering can be reproduced - even if it was originally seeded from ```std::time(0)```. + +<a id="usage"></a> +## Usage +<pre>-h, -?, --help</pre> + +Prints the command line arguments to stdout + +--- + +[Home](Readme.md)
\ No newline at end of file diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..3305219 --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,73 @@ +Catch is designed to "just work" as much as possible. For most people the only configuration needed is telling Catch which source file should host all the implementation code (```CATCH_CONFIG_MAIN```). + +Nonetheless there are still some occasions where finer control is needed. For these occasions Catch exposes a small set of macros for configuring how it is built. + +# main()/ implementation + + CATCH_CONFIG_MAIN // Designates this as implementation file and defines main() + CATCH_CONFIG_RUNNER // Designates this as implementation file + +Although Catch is header only it still, internally, maintains a distinction between interface headers and headers that contain implementation. Only one source file in your test project should compile the implementation headers and this is controlled through the use of one of these macros - one of these identifiers should be defined before including Catch in *exactly one implementation file in your project*. + +# Prefixing Catch macros + + CATCH_CONFIG_PREFIX_ALL + +To keep test code clean and uncluttered Catch uses short macro names (e.g. ```TEST_CASE``` and ```REQUIRE```). Occasionally these may conflict with identifiers from platform headers or the system under test. In this case the above identifier can be defined. This will cause all the Catch user macros to be prefixed with ```CATCH_``` (e.g. ```CATCH_TEST_CASE``` and ```CATCH_REQUIRE```). + + +# Terminal colour + + CATCH_CONFIG_COLOUR_NONE // completely disables all text colouring + CATCH_CONFIG_COLOUR_WINDOWS // forces the Win32 console API to be used + CATCH_CONFIG_COLOUR_ANSI // forces ANSI colour codes to be used + +Yes, I am English, so I will continue to spell "colour" with a 'u'. + +When sending output to the terminal, if it detects that it can, Catch will use colourised text. On Windows the Win32 API, ```SetConsoleTextAttribute```, is used. On POSIX systems ANSI colour escape codes are inserted into the stream. + +For finer control you can define one of the above identifiers (these are mutually exclusive - but that is not checked so may behave unexpectedly if you mix them): + +Note that when ANSI colour codes are used "unistd.h" must be includable - along with a definition of ```isatty()``` + +Typically you should place the ```#define``` before #including "catch.hpp" in your main source file - but if you prefer you can define it for your whole project by whatever your IDE or build system provides for you to do so. + +# Console width + + CATCH_CONFIG_CONSOLE_WIDTH = x // where x is a number + +Catch formats output intended for the console to fit within a fixed number of characters. This is especially important as indentation is used extensively and uncontrolled line wraps break this. +By default a console width of 80 is assumed but this can be controlled by defining the above identifier to be a different value. + +# stdout + + CATCH_CONFIG_NOSTDOUT + +Catch does not use ```std::cout``` and ```std::cerr``` directly but gets them from ```Catch::cout()``` and ```Catch::cerr()``` respectively. If the above identifier is defined these functions are left unimplemented and you must implement them yourself. Their signatures are: + + std::ostream& cout(); + std::ostream& cerr(); + +This can be useful on certain platforms that do not provide ```std::cout``` and ```std::cerr```, such as certain embedded systems. + +# C++ conformance toggles + + CATCH_CONFIG_CPP11_NULLPTR // nullptr is supported? + CATCH_CONFIG_CPP11_NOEXCEPT // noexcept is supported? + CATCH_CONFIG_CPP11_GENERATED_METHODS // delete and default keywords for methods + CATCH_CONFIG_CPP11_IS_ENUM // std::is_enum is supported? + CATCH_CONFIG_CPP11_TUPLE // std::tuple is supported + CATCH_CONFIG_VARIADIC_MACROS // Usually pre-C++11 compiler extensions are sufficient + CATCH_CONFIG_CPP11_LONG_LONG // generates overloads for the long long type + CATCH_CONFIG_CPP11_OVERRIDE // CATCH_OVERRIDE expands to override (for virtual function implementations) + CATCH_CONFIG_CPP11_UNIQUE_PTR // Use std::unique_ptr instead of std::auto_ptr + +Catch has some basic compiler detection that will attempt to select the appropriate mix of these macros. However being incomplete - and often without access to the respective compilers - this detection tends to be conservative. +So overriding control is given to the user. If a compiler supports a feature (and Catch does not already detect it) then one or more of these may be defined to enable it (or suppress it, in some cases). If you do do this please raise an issue, specifying your compiler version (ideally with an idea of how to detect it) and stating that it has such support. +You may also suppress any of these features by using the `_NO_` form, e.g. `CATCH_CONFIG_CPP11_NO_NULLPTR`. + +All C++11 support can be disabled with `CATCH_CONFIG_NO_CPP11` + +--- + +[Home](Readme.md)
\ No newline at end of file diff --git a/docs/contributing.md b/docs/contributing.md new file mode 100644 index 0000000..49a663c --- /dev/null +++ b/docs/contributing.md @@ -0,0 +1,23 @@ +# Contributing to Catch + +So you want to contribute something to Catch? That's great! Whether it's a bug fix, a new feature, support for additional compilers - or just a fix to the documentation - all contributions are very welcome and very much appreciated. Of course so are bug reports and other comments and questions. + +If you are contributing to the code base there are a few simple guidelines to keep in mind. This also includes notes to help you find your way around. As this is liable to drift out of date please raise an issue or, better still, a pull request for this file, if you notice that. + +## Branches + +Ongoing development is on the "develop" branch (if there is one, currently), or on feature branches that are branched off of develop. Please target any pull requests at develop, or, for larger chunks of work, a branch off of develop. + +## Directory structure + +Users of Catch primarily use the single header version. Maintainers should work with the full source (which is still, primarily, in headers). This can be found in the ```include``` folder, but you may prefer to use one of the IDE project files (for MSVC or XCode). These can be found under ```projects/```*IDE Name*```/```*project name*. A number of contributors have proposed make files, and submitted their own versions. At some point these should be made available too. + +In addition to the include files and IDE projects there are a number of tests in cpp files. These can all be found in ```projects/SelfTest```. You'll also see a ```SurrogateCpps``` directory in there. This contains a set of cpp files that each ```#include``` a single header. While these files are not essential to compilation they help to keep the implementation headers self-contained. At time of writing this set is not complete but has reasonable coverage. If you add additional headers please try to remember to add a surrogate cpp for it. + +The other directories are ```scripts``` which contains a set of python scripts to help in testing Catch as well as generating the single include, and docs, which contains the documentation as a set of markdown files. + + *this document is in-progress...* + +--- + +[Home](Readme.md)
\ No newline at end of file diff --git a/docs/logging.md b/docs/logging.md new file mode 100644 index 0000000..a659b3e --- /dev/null +++ b/docs/logging.md @@ -0,0 +1,52 @@ +# Logging macros + +Additional messages can be logged during a test case. + +## Streaming macros + +All these macros allow heterogenous sequences of values to be streaming using the insertion operator (```<<```) in the same way that std::ostream, std::cout, etc support it. + +E.g.: +```c++ +INFO( "The number is " << i ); +``` + +(Note that there is no initial ```<<``` - instead the insertion sequence is placed in parentheses.) +These macros come in three forms: + +**INFO(** _message expression_ **)** + +The message is logged to a buffer, but only reported with the next assertion that is logged. This allows you to log contextual information in case of failures which is not shown during a successful test run (for the console reporter, without -s). Messages are removed from the buffer at the end of their scope, so may be used, for example, in loops. + +**WARN(** _message expression_ **)** + +The message is always reported but does not fail the test. + +**FAIL(** _message expression_ **)** + +The message is reported and the test case fails. + +## Quickly capture a variable value + +**CAPTURE(** _expression_ **)** + +Sometimes you just want to log the name and value of a variable. While you can easily do this with the INFO macro, above, as a convenience the CAPTURE macro handles the stringising of the variable name for you (actually it works with any expression, not just variables). + +E.g. +```c++ +CAPTURE( theAnswer ); +``` + +This would log something like: + +<pre>"theAnswer := 42"</pre> + +## Deprecated macros + +**SCOPED_INFO and SCOPED_CAPTURE** + +These macros are now deprecated and are just aliases for INFO and CAPTURE (which were not previously scoped). + +--- + +[Home](Readme.md)
\ No newline at end of file diff --git a/docs/own-main.md b/docs/own-main.md new file mode 100644 index 0000000..f8c836e --- /dev/null +++ b/docs/own-main.md @@ -0,0 +1,68 @@ +# Supplying main() yourself + +The easiest way to use Catch is to let it supply ```main()``` for you and handle configuring itself from the command line. + +This is achieved by writing ```#define CATCH_CONFIG_MAIN``` before the ```#include "catch.hpp"``` in *exactly one* source file. + +Sometimes, though, you need to write your own version of main(). You can do this by writing ```#define CATCH_CONFIG_RUNNER``` instead. Now you are free to write ```main()``` as normal and call into Catch yourself manually. + +You now have a lot of flexibility - but here are three recipes to get your started: + +## Let Catch take full control of args and config + +If you just need to have code that executes before and/ or after Catch this is the simplest option. + +```c++ +#define CATCH_CONFIG_RUNNER +#include "catch.hpp" + +int main( int argc, char* const argv[] ) +{ + // global setup... + + int result = Catch::Session().run( argc, argv ); + + // global clean-up... + + return result; +} +``` + +## Amending the config + +If you still want Catch to process the command line, but you want to programatically tweak the config, you can do so in one of two ways: + +```c++ +#define CATCH_CONFIG_RUNNER +#include "catch.hpp" + +int main( int argc, char* const argv[] ) +{ + Catch::Session session; // There must be exactly once instance + + // writing to session.configData() here sets defaults + // this is the preferred way to set them + + int returnCode = session.applyCommandLine( argc, argv ); + if( returnCode != 0 ) // Indicates a command line error + return returnCode; + + // writing to session.configData() or session.Config() here + // overrides command line args + // only do this if you know you need to + + return session.run(); +} +``` + +Take a look at the definitions of Config and ConfigData to see what you can do with them. + +To take full control of the config simply omit the call to ```applyCommandLine()```. + +## Adding your own command line options + +Catch embeds a powerful command line parser which you can also use to parse your own options out. This capability is still in active development but will be documented here when it is ready. + +--- + +[Home](Readme.md)
\ No newline at end of file diff --git a/docs/slow-compiles.md b/docs/slow-compiles.md new file mode 100644 index 0000000..5291aec --- /dev/null +++ b/docs/slow-compiles.md @@ -0,0 +1,22 @@ +# Why do my tests take so long to compile? + +Several people have reported that test code written with Catch takes much longer to compile than they would expect. Why is that? + +Catch is implemented entirely in headers. There is a little overhead due to this - but not as much as you might think - and you can minimise it simply by organising your test code as follows: + +## Short answer +Exactly one source file must ```#define``` either ```CATCH_CONFIG_MAIN``` or ```CATCH_CONFIG_RUNNER``` before ```#include```-ing Catch. In this file *do not write any test cases*! In most cases that means this file will just contain two lines (the ```#define``` and the ```#include```). + +## Long answer + +Usually C++ code is split between a header file, containing declarations and prototypes, and an implementation file (.cpp) containing the definition, or implementation, code. Each implementation file, along with all the headers that it includes (and which those headers include, etc), is expanded into a single entity called a translation unit - which is then passed to the compiler and compiled down to an object file. + +But functions and methods can also be written inline in header files. The downside to this is that these definitions will then be compiled in *every* translation unit that includes the header. + +Because Catch is implemented *entirely* in headers you might think that the whole of Catch must be compiled into every translation unit that uses it! Actually it's not quite as bad as that. Catch mitigates this situation by effectively maintaining the traditional separation between the implementation code and declarations. Internally the implementation code is protected by ```#ifdef```s and is conditionally compiled into only one translation unit. This translation unit is that one that ```#define```s ```CATCH_CONFIG_MAIN``` or ```CATCH_CONFIG_RUNNER```. Let's call this the main source file. + +As a result the main source file *does* compile the whole of Catch every time! So it makes sense to dedicate this file to *only* ```#define```-ing the identifier and ```#include```-ing Catch (and implementing the runner code, if you're doing that). Keep all your test cases in other files. This way you won't pay the recompilation cost for the whole of Catch + +--- + +[Home](Readme.md)
\ No newline at end of file diff --git a/docs/test-cases-and-sections.md b/docs/test-cases-and-sections.md new file mode 100644 index 0000000..86644f0 --- /dev/null +++ b/docs/test-cases-and-sections.md @@ -0,0 +1,86 @@ +# Test cases and sections + +While Catch fully supports the traditional, xUnit, style of class-based fixtures containing test case methods this is not the preferred style. + +Instead Catch provides a powerful mechanism for nesting test case sections within a test case. For a more detailed discussion see the [tutorial](tutorial.md#testCasesAndSections). + +Test cases and sections are very easy to use in practice: + +* **TEST_CASE(** _test name_ \[, _tags_ \] **)** +* **SECTION(** _section name_ **)** + +_test name_ and _section name_ are free form, quoted, strings. The optional _tags_ argument is a quoted string containing one or more tags enclosed in square brackets. Tags are discussed below. Test names must be unique within the Catch executable. + +For examples see the [Tutorial](tutorial.md) + +## Tags + +Tags allow an arbitrary number of additional strings to be associated with a test case. Test cases can be selected (for running, or just for listing) by tag - or even by an expression that combines several tags. At their most basic level they provide a simple way to group several related tests together. + +As an example - given the following test cases: + + TEST_CASE( "A", "[widget]" ) { /* ... */ } + TEST_CASE( "B", "[widget]" ) { /* ... */ } + TEST_CASE( "C", "[gadget]" ) { /* ... */ } + TEST_CASE( "D", "[widget][gadget]" ) { /* ... */ } + +The tag expression, ```"[widget]"``` selects A, B & D. ```"[gadget]"``` selects C & D. ```"[widget][gadget]"``` selects just D and ```"[widget],[gadget]"``` selects all four test cases. + +For more detail on command line selection see [the command line docs](command-line.md#specifying-which-tests-to-run) + +Tag names are not case sensitive. + +### Special Tags + +All tag names beginning with non-alphanumeric characters are reserved by Catch. Catch defines a number of "special" tags, which have meaning to the test runner itself. These special tags all begin with a symbol character. Following is a list of currently defined special tags and their meanings. + +* `[!hide]` or `[.]` (or, for legacy reasons, `[hide]`) - causes test cases to be skipped from the default list (ie when no test cases have been explicitly selected through tag expressions or name wildcards). The hide tag is often combined with another, user, tag (for example `[.][integration]` - so all integration tests are excluded from the default run but can be run by passing `[integration]` on the command line). As a short-cut you can combine these by simply prefixing your user tag with a `.` - e.g. `[.integration]`. Because the hide tag has evolved to have several forms, all forms are added as tags if you use one of them. + +* `[!throws]` - lets Catch know that this test is likely to throw an exception even if successful. This causes the test to be exluded when running with `-e` or `--nothrow`. + +* `[!shouldfail]` - reverse the failing logic of the test: if the test is successful if it fails, and vice-versa. + +* `[!mayfail]` - doesn't fail the test if any given assertion fails (but still reports it). This can be useful to flag a work-in-progress, or a known issue that you don't want to immediately fix but still want to track in the your tests. + +* `[#<filename>]` - running with `-#` or `--filenames-as-tags` causes Catch to add the filename, prefixed with `#` (and with any extension stripped) as a tag. e.g. tests in testfile.cpp would all be tagged `[#testfile]`. + +* `[@<alias>]` - tag aliases all begin with `@` (see below). + +## Tag aliases + +Between tag expressions and wildcarded test names (as well as combinations of the two) quite complex patterns can be constructed to direct which test cases are run. If a complex pattern is used often it is convenient to be able to create an alias for the expression. this can be done, in code, using the following form: + + CATCH_REGISTER_TAG_ALIAS( <alias string>, <tag expression> ) + +Aliases must begining with the `@` character. An example of a tag alias is: + + CATCH_REGISTER_TAG_ALIAS( "[@nhf]", "[failing]~[.]" ) + +Now when `[@nhf]` is used on the command line this matches all tests that are tagged `[failing]`, but which are not also hidden. + +## BDD-style test cases + +In addition to Catch's take on the classic style of test cases, Catch supports an alternative syntax that allow tests to be written as "executable specifications" (one of the early goals of [Behaviour Driven Development](http://dannorth.net/introducing-bdd/)). This set of macros map on to ```TEST_CASE```s and ```SECTION```s, with a little internal support to make them smoother to work with. + +* **SCENARIO(** _scenario name_ \[, _tags_ \] **)** + +This macro maps onto ```TEST_CASE``` and works in the same way, except that the test case name will be prefixed by "Scenario: " + +* **GIVEN(** _something_ **)** +* **WHEN(** _something_ **)** +* **THEN(** _something_ **)** + +These macros map onto ```SECTION```s except that the section names are the _something_s prefixed by "given: ", "when: " or "then: " respectively. + +* **AND_WHEN(** _something_ **)** +* **AND_THEN(** _something_ **)** + +Similar to ```WHEN``` and ```THEN``` except that the prefixes start with "and ". These are used to chain ```WHEN```s and ```THEN```s together. + +When any of these macros are used the console reporter recognises them and formats the test case header such that the Givens, Whens and Thens are aligned to aid readability. + +Other than the additional prefixes and the formatting in the console reporter these macros behave exactly as ```TEST_CASE```s and ```SECTION```s. As such there is nothing enforcing the correct sequencing of these macros - that's up to the programmer! + +--- + +[Home](Readme.md)
\ No newline at end of file diff --git a/docs/test-fixtures.md b/docs/test-fixtures.md new file mode 100644 index 0000000..6bef762 --- /dev/null +++ b/docs/test-fixtures.md @@ -0,0 +1,32 @@ +Although Catch allows you to group tests together as sections within a test case, it can still convenient, sometimes, to group them using a more traditional test fixture. Catch fully supports this too. You define the test fixture as a simple structure: + +```c++ +class UniqueTestsFixture { + private: + static int uniqueID; + protected: + DBConnection conn; + public: + UniqueTestsFixture() : conn(DBConnection::createConnection("myDB")) { + } + protected: + int getID() { + return ++uniqueID; + } + }; + + int UniqueTestsFixture::uniqueID = 0; + + TEST_CASE_METHOD(UniqueTestsFixture, "Create Employee/No Name", "[create]") { + REQUIRE_THROWS(conn.executeSQL("INSERT INTO employee (id, name) VALUES (?, ?)", getID(), "")); + } + TEST_CASE_METHOD(UniqueTestsFixture, "Create Employee/Normal", "[create]") { + REQUIRE(conn.executeSQL("INSERT INTO employee (id, name) VALUES (?, ?)", getID(), "Joe Bloggs")); + } +``` + +The two test cases here will create uniquely-named derived classes of UniqueTestsFixture and thus can access the `getID()` protected method and `conn` member variables. This ensures that both the test cases are able to create a DBConnection using the same method (DRY principle) and that any ID's created are unique such that the order that tests are executed does not matter. + +--- + +[Home](Readme.md)
\ No newline at end of file diff --git a/docs/tostring.md b/docs/tostring.md new file mode 100644 index 0000000..dbb6cb8 --- /dev/null +++ b/docs/tostring.md @@ -0,0 +1,70 @@ +# String conversions + +Catch needs to be able to convert types you use in assertions and logging expressions into strings (for logging and reporting purposes). +Most built-in or std types are supported out of the box but there are three ways that you can tell Catch how to convert your own types (or other, third-party types) into strings. + +## operator << overload for std::ostream + +This is the standard way of providing string conversions in C++ - and the chances are you may already provide this for your own purposes. If you're not familiar with this idiom it involves writing a free function of the form: + +``` +std::ostream& operator << ( std::ostream& os, T const& value ) { + os << convertMyTypeToString( value ); + return os; +} +``` + +(where ```T``` is your type and ```convertMyTypeToString``` is where you'll write whatever code is necessary to make your type printable - it doesn't have to be in another function). + +You should put this function in the same namespace as your type. + +Alternatively you may prefer to write it as a member function: + +``` +std::ostream& T::operator << ( std::ostream& os ) const { + os << convertMyTypeToString( *this ); + return os; +} +``` + +## Catch::toString overload + +If you don't want to provide an ```operator <<``` overload, or you want to convert your type differently for testing purposes, you can provide an overload for ```Catch::toString()``` for your type. + +``` +namespace Catch { + std::string toString( T const& value ) { + return convertMyTypeToString( value ); + } +} +``` + +Again ```T``` is your type and ```convertMyTypeToString``` is where you'll write whatever code is necessary to make your type printable. Note that the function must be in the Catch namespace, which itself must be in the global namespace. + +## Catch::StringMaker<T> specialisation + +There are some cases where overloading toString does not work as expected. Specialising StringMaker<T> gives you more precise, and reliable, control - but at the cost of slightly more code and complexity: + +``` +namespace Catch { + template<> struct StringMaker<T> { + static std::string convert( T const& value ) { + return convertMyTypeToString( value ); + } + }; +} +``` + +## Exceptions + +By default all exceptions deriving from `std::exception` will be translated to strings by calling the `what()` method. For exception types that do not derive from `std::exception` - or if `what()` does not return a suitable string - use `CATCH_TRANSLATE_EXCEPTION`. This defines a function that takes your exception type, by reference, and returns a string. It can appear anywhere in the code - it doesn't have to be in the same translation unit. For example: + +``` +CATCH_TRANSLATE_EXCEPTION( MyType& ex ) { + return ex.message(); +} +``` + +--- + +[Home](Readme.md) diff --git a/docs/tutorial.md b/docs/tutorial.md new file mode 100644 index 0000000..0fdaff9 --- /dev/null +++ b/docs/tutorial.md @@ -0,0 +1,249 @@ +# Getting Catch + +The simplest way to get Catch is to download the single header version from [http://builds.catch-lib.net](http://builds.catch-lib.net). Don't be put off by the word "builds" there. The single header is generated by merging a set of individual headers but it is still just normal source code in a header file. + +The full source for Catch, including test projects, documentation, and other things, is hosted on GitHub. [http://catch-lib.net](http://catch-lib.net) will redirect you there. + + +## Where to put it? + +Catch is header only. All you need to do is drop the file(s) somewhere reachable from your project - either in some central location you can set your header search path to find, or directly into your project tree itself! This is a particularly good option for other Open-Source projects that want to use Catch for their test suite. See [this blog entry for more on that](http://www.levelofindirection.com/journal/2011/5/27/unit-testing-in-c-and-objective-c-just-got-ridiculously-easi.html). + +The rest of this tutorial will assume that the Catch single-include header (or the include folder) is available unqualified - but you may need to prefix it with a folder name if necessary. + +# Writing tests + +Let's start with a really simple example. Say you have written a function to calculate factorials and now you want to test it (let's leave aside TDD for now). + +```c++ +unsigned int Factorial( unsigned int number ) { + return number <= 1 ? number : Factorial(number-1)*number; +} +``` + +To keep things simple we'll put everything in a single file (<a href="#scaling-up">see later for more on how to structure your test files</a>) + +```c++ +#define CATCH_CONFIG_MAIN // This tells Catch to provide a main() - only do this in one cpp file +#include "catch.hpp" + +unsigned int Factorial( unsigned int number ) { + return number <= 1 ? number : Factorial(number-1)*number; +} + +TEST_CASE( "Factorials are computed", "[factorial]" ) { + REQUIRE( Factorial(1) == 1 ); + REQUIRE( Factorial(2) == 2 ); + REQUIRE( Factorial(3) == 6 ); + REQUIRE( Factorial(10) == 3628800 ); +} +``` + +This will compile to a complete executable which responds to [command line arguments](command-line.md). If you just run it with no arguments it will execute all test cases (in this case there is just one), report any failures, report a summary of how many tests passed and failed and return the number of failed tests (useful for if you just want a yes/ no answer to: "did it work"). + +If you run this as written it will pass. Everything is good. Right? +Well, there is still a bug here. In fact the first version of this tutorial I posted here genuinely had the bug in! So it's not completely contrived (thanks to Daryle Walker (```@CTMacUser```) for pointing this out). + +What is the bug? Well what is the factorial of zero? +[The factorial of zero is one](http://mathforum.org/library/drmath/view/57128.html) - which is just one of those things you have to know (and remember!). + +Let's add that to the test case: + +```c++ +TEST_CASE( "Factorials are computed", "[factorial]" ) { + REQUIRE( Factorial(0) == 1 ); + REQUIRE( Factorial(1) == 1 ); + REQUIRE( Factorial(2) == 2 ); + REQUIRE( Factorial(3) == 6 ); + REQUIRE( Factorial(10) == 3628800 ); +} +``` + +Now we get a failure - something like: + +``` +Example.cpp:9: FAILED: + REQUIRE( Factorial(0) == 1 ) +with expansion: + 0 == 1 +``` + +Note that we get the actual return value of Factorial(0) printed for us (0) - even though we used a natural expression with the == operator. That let's us immediately see what the problem is. + +Let's change the factorial function to: + +```c++ +unsigned int Factorial( unsigned int number ) { + return number > 1 ? Factorial(number-1)*number : 1; +} +``` + +Now all the tests pass. + +Of course there are still more issues to do deal with. For example we'll hit problems when the return value starts to exceed the range of an unsigned int. With factorials that can happen quite quickly. You might want to add tests for such cases and decide how to handle them. We'll stop short of doing that here. + +## What did we do here? + +Although this was a simple test it's been enough to demonstrate a few things about how Catch is used. Let's take moment to consider those before we move on. + +1. All we did was ```#define``` one identifier and ```#include``` one header and we got everything - even an implementation of ```main()``` that will [respond to command line arguments](command-line.md). You can only use that ```#define``` in one implementation file, for (hopefully) obvious reasons. Once you have more than one file with unit tests in you'll just ```#include "catch.hpp"``` and go. Usually it's a good idea to have a dedicated implementation file that just has ```#define CATCH_CONFIG_MAIN``` and ```#include "catch.hpp"```. You can also provide your own implementation of main and drive Catch yourself (see [Supplying-your-own-main()](own-main.md)). +2. We introduce test cases with the ```TEST_CASE``` macro. This macro takes one or two arguments - a free form test name and, optionally, one or more tags (for more see <a href="#test-cases-and-sections">Test cases and Sections</a>, ). The test name must be unique. You can run sets of tests by specifying a wildcarded test name or a tag expression. See the [command line docs](command-line.md) for more information on running tests. +3. The name and tags arguments are just strings. We haven't had to declare a function or method - or explicitly register the test case anywhere. Behind the scenes a function with a generated name is defined for you, and automatically registered using static registry classes. By abstracting the function name away we can name our tests without the constraints of identifier names. +4. We write our individual test assertions using the ```REQUIRE``` macro. Rather than a separate macro for each type of condition we express the condition naturally using C/C++ syntax. Behind the scenes a simple set of expression templates captures the left-hand-side and right-hand-side of the expression so we can display the values in our test report. As we'll see later there _are_ other assertion macros - but because of this technique the number of them is drastically reduced. + +<a id="test-cases-and-sections"></a> +## Test cases and sections + +Most test frameworks have a class-based fixture mechanism. That is, test cases map to methods on a class and common setup and teardown can be performed in ```setup()``` and ```teardown()``` methods (or constructor/ destructor in languages, like C++, that support deterministic destruction). + +While Catch fully supports this way of working there are a few problems with the approach. In particular the way your code must be split up, and the blunt granularity of it, may cause problems. You can only have one setup/ teardown pair across a set of methods, but sometimes you want slightly different setup in each method, or you may even want several levels of setup (a concept which we will clarify later on in this tutorial). It was <a href="http://jamesnewkirk.typepad.com/posts/2007/09/why-you-should-.html">problems like these</a> that led James Newkirk, who led the team that built NUnit, to start again from scratch and <a href="http://jamesnewkirk.typepad.com/posts/2007/09/announcing-xuni.html">build xUnit</a>). + +Catch takes a different approach (to both NUnit and xUnit) that is a more natural fit for C++ and the C family of languages. This is best explained through an example: + +```c++ +TEST_CASE( "vectors can be sized and resized", "[vector]" ) { + + std::vector<int> v( 5 ); + + REQUIRE( v.size() == 5 ); + REQUIRE( v.capacity() >= 5 ); + + SECTION( "resizing bigger changes size and capacity" ) { + v.resize( 10 ); + + REQUIRE( v.size() == 10 ); + REQUIRE( v.capacity() >= 10 ); + } + SECTION( "resizing smaller changes size but not capacity" ) { + v.resize( 0 ); + + REQUIRE( v.size() == 0 ); + REQUIRE( v.capacity() >= 5 ); + } + SECTION( "reserving bigger changes capacity but not size" ) { + v.reserve( 10 ); + + REQUIRE( v.size() == 5 ); + REQUIRE( v.capacity() >= 10 ); + } + SECTION( "reserving smaller does not change size or capacity" ) { + v.reserve( 0 ); + + REQUIRE( v.size() == 5 ); + REQUIRE( v.capacity() >= 5 ); + } +} +``` + +For each ```SECTION``` the ```TEST_CASE``` is executed from the start - so as we enter each section we know that size is 5 and capacity is at least 5. We enforced those requirements with the ```REQUIRE```s at the top level so we can be confident in them. +This works because the ```SECTION``` macro contains an if statement that calls back into Catch to see if the section should be executed. One leaf section is executed on each run through a ```TEST_CASE```. The other sections are skipped. Next time through the next section is executed, and so on until no new sections are encountered. + +So far so good - this is already an improvement on the setup/teardown approach because now we see our setup code inline and use the stack. + +The power of sections really shows, however, when we need to execute a sequence of, checked, operations. Continuing the vector example, we might want to verify that attempting to reserve a capacity smaller than the current capacity of the vector changes nothing. We can do that, naturally, like so: + +```c++ + SECTION( "reserving bigger changes capacity but not size" ) { + v.reserve( 10 ); + + REQUIRE( v.size() == 5 ); + REQUIRE( v.capacity() >= 10 ); + + SECTION( "reserving smaller again does not change capacity" ) { + v.reserve( 7 ); + + REQUIRE( v.capacity() >= 10 ); + } + } +``` + +Sections can be nested to an arbitrary depth (limited only by your stack size). Each leaf section (i.e. a section that contains no nested sections) will be executed exactly once, on a separate path of execution from any other leaf section (so no leaf section can interfere with another). A failure in a parent section will prevent nested sections from running - but then that's the idea. + +## BDD-Style + +If you name your test cases and sections appropriately you can achieve a BDD-style specification structure. This became such a useful way of working that first class support has been added to Catch. Scenarios can be specified using ```SCENARIO```, ```GIVEN```, ```WHEN``` and ```THEN``` macros, which map on to ```TEST_CASE```s and ```SECTION```s, respectively. For more details see [Test cases and sections](test-cases-and-sections.md). + +The vector example can be adjusted to use these macros like so: + +```c++ +SCENARIO( "vectors can be sized and resized", "[vector]" ) { + + GIVEN( "A vector with some items" ) { + std::vector<int> v( 5 ); + + REQUIRE( v.size() == 5 ); + REQUIRE( v.capacity() >= 5 ); + + WHEN( "the size is increased" ) { + v.resize( 10 ); + + THEN( "the size and capacity change" ) { + REQUIRE( v.size() == 10 ); + REQUIRE( v.capacity() >= 10 ); + } + } + WHEN( "the size is reduced" ) { + v.resize( 0 ); + + THEN( "the size changes but not capacity" ) { + REQUIRE( v.size() == 0 ); + REQUIRE( v.capacity() >= 5 ); + } + } + WHEN( "more capacity is reserved" ) { + v.reserve( 10 ); + + THEN( "the capacity changes but not the size" ) { + REQUIRE( v.size() == 5 ); + REQUIRE( v.capacity() >= 10 ); + } + } + WHEN( "less capacity is reserved" ) { + v.reserve( 0 ); + + THEN( "neither size nor capacity are changed" ) { + REQUIRE( v.size() == 5 ); + REQUIRE( v.capacity() >= 5 ); + } + } + } +} +``` + +Conveniently, these tests will be reported as follows when run: + +``` +Scenario: vectors can be sized and resized + Given: A vector with some items + When: more capacity is reserved + Then: the capacity changes but not the size +``` + +<a id="scaling-up"></a> +## Scaling up + +To keep the tutorial simple we put all our code in a single file. This is fine to get started - and makes jumping into Catch even quicker and easier. As you write more real-world tests, though, this is not really the best approach. + +The requirement is that the following block of code ([or equivalent](own-main.md)): + +```c++ +#define CATCH_CONFIG_MAIN +#include "catch.hpp" +``` + +appears in _exactly one_ source file. Use as many additional cpp files (or whatever you call your implementation files) as you need for your tests, partitioned however makes most sense for your way of working. Each additional file need only ```#include "catch.hpp"``` - do not repeat the ```#define```! + +In fact it is usually a good idea to put the block with the ```#define``` [in it's own source file](slow-compiles.md). + +Do not write your tests in header files! + + +## Next steps + +This has been a brief introduction to get you up and running with Catch, and to point out some of the key differences between Catch and other frameworks you may already be familiar with. This will get you going quite far already and you are now in a position to dive in and write some tests. + +Of course there is more to learn - most of which you should be able to page-fault in as you go. Please see the ever-growing [Reference section](Readme.md) for what's available. + +--- + +[Home](Readme.md) diff --git a/docs/why-catch.md b/docs/why-catch.md new file mode 100644 index 0000000..93488d2 --- /dev/null +++ b/docs/why-catch.md @@ -0,0 +1,42 @@ +# Why do we need yet another C++ test framework? + +Good question. For C++ there are quite a number of established frameworks, including (but not limited to), [CppUnit](http://sourceforge.net/apps/mediawiki/cppunit/index.php?title=Main_Page), [Google Test](http://code.google.com/p/googletest/), [Boost.Test](http://www.boost.org/doc/libs/1_49_0/libs/test/doc/html/index.html), [Aeryn](https://launchpad.net/aeryn), [Cute](http://r2.ifs.hsr.ch/cute), [Fructose](http://fructose.sourceforge.net/) and [many, many more](http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#C.2B.2B). Even for Objective-C there are a few, including OCUnit - which now comes bundled with XCode. + +So what does Catch bring to the party that differentiates it from these? Apart from a Catchy name, of course. + +## Key Features + +* Really easy to get started. Just download catch.hpp, #include it and you're away. +* No external dependencies. As long as you can compile C++98 and have a C++ standard library available. +* Write test cases as, self-registering, functions or methods. +* Divide test cases into sections, each of which is run in isolation (eliminates the need for fixtures!) +* Use BDD-style Given-When-Then sections as well as traditional unit test cases. +* Only one core assertion macro for comparisons. Standard C/C++ operators are used for the comparison - yet the full expression is decomposed and lhs and rhs values are logged. + +## Other core features + +* Tests are named using free-form strings - no more couching names in legal identifiers. +* Tests can be tagged for easily running ad-hoc groups of tests. +* Failures can (optionally) break into the debugger on Windows and Mac. +* Output is through modular reporter objects. Basic textual and XML reporters are included. Custom reporters can easily be added. +* JUnit xml output is supported for integration with third-party tools, such as CI servers. +* A default main() function is provided (in a header), but you can supply your own for complete control (e.g. integration into your own test runner GUI). +* A command line parser is provided and can still be used if you choose to provided your own main() function. +* Catch can test itself. +* Alternative assertion macro(s) report failures but don't abort the test case +* Floating point tolerance comparisons are built in using an expressive Approx() syntax. +* Internal and friendly macros are isolated so name clashes can be managed +* Support for Matchers (early stages) + +## Objective-C-specific features + +* Automatically detects if you are using it from an Objective-C project +* Works with and without ARC with no additional configuration +* Implement test fixtures using Obj-C classes too (like OCUnit) +* Additional built in matchers that work with Obj-C types (e.g. string matchers) + +See the [tutorial](tutorial.md) to get more of a taste of using CATCH in practice + +--- + +[Home](Readme.md)
\ No newline at end of file |