This repository has been archived on 2025-08-18. You can view files and clone it, but you cannot make any changes to it's state, such as pushing and creating new issues, pull requests or comments.
GGS/lib/eqc/doc/eqc.html
2011-05-05 02:46:21 +02:00

385 lines
23 KiB
HTML

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>Module eqc</title>
<link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
</head>
<body bgcolor="white">
<div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
<hr>
<h1>Module eqc</h1>
<ul class="index"><li><a href="#description">Description</a></li><li><a href="#types">Data Types</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>This module defines functions for writing and testing QuickCheck properties.
<p><b>Version:</b> 1.0.1</p>
<h2><a name="description">Description</a></h2>This module defines functions for writing and testing QuickCheck properties.
Much of the interface is provided via macros (defined in <tt>eqc.hrl</tt>).
These are documented below:
<h2><tt>?FORALL(X,Gen,Prop)</tt></h2>
Property that holds if <tt>Prop</tt> holds for all values <tt>X</tt> that
can be generated by <tt>Gen</tt>. For example,
<pre>
prop_reverse() -&gt;
?FORALL(Xs,list(int()),
lists:reverse(lists:reverse(Xs)) == Xs).
</pre>
Generators are defined using the module <a href="eqc_gen.html"><code>eqc_gen</code></a>.
<h2><tt>?IMPLIES(Pre,Prop)</tt></h2>
Property that holds if <tt>Prop</tt> holds whenever the precondition
<tt>Pre</tt> is true. The precondition must be a boolean, but <tt>Prop</tt>
can be any QuickCheck property. An implication is tested by discarding test
cases which do not satisfy the precondition. This can make testing slow,
since many more test cases may need to be generated to find 100 which
satisfy the precondition. In the worst case, QuickCheck may not be able
to find enough test cases that do satisfy the precondition, in which case
the number actually found is reported. Some preconditions may also skew
the test data badly--for example, a precondition that a list is sorted
skews the test data towards short lists, since random longer lists are
extremely unlikely to be sorted just by chance. <tt>?IMPLIES</tt> works
well for preconditions which are true with a high probability, but if the
precondition is unlikely to hold, then it is better to write a custom
generator which generates test cases where the precondition is true.
<h2><tt>?WHENFAIL(Action,Prop)</tt></h2><p>
Property that is equivalent to <tt>Prop</tt>, but performs <tt>Action</tt>
(for its side effects) when <tt>Prop</tt> fails. This can be used to
print additional information when a test case fails.</p>
<h2><a name="types">Data Types</a></h2>
<h3 class="typedecl"><a name="type-counterexample">counterexample()</a></h3>
<p><b>abstract datatype</b>: <tt>counterexample()</tt></p>
<p>A counter-example to a QuickCheck property, which can be obtained
using <a href="#counterexample-0"><code>counterexample/0</code></a> or <a href="#counterexample-1"><code>counterexample/1</code></a>, and used to repeat a test,
or test a different property in the same case. Counterexamples are represented by the values
bound by ?FORALL--for the counterexample to make sense independently, it's important that
these were generated without side-effects.</p>
<h3 class="typedecl"><a name="type-print_method">print_method()</a></h3>
<p><tt>print_method() = (list(term())) -&gt; any()</tt></p>
<p>A function for
printing statistics, which is passed a list of samples and is
expected to print statistical information about them. Print methods
are used by <a href="#collect-3"><code>collect/3</code></a> and <a href="#aggregate-3"><code>aggregate/3</code></a>.</p>
<h3 class="typedecl"><a name="type-property">property()</a></h3>
<p><b>abstract datatype</b>: <tt>property()</tt></p>
<p>QuickCheck properties, which can either be boolean
expressions, or constructed using the functions in this module.
QuickCheck properties are tested using <a href="#quickcheck-1"><code>quickcheck/1</code></a>.</p>
<h2><a name="index">Function Index</a></h2>
<table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#aggregate-2">aggregate/2</a></td><td>A property logically equivalent to <tt>Prop</tt>, but which collects a list of values in
each test, and displays the distribution of these values once
testing is complete.</td></tr>
<tr><td valign="top"><a href="#aggregate-3">aggregate/3</a></td><td>Like <a href="#aggregate-2"><code>aggregate/2</code></a>, but allows the user to specify how
the collected values should be printed.</td></tr>
<tr><td valign="top"><a href="#backtrace-0">backtrace/0</a></td><td>Displays a stack backtrace from the last exception QuickCheck caught.</td></tr>
<tr><td valign="top"><a href="#check-2">check/2</a></td><td>Tests the property in the case given.</td></tr>
<tr><td valign="top"><a href="#classify-3">classify/3</a></td><td>Property which is logically equivalent to <tt>Prop</tt>, but also
classifies test cases and displays the distribution of test case classes
when testing is complete.</td></tr>
<tr><td valign="top"><a href="#collect-2">collect/2</a></td><td>Equivalent to <a href="#aggregate-2"><tt>aggregate([S], Prop)</tt></a>.
</td></tr>
<tr><td valign="top"><a href="#collect-3">collect/3</a></td><td>Equivalent to <a href="#aggregate-3"><tt>aggregate(PrintMethod, [S], Prop)</tt></a>.
</td></tr>
<tr><td valign="top"><a href="#counterexample-0">counterexample/0</a></td><td>Returns the last counter-example found.</td></tr>
<tr><td valign="top"><a href="#counterexample-1">counterexample/1</a></td><td>Tests the property in the same way as <a href="#quickcheck-1"><code>quickcheck/1</code></a>, but if
a test fails, then the failing test case is returned as a counterexample.</td></tr>
<tr><td valign="top"><a href="#counterexamples-0">counterexamples/0</a></td><td>Returns a list of the counterexamples found by the last call
of <code>eqc:module</code>, paired with the name of the property that failed.</td></tr>
<tr><td valign="top"><a href="#current_counterexample-0">current_counterexample/0</a></td><td>Returns the most recent
counterexample found by QuickCheck.</td></tr>
<tr><td valign="top"><a href="#equals-2">equals/2</a></td><td>A property which holds if X and Y are equal...</td></tr>
<tr><td valign="top"><a href="#fails-1">fails/1</a></td><td>A property which succeeds when its argument fails.</td></tr>
<tr><td valign="top"><a href="#measure-3">measure/3</a></td><td>Collects the values of X while testing Prop, and if all tests
pass, displays statistics such as the minimum, average, and maximum
values, identified by the name Name.</td></tr>
<tr><td valign="top"><a href="#module-1">module/1</a></td><td>Tests all the properties exported from a module, given the module name.</td></tr>
<tr><td valign="top"><a href="#numtests-2">numtests/2</a></td><td>Property which is logically equivalent to <tt>Prop</tt>, but is
tested <tt>N</tt> times rather than 100.</td></tr>
<tr><td valign="top"><a href="#on_output-2">on_output/2</a></td><td>Supplies an output function to be used instead of io:format
when QuickCheck generates output.</td></tr>
<tr><td valign="top"><a href="#on_test-2">on_test/2</a></td><td>Attaches a function to a property which is called every time a
test passes or fails.</td></tr>
<tr><td valign="top"><a href="#quickcheck-1">quickcheck/1</a></td><td>Tests the property in 100 random cases, printing a counter-example
if one is found.</td></tr>
<tr><td valign="top"><a href="#recheck-1">recheck/1</a></td><td>Tests the property with the <i>same</i> random number seed as
the last failing call of <a href="#quickcheck-1"><code>quickcheck/1</code></a>.</td></tr>
<tr><td valign="top"><a href="#start-0">start/0</a></td><td>Equivalent to <a href="#start-1"><tt>start(true)</tt></a>.
</td></tr>
<tr><td valign="top"><a href="#start-1">start/1</a></td><td>Starts the QuickCheck server.</td></tr>
<tr><td valign="top"><a href="#stop-0">stop/0</a></td><td>Stops the QuickCheck server.</td></tr>
<tr><td valign="top"><a href="#version-0">version/0</a></td><td></td></tr>
<tr><td valign="top"><a href="#with_title-1">with_title/1</a></td><td>A printing method for collected data, which displays a title
before
the percentages of each value in the data.</td></tr>
</table>
<h2><a name="functions">Function Details</a></h2>
<h3 class="function"><a name="aggregate-2">aggregate/2</a></h3>
<div class="spec">
<p><tt>aggregate(L::list(term()), Prop::<a href="#type-property">property()</a>) -&gt; <a href="#type-property">property()</a></tt></p>
</div><p>A property logically equivalent to <tt>Prop</tt>, but which collects a list of values in
each test, and displays the distribution of these values once
testing is complete. A typical use would be to aggregate the list of command names generated
by <a href="eqc_statem.html#commands-1"><code>eqc_statem:commands/1</code></a>, in order to see how often each individual
command appeared in generated tests:
<pre>aggregate(command_names(Cmds), ...) </pre>
<p>
See also <a href="#aggregate-3"><code>aggregate/3</code></a>.
</p></p>
<h3 class="function"><a name="aggregate-3">aggregate/3</a></h3>
<div class="spec">
<p><tt>aggregate(PrintMethod::(list(term())) -&gt; any(), L::list(term()), Prop::<a href="#type-property">property()</a>) -&gt; <a href="#type-property">property()</a></tt></p>
</div><p>Like <a href="#aggregate-2"><code>aggregate/2</code></a>, but allows the user to specify how
the collected values should be printed. The <tt>PrintMethod</tt> parameter
is called with a sorted list of the collected data as an argument,
and is expected to print some statistics. A predefined printing
methods is provided to add a title to the statistics:
<pre>aggregate(with_title(T),L,Prop)</pre>. This is useful when a property contains
several calls to aggregate or collect.</p>
<h3 class="function"><a name="backtrace-0">backtrace/0</a></h3>
<div class="spec">
<p><tt>backtrace() -&gt; ok</tt></p>
</div><p>Displays a stack backtrace from the last exception QuickCheck caught. Note that
this is only possible if the exception is raised in the process in which the test
case starts. If a test case fails because of an exception in another, linked,
process, then no backtrace is available. Calls to functions in the implementation
of QuickCheck itself are not included in the backtrace.
<p>If you really need to see a backtrace from a linked process, then you can do so by
catching
the exception yourself in that process, using erlang:get_stacktrace() to obtain the
backtrace, and printing it yourself.</p></p>
<h3 class="function"><a name="check-2">check/2</a></h3>
<div class="spec">
<p><tt>check(P::<a href="#type-property">property()</a>, Values::<a href="#type-counterexample">counterexample()</a>) -&gt; bool()</tt></p>
</div><p>Tests the property in the case given. Counterexamples are generated by testing a
property using <a href="#counterexample-1"><code>counterexample/1</code></a> or <a href="#counterexample-0"><code>counterexample/0</code></a>, and contain a list
of the values bound by ?FORALL. A property tested by check should begin with the <i>same</i>
sequence of ?FORALL s as the property from which the counterexample was generated, otherwise
the results will be unpredictable. In particular, there is no check that the values
in the counterexample could actually have been generated by the ?FORALL s in the property under
test.
<p><a href="#check-2"><code>check/2</code></a> can be used without a QuickCheck licence, allowing anyone to run
tests that a licenced user has generated.</p></p>
<h3 class="function"><a name="classify-3">classify/3</a></h3>
<div class="spec">
<p><tt>classify(B::bool(), S::term(), Prop::<a href="#type-property">property()</a>) -&gt; <a href="#type-property">property()</a></tt></p>
</div><p>Property which is logically equivalent to <tt>Prop</tt>, but also
classifies test cases and displays the distribution of test case classes
when testing is complete. If the boolean is true then the current test case is
labelled with the term <tt>S</tt>,
and, after testing is complete, QuickCheck prints out the percentage of
test cases carrying each label. This can be used to check that the space
of possible test cases has been covered reasonably well. For example,
classifying test cases according to the length of a list enables one to
see whether unreasonably many lists were short. Classifying
test cases is a way to discover skewed distributions, such as can arise
from using <tt>?IMPLIES</tt>. It is good practice to check the distribution
of test data using <tt>classify</tt> or <a href="#collect-2"><code>collect/2</code></a>, at least while
properties are being developed.
<p>
Each test case can be labelled with any number of labels: QuickCheck then
displays the percentage of each label in the generated
test data.
</p>
<p>
Calls of classify or collect can be nested, in which case each call
generates its own table of distributions.
</p></p>
<h3 class="function"><a name="collect-2">collect/2</a></h3>
<div class="spec">
<p><tt>collect(S::term(), Prop::<a href="#type-property">property()</a>) -&gt; <a href="#type-property">property()</a></tt></p>
</div><p>Equivalent to <a href="#aggregate-2"><tt>aggregate([S], Prop)</tt></a>.</p>
<h3 class="function"><a name="collect-3">collect/3</a></h3>
<div class="spec">
<p><tt>collect(PrintMethod::(list(term())) -&gt; any(), S::term(), Prop::<a href="#type-property">property()</a>) -&gt; <a href="#type-property">property()</a></tt></p>
</div><p>Equivalent to <a href="#aggregate-3"><tt>aggregate(PrintMethod, [S], Prop)</tt></a>.</p>
<h3 class="function"><a name="counterexample-0">counterexample/0</a></h3>
<div class="spec">
<p><tt>counterexample() -&gt; undefined | <a href="#type-counterexample">counterexample()</a></tt></p>
</div><p>Returns the last counter-example found. See <a href="#counterexample-1"><code>counterexample/1</code></a>.</p>
<h3 class="function"><a name="counterexample-1">counterexample/1</a></h3>
<div class="spec">
<p><tt>counterexample(P::<a href="#type-property">property()</a>) -&gt; true | <a href="#type-counterexample">counterexample()</a></tt></p>
</div><p>Tests the property in the same way as <a href="#quickcheck-1"><code>quickcheck/1</code></a>, but if
a test fails, then the failing test case is returned as a counterexample.</p>
<h3 class="function"><a name="counterexamples-0">counterexamples/0</a></h3>
<div class="spec">
<p><tt>counterexamples() -&gt; list({atom(), <a href="#type-counterexample">counterexample()</a>})</tt></p>
</div><p>Returns a list of the counterexamples found by the last call
of <code>eqc:module</code>, paired with the name of the property that failed.</p>
<h3 class="function"><a name="current_counterexample-0">current_counterexample/0</a></h3>
<div class="spec">
<p><tt>current_counterexample() -&gt; <a href="#type-counterexample">counterexample()</a></tt></p>
</div><p>Returns the most recent
counterexample found by QuickCheck. This can be used while
QuickCheck is shrinking a failed test case to follow progress, or if
shrinking must be interrupted, to recover the last failed test case
that QuickCheck had found. The counterexample is fetched from a file
in the current directory.</p>
<h3 class="function"><a name="equals-2">equals/2</a></h3>
<div class="spec">
<p><tt>equals(X::any(), Y::any()) -&gt; <a href="#type-property">property()</a></tt></p>
</div><p>A property which holds if X and Y are equal... and displays
their values when a test fails.</p>
<h3 class="function"><a name="fails-1">fails/1</a></h3>
<div class="spec">
<p><tt>fails(P::<a href="#type-property">property()</a>) -&gt; <a href="#type-property">property()</a></tt></p>
</div><p>A property which succeeds when its argument fails.
Sometimes it is useful to write down properties which do <i>not</i> hold
(even though one might expect them to). This can help prevent misconceptions.
<tt>fails(P)</tt> is tested in the same way as <tt>P</tt>, but
fails only if <tt>P</tt> <i>succeeds</i> 100 times. Thus
<tt>fails(P)</tt> declares that QuickCheck should be able to find
a counter-example to property <tt>P</tt>.</p>
<h3 class="function"><a name="measure-3">measure/3</a></h3>
<div class="spec">
<p><tt>measure(Name::atom() | string(), X::number() | list(number()), Prop::<a href="#type-property">property()</a>) -&gt; <a href="#type-property">property()</a></tt></p>
</div><p>Collects the values of X while testing Prop, and if all tests
pass, displays statistics such as the minimum, average, and maximum
values, identified by the name Name. X can also be a list of values,
in which case all of them are included in the measurements.</p>
<h3 class="function"><a name="module-1">module/1</a></h3>
<div class="spec">
<p><tt>module(Mod::atom()) -&gt; list(atom())</tt></p>
</div><p>Tests all the properties exported from a module, given the module name.
Any function with arity zero whose name begins with "prop_" is treated as a
property. The result is a list of the names of the properties that
failed. See also <a href="#module-2"><code>module/2</code></a>.</p>
<h3 class="function"><a name="numtests-2">numtests/2</a></h3>
<div class="spec">
<p><tt>numtests(N::<a href="#type-int">int()</a>, Prop::<a href="#type-property">property()</a>) -&gt; <a href="#type-property">property()</a></tt></p>
</div><p>Property which is logically equivalent to <tt>Prop</tt>, but is
tested <tt>N</tt> times rather than 100. If numtests appears more than once
in a property, then the outermost use takes precedence.</p>
<h3 class="function"><a name="on_output-2">on_output/2</a></h3>
<div class="spec">
<p><tt>on_output(Fun::(string(), list(term())) -&gt; any(), Prop::<a href="#type-property">property()</a>) -&gt; <a href="#type-property">property()</a></tt></p>
</div><p>Supplies an output function to be used instead of io:format
when QuickCheck generates output. All output generated by
QuickCheck is passed to <tt>Fun</tt>, in the form of a format
string and a list of terms--the same arguments expected by
<tt>io:format</tt>. By supplying a function which does nothing,
QuickCheck can be run silently. By supplying a function which
writes to a file, all QuickCheck output can be saved.
<p>Note that output generated by user code is <i>not</i> passed to
this output function. For example, calls to io:format in the
property, or in the code under test, will generate output in the
shell as usual. This applies even to calls inside a
<tt>?WHENFAIL</tt>. If you want to redirect such output also, then
you need to modify your own code appropriately.</p>
<p>The reason that <tt>Fun</tt> is passed a format string and
arguments, rather than an already formatted string, is to make it
easier to extract information from the output without parsing
it. However, there is no guarantee that different versions of
QuickCheck will use the same format strings and term lists--you use
this information at your own risk, in other words.</p></p>
<h3 class="function"><a name="on_test-2">on_test/2</a></h3>
<div class="spec">
<p><tt>on_test(Fun::(<a href="#type-counterexample">counterexample()</a>, bool()) -&gt; any(), Prop::<a href="#type-property">property()</a>) -&gt; <a href="#type-property">property()</a></tt></p>
</div><p>Attaches a function to a property which is called every time a
test passes or fails. The arguments are the test case (a list of
values), and a boolean indicating whether or not the test
passed. Tests which are skipped (because of an
<tt>?IMPLIES(false,...)</tt>) are not included.</p>
<h3 class="function"><a name="quickcheck-1">quickcheck/1</a></h3>
<div class="spec">
<p><tt>quickcheck(P::<a href="#type-property">property()</a>) -&gt; bool()</tt></p>
</div><p>Tests the property in 100 random cases, printing a counter-example
if one is found. Initially small test cases are generated, then the
size increases as testing progresses (see <a href="eqc_gen.html"><code>eqc_gen</code></a>, <tt>?SIZED</tt>,
<a href="eqc_gen.html#resize-2"><code>eqc_gen:resize/2</code></a> for the way size affects test data generation).
The result is <tt>true</tt> if all tests succeeded (or if one failed,
and failure was expected). On success, <tt>quickcheck</tt> analyses
the distribution of test case labels. On failure, <tt>quickcheck</tt>
tries to simplify the counter-example found as far as possible (see <i>
shrinking</i>, described in <a href="eqc_gen.html"><code>eqc_gen</code></a>).</p>
<h3 class="function"><a name="recheck-1">recheck/1</a></h3>
<div class="spec">
<p><tt>recheck(Prop::<a href="#type-property">property()</a>) -&gt; bool()</tt></p>
</div><p>Tests the property with the <i>same</i> random number seed as
the last failing call of <a href="#quickcheck-1"><code>quickcheck/1</code></a>. If the property is
the same as in that last call, then the same test case will be
generated. Note that recheck repeats the test <i>and its
shrinking</i>. This can be used to adjust the shrinking strategy in
the property, then reshrink the same counterexample, perhaps to a
better result. If you just
want to repeat the <i>shrunk</i> test, then use
<pre>eqc:check(Prop,eqc:counterexample())</pre> instead.
<p><b>Note:</b> the type and behaviour of recheck changed in version 1.19.</p></p>
<h3 class="function"><a name="start-0">start/0</a></h3>
<div class="spec">
<p><tt>start() -&gt; any()</tt></p>
</div><p>Equivalent to <a href="#start-1"><tt>start(true)</tt></a>.</p>
<h3 class="function"><a name="start-1">start/1</a></h3>
<div class="spec">
<p><tt>start(Force::bool()) -&gt; pid()</tt></p>
</div><p><p>Starts the QuickCheck server. If it is already running on this
node, nothing is done.</p>
Each user can run only one instance of the QuickCheck server at a
time. If the server is already running on another Erlang node, it
will be terminated automatically if <tt>Force</tt> is
<tt>true</tt>. If another instance is running, and <tt>Force</tt> is
<tt>false</tt>, then the new instance will not start.</p>
<h3 class="function"><a name="stop-0">stop/0</a></h3>
<div class="spec">
<p><tt>stop() -&gt; any()</tt></p>
</div><p>Stops the QuickCheck server.
QuickCheck properties are tested in the QuickCheck server process, which is
spawned automatically when quickcheck is first called. Usually there is no
need to stop the QuickCheck server explicitly, but if a need does arise
then this function can be used. For example, if the shell process crashes
and is restarted, then the QuickCheck server should be stopped and restarted
too, since otherwise the server will crash when it attempts to write to the
console.</p>
<h3 class="function"><a name="version-0">version/0</a></h3>
<div class="spec">
<p><tt>version() -&gt; any()</tt></p>
</div>
<h3 class="function"><a name="with_title-1">with_title/1</a></h3>
<div class="spec">
<p><tt>with_title(Title::atom() | string()) -&gt; <a href="#type-print_method">print_method()</a></tt></p>
</div><p>A printing method for collected data, which displays a title
before
the percentages of each value in the data. It is intended to be
passed to <a href="#collect-3"><code>collect/3</code></a> or <a href="#aggregate-3"><code>aggregate/3</code></a>.</p>
<hr>
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
<p><i>Generated by EDoc, Jun 13 2010, 13:15:30.</i></p>
</body>
</html>