Added QuickCheck references, moved Testing section around and added some text
This commit is contained in:
parent
006bc912b2
commit
615e9d163f
2 changed files with 157 additions and 59 deletions
|
@ -308,4 +308,22 @@ This paper details the motivations and design choices we made in Thrift, as well
|
|||
month = "04",
|
||||
day = "24",
|
||||
url = "http://news.cnet.com/World-of-Warcraft-battles-server-problems/2100-1043_3-6063990.html"
|
||||
}
|
||||
}
|
||||
|
||||
@inproceedings{Arts:2006:TTS:1159789.1159792,
|
||||
author = {Arts, Thomas and Hughes, John and Johansson, Joakim and Wiger, Ulf},
|
||||
title = {Testing telecoms software with quviq QuickCheck},
|
||||
booktitle = {Proceedings of the 2006 ACM SIGPLAN workshop on Erlang},
|
||||
series = {ERLANG '06},
|
||||
year = {2006},
|
||||
isbn = {1-59593-490-1},
|
||||
location = {Portland, Oregon, USA},
|
||||
pages = {2--10},
|
||||
numpages = {9},
|
||||
url = {http://doi.acm.org/10.1145/1159789.1159792},
|
||||
doi = {http://doi.acm.org/10.1145/1159789.1159792},
|
||||
acmid = {1159792},
|
||||
publisher = {ACM},
|
||||
address = {New York, NY, USA},
|
||||
keywords = {property based testing, test automation},
|
||||
}
|
||||
|
|
196
report.lyx
196
report.lyx
|
@ -6431,6 +6431,144 @@ Hot code replacement is a technique used to update systems while they are
|
|||
some work it could be implemented into the GGS.
|
||||
\end_layout
|
||||
|
||||
\begin_layout Section
|
||||
Software testing
|
||||
\end_layout
|
||||
|
||||
\begin_layout Standard
|
||||
In order to make sure the GGS prototype adheres to the specification set
|
||||
two different approaches to software testing are used.
|
||||
For simpler testing the GGS prototype uses unit tests.
|
||||
Modules are tested on a high level, making sure each function in the module
|
||||
tested functiions according to specification.
|
||||
\end_layout
|
||||
|
||||
\begin_layout Standard
|
||||
Unit testing is not employed to test the system from the client side.
|
||||
In order to more accurately simulate real users some randomization is needed
|
||||
\begin_inset Note Note
|
||||
status open
|
||||
|
||||
\begin_layout Plain Layout
|
||||
citation needed
|
||||
\end_layout
|
||||
|
||||
\end_inset
|
||||
|
||||
, as users do not always act rationally.
|
||||
In order to introduce random data, the client side of the GGS is simulated
|
||||
by QuickCheck tests.
|
||||
\end_layout
|
||||
|
||||
\begin_layout Subsection
|
||||
Unit testing
|
||||
\end_layout
|
||||
|
||||
\begin_layout Standard
|
||||
Unit testing is a way to check if the functionality adheres to the specification
|
||||
of the system by manually creating test cases for sections of code.
|
||||
In most cases whole functions.
|
||||
Unit testing is good, not only for revealing software bugs, but also to
|
||||
state that a feature is working according to the specification.
|
||||
Unit testing is a common way to test software and has proven useful within
|
||||
the GGS when functions take complicated arguments.
|
||||
In these cases it is easy to set up a scenario that should work.
|
||||
|
||||
\end_layout
|
||||
|
||||
\begin_layout Standard
|
||||
Unit testing is a useful way to create regression tests.
|
||||
Regression tests are used to make sure changes made to the GGS do not introduce
|
||||
new bugs or break the specification.
|
||||
The regression tests are optimally run very often, such as after each change
|
||||
to the code.
|
||||
\end_layout
|
||||
|
||||
\begin_layout Standard
|
||||
\begin_inset Note Note
|
||||
status open
|
||||
|
||||
\begin_layout Plain Layout
|
||||
Erlang provides a module for unit testing called eunit.
|
||||
Eunit, being a part of OTP, is rich in functionality and well documented
|
||||
yet it doesn't allow any means of testing asynchronous behaviours as opposed
|
||||
to other means of software testing.
|
||||
\end_layout
|
||||
|
||||
\end_inset
|
||||
|
||||
|
||||
\end_layout
|
||||
|
||||
\begin_layout Subsection
|
||||
Automated test case generation
|
||||
\end_layout
|
||||
|
||||
\begin_layout Standard
|
||||
The problem of writing software tests manually, is that it takes a lot of
|
||||
time.
|
||||
There exists other ways to test software that address this problem by generatin
|
||||
g test cases with certain properties.
|
||||
This allows for testing functions with a lot of different input parameters
|
||||
without having to implement each specific test itself.
|
||||
|
||||
\end_layout
|
||||
|
||||
\begin_layout Standard
|
||||
By having each test automatically generated, each test can be very complex
|
||||
and long.
|
||||
In order to generate random, complex tests the GGS uses QuickCheck.
|
||||
By using QuickCheck the GGS can be tested with input which would be extremely
|
||||
difficult to construct using manual testing methods.
|
||||
Regression tests, such as the unit tests used by the GGS are more useful
|
||||
for ensuring the system does not diverge from a working scenario than for
|
||||
finding new cases where the specification does not hold
|
||||
\begin_inset CommandInset citation
|
||||
LatexCommand citet
|
||||
key "Arts:2006:TTS:1159789.1159792"
|
||||
|
||||
\end_inset
|
||||
|
||||
.
|
||||
\end_layout
|
||||
|
||||
\begin_layout Standard
|
||||
The entire GGS was not tested using QuickCheck, nor was the entire client
|
||||
protocol for a game tested using QuickCheck, however the tests performed
|
||||
using QuickCheck show that an automated testing system such as QuickCheck
|
||||
is a very viable testing method for the GGS.
|
||||
\end_layout
|
||||
|
||||
\begin_layout Standard
|
||||
QuickCheck has features to generate very large and complex tests, the results
|
||||
of which can be hard to analyze.
|
||||
The solution to reading these complex test is to extract a
|
||||
\emph on
|
||||
minimal failing test case
|
||||
\emph default
|
||||
which contains the smalles failing test sequence.
|
||||
By applying a very large test and gradually simplifying the test to find
|
||||
the smallest failing sequence, many bugs which would other wise have been
|
||||
hard to catch can be caught
|
||||
\begin_inset CommandInset citation
|
||||
LatexCommand citet
|
||||
key "Arts:2006:TTS:1159789.1159792"
|
||||
|
||||
\end_inset
|
||||
|
||||
.
|
||||
\end_layout
|
||||
|
||||
\begin_layout Standard
|
||||
QuickCheck was originally made for the programming language Haskell.
|
||||
There are a lot of reimplementations of QuickCheck in various programming
|
||||
languages.
|
||||
Erlang QuickCheck (EQC) and Triq are two variants of QuickCheck for Erlang.
|
||||
EQC was chosen for testing the GGS.
|
||||
Besides the standard functionality that QuickCheck provides, EQC is capable
|
||||
of testing concurrency within a program.
|
||||
\end_layout
|
||||
|
||||
\begin_layout Section
|
||||
Case studies
|
||||
\end_layout
|
||||
|
@ -7366,64 +7504,6 @@ The use of Thrift, Google protocol buffers - which is a different approach
|
|||
the writing of this thesis.
|
||||
\end_layout
|
||||
|
||||
\begin_layout Section
|
||||
Software testing
|
||||
\end_layout
|
||||
|
||||
\begin_layout Subsection
|
||||
Unit testing
|
||||
\end_layout
|
||||
|
||||
\begin_layout Standard
|
||||
Unit testing is a way to check if functionality works by manually creating
|
||||
test cases for sections of code.
|
||||
In most cases whole functions.
|
||||
Unit testing is good, not only for revealing software bugs, but also to
|
||||
state that a feature is working according to the specification.
|
||||
Unit testing is a common way to test software and has proven useful within
|
||||
the GGS when functions takes complicated arguments.
|
||||
In these cases it is easy to set up a scenario that should work.
|
||||
|
||||
\end_layout
|
||||
|
||||
\begin_layout Standard
|
||||
Erlang provides a module for unit testing called eunit.
|
||||
Eunit, being a part of OTP, is rich in functionality and well documented
|
||||
yet it doesn't allow any means of testing asynchronous behaviours as opposed
|
||||
to other means of software testing.
|
||||
\end_layout
|
||||
|
||||
\begin_layout Subsection
|
||||
Automated test case generation
|
||||
\end_layout
|
||||
|
||||
\begin_layout Standard
|
||||
The problem of writing software tests manually, is that it takes a lot of
|
||||
time.
|
||||
There exists other ways to test software that address this problem by generatin
|
||||
g test cases with certain properties.
|
||||
This allows for testing functions with a lot of different input parameters
|
||||
without having to implement each specific test itself.
|
||||
The drawback of generating test cases instead of manually creating them,
|
||||
is when the types or structures of the parameters of a function are complicated.
|
||||
This makes it hard to find suitable rules for the tests to generate.
|
||||
|
||||
\end_layout
|
||||
|
||||
\begin_layout Standard
|
||||
One tool used for automatic test case generation is QuickCheck originally
|
||||
made for the programming language Haskell.
|
||||
There are a lot of reimplementations of QuickCheck in various programming
|
||||
languages.
|
||||
Erlang QuickCheck(EQC) and Triq are two variants of QuickCheck for Erlang.
|
||||
The one desided to be used within the GGS is EQC.
|
||||
Besides the standard functionality that QuickCheck provides, EQC is capable
|
||||
of testing concurrency within a program.
|
||||
Because of EQC being a commercial product, a lot of features of EQC are
|
||||
unused, including the means of testing concurrency.
|
||||
|
||||
\end_layout
|
||||
|
||||
\begin_layout Chapter
|
||||
Results and discussion
|
||||
\begin_inset CommandInset label
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue