You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/src/code/writing-tests.adoc
+33-58Lines changed: 33 additions & 58 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,23 +3,17 @@
3
3
4
4
== The test framework
5
5
6
-
The code base has unit and integration tests that can be executed
7
-
automatically to ensure the program works as intended. Such tests are
8
-
often written to trigger a bug and to ensure the bug is detected if it
9
-
resurfaces in the future, but also to validate behavior of components
10
-
and interfaces.
6
+
The code base has unit and integration tests that can be executed automatically to ensure the program works as intended.
7
+
Such tests are often written to trigger a bug and to ensure the bug is detected if it resurfaces in the future, but also to validate behavior of components and interfaces.
11
8
12
-
The tests are collected in the `tests/` directory. The individual tests
13
-
are in subdirectories of this directory. The tests are grouped in
14
-
directories.
9
+
The tests are collected in the `tests/` directory.
10
+
The individual tests are in subdirectories of this directory.
11
+
The tests are grouped in directories.
15
12
16
13
== Running tests
17
14
18
-
The tests are executed by the `scripts/runtests` script generated from
19
-
`scripts/runtests.in` during build. The runtests script will by default
20
-
locate tests to run under `tests/`, but can be limited to only run a
21
-
limited set of tests by specifying the directory of the test or tests
22
-
as argument(s).
15
+
The tests are executed by the `scripts/runtests` script generated from `scripts/runtests.in` during build.
16
+
The runtests script will by default locate tests to run under `tests/`, but can be limited to only run a limited set of tests by specifying the directory of the test or tests as argument(s).
23
17
24
18
Here is an example running only the tests in `tests/lathe/`
The runtests script looks for all files named _test_, _test.sh_ and
33
-
_test.hal_ below the directories specified on the command line, or under
34
-
`tests/` if no command line argument is specified. These files
35
-
represent three different ways to run the tests.
26
+
The runtests script looks for all files named `test`, `test.sh` or `test.hal` below the directories specified on the command line, or under `tests/` if no command line argument is specified.
27
+
These files represent three different ways to run the tests.
36
28
37
-
The _runtests_ script accepts the following arguments, see the output
38
-
from `scripts/runtests -h` for the authorative list:
29
+
The _runtests_ script accepts the following arguments, see the output from `scripts/runtests -h` for the authorative list:
39
30
----
40
31
-n do not remove temporary files for successful tests.
41
32
-s stop after any failed test.
@@ -46,79 +37,63 @@ from `scripts/runtests -h` for the authorative list:
46
37
----
47
38
== Writing tests
48
39
49
-
Make sure the test can run successfully without a working X11 display,
50
-
i.e. with the DISPLAY environment variable unset.
40
+
Make sure the test can run successfully without a working X11 display, i.e. with the DISPLAY environment variable unset.
51
41
52
42
1. Create a folder in `tests/`.
53
43
2. Provide one test script.
54
44
3. Evaluate the output with one of the options below.
55
45
56
-
These are the files considered in the directory with the individual
57
-
tests:
46
+
These are the files considered in the directory with the individual tests:
58
47
59
48
.Test script (only one of these three)
60
49
61
50
test::
62
-
A program that is executed and its exit code and output checked using
63
-
either checkresult or expected.
51
+
A program that is executed and its exit code and output checked using either checkresult or expected.
64
52
65
53
test.sh::
66
-
A bash script that is executed and its exit code and output checked using
67
-
either checkresult or expected.
54
+
A bash script that is executed and its exit code and output checked using either checkresult or expected.
68
55
69
56
test.hal::
70
-
A HAL script that is executed using `halrun -f test.hal` and its exit code
71
-
and output checked using either checkresult or expected.
57
+
A HAL script that is executed using `halrun -f test.hal` and its exit code and output checked using either checkresult or expected.
72
58
73
59
.Test evaluation
74
60
75
61
expected::
76
-
A file whose content is compared to the output from running the test
77
-
scripts. If the test output is identical to the content of the
78
-
expected file, the test succeeds.
62
+
A file whose content is compared to the output from running the test scripts.
63
+
If the test output is identical to the content of the expected file, the test succeeds.
79
64
80
65
checkresult::
81
-
An excutable file to perform more complex validation than just comparing
82
-
the output of a test script. It gets the filename of the test program as
83
-
its command line argument. The exit code of this program controls the result
84
-
of the test. If both `expected` and `checkresult` exist, only `checkresult`
85
-
is consulted to validate the test output.
66
+
An excutable file to perform more complex validation than just comparing the output of a test script.
67
+
It gets the filename of the test program as its command line argument.
68
+
The exit code of this program controls the result of the test.
69
+
If both `expected` and `checkresult` exist, only `checkresult` is consulted to validate the test output.
86
70
87
71
xfail::
88
-
If this file exist, a test failure is expected and does not cause
89
-
runtests to return an exit code signaling an error.
72
+
If this file exist, a test failure is expected and does not cause runtests to return an exit code signaling an error.
90
73
91
74
skip::
92
75
If this file exist, the test is skipped and not executed at all.
93
76
94
77
control::
95
-
This file can be used to flag specific needs in the test. At the
96
-
moment, the use of _sudo_ can be flagged, and tests requiring sudo
97
-
can be skipped when using `runtests -u`. To flag such requirements,
98
-
add a line with `Restrictions: sudo` to this file.
78
+
This file can be used to flag specific needs in the test.
79
+
At the moment, the use of _sudo_ can be flagged, and tests requiring sudo can be skipped when using `runtests -u`.
80
+
To flag such requirements, add a line with `Restrictions: sudo` to this file.
99
81
100
82
== Some testing approaches
101
83
102
-
There are various ways to structure a test, depending on what one wants
103
-
to test. Here are a few ideas on how to do it.
84
+
There are various ways to structure a test, depending on what one wants to test.
85
+
Here are a few ideas on how to do it.
104
86
105
87
=== Non-interactive "GUI"
106
88
107
-
If you want to test some operations in the user interface, a useful
108
-
approach is is to write a custom "GUI" simulating the operations.
109
-
This can be done by creating a normal LinuxCNC setup and pointing the
110
-
[DISPLAY] DISPLAY value to a script that does the operations needed to
111
-
test the behaviour.
89
+
If you want to test some operations in the user interface, a useful approach is is to write a custom "GUI" simulating the operations.
90
+
This can be done by creating a normal LinuxCNC setup and pointing the [DISPLAY] DISPLAY value to a script that does the operations needed to test the behaviour.
112
91
113
-
Examples of this approach can be found in `tests/halui/jogging/` and
114
-
`tests/interp/subroutine-return/`.
92
+
Examples of this approach can be found in `tests/halui/jogging/` and `tests/interp/subroutine-return/`.
115
93
116
94
=== Recording HAL pin transitions
117
95
118
-
Using the _sampler_ and _halsampler_ HAL components, one can set up a
119
-
HAL configuration and collect pin value settings and changes and dump
120
-
the result to stdout (or a file). The end result can then be compared
121
-
with the expected output to verify if HAL behaves as expected.
96
+
Using the _sampler_ and _halsampler_ HAL components, one can set up a HAL configuration and collect pin value settings and changes and dump the result to stdout (or a file).
97
+
The end result can then be compared with the expected output to verify if HAL behaves as expected.
122
98
123
-
Examples of this approach can be found in `tests/multiclick/` and
124
-
`tests/stepgen.2/`.
99
+
Examples of this approach can be found in `tests/multiclick/` and `tests/stepgen.2/`.
0 commit comments