You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: joss/paper.md
+9-20Lines changed: 9 additions & 20 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -41,7 +41,7 @@ affiliations:
41
41
index: 2
42
42
date: February 2022
43
43
44
-
[//]: # (bibliography: paper.bib)
44
+
bibliography: paper.bib
45
45
---
46
46
47
47
# Summary
@@ -58,7 +58,7 @@ If a SUT uses any external services (e.g., a SQL database), these will be automa
58
58
59
59
# Statement of Need
60
60
61
-
This collection of SUTs was originally assembled for easing experimentation with the fuzzer called [EvoMaster](http://evomaster.org).
61
+
This collection of SUTs was originally assembled for easing experimentation with the fuzzer called _EvoMaster_[@arcuri2021evomaster].
62
62
However, finding this type of applications is not trivial among open-source projects.
63
63
Furthermore, it is not simple to sort out all the technical details on how to set these applications up and start them in a simple, uniform approach.
64
64
@@ -76,18 +76,11 @@ Note that some of these open-source projects might be no longer supported, where
76
76
Once a system is added to EMB, we do not modify nor keep it updated with its current version under development.
77
77
The reason is that we want to keep an easy to use, constant set of case studies for experimentation that can be reliably used throughout the years.
78
78
79
-
The SUTs called _NCS_ (Numerical Case Study) and _SCS_ (String Case study) are artificial, developed by us.
79
+
The SUTs called _NCS_ (Numerical Case Study) and _SCS_ (String Case Study) are artificial, developed by us.
80
80
They are based on numerical and string-based functions previously used in the literature of unit test generation.
81
81
We just re-implemented in different languages, and put them behind a web service.
82
82
83
83
84
-
Due to several reasons, the software in this repository is not published as a library (e.g., on Maven and NPM).
85
-
To use EMB, you need to clone this repository:
86
-
87
-
```
88
-
git clone https://github.com/EMResearch/EMB.git
89
-
```
90
-
91
84
There are 2 main use cases for EMB:
92
85
93
86
* Run experiments with _EvoMaster_
@@ -102,17 +95,9 @@ Once the script is completed, all the SUTs will be available under the `dist` fo
102
95
103
96
Note that here the drivers will be built as well besides the SUTs, and the SUT themselves will also have an instrumented version (for white-box testing heuristics) for _EvoMaster_ (this is for JavaScript and .Net, whereas instrumentation for JVM is done at runtime, via an attached JavaAgent).
104
97
105
-
In the built `dist` folder, the files will be organized as follows:
106
-
107
-
* For JVM: `<name>-sut.jar` will be the non-instrumented SUTs, whereas their executable drivers will be called `<name>-evomaster-runner.jar`.
108
-
Instrumentation can be done at runtime by attaching the `evomaster-agent.jar` JavaAgent. If you are running experiments with EvoMaster, this will be automatically attached when running experiments with `exp.py` (available in the EvoMaster's repository). Or it can be attached manually with JVM option `-Devomaster.instrumentation.jar.path=evomaster-agent.jar` when starting the driver.
109
-
* For NodeJS: under the folder `<name>` (for each NodeJS SUT), the SUT is available under `src`, whereas the instrumented version is under `build`.
110
-
* For .Net: currently only the instrumented version is available (WORK IN PROGRESS)
111
98
112
-
113
-
114
-
For running experiments with EvoMaster, you can also "start" each driver directly from an IDE (e.g., IntelliJ).
115
-
Each of these drivers has a "main" method that is running a REST API (binding on default port 40100), where each operation (like start/stop/reset the SUT) can be called via an HTTP message by EvoMaster.
99
+
For running experiments with _EvoMaster_, you can also "start" each driver directly from an IDE (e.g., IntelliJ).
100
+
Each of these drivers has a "main" method that is running a REST API (binding on default port 40100), where each operation (like start/stop/reset the SUT) can be called via an HTTP message by _EvoMaster_.
116
101
For JavaScript, you need to use the files `em-main.js`.
117
102
118
103
@@ -142,8 +127,12 @@ The difference is that in External the SUT is started on a separated process, an
142
127
143
128
# Published Results
144
129
130
+
The software implemented for this repository has been already used for some years when carrying out scientific research with the _EvoMaster_ fuzzer (e.g., [@arcuri2019restful; @arcuri2021tt]).
131
+
Other research groups have started to use EMB as well for their experiments (e.g., [@stallenberg2021improving]).
132
+
145
133
# Related Work
146
134
135
+
As far as we know, there is no existing corpus of web/enterprise applications where scripts/libraries have been provided to easily handle them (e.g., to start/stop/reset them in a programmatic way, including running necessary dependencies like databases via Docker).
0 commit comments