Skip to content

Commit a894f16

Browse files
committed
JOSS paper
1 parent dd7ae97 commit a894f16

2 files changed

Lines changed: 51 additions & 20 deletions

File tree

joss/paper.bib

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
@article{arcuri2021evomaster,
2+
title={EvoMaster: A Search-Based System Test Generation Tool},
3+
author={Arcuri, Andrea and Galeotti, Juan Pablo and Marculescu, Bogdan and Zhang, Man},
4+
journal={Journal of Open Source Software},
5+
volume={6},
6+
number={57},
7+
pages={2153},
8+
year={2021}
9+
}
10+
11+
@article{arcuri2019restful,
12+
title={{RESTful API Automated Test Case Generation with EvoMaster}},
13+
author={Arcuri, Andrea},
14+
journal={{ACM Transactions on Software Engineering and Methodology (TOSEM)}},
15+
volume={28},
16+
number={1},
17+
pages={3},
18+
year={2019},
19+
publisher={ACM},
20+
doi={10.1145/3293455}
21+
}
22+
23+
24+
@article{arcuri2021tt,
25+
title={Enhancing Search-based Testing with Testability Transformations for Existing APIs},
26+
author={Arcuri, Andrea and Galeotti, Juan P},
27+
journal={ACM Transactions on Software Engineering and Methodology (TOSEM)},
28+
volume={31},
29+
number={1},
30+
pages={1--34},
31+
year={2021},
32+
publisher={ACM New York, NY}
33+
}
34+
35+
@inproceedings{stallenberg2021improving,
36+
title={Improving Test Case Generation for REST APIs Through Hierarchical Clustering},
37+
author={Stallenberg, Dimitri and Olsthoorn, Mitchell and Panichella, Annibale},
38+
booktitle={2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)},
39+
pages={117--128},
40+
year={2021},
41+
organization={IEEE}
42+
}

joss/paper.md

Lines changed: 9 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ affiliations:
4141
index: 2
4242
date: February 2022
4343

44-
[//]: # (bibliography: paper.bib)
44+
bibliography: paper.bib
4545
---
4646

4747
# Summary
@@ -58,7 +58,7 @@ If a SUT uses any external services (e.g., a SQL database), these will be automa
5858

5959
# Statement of Need
6060

61-
This collection of SUTs was originally assembled for easing experimentation with the fuzzer called [EvoMaster](http://evomaster.org).
61+
This collection of SUTs was originally assembled for easing experimentation with the fuzzer called _EvoMaster_ [@arcuri2021evomaster].
6262
However, finding this type of applications is not trivial among open-source projects.
6363
Furthermore, it is not simple to sort out all the technical details on how to set these applications up and start them in a simple, uniform approach.
6464

@@ -76,18 +76,11 @@ Note that some of these open-source projects might be no longer supported, where
7676
Once a system is added to EMB, we do not modify nor keep it updated with its current version under development.
7777
The reason is that we want to keep an easy to use, constant set of case studies for experimentation that can be reliably used throughout the years.
7878

79-
The SUTs called _NCS_ (Numerical Case Study) and _SCS_ (String Case study) are artificial, developed by us.
79+
The SUTs called _NCS_ (Numerical Case Study) and _SCS_ (String Case Study) are artificial, developed by us.
8080
They are based on numerical and string-based functions previously used in the literature of unit test generation.
8181
We just re-implemented in different languages, and put them behind a web service.
8282

8383

84-
Due to several reasons, the software in this repository is not published as a library (e.g., on Maven and NPM).
85-
To use EMB, you need to clone this repository:
86-
87-
```
88-
git clone https://github.com/EMResearch/EMB.git
89-
```
90-
9184
There are 2 main use cases for EMB:
9285

9386
* Run experiments with _EvoMaster_
@@ -102,17 +95,9 @@ Once the script is completed, all the SUTs will be available under the `dist` fo
10295

10396
Note that here the drivers will be built as well besides the SUTs, and the SUT themselves will also have an instrumented version (for white-box testing heuristics) for _EvoMaster_ (this is for JavaScript and .Net, whereas instrumentation for JVM is done at runtime, via an attached JavaAgent).
10497

105-
In the built `dist` folder, the files will be organized as follows:
106-
107-
* For JVM: `<name>-sut.jar` will be the non-instrumented SUTs, whereas their executable drivers will be called `<name>-evomaster-runner.jar`.
108-
Instrumentation can be done at runtime by attaching the `evomaster-agent.jar` JavaAgent. If you are running experiments with EvoMaster, this will be automatically attached when running experiments with `exp.py` (available in the EvoMaster's repository). Or it can be attached manually with JVM option `-Devomaster.instrumentation.jar.path=evomaster-agent.jar` when starting the driver.
109-
* For NodeJS: under the folder `<name>` (for each NodeJS SUT), the SUT is available under `src`, whereas the instrumented version is under `build`.
110-
* For .Net: currently only the instrumented version is available (WORK IN PROGRESS)
11198

112-
113-
114-
For running experiments with EvoMaster, you can also "start" each driver directly from an IDE (e.g., IntelliJ).
115-
Each of these drivers has a "main" method that is running a REST API (binding on default port 40100), where each operation (like start/stop/reset the SUT) can be called via an HTTP message by EvoMaster.
99+
For running experiments with _EvoMaster_, you can also "start" each driver directly from an IDE (e.g., IntelliJ).
100+
Each of these drivers has a "main" method that is running a REST API (binding on default port 40100), where each operation (like start/stop/reset the SUT) can be called via an HTTP message by _EvoMaster_.
116101
For JavaScript, you need to use the files `em-main.js`.
117102

118103

@@ -142,8 +127,12 @@ The difference is that in External the SUT is started on a separated process, an
142127

143128
# Published Results
144129

130+
The software implemented for this repository has been already used for some years when carrying out scientific research with the _EvoMaster_ fuzzer (e.g., [@arcuri2019restful; @arcuri2021tt]).
131+
Other research groups have started to use EMB as well for their experiments (e.g., [@stallenberg2021improving]).
132+
145133
# Related Work
146134

135+
As far as we know, there is no existing corpus of web/enterprise applications where scripts/libraries have been provided to easily handle them (e.g., to start/stop/reset them in a programmatic way, including running necessary dependencies like databases via Docker).
147136

148137

149138
# Acknowledgements

0 commit comments

Comments
 (0)