declarativeIO

This repository is an artifact for the OSDI'26 paper Scaling the IO wall with Declarative IO.

Simulator Dependencies

• C++ 17 (e.g., GCC 10+ or Clang 11+ for <execution> support)
• CMake >= 3.17
• spdlog: Fast C++ logging library. (Install from source on Ubuntu18)
• libconfig++: C++ Configuration File Library.
• TBB (libtbb-dev): Threading Building Blocks for parallel execution.
• ZLIB (zlib1g-dev): Compression library.
• SQLite3 (libsqlite3-dev)

On Ubuntu/Debian, you can install most of the simulator dependencies with:

cd simulator
./install_dependencies.sh

Simulator Build

Navigate to the simulator/ directory to build:

cd simulator
mkdir build
cd build
cmake ..
make -j $(nproc)

To build for debugging, configure CMake with the debug flag:

cmake -DCMAKE_BUILD_TYPE=DEBUG ..

Simulator Usage

Make sure you are in the simulator/ directory when running the simulator:

cd simulator
./run.sh <configpath>

Example:

./run.sh ../configs/configs/clusterA.cfg

clusterA.cfg simulates a small cluster for quick verification. Running the full 100 PB simulation for results shown in the paper takes several hours.

By default, the log file can be found in the logs/ directory, and the simulation results JSON file can be found in the output/ directory. The generated log and JSON files will share the same base name as the provided config file.

Analysis

To generate a graph showing the breakdown of savings, navigate to the analysis/ folder from the root directory and run the scripts using the following generic format:

cd analysis
python3 io.py <output_directory> <outputfile.json>
python3 cache-hit-sources.py <output_directory> <outputfile.json>

Example (using the output from the clusterA.cfg run):

cd analysis
python3 io.py results/test/ae ../simulator/output/clusterA.json
python3 cache-hit-sources.py results/test/ae_breakdown ../simulator/output/clusterA.json