Skip to content

Use the full SV-COMP time and memory budget #51

Open

Description

@nils-loose

Make sure SWAT uses the full SV-COMP resource budget. bench-defs swat.xml: timelimit="15 min" (CPU time), hardtimelimit="16 min", memlimit="15 GB", cpuCores="4".

Already done on dev: internal 180 s run cutoff and 60 s Z3 timeout removed (#32).

Still to check:

  • CPU time vs. wall time: JVM (GC/JIT threads) + explorer + Z3 consume CPU time in parallel, so the 15 min CPU budget may run out well before 15 min wall time — measure on long-running tasks
  • Memory: SVCompDriver.build_command passes -Xmx32g (default mem=32), above the 15 GB limit, and explorer memory comes on top — set heap so JVM + explorer stay under 15 GB
  • No remaining caps that end exploration early (iteration limits, per-run timeouts, e.g. the unused 60 s default in svcomp/SVCompHandler.py)
  • Verify with a BenchExec run using the competition limits (count TIMEOUT / OUT OF MEMORY vs. unknown)

Activity

  1. added this to the SV-COMP 2027 milestone on Oct 15, 2026
  2. nils-loose commented on Oct 15, 2026

    @nils-loose
    ContributorAuthor

    Data from the full Java run (2026-10-15, no memory limit enforced): 21 runs grew past 15 GB RSS with -Xmx32g (up to 72 GB). In SV-COMP they would end as OUT OF MEMORY, including 8 currently correct SAFE results (BellmanFord-, Tsp-). Tasks: BellmanFord-, Tsp-, BufferedBlockCipher, DistinguishedNameParser, jbmc-regression/array1. Suggest -Xmx12g or deriving it from the limit. Full list: section "Memory" in the local campaign report.

  3. nils-loose commented on Oct 20, 2026

    @nils-loose
    ContributorAuthor

    #88 (merged into dev through #92) covers most of this:

    • a shared memory and CPU budget monitor;
    • the executor heap is derived from the limit;
    • the CPU time of the whole cgroup is counted, and the run ends with UNKNOWN 20 s before the limit;
    • the dead SVCompHandler.py is removed.

    Still open:

    • a full BenchExec campaign with the competition limits;
    • a per-round executor budget (runs with 2^30 iterations still use CPU until the deadline).
  4. solid-eureka commented on Oct 27, 2026

    @solid-eureka
    Contributor

    Part of this is in #109: a wall-clock deadline in ResourceBudget (a run stops before the wall limit as well as the CPU limit) and an executor DeadlockWatchdog (a deadlocked run ends with UNKNOWN instead of hanging until the wall timeout). The BenchExec campaign and a per-round executor budget are still open.

    🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions