Skip to content

Local Files

Jump to bottom
Petrus Pradella edited this page Jul 5, 2026 · 6 revisions

Local Files

What this page covers: opening the local-file backend with Storages.createLocalFile, its one-arg LocalFileConfig (a base directory), the one-file-per-entity on-disk layout, the fact that this backend accepts a non-JSON (YAML) codec (one of the two file backends that do — see also Grouped Files), how queries work as a correct-but-slow full scan (no real index, yet still validating index declarations), crash-safe atomic writes, and the two things it can't do — no transactions, no index acceleration.

📌 Note — switching to local files is a one-line change at construction. From storage.repository(...) onward your code is identical to every other backend. See Choosing a Backend for the full capability matrix.

The 30-second version

import br.com.finalcraft.everydatabase.*;
import br.com.finalcraft.everydatabase.codec.JacksonJsonCodec;
import br.com.finalcraft.everydatabase.modules.localfile.LocalFileConfig;

import java.nio.file.Paths;
import java.util.Optional;
import java.util.UUID;

// 1. Describe the entity once — key type FIRST, entity type second.
EntityDescriptor<UUID, PlayerData> PLAYERS = EntityDescriptor.builder(UUID.class, PlayerData.class)
        .collection("players")                                 // becomes a sub-directory on disk
        .keyExtractor(PlayerData::getUuid)
        .codec(new JacksonJsonCodec<>(PlayerData.class))       // JSON here; YAML also allowed (below)
        .build();

// 2. Open the backend — just point it at a directory.
LocalFileStorage storage = Storages.createLocalFile(
        new LocalFileConfig(Paths.get("data")));
storage.init().join();                                         // creates the base directory if absent

// 3. Use it — exactly like any other backend.
Repository<UUID, PlayerData> repo = storage.repository(PLAYERS);
UUID id = UUID.randomUUID();
repo.save(new PlayerData(id, "Alice", 100)).join();           // writes data/players/<uuid>.json
Optional<PlayerData> alice = repo.find(id).join();            // -> Optional[Alice]

storage.close().join();

createLocalFile returns the concrete LocalFileStorage type. The PlayerData entity is the same plain Jackson POJO used throughout the wiki (Quick Start).

📌 Note — every I/O call returns a CompletableFuture. .join() is shown for brevity; compose with thenApply / thenCompose in real code. There are no blocking variants — see The Async API.

Configuration: LocalFileConfig

One constructor — just the base directory:

import java.nio.file.Paths;

new LocalFileConfig(Paths.get("data"));
Argument Type Meaning
baseDirectory Path root directory; each collection is a sub-directory under it

💡 Tip — file formatting is the codec's job, not the config's. For readable JSON use JacksonJsonCodec.pretty(Type.class) (the plain JacksonJsonCodec is compact); JacksonYamlCodec is inherently readable. See the YAML section below and Codecs.

On disk: one file per entity

Each collection is a sub-directory of the base directory; each entity is its own file named after its key:

data/
  _schema_migrations.json          # reserved: applied migration versions
  players/
    5f1e8400-e29b-41d4-a716-446655440000.json
    9a2c…-….json

The filename is key.toString() plus the codec's file extension (.json / .yml). Path-separator characters (/, \, :) are sanitised to _ to prevent directory traversal; when two distinct keys would sanitise to the same name, a short hash of the original key is appended to keep one file per key. An over-long key is hash-truncated — a short prefix plus a stable hash of the full key — so it still maps to exactly one file and stays portable across filesystems. The key contract is otherwise the usual one — a stable, unique toString() of ≤ 255 characters — see Entities, Keys & Collections. Collection names must match ^[a-zA-Z][a-zA-Z0-9_]*$.

📌 Note — writes are crash-atomic. Each save goes to a sibling .tmp file and is then moved over the target with ATOMIC_MOVE (falling back when the filesystem can't do an atomic move), so a crash mid-write never leaves a truncated entity file — at worst an orphan .tmp. This guards against torn files, not power loss: the write is not fsynced, so if the OS hasn't yet flushed its page cache, the very last save can still be lost on a power cut. Crash-atomic, not durable.

A file backend that accepts YAML

Local files treat the serialized payload as opaque bytes, so — together with Grouped Files — this is one of the two backends that accept a non-JSON codec. Pair it with JacksonYamlCodec to get human-friendly .yml files:

import br.com.finalcraft.everydatabase.codec.JacksonYamlCodec;

EntityDescriptor<UUID, PlayerData> PLAYERS = EntityDescriptor.builder(UUID.class, PlayerData.class)
        .collection("players")
        .keyExtractor(PlayerData::getUuid)
        .codec(new JacksonYamlCodec<>(PlayerData.class))       // -> data/players/<uuid>.yml
        .build();
# data/players/5f1e8400-….yml
uuid: "5f1e8400-e29b-41d4-a716-446655440000"
name: "Alice"
score: 100

⚠️ Gotcha — JacksonYamlCodec works only on the file backends (here and Grouped Files). SQL, Mongo, and in-memory require a JSON codec (codec.isJsonCodec()) and throw IllegalArgumentException from repository(...) if given YAML. If you later migrate a YAML local-file store into SQL, change the codec mid-transfer with the two-arg descriptor(srcDesc, dstDesc) form — see Moving Data Between Backends and Codecs.

For JSON, JacksonJsonCodec.pretty(Type.class) gives indented output (the plain JacksonJsonCodec is compact).

Queries: a correct full scan (no real index)

Local files have no real index. Queries are answered by a full scan — every entity in the collection is read and matched in memory. It's O(n) and slow on large collections, but it returns the correct result, so a query that works here keeps working when you swap to SQL or Mongo.

repo.findBy("score", 100).join();
repo.query(Query.range("score", 50, null)).join();   // score >= 50 (full scan, correct result)

Crucially, the backend still validates index declarations before scanning:

⚠️ Gotcha — querying a field that was not declared as an IndexHint throws IllegalArgumentException here too — even though the scan could answer it. This is deliberate: it stops a query that works on local files from silently breaking when the storage is swapped for SQL/Mongo (which genuinely require the declaration). Declare the field with .index(IndexHint.<type>("...")) or @Indexed. See Indexing & Queries.

Declare queryable fields the same way you would for any backend — the declaration keeps your data-access code portable, even though local files build no actual index from it.

No transactions

LocalFileStorage does not implement tx.TransactionalStorage. There is no inTransaction.

⚠️ Gotcha — individual save operations are atomic (the .tmp + ATOMIC_MOVE write), but there is no multi-entity transaction — you can't commit/rollback a group of writes as a unit. If you need ACID across multiple entities, use a SQL backend (MySQL & MariaDB / PostgreSQL / H2) or Mongo with a replica set. See Transactions.

Optimistic locking isn't enforced either (local files and in-memory don't enforce it — see Optimistic Locking and Choosing a Backend).

Schema migrations are supported

Unlike transactions, migrations work. LocalFileStorage implements schema.SchemaAwareStorage; applied versions are tracked in a reserved _schema_migrations.json file at the base directory, and migrations are forward-only. Extend LocalFileMigration and override executeOnStorage(LocalFileStorage):

import br.com.finalcraft.everydatabase.modules.localfile.LocalFileMigration;

class V1_Seed extends LocalFileMigration {
    @Override public String version()     { return "001"; }
    @Override public String description() { return "seed defaults"; }
    @Override protected void executeOnStorage(LocalFileStorage storage) {
        // mutate via repositories obtained from `storage`
    }
}

storage.register(new V1_Seed()).migrate().join();

MigrationContext.getNativeClient(...) also exposes the LocalFileStorage and its base Path. See Schema Migrations.

When to pick local files

🧭 Decision — choose local files when you want to read, diff, or hand-edit your data, or have no database server at all. One human-readable file per entity (YAML or pretty JSON), crash-atomic, zero ops. The trade-offs: no transactions, no real index (full-scan queries), no optimistic-locking enforcement — fine for small or cold datasets, configuration-like data, and single-server deployments. For large or write-heavy data, or anything needing ACID, prefer a SQL backend or Mongo.

A common pattern: ship small deployments on local files, let operators flip to MariaDB/Mongo for large ones and move the live data with Moving Data Between Backends — no code changes, source untouched.

See also

EveryDatabase · Home · made by Petrus Pradella

EveryDatabase

🏠 Home

Getting Started

Core Concepts

Working with Data

Backends

Manager Module

Operations

Advanced

Reference

Contributing

Clone this wiki locally