Skip to content

Latest commit

 

History

105 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Dataset Tools: An AI Metadata Viewer


Dataset Tools NextJS Edition is a local-first web application for viewing EXIF metadata from AI-generated images and camera photos, including the software used to create AI images and the camera details embedded in photos.


Community-Driven Development

This project is inspired by stable-diffusion-prompt-reader and thrives on community contributions. Found a bug? Have a workflow that won't parse? Want to add support for a new tool? We welcome forks, fixes, and pull requests!



Installation

Clone repo

git clone https://github.com/Ktiseos-Nyx/Dataset-Tools.git
cd Dataset-Tools

Install dependencies (Node.js 18+ required)

npm install

Start dev server (For Local Testing)

npm run dev

For production:

npm run build && npm start

Usage

  1. Start the app: npm run dev → open http://localhost:3000
  2. Browse files: Use the file tree sidebar, or click the folder icon to pick any directory.
  3. Drag & drop: Drop an image anywhere in the app — it'll find the folder and load thumbnails.
  4. Inspect metadata: Click any image → metadata panel shows prompts, parameters, LoRAs, and workflow info. Safetensors files show model architecture, training details, and dataset tags.
  5. Edit metadata: Click the edit button on any text field to modify prompts and parameters directly in PNG files.
  6. Customize: Settings panel has theme, accent colors, font size, thumbnail size, and file display options.

When metadata fails to parse

  1. Check browser console for parser logs.
  2. Note the workflow structure (ComfyUI? A1111? Custom nodes?).
  3. File an issue with:
    • Console error snippet
    • Workflow type + custom nodes used
    • Minimal repro image (if shareable)

Current Capabilities

Feature Status Details
Metadata Parsing 90% success rate. Graph-tracing engine for ComfyUI, field-based detection for A1111/Forge/NovelAI.
Image Viewing PNG, JPG, JPEG, WebP. Zoom (25-400%), rotation, fit-to-container.
File Browsing Recursive lazy-loading file tree. Browse any folder on your system.
Drag & Drop Drop an image to auto-detect its folder and extract metadata.
Thumbnails Sharp-powered WebP thumbnails with disk cache (.thumbcache/).
Sorting Sort by name, date modified, or file size in both file tree and thumbnail viewport.
Accent Colors 7 color themes (zinc, red, orange, green, blue, violet, pink) with dark mode support.
WebP Viewing Viewing works for static and animated WebP files. Metadata extraction is format-dependent.
ComfyUI Workflows 3-phase extraction + ComfyUI ≥1.26 provenance (cnr_id/aux_id). Node graph tracing, custom node resolution, service detection.
GitHub Lookup Fallback search for unknown ComfyUI custom nodes via GitHub code search.
Safetensors Metadata Full metadata extraction for LoRA and model files — rank, alpha, training params, dataset tags.
Metadata Editing Edit prompts and parameters in PNG files, write changes back to disk.

Supported Formats

  • A1111 / Forge — PNG tEXt chunks, JPEG EXIF
  • ComfyUI — JSON workflow with node graph resolution
  • NovelAI — PNG metadata
  • Civitai — UTF-16-LE JPEG UserComment
  • Standard EXIF/IPTC/XMP — All image formats
  • Safetensors — LoRA rank/alpha, training parameters, dataset tags, base model info
  • PNG as JPEG - Magic Byte Detection.

Tech Stack

  • Framework: Next.js 16 (App Router), React 19, TypeScript 5
  • UI: shadcn/ui + Radix UI + Base UI, Lucide icons, Tailwind CSS v4 (OKLch color space)
  • Thumbnails: Sharp (libvips) with WebP disk caching
  • Metadata: Pure JS parsing — PNG chunks, JPEG EXIF (exif-parser), ComfyUI graph traversal with node provenance
  • Node Resolution: ComfyUI Node Finder port (extension-node-map registry + GitHub code search fallback)

Service Detection

As we had in the python edition our journey is marked with making sure as many of the popular and niche sites with generation services are tagged for easy detection. If your workflow or website has a custom pattern for resource detection, our tool is likely to be able to find it. If it hasn't been hashed and dashed through to the tool yet, just flick an issue up and we'll hook it up ASAP.

Examples of this are Tensorart, Forge, ArcEncCiel, CivitAI and Yodayo.

Many tools desktop or remote based have patterns, so the key is either in the metadata handling, or the resource identification.

Why 90% > 65% Matters

The Python edition relied on fragile heuristics. This engine uses deterministic graph traversal with proper node relationship mapping. It follows wires backwards from sampler nodes to find prompts, identifies nodes by their data (not just class_type), and handles platform-wrapped node names via substring matching. When it fails, logs show exactly why.

Why this exists

The Python edition worked at 65% success rate with heuristic spaghetti. This NextJS engine hits 90% success rate on complex ComfyUI workflows using deterministic graph traversal. Metadata is parsed in pure JavaScript — no waiting for Python to boot, no OpenCV overhead.

Development Stage

This engine has been battle-tested across thousands of images and is in active daily use. While there will always be edge cases with new custom nodes and formats, the core parser is stable and handles the vast majority of real-world workflows. Bug reports with repro images remain the best way to push that success rate higher.


Contributing

Found a parsing failure?

Open an issue with the details above. Real-world edge cases are how we push past 90%.

Want to improve the parser?

  1. Fork repo → npm installnpm run dev
  2. Metadata extraction lives in app/api/metadata/route.ts
  3. Test with your own images or AI-generated samples
  4. Submit a PR with before/after evidence

Ideas for contributors

  • WebP metadata chunk parser
  • Editable metadata (write back to files)
  • SQLite indexing for faster folder browsing
  • ComfyUI workflow visualization
  • Batch metadata export (CSV/JSON)
  • Parser debugger panel showing traversal steps
  • [ ]

Q&A

Q: Are you planning a desktop app?

A: Yes — we're evaluating both Electron and Tauri. No firm decision yet, but the goal is a standalone executable that runs without needing Node.js installed.

Q: Isn't Node.js a security concern?

A: Security is taken seriously in this project. Unlike ML-heavy tools that pull in massive native dependency trees, Dataset Tools has a minimal, auditable surface area. All file access is validated server-side, and API keys are stored in .env.local, never exposed to the browser.

Q: Does it only work in Chrome?

A: The app works in any modern browser. Firefox, Edge, and Chromium-based browsers like Vivaldi are all supported.

Q: Is this project actively maintained?

A: Yes. The parser is stable and handles the vast majority of real-world workflows. Maintenance and improvements are ongoing — bug reports and contributions are always welcome.

AI-Assisted Development

This project was built with significant assistance from large language models — primarily Claude, with additional contributions from Gemini, DeepSeek, and Qwen. The role of the project maintainer has been focused on research, architecture decisions, testing across diverse AI-generated images and workflows, and directing the scope and quality of the codebase.

Transparency note: while LLMs generated a substantial portion of the implementation, every line has been reviewed, tested against real-world datasets, and refined through iterative prompting and validation. The 90%+ metadata parsing success rate is the result of detailed research into AI image generation formats combined with LLM-assisted implementation.


License

GNU General Public License v3.0

Acknowledgements

  • Core Parsing Logic: This project incorporates and adapts parsing functionality from Stable Diffusion Prompt Reader by receyuki. The original MIT license for vendored code is included in NOTICE.md.
  • traugdor — Project supervision and the ComfyUI Node Finder (Python), whose extension-node-map registry and node classification logic are now built directly into Dataset Tools.
  • Everyone at Arc En Ciel for continued support.
  • Anzhc for ongoing support and motivation.
  • The wider AI and open-source communities for feedback, testing, and contributions.

SPECIAL THANKS

  • Supervised by: traugdor
  • Contributors: open-source community, Whitevamp, Exdysa, and many more.
  • Anthropic for Claude API credits supporting development.

Releases

Packages

Used by

Contributors

Languages