Skip to content

LLM.md

Latest commit

 

History

History
252 lines (143 loc) · 11.6 KB

LLM.md

File metadata and controls

252 lines (143 loc) · 11.6 KB

ChrysaLisp: A Guide for Large Language Models

This document serves as the primary entry point for LLMs seeking to understand ChrysaLisp. The documents are organized in a suggested reading order, moving from high-level concepts to implementation details, then to practical applications.

Getting Started

If you're new to ChrysaLisp, start here to understand what it is and why it exists:

  1. Contributions - for hard rules about PRs and testing.

  2. Summary - A comprehensive overview of the entire ChrysaLisp system, its core components, architecture, and capabilities.

  3. The Philosophy - Understanding computation as sequence transformation and why the "Four Horsemen" (map, filter, reduce, some) are central to the design.

  4. Genesis - The origin story: building guarantees from an unreliable sea, and how ChrysaLisp achieves absolute resilience through ephemeral identity.

  5. Buda Palm Manual - A newcomer's primer explaining ChrysaLisp's core concepts and how it differs from other languages.

Core Architectural Principles

These documents explain the fundamental design decisions that make ChrysaLisp unique:

  1. Know Thyself - The Tao of ChrysaLisp: cooperative design, iteration over recursion, and the synergy between small stacks and O(1) caching.

  2. All is One - How the O(1) hmap unifies lexical scoping, class inheritance, and property inheritance into a single elegant primitive.

  3. Memory Architecture - The complete memory model from low-level allocation (sys_mem) to Lisp objects, reference counting, and the boot image.

The Virtual Processor (VP) Layer

Understanding the VP is key to understanding ChrysaLisp's performance:

  1. VP Translation - How the Virtual Processor architecture works and how VP instructions are translated to native code for different CPU targets.

  2. VP Classes - The inheritance hierarchy of VP classes from obj to specialized types like streams, numbers, and GUI views.

  3. VP Functions - The unified binary format for functions and vtables, and the "linkerless" build process.

  4. VP SIMD - How vp-simd transforms multi-dimensional thinking into linear, high-performance code through atomic read-execute-write operations.

The Lisp Language

ChrysaLisp's Lisp dialect is modern, performant, and pragmatic:

  1. Modern Lisp - ChrysaLisp's sequence-centric Lisp dialect: no cons cells, reference counting instead of GC, and powerful iteration primitives.

  2. Lisp Forth - How ChrysaLisp bridges Forth and Lisp, offering Forth's speed with Lisp's symbolic grounding.

  3. Lisp Primitives - The built-in VP Lisp primitives organized by function family.

  4. Library Primitives - System and library primitives beyond the core language primitives.

Advanced Lisp Concepts

Deeper understanding of how Lisp works in ChrysaLisp:

  1. On Classes - The dual vtable architecture: static VP classes for the "Engine" and dynamic Lisp classes for the "Script" layer.

  2. Closure or Not - Why ChrysaLisp deliberately rejects implicit closures in favor of explicit object-oriented state management.

  3. Art of the Call - How ChrysaLisp achieves O(1) performance through pre-binding, str_hashslot caching, and the evaluation pipeline.

  4. Import Modules - The module system using import, include, and the export pattern for encapsulation.

Data Types and Processing

How ChrysaLisp handles different types of data:

  1. Numerics - Numeric types (Num, Fixed, Real) and vectorized operations (nums, fixeds, reals).

  2. Text Parsing - High-performance text processing using char-class and binary search primitives (bfind, bskip).

  3. Streams - The stream system for I/O, including file streams, string streams, and IPC streams (in/out).

  4. Pipe Commands - The distributed pipe system for building complex command-line processing pipelines across nodes.

GUI System

ChrysaLisp's sophisticated graphical interface:

  1. GUI Views - The View class as the foundation for all GUI widgets, with its non-recursive layout system.

  2. GUI Services - GUI widgets, event system, and the UI builder macros.

  3. GUI Composition - How the compositor efficiently redraws using dirty regions, opaque regions, and multi-pass rendering.

System Architecture

Understanding ChrysaLisp as a distributed operating system:

  1. Fault Tolerant - How ChrysaLisp achieves fault tolerance through restartable task trees, job requeueing, and ephemeral mailboxes.

  2. Dynamic Code - The assembler as a Lisp library: how dynamic code generation works (demonstrated with pixmap format conversion).

  3. Coding Style - Guidelines for writing idiomatic ChrysaLisp: naming conventions, state management, iteration patterns, and the three-stage optimization path.

Philosophy and Performance

Understanding the "why" behind ChrysaLisp's design:

  1. Evidence Not Faith - Benchmarks proving the sub-second rebuild claims and demonstrating performance across platforms.

  2. Malleability - How ChrysaLisp delivers Lisp's dynamic malleability without sacrificing C-like performance.

  3. More Haste Less Speed - The philosophy of deliberate craftsmanship: achieving performance through thoughtful design rather than raw speed.

  4. The Tao of Now - It's all a dream. The system is a concensous reality.

Practical Implementation

Getting ChrysaLisp running and building applications:

  1. Host Interface - The bridge between ChrysaLisp and the host OS: PII functions, GUI/audio layers, and the VP64 emulator.

  2. Porting - Comprehensive guide to porting ChrysaLisp to new operating systems and CPU architectures.

  3. App Configuration - The standard pattern for managing application state and user preferences.

Advanced Topics

Deep dives into specialized areas:

  1. Inner Thoughts - The REPL as a JIT compiler: understanding the read-expand-bind-eval pipeline and the static-q* family.

  2. Rocinante - Mastering the "Four Horsemen" primitives (each!, map!, reduce!, some!, filter!) and the ! special form.

  3. Udat AI Shares - AI commentary on ChrysaLisp's architectural coherence and design philosophy.

  4. Coding Domains - Know your coding domain and stick to it.

  5. Sequence Indexing - Sequence indexing and slicing.

  6. Application Acceleration - Adding native functions to applications..

  7. The Dual VTable - A Study of ChrysaLisp's Static and Dynamic Object Models.

  8. Network & IPC - ChrysaLisp Network & IPC Architecture.

  9. Vector Graphics - ChrysaLisp Vector Graphics Architecture.

  10. Slicing and Dicing - Deep Analysis: Vectorized Slicing and Dicing in ChrysaLisp.

  11. Text Buffers - The Text Buffer and Edit Architecture.

  12. The Text Stack - A Comprehensive Analysis.

  13. Edit Command - The ChrysaLisp Parallel Programmable Editor.

  14. VP64 Emulator - The VP64 C/C++ Emulator Implementation.

  15. Task Farming - The Task farm libraries.

Reading Paths

Depending on your goals, you might follow different reading paths:

For Understanding the System

Follow the order above from 0-48 for a complete understanding.

For Application Development

Focus on the language, standard libraries, and GUI framework.

Read: 1, 4, 12, 29, 38, 14, 15, 19, 20, 22, 24, 25, 26, 36, 27, 33

For System Programming

Focus on the Virtual Processor, memory model, and host integration.

Read: 1, 3, 5, 7, 8, 9, 10, 11, 34, 35, 18, 28, 37

For Language Design Insights

Focus on the philosophical divergences from traditional Lisp and architectural unification.

Read: 2, 3, 6, 13, 16, 17, 30, 31, 32, 33, 39

Quick Start

Read: 1, 4, 12, 14, 29, 38, 40, 41

LLM Test Scripts

Running test scripts via an LLM is easy to accomplish via the -s option of the ChrysaLisp launch bash file.

./run_tui.sh -n 1 -f -s script_name.lisp

This will launch the system on a single VP node, in the foreground, and run the raw script_name.lisp file.

Keep any test scripts and associated files in the tests/ folder !

There is a compreshensive test suite that can be run via.

./run_tui.sh -n 1 -f -s tests/run_all.lisp

If you are not changing the base VM or any VP level system files, which most app coding should NOT be doing ! There is no need to keep making the system from scratch each time ! Running your tests via this script launcher is what you should be doing.

In order to run a cmd/ app, from a raw script, you need to wrap the cmd/appname.lisp in a (pipe-run command_line) function, from the (import "lib/task/pipe.inc") library.

And to be robust around catching errors, should wrap your tests in a catch block, and end the script in a call to the host shutdown code, like so.

To throw an error, use (throw "Description !" obj), if no object of interest, use :nil. Do NOT use catch or throw in runtime ChrysaLisp code ! That feature is compiled out in release mode and is only available for the debug builds and testing.

;use of (pipe-run command_line)
(import "lib/task/pipe.inc")

(defun my-test ()
    (defq test_string "string with new line\n")
    ...
    (pipe-run appname)
    ...
    )

(catch
    (my-test)
    (progn
        ;report error
        (print "Test failed with error" _)
        ;signal to abort the catch
        :t))

;clean shutdown of the VP node
((ffi "service/gui/lisp_deinit"))

ChrysaLisp represents a radical rethinking of how a Lisp system can be built for maximum performance and resilience. It proves that the dichotomy between dynamic malleability and static performance is a false one, born of historical implementation choices rather than fundamental constraints.