NLP Model Evaluation

A collection of standalone evaluation scripts for comparing local/hosted LLM outputs (BLEU, ROUGE, embedding similarity, GPU/VRAM usage via pynvml), plus a couple of one-off notebooks and a ViT-MAE fine-tuning experiment (python_agentic_training_pipeline.py, Untitled3.ipynb).

Recovered from a personal backup — this is a script collection, not a single deployable app. Each file is run independently.

Known issues

• No single entrypoint — Model_valid.py, Model_test.py, model_test_root.py, Valid.py, Valid_1.py overlap in purpose and were likely different iterations of the same evaluation harness. Compare before picking one to keep.
• scatter_plot_*.png are prior run outputs kept for reference — regenerate rather than edit.
• pynvml requires an NVIDIA GPU + drivers; scripts using it will fail on CPU-only or non-NVIDIA machines unless that import is guarded.

Setup

pip install -r requirements.txt
playwright install --with-deps chromium   # only needed by scripts that crawl web pages
python -c "import nltk; nltk.download('punkt')"

Run

python Model_valid.py     # or whichever script you're using for a given evaluation run

Docker

Provided for a reproducible CPU environment to run any of the scripts in; edit the CMD to point at the script you want.

docker build -t nlp-model-evaluation .
docker run --rm nlp-model-evaluation python Model_valid.py