Debmalya/HAMVCC_V5_HuggingFace_Private_Release_Kit

HAMV-CC V5 — Python package and visualization bundle

Contents

• src/hamvcc_v5_full_export.py: all notebook cells exported in execution order, with markdown headings preserved as comments.
• src/hamvcc_visualizations.py: standalone visualization pipeline from actual image corpus through PCA and exploratory K-means clustering.
• src/run_full_pipeline.py: launcher for the full sequential export.
• notebook_cells/: individual .py exports for every code cell, with a manifest mapping them to original notebook cell numbers.
• visualizations/: diagnostic charts generated from the existing diagnostic report's published aggregate values. They are explicitly marked as report-derived, not raw-document results.
• requirements.txt: core dependencies.

1. Install

Use a virtual environment recommended for your OS:

python -m venv .venv
# Windows:
.venv\Scripts\activate
# macOS/Linux:
source .venv/bin/activate
pip install -r requirements.txt

2. Run visualizations on your real image corpus

python src/hamvcc_visualizations.py --dataset "PATH_TO_IMAGE_DATASET" --output visualizations

Alternatively set HAMVCC_DATASET_OVERRIDE before running. Optional --limit 200 makes a quick smoke run; omit for the full corpus. The script writes PNG plots, CSV tables, and a JSON run report.

3. Run the full notebook export

python src/run_full_pipeline.py

The full export executes code cells in original order and may require additional project files, model weights, OCR binaries, and packages present in the original notebook environment. Set the correct dataset path first. TEST remains gated by the source notebook's safeguards; do not enable TEST without a reviewed freeze manifest and explicit approval.

Important evidence and safety notes

• The included PNG charts are based only on aggregate metrics in the supplied diagnostic report. They are not computed from raw per-document records.
• The PCA/K-means visualizations generated by hamvcc_visualizations.py are exploratory and do not establish semantic correctness or trained HAMV-CC performance.
• The individual notebook_cells/cell_*.py files are for audit/reuse and may depend on state from preceding cells. Prefer the full export for sequential execution.

One-click Windows execution

Double-click RUN_HAMVCC.bat. On first run, choose Yes when asked to create .venv and install dependencies. Then select:

1. Generate visualizations from your dataset (recommended first run).
2. Run the full exported notebook pipeline.
3. Install/repair dependencies.
4. Open the visualizations folder.

For PowerShell users, run RUN_HAMVCC.ps1 from the package folder. If Windows blocks the script, use the .bat launcher instead.

The launcher asks for the dataset folder path and does not assume a fixed user-specific Windows directory.

Standalone Windows execution fixes

This corrected package defines a safe display() compatibility helper so notebook-exported tables can also be printed when the pipeline is run from Command Prompt. On Python versions older than 3.10, DINOv2 is skipped with an explanatory message because the current upstream hub source may use newer type-annotation syntax; the other available feature backbones can still run. DINOv2 is optional and is not required for the pipeline to continue.

Run from the package root in Command Prompt:

set LOKY_MAX_CPU_COUNT=8
python src/run_full_pipeline.py

The two commands above should be entered on separate lines. If another exception occurs, save the complete traceback; this patch addresses the NameError: name 'display' is not defined reported during corpus diagnostics.

HAMV-CC / FUNSD Component Test Pack This pack provides executable Python diagnostics for testing the pipeline components identified as missing or inconsistent in the supplied FUNSD diagnostic report. Important scope These scripts are component-test scaffolding, not the missing HAMV-CC implementation. They do not fabricate model results or claim FUNSD benchmark validity. Tests that require the original project functions/models are marked as integration tests and will report SKIP unless you connect your real implementations. Requirements Python 3.10+ recommended (the diagnostic log says DINOv2 was skipped under Python 3.8.10). Core tests use the Python standard library, NumPy, pandas, scikit-learn, Pillow, and pytest. Optional OCR tests need pytesseract and a locally installed Tesseract OCR engine. Install:

python -m pip install -r requirements.txt

Run the core checks:

python run_all_tests.py

Run with a dataset folder:

python run_all_tests.py --data-dir "PATH_TO_IMAGE_FOLDER"

Run OCR checks (requires Tesseract installed):

python run_all_tests.py --data-dir "PATH_TO_IMAGE_FOLDER" --ocr

Run only tests:

python -m pytest -v

Dataset expectations The default scanner accepts image files recursively (.png, .jpg, .jpeg, .tif, .tiff, .bmp, .webp). It computes hashes and basic image/OCR coverage statistics. It does not prove that files are from the official FUNSD release. Supply annotation files separately for provenance/annotation checks. Connecting your actual HAMV-CC code Edit project_adapter.py and set PROJECT_MODULE to the importable module containing your original functions. The adapter checks for these reported components: fit_reliability_models build_condition_v4 train_candidate partition_metrics_v4 build_severity_features predict_rel The exact function signatures are not inferred here. After connecting the real module, inspect and adapt integration_checks.py to your API before interpreting any test result. Missing symbols are reported; no replacement model is silently substituted. Outputs A run creates reports/ containing: component_test_report.json dataset_manifest.csv (when --data-dir is supplied) feature_schema_report.json (when a feature CSV is supplied) pca_report.json (when a feature CSV is supplied) pytest_output.txt Suggested feature CSV Use one row per document and numeric feature columns only. Exclude document IDs, labels, split names, paths, and target columns from the PCA feature matrix. If an ID column is needed for joins, keep it as metadata rather than a feature.