Compress .lapis-plugin bundles and verify installs in a worker #349

New issue

Closed

opened 2026-06-06 22:11:53 +00:00 by steve · 0 comments

steve commented

2026-06-06 22:11:53 +00:00

Owner

Problem

The current .lapis-plugin bundle format uses a hand-written stored-ZIP subset
and app-side verification runs on the installer thread. Large official plugins
therefore download as uncompressed bundles and web installs can spend noticeable
time parsing/verifying archive bytes on the main thread.

Goal

Use fflate for deterministic .lapis-plugin archive creation/extraction,
allow DEFLATE-compressed plugin file entries, and verify/extract bundles in a
web worker when available while preserving the signed-manifest security model.

Scope

App release tooling, API installer, worker verification path, progress phases,
tests, and specs.
Registry sync/validation/site metadata support in
/Users/stevejuma/code/lapis-plugin-registry.
Direct fflate dependencies where archive code imports it.

Non-goals

No return to multi-asset Forgejo releases or per-file post-install downloads.
No network-to-vault streaming install; the first implementation may still
buffer the downloaded bundle before verification.
No ZIP64, encryption, split archive, or arbitrary compression method support.

Acceptance Criteria

Official release tooling writes deterministic .lapis-plugin bundles whose
root release.signed.json entry is stored and whose plugin file entries are
DEFLATE-compressed with fixed options.
Existing stored .lapis-plugin bundles and new compressed bundles both verify
and install through the same signed official path.
Web installs use worker verification by default when available and fall back
to main-thread verification; tests can force main-thread verification.
Bundle verification rejects tampered bundle hash, invalid signature, changed
file bytes/size, unsafe path, duplicate path, missing signed file, unsigned
extra file, encrypted entries, and unsupported methods.
Registry sync/validation consumes compressed bundles, rejects malformed
bundles, and records compressed bundle sha256 and size.
Plugin registry UI behavior and size display remain unchanged except for the
new verifying-bundle progress phase.

Implementation Notes

Add fflate@^0.8.3 as a direct dependency for app API runtime, root release
scripts, and registry archive tooling.
Replace the manual bundle writer with fflate.Zip, using
ZipPassThrough for release.signed.json and ZipDeflate level 6 for
plugin files.
Replace the manual parser with a strict fflate.Unzip based extractor that
accepts only stored or DEFLATE entries and preserves the current path,
duplicate, missing-file, extra-file, size, hash, and signature checks.
Add an API worker module and a manager/installer option for
bundleVerification: "auto" | "worker" | "main-thread", defaulting to
"auto".
Update specs to describe compressed bundle entries and worker verification.

Suggested Files or Specs To Inspect

scripts/plugin-release.mjs
packages/api/src/lib/plugin-distribution/
packages/workspace/src/lib/components/configuration/plugins-registry-panel.svelte
spec/src/30-cross-package-contracts/plugin-registry.md
/Users/stevejuma/code/lapis-plugin-registry/scripts/

Validation Commands

App repo: pnpm test:scripts
App repo: pnpm --filter @lapis-notes/api check:all
App repo: pnpm --filter @lapis-notes/api test
App repo: pnpm --filter @lapis-notes/workspace check:all
App repo: pnpm --filter @lapis-notes/workspace test
App repo: pnpm check:all
App repo: pnpm test
App repo: pnpm test:smoke
App repo: mdbook build spec
Registry repo: pnpm test
Registry repo: pnpm check

Follow-up to #348 — Replace official plugin release assets with single .lapis-plugin bundles

Follow-up Tasks

Implementation Summary

Implemented fflate-based deterministic .lapis-plugin compression, shared bundle verification with web worker support and abort handling, updated installer progress and specs/tests, and updated registry validation/sync tooling to accept stored or DEFLATE bundle entries.

## Problem The current `.lapis-plugin` bundle format uses a hand-written stored-ZIP subset and app-side verification runs on the installer thread. Large official plugins therefore download as uncompressed bundles and web installs can spend noticeable time parsing/verifying archive bytes on the main thread. ## Goal Use `fflate` for deterministic `.lapis-plugin` archive creation/extraction, allow DEFLATE-compressed plugin file entries, and verify/extract bundles in a web worker when available while preserving the signed-manifest security model. ## Scope - App release tooling, API installer, worker verification path, progress phases, tests, and specs. - Registry sync/validation/site metadata support in `/Users/stevejuma/code/lapis-plugin-registry`. - Direct `fflate` dependencies where archive code imports it. ## Non-goals - No return to multi-asset Forgejo releases or per-file post-install downloads. - No network-to-vault streaming install; the first implementation may still buffer the downloaded bundle before verification. - No ZIP64, encryption, split archive, or arbitrary compression method support. ## Acceptance Criteria - Official release tooling writes deterministic `.lapis-plugin` bundles whose root `release.signed.json` entry is stored and whose plugin file entries are DEFLATE-compressed with fixed options. - Existing stored `.lapis-plugin` bundles and new compressed bundles both verify and install through the same signed official path. - Web installs use worker verification by default when available and fall back to main-thread verification; tests can force main-thread verification. - Bundle verification rejects tampered bundle hash, invalid signature, changed file bytes/size, unsafe path, duplicate path, missing signed file, unsigned extra file, encrypted entries, and unsupported methods. - Registry sync/validation consumes compressed bundles, rejects malformed bundles, and records compressed bundle `sha256` and `size`. - Plugin registry UI behavior and size display remain unchanged except for the new `verifying-bundle` progress phase. ## Implementation Notes - Add `fflate@^0.8.3` as a direct dependency for app API runtime, root release scripts, and registry archive tooling. - Replace the manual bundle writer with `fflate.Zip`, using `ZipPassThrough` for `release.signed.json` and `ZipDeflate` level `6` for plugin files. - Replace the manual parser with a strict `fflate.Unzip` based extractor that accepts only stored or DEFLATE entries and preserves the current path, duplicate, missing-file, extra-file, size, hash, and signature checks. - Add an API worker module and a manager/installer option for `bundleVerification: "auto" | "worker" | "main-thread"`, defaulting to `"auto"`. - Update specs to describe compressed bundle entries and worker verification. ## Suggested Files or Specs To Inspect - `scripts/plugin-release.mjs` - `packages/api/src/lib/plugin-distribution/` - `packages/workspace/src/lib/components/configuration/plugins-registry-panel.svelte` - `spec/src/30-cross-package-contracts/plugin-registry.md` - `/Users/stevejuma/code/lapis-plugin-registry/scripts/` ## Validation Commands - App repo: `pnpm test:scripts` - App repo: `pnpm --filter @lapis-notes/api check:all` - App repo: `pnpm --filter @lapis-notes/api test` - App repo: `pnpm --filter @lapis-notes/workspace check:all` - App repo: `pnpm --filter @lapis-notes/workspace test` - App repo: `pnpm check:all` - App repo: `pnpm test` - App repo: `pnpm test:smoke` - App repo: `mdbook build spec` - Registry repo: `pnpm test` - Registry repo: `pnpm check` ## Related Issues - Follow-up to #348 — [Replace official plugin release assets with single .lapis-plugin bundles](../closed/348-replace-official-plugin-release-assets-with-single-lapis-plugin-bundles.md) ## Follow-up Tasks ## Implementation Summary Implemented fflate-based deterministic .lapis-plugin compression, shared bundle verification with web worker support and abort handling, updated installer progress and specs/tests, and updated registry validation/sync tooling to accept stored or DEFLATE bundle entries.

steve added this to the Full Registry V1 milestone

2026-06-06 22:11:53 +00:00

steve

2026-06-06 22:11:53 +00:00