Skip to content

Origin notebook packaging

Tutorials remain in simulation_templates/, with their existing upload workflow and tutorial endpoint. App origins live beside them in a separate authoring tree:

origin_simulation_templates/
  femwell/heater_steady_state_vswa/femwell_heater_steady_state_vswa.ipynb
  femwell/waveguide_modes_vswa/femwell_waveguide_modes_vswa.ipynb
  meep/generic_3d/meep_generic_3d.ipynb
  meow/mmi1x2_vswa/meow_mmi1x2_vswa.ipynb
  meow/mmi2x2_vswa/meow_mmi2x2_vswa.ipynb
  meow/taper_vswa/meow_taper_vswa.ipynb
  palace/driven_sparams/palace_driven_sparams.ipynb
  sax/circuit_wavelength_sweep/sax_circuit_wavelength_sweep.ipynb
  zapfdtd/generic_3d/zapfdtd_generic_3d.ipynb

The rule is <simulator>/<template>/<simulator>_<template>.ipynb. Add a new simulator or template directory without changing a registry or packaging list. Keep paired .py authoring sources where needed. Stable template IDs and engine IDs are independent of paths: for example, the zap origin keeps id='zapfdtd_3d' and engine='fdtd'. The existing Config literal declarations, including optional package requirements, are its app metadata authority. User-input declarations stay next to the settings they configure; see compact metadata.

Run from the repository root:

python -m scripts.check_origin_templates
python -m scripts.package_origin_templates
python -m pytest tests/test_origin_publish_contract.py -q

The second command creates the ignored origin_simulation_templates.zip with canonical notebooks at the ZIP root, such as zapfdtd_generic_3d.ipynb and meep_generic_3d.ipynb. It copies validated notebook bytes without changing them. There is no enclosing folder inside the ZIP:

origin_simulation_templates.zip
  femwell_heater_steady_state_vswa.ipynb
  femwell_waveguide_modes_vswa.ipynb
  meep_generic_3d.ipynb
  meow_mmi1x2_vswa.ipynb
  meow_mmi2x2_vswa.ipynb
  meow_taper_vswa.ipynb
  palace_driven_sparams.ipynb
  sax_circuit_wavelength_sweep.ipynb
  zapfdtd_generic_3d.ipynb

No sidecars, paired Python sources, shared JSON/Python implementations, README files, directory entries or target exports are delivered. Do not commit generated ZIPs or flattened source copies. Target exports retain their generated route stamp and are excluded; a target export using a canonical filename is an error.

Packaging validates the nested naming rule, portable names, case-insensitive flattening collisions, unique template IDs, notebook JSON/schema, unique cell IDs, the .venv kernelspec, embedded metadata/binding constraints, the canonical target marker and a nonempty notebook set. Unsafe paths or symlink escapes fail. The existing 512-file, 64 MiB aggregate and 10 MiB per-notebook bounds apply. Nonempty assets declarations fail explicitly because this transport delivers notebooks only. Project input capture by the app is independent of archive assets.

upload_origin_simulation_templates.yml validates on pull requests, pushes to main and manual dispatch. Its validation job checks inline declarations, binds all supported source routes without executing sources, checks composition syntax, and builds the archive. The resulting ZIP is available as the origin-simulation-templates workflow artifact for seven days. The upload job downloads that same validated artifact instead of rebuilding a second ZIP.

Its separate upload job is disabled unless the repository variable ORIGIN_SIMULATION_TEMPLATES_UPLOAD_ENABLED is exactly true. Leave it unset until the origin endpoint from DataLab PR #4034 is deployed in both environments. The templates PR may merge first; enabling upload is a separate rollout action. Upload uses existing dev/prod credentials, multipart field file, and only:

/api/portal/admin/sdk/v1/origin-simulation-templates/upload

The origin ZIP never goes to /simulation-templates/upload. No repository variable is enabled by this implementation. App-owned adapters, schemas and binding rules remain bundled with the matching app implementation. Packaging validation alone does not establish deployed DataLab/app compatibility or live notebook creation. The reference binder reads either this nested authoring tree or an extracted flat ZIP; the current app development provider consumes the extracted flat catalog.

DataLab contract and rollout

This publisher follows the DataLab origin delivery contract introduced by PR #4034. DataLab stores canonical notebook bytes under its fixed origin-simulation-templates/ prefix. It does not interpret their metadata.

The app-side integration must use the origin SDK list/download endpoints, then read the embedded declarations using its own binding rules and adapters. The tutorial menu continues using the tutorial collection. That acquisition work does not belong in this publisher.

Upload replaces origin collection membership: notebooks missing from the complete ZIP are deleted from that collection. Do not publish a subset as an incremental update. Publication is not atomic; after a failed publication, retry the same intended complete ZIP. No automatic retry, live upload, repository-variable change, or deployment is performed by the local checks.

The contract test sends the workflow's actual curl command to a loopback HTTP receiver, using a dummy key. It checks the multipart file, exact canonical ZIP contents, and failure on HTTP errors. This verifies the publisher request shape; it does not establish deployed Portal authentication, S3 behavior, or app delivery.

Retired source-specific origin exports, binding JSON/schema files, and adapter implementations have been removed from the tutorial delivery folder as well. Current authoring policy lives under scripts/contracts; legacy validation helpers retained for historical development harnesses live under scripts/legacy. Neither is included by either notebook publisher.