Eve: Simulation templates app/UI handoff¶
Historical integration/qualification notes: source-specific origin exports and shared delivery sidecars described below are retired. For current authoring and delivery, use compact metadata and origin packaging.
Current AUTO/USER tokens, metadata defaults, migration status and app dependencies:
template token author guide. Each analysis uses one canonical notebook, shared source adapters and
the generic auto_user_tokens binder; app integration remains separate.
Implement a shared app flow for generating simulation notebooks from true templates. The full spec owns catalog/publication, representation/PDK contracts, readiness gates, roadmap and notebook audit. Development notes retain test evidence, solver limitations and temporary compatibility code. Current notebooks are drafts; this reference generator is not production app orchestration.
The settled app behavior is recorded in GDSFactory+'s
specs/simulation/notebook-launcher.md for issue 5167. This handoff follows that
contract. In particular, notebooks follow saved original design files; existing
reference-binder captures and Config loading still need the live-Prepare changes
below. Passing binder tests does not establish that those changes or the app
creation coordinator are implemented.
The app runtime uses a standalone Python worker under the owning project's environment interpreter. Extend its Rust-authoritative operation registry and generated Python contract; do not add handlers to the removed embedded runtime. Normal worker startup can install missing runtime dependencies, so notebook Resolve/Preflight needs a nonprovisioning boundary to preserve the no-install creation policy. Read-only Worker Health alone cannot prevent a later startup from repairing the environment. Kernel selection follows the source project, including when Save to is outside that project. Keep indexed partial aliases and project PDK cell overrides intact, and reuse current SDK materialization for repeated GSCH Prepare. Conversion warnings are not evidence of physical/LVS readiness. Template-registry API-key access remains separate from private-package session tokens; structured catalog/release endpoints are still proposed.
The app now has the internal WorkerManager::call_existing_project boundary:
fresh project workers retain each job's PDK manifest, inspect the existing venv
under the preparation lock, never provision/retry through repair, and preserve
Python error diagnostics. Its standalone notebook_inspect_python operation
executes the exact alias with raw JSON arguments and reports full bounds, ports
and retained PDK identities; active-PDK changes fail. This is not a complete
runtime Resolve/Preflight endpoint. The internal app runtime service now retains
saved source/configuration, completed-index fingerprints and the exact indexed
owning PDK/cell/model manifest, revalidating before/after inspection and staged
parameterized export. Export writes embedded-metadata GDS from the exact alias
and JSON arguments into exclusively owned staging; it has no public export RPC
or publication endpoint. Windows notebook worker descendants are contained and
output ownership survives caller cancellation through process termination.
Conservative indexed-source checks are not physical/material qualification.
Physical/parameter/port qualification and notebook
creation/kernel orchestration remain required; catalog drafts stay unavailable.
The app's internal typed physical-preflight context now retains that private
runtime and revalidates before/after derivation and again on demand. It records
source-port selection/exclusion provenance and explicit project sim.x/legacy
sim.wls wavelength provenance, with validated deliberate overrides. It never
uses GUI fallback ranges or sample counts. Port display observations supply no
excitation/material/units proof. Required physical providers (units, layers,
stack, materials, cladding, port conventions or models) remain typed unavailable
with corrective guidance; even a context with no such requirements is not Ready.
There is no public preflight RPC or qualified generation adapter. Existing
sidecar route requires strings are not the shared typed requirements contract;
an adapter must declare source-port binding policy, accepted types/capabilities,
wavelength use and reconstructable physical providers before it can be enabled.
Legacy reference-loader wavelength/cladding defaults and short-name/snapshot
behavior are not used by this app service. No authored origins or MEOW files are
changed by this boundary.
Internal app adapter contracts now pin the exact template ID and metadata/asset content hash as well as adapter identity, binding ID, source route and shared typed requirements. Retained preflight rechecks current server requirements, reacquires the original catalog projection and exact release, then revalidates saved source/runtime authority. Catalog support declarations do not register a runtime implementation. Changed content cannot pass solely by retaining the same adapter name: retained bytes and hashes must still agree.
The app also validates one explicit stack-backed native provider declaration family: PDK identity/activation, micron/DBU conventions, stack/material source references and digests, exact cladding role/key/material and physical-port plus excitation evidence declarations. This is coherence checking, without importing references, verifying source digests or qualifying material bands, modes or geometry. Layer-map/model providers are not represented by that family. No provider loader, physical worker qualifier or generation adapter is enabled. A supported reviewed backend and authoritative evidence from the selected project remain required; current sample verification and native material-name fallbacks cannot supply it.
The zapFDTD Python/GDS/GSCH origins retain the selected module as pdk_provider
and construct fdtd.Simulation(pdk=pdk_provider). Passing only its PDK object
would discard module-owned MATERIAL_CARDS supported by the explicit declaration
family and pinned backend. PDK activation and native attached-card precedence
remain intact; this provider handoff fix does not qualify materials or the
backend. Meep's get_stack() uses a separate conversion/material path.
Notebook creation reads only the PDK values required by the selected origin. There is no additional worker audit of stack/card ownership, native units or thickness, material-card membership, or provider/parent source hashes. Native physical validation remains in manual notebook preparation. Ordinary saved-source, project/runtime and exact template-release freshness still protect binding.
The app's simulationNotebook.catalog / catalogRelease operations have an
explicit server-configured local development provider for these authored
sidecars. It reports drafts, unsupported adapters and content hashes without
running notebooks or claiming production qualification. Production registry
integration remains separate; the legacy filename list/download is preserved.
Current drafts are Meep → Generic 3D Template and zapFDTD → Generic 3D Template;
MEOW, then Femwell, are next. Reuse the launcher, source selection,
generation and Config initialization across engines. Read each template's metadata
for accepted routes and its native settings/results contract; do not hard-code
Meep's capabilities into the app. zapFDTD details are in its
implementation notes; its gsim/cloud engine ID is fdtd.
zapFDTD cloud execution is currently unqualified: two real diagnostic jobs crashed with exit 139; Eve subsequently identified CRLF inputs as the cause. The development notes record a temporary LF upload workaround; patched execution still needs verification here. The local writer/parser also disagree on required grid bounds; see its implementation notes. Do not present field/mode images or a successful solve as available until the solver integration has passed that gate.
1. Entry points and source choice¶
Add sidebar Simulate in Notebook; rename existing Simulate to Simulate in GUI without changing its behavior. Dispatch a shared launcher into template selection, with the clicked design selected. Sidebar/wizard owners must agree on the launcher; Eve owns menu/launcher UI, not automatically every wizard/runtime task.
Meep and zapFDTD are both in the initial app scope, each with Python, GSCH layout and direct-GDS inputs. GSCH schematic is a declared route with shared scaffolding; incompatible geometry templates stay visible with a reason and cannot advance. Additional compatible templates plug in through metadata and app-owned binding/adapter contracts rather than sidebar engine branches.
Catalog/capabilities, source/context resolution, preflight and generation are general Rust runtime operations with serializable source descriptors, structured choices/errors, progress/cancellation and revision checks. Python operations use the worker protocol. Sidebar, future editor top bar and other UI sections call the same operations. The extension owns Quick Picks, notebook opening and kernel/setup-cell actions through one shared coordinator.
| Sidebar row | Default | Alternative |
|---|---|---|
| Python PCell | From Python (.py) | From GDS (.gds) |
| GSCH | From Layout Editor | From Schematic Editor |
Filter eligible templates by the selected route and display each incompatible
template's reason. Meep 3D and zapFDTD currently declare
input_routes.gsch_schematic.accepted=false because they require physical layout;
future circuit templates may accept schematic inputs. Keep internal editor names out of notebook/menu
text. The sidebar action opens sequential native VS Code Quick Picks at the top
of the window, starting with source route, then engine and eligible analysis/template.
Resolve PDK, physical/material providers and paths automatically from the clicked
design and its owning project/indexed PDK data; ask a targeted question only when
there are genuinely ambiguous valid choices. Do not guess physical defaults from
filenames. Use one shared launcher, not per-engine submenus.
Eventually add the same action near the .py/.gds eyeball and the .gsch top-right
GUI simulation button. Assess placement/feasibility. Sidebar and editor must share
one workflow, resolving the clicked item or active editor. Preserve selected cell
and subcells; collect selection only when ambiguous. Top-bar work may follow the
initial sidebar release. Both entry points follow the full spec's freshness,
independent-copy and no-automatic-execution rules.
2. Files and cells to generate¶
Keep the PDK import and PDK.activate() uncommented in every Meep/zapFDTD
origin and generated notebook. Origins retain the active placeholder import
from <INSERT_AUTO_PDK_MODULE_HERE> import PDK; generation substitutes the validated
module before execution. Never comment either line out during future edits.
Use engine-first selection, followed by the eligible generic template and future
presets. Each origin notebook has a paired .py and capability metadata. Current origin notebooks:
| Engine / choice | Origin notebook in simulation_templates/ |
Template ID |
|---|---|---|
| Meep / Generic 3D Template | meep_3d.ipynb |
meep_3d |
| zapFDTD / Generic 3D Template | Three separate origins: zapfdtd_template_py.ipynb, zapfdtd_template_gsch_layouteditor.ipynb, zapfdtd_template_gds.ipynb |
zapfdtd_3d |
The Meep display title is Generic 3D Template, declared in
meep_3d_template.metadata.json; the builder and binder propagate it into notebook
metadata and generated headings. Keep the stable meep_3d ID and filenames.
The compact native review uses Simulator & Template: Meep, Generic 3D Template;
other review labels use colons, including Simulation target:, Source port:
and Save to:. Simulation target shows the file plus selected factory/cell; its
edit action changes only the representation of the same clicked design, not the
design identity. The representation picker uses matching two-line rows with paths
beginning at the owning project folder, using the source filesystem's separators.
Python includes its explicit factory call; exported GDS omits a redundant cell
suffix. Direct multicell GDS retains cell identity when needed.
Keep the review enabled by default, with editable rows and no tagline beneath
the selected Review Completed, Create Notebook action.
Current generated filenames and geometry handling:
| Route | Meep filename | zapFDTD filename | Required loading placement |
|---|---|---|---|
| Python | meep_3dtemplate_py.ipynb |
zapfdtd_template_py.ipynb |
Factory import and component call together in Settings; refresh on Prepare. |
| Layout Editor | meep_3dtemplate_gsch_layouteditor.ipynb |
zapfdtd_template_gsch_layouteditor.ipynb |
Saved GSCH → rebuilt GDS → selected cell import. |
| Direct GDS | meep_3dtemplate_gds.ipynb |
zapfdtd_template_gds.ipynb |
Selected cell import in Settings, retaining ports; reread on Prepare. |
| Schematic Editor | No runnable notebook | No runnable notebook | Show incompatibility; the Meep _gsch_schematiceditor preview is explanatory only. |
Use matching paired-source suffixes and source-specific titles. Layout: one collapsed Config setup cell → Settings → Prepare/preview → Run → Results. Config is notebook cell 2 (code-cell ordinal 0), collapsed but accessible. It holds paths, ordinary imports and PDK activation. Python component-producing imports/calls and GDS loading move together to Settings. GSCH keeps its authored build, warnings, LVS, import and component-assignment chain together in Config, made reusable. Settings is the first visible code. These loading moves are specified work, not a claim that current origins already implement them.
For zapFDTD, each route uses its separately authored October 14 origin, including
native sim.add_monitors() and Results. Do not collapse these into one combined origin or
replace authored setup/settings cells. Bind only declared placeholders, retaining
comments, cell order and metadata. All five user settings are explicit slots:
source_port, wavelength_um, wavelength_span_um, num_wavelengths, cell_size_nm.
Settings and result methods are engine-specific. Meep exposes wavelength/span,
frequency count, one source port and resolution (pixels/µm); wavelengths are in µm.
Keep native meep.Simulation() setup/validation,
get_stack(), configuration write and interactive preview readable; results call
plot_interactive() / show_diagnostics(). zapFDTD uses native fdtd.Simulation(pdk=pdk_provider),
grid spacing in nm, PDK cladding material and its own preview/results API; see its
implementation notes. Future engines define their own controls and units.
The current Meep/zapFDTD Run cell is:
# Config binds results_path to this notebook's selected bundle.
# Results are downloaded into results_path/sim-data-<job_name>/.
# None would instead use the kernel's current working directory.
result = sim.run(parent_dir=results_path)
No execution toggle or cell tags. Only Config is collapsed; later code,
outputs and execution timing remain visible. Mark its Jupyter input/output hidden,
and apply the editor's actual hidden state explicitly: VS Code may not honor that
metadata alone. Bind by checked code-cell order; preserve metadata and clear counts/
outputs on generation. Use .venv kernelspec name/display, language python.
Creation hook: after writing a newly generated notebook, choose its project
Python kernel, set cwd to the notebook directory, and execute only code-cell ordinal 0.
Wait for completion before treating the notebook as ready. Never execute Settings,
preparation, Run or Results automatically. Do this once per creation, not on every
open/focus or for raw downloads/existing notebooks. Surface setup errors and its
LVS report; do not continue with stale geometry. The notebook metadata declares
creation_setup={code_cell:0, execute_on_creation:true}; the app implements the hook,
not Jupyter itself. Missing kernel/dependencies leave the created/open notebook
Setup pending; no automatic installs. Setup errors are recovered by manually
rerunning Config, with no separate Retry setup action.
Prepare follows the latest saved original design through compact shared support. Python refreshes local implementation/helpers/caches while retaining explicit notebook arguments. Direct GDS rereads the original file; Python-derived GDS first refreshes/re-exports the exact upstream factory/arguments. GSCH defaults to the full shared rebuild/LVS chain on every Prepare, with one shared switch to manual Config reload if too slow. Run uses prepared state without freshness guards, automatic refresh or extra confirmation.
3. Concrete app-supplied values¶
| Value | Resolution / destination |
|---|---|
| Source route / design / cell | Clicked item or active editor and user choice; preserve exported cell identity/hierarchy. |
gsch_path |
Absolute saved GSCH path, passed as nyancir_path. |
gds_path |
Original direct GDS or a generated bundle preparation/export artifact; a captured copy cannot become the authoritative live input. |
dschematic_path, lvs_path |
Intermediate DSchematic and LVS LYRDB destinations inside inputs/. |
source_path, project_path |
Original selected file and project root. |
prepare_path |
Local preparation directory inside the notebook bundle. |
results_path |
Absolute selected bundle path; default project simulations/<design>/<template>-NNN/. An edited Save to means exactly that folder, including existing/outside-project folders. Run passes it as parent_dir; gsim adds sim-data-<job_name>/. |
| PDK module / qualified name | Resolve through full spec's project/indexed activation contract; import module and <module>.PDK build argument. |
| Python module/factory/settings | Exact indexed clicked factory and current card arguments; import/call go together in Settings, retaining explicit notebook arguments on refresh. |
| Ports/source/monitors | Template-declared required/optional/unused and optical/electrical eligibility; missing required direct-GDS ports block with an explanation, without a new port editor. |
| Heading/filename/bundle/cwd | Template/route-specific names; default numbered destination or exact user-selected folder, kernel cwd at notebook, owning project/PDK stays tied to source. |
| Physical/solver overrides | Declared stack/materials/source/domain/numerical settings from setup. |
| Initial wavelength/span | Resolve project pyproject.toml simulation sweep through the same settings path as SAX. Convert min/max to center/full span per the full spec; write editable literals in Settings. |
| zapFDTD cladding layer | Ask the solver/PDK integration to return the resolved layer-stack key (cladding_layer_name), material and provenance. Bind that key and layers = PDK.layer_stack.layers once in Config; Settings uses layers[cladding_layer_name].material. Do not have the sidebar guess arbitrary layer names. |
Create inputs directories. Quote paths/data correctly; prefer /, including Windows
absolute paths. Origin notebook files have no user paths; review copies contain concrete
test bindings. Omit unused None assignments from Python/GDS Config cells; GDS
imports still require populated gds_path and cell_name. Keep GSCH rebuild/LVS
paths on the Layout Editor route. design.ports=[] preserves embedded ports in the current Meep/zapFDTD binders;
a nonempty list currently replaces them. The live-source implementation must not
replay captured ports.clear()/add_port(...) definitions over fresh GSCH ports.
Explicit user-authored overrides remain editable Python. Overrides specify name, center, width,
orientation, layer and optical type. Meep's get_stack() uses activated PDK: PH18DA providers
are in ph18da.layers, Intel 0.4.0 in intel.tech, registered on PDK.layer_stack.
Intel boolean-derived layer expressions need geometry qualification. Factory imports
remain live project references; shared automatic refresh on Prepare must handle
local helpers and component caches, rather than relying on a repeated import.
Origin notebooks expose named placeholders such as
input_path = "<INSERT_INPUT_PATH_HERE>", with corresponding project, preparation,
results, PDK-module and applicable material/layer bindings. Generation replaces
declared slots with concrete editable values, not global text
replacement over arbitrary cells. No required unresolved placeholder reaches
setup execution. New bundles default under project simulations/; reference
binders accept an explicit destination and bind results to that destination.
Reference binders scripts/bind_meep_template.py and
scripts/bind_zapfdtd_template.py read app-generation simulation.json against
their template schemas, stage inputs and refuse existing destinations. Generated
notebooks do not need JSON or runtime assets. The Meep context includes
validated structure, template/engine, PDK, provenance, design, source and monitors;
optional stack/materials, port vertical overrides, frequencies and native domain/
solver controls. Provenance revision is not an automatic freshness check. Production
coordinator must perform the full spec's validation and publication. Its destination
policy differs from these prototypes: existing folders are valid, without sibling
redirection, forced suffixing or an empty-folder requirement. Exact notebook-file
collision handling is the remaining narrow publication decision; future
sim-data-<job_name>/ results are separate and require no naming redesign. Use
zapFDTD's documented native contract for its fields; new engines supply their own
schemas rather than copying Meep-only controls.
The reference binders read project_root/pyproject.toml during creation; production
should reuse the app's resolved settings.sim.x value (including sim.wls
normalization). wavelength_overrides carries deliberate wizard center/span edits;
old example values in source must not supersede project defaults. Keep num_freqs
or num_wavelengths independent of the GUI sweep's num. Config reruns refresh
geometry, not overwrite the user's edited Settings; create a new notebook or edit
the wavelength values to apply later project-default changes.
4. GSCH build and direct-GDS behavior¶
The app checks source route/selection and resolves saved edits before creation.
Full build/LVS runs in the hidden setup. Never accept an existing build/gds artifact
as fresh. Keep physical-placement qualification via existing SDK/app semantics;
schematic-coordinate fallback is not layout. Hierarchical dialects remain unqualified.
GSCH setup calls build_nyancir_gds with gsch_path, gds_path, pdk_name and
dschematic_path (all four SDK arguments required). Reject returned warnings. Then
use await asyncio.to_thread(check_lvs, ...), write lvs_path, load it with KLayout
ReportDatabase, and stop if num_items() is nonzero. Only then import geometry.
No ET parser, hidden full-LVS inside the build function, or custom comparison logic.
Prepare invokes the same reusable chain by default; manual mode uses an explicit
Config rerun. Opening an existing notebook does not execute it. All concrete paths
occur as Config variables, not repeated literals.
Direct GDS imports preserve embedded ports. Use the clicked valid cell, otherwise the sole top cell, otherwise a cell Quick Pick; never infer it from a filename. Missing required ports stop creation with guidance to existing project/PDK mechanisms, without sample ports or a new port-definition editor. Standalone GDS has no reference schematic for LVS. Import error text: “This GDS file is corrupt or unreadable. Export it again or select another GDS file.” Material/stack errors identify the unresolved property.
5. Implementation checks and remaining tasks¶
Apply the full spec's release gates; the app-specific checks are:
- Missing/stale project GDS still rebuilds from valid GSCH; changed saved source reaches a fresh bundle. Invalid placement/connections/LVS prevents successful initialization.
- Repeating generation preserves previous notebooks/results. Explicit execution of the hidden setup rebuilds its own input; reopening does not auto-run it.
- Only setup auto-executes once on creation; paths/PDK/geometry resolve, settings remain first visible, and setup failures are clearly reported. Kernel/cwd are correct.
- Sidebar and eventual top bar reach equivalent setup with the correct selected cell.
Latest recorded shared Meep/zapFDTD/project checks: 88 passed (October 15), not proof of end-to-end app integration. Supported source routes and valid/broken rebuild fixtures were verified on October 12 (see each engine's notes). Finish shared worker/registry/wizard orchestration and the creation hook. Current Meep/zapFDTD templates monitor all exported optical ports; no editable monitor selector. The revised hidden/LVS setup was also verified: supported routes pass and both broken fixtures stop before import. Editor auto-execution/hiding remains app work. Solver convergence/mode diagnostics remain SDK/solver follow-up, recorded in development notes, not sidebar implementation.
If Config fails, an earlier component may remain in the kernel. Ordinary recovery is to fix and manually rerun Config. Do not add a Run guard or separate retry UI.
Source-port preference order is explicit choice, eligible remembered choice for the
exact canonical project/design, project preferred choice, sole eligible port,
otherwise a picker. Remember is opt-in, staged until publication; cancel writes no
new preference. Change/Forget remain accessible even with review disabled. No
o1 or alphabetical guessing. A removed explicit notebook port raises the ordinary
Prepare error. Only multiple valid PDK candidates cause a PDK picker, with:
warning: multiple PDKs in the same project file may result in unpredictable behavior.
Missing PDK/physical data is an actionable error, not a transcription prompt or
filename-based guess.
Shared template edits: use C:/Users/eve/Documents/Codex/worktrees/simulation-template-spec
for these route-specific origins. Other worktrees do not synchronize edits.
Coordinate edits to the same files; preserve the declared <INSERT_..._HERE> slots.
The zap origins require gsim PR #371 (9c9975b or later); the app must not
reinsert the retired notebook-local monitor helper.