Add Manual Notebook CI workflow for executing and rebuilding Nexus-ready notebooks

Author: AmberLee2427

.github/workflows/manual-notebook-ci.yml

@@ -0,0 +1,58 @@
+name: Manual Notebook CI
+
+on:
+  workflow_dispatch:
+    inputs:
+      profile:
+        description: "Execution profile"
+        required: true
+        default: "fast"
+        type: choice
+        options:
+          - fast
+          - full
+
+jobs:
+  execute-notebooks:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          submodules: true
+
+      - name: Setup micromamba
+        uses: mamba-org/setup-micromamba@v2
+        with:
+          environment-file: env.yml
+          environment-name: rges-pit-dc
+          cache-environment: true
+
+      - name: Install runner + transformer deps
+        shell: bash -l {0}
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install pyyaml nbclient nbformat
+
+      - name: Rebuild Nexus-ready notebooks (RRN/build)
+        shell: bash -l {0}
+        run: |
+          rm -rf RRN/build
+          python scripts/notebook_transformer.py --force
+
+      - name: Execute notebooks
+        shell: bash -l {0}
+        run: |
+          SKIP_TAGS=""
+          if [ "${{ inputs.profile }}" = "fast" ]; then
+            SKIP_TAGS="slow,ci-skip"
+          else
+            SKIP_TAGS="ci-skip"
+          fi
+
+          python scripts/execute_notebooks_ci.py \
+            --glob "RRN/build/*.ipynb" \
+            --skip-tags "$SKIP_TAGS" \
+            --timeout 1800

TODO.md

@@ -28,11 +28,13 @@
   - `reges-pit/nexus-notebooks`
 - [ ] add global `refdata_dependencies.yaml` (match `spacetelescope/roman_notebooks` format)
 - [ ] decide notebook discovery regexes (paths/substrings STScI wrangler should scan)
-- [ ] organize notebooks in directories where each dir has its own `requirements.txt`
-- [ ] add optional top-level `environment.yml` (mamba spec for non-pip / low-level deps only)
-- [ ] move any per-notebook helper `.py` into the same notebook directories
-- [ ] if using shared modules, create a top-level shared dir + symlink into each notebook dir that imports it
-- [ ] update automation: publish/update the Nexus repo from `RRN/export/` (no PRs to `roman_notebooks`)
+  - `notebooks/*.ipynb`
+- [-] organize notebooks in directories where each dir has its own `requirements.txt`
+- [x] add optional top-level `environment.yml` (mamba spec for non-pip / low-level deps only)
+- [-] move any per-notebook helper `.py` into the same notebook directories
+   - they are downloaded in cells
+- [-] if using shared modules, create a top-level shared dir + symlink into each notebook dir that imports it
+- [x] update automation: publish/update the Nexus repo from `RRN/export_submodule/` (no PRs to `roman_notebooks`)
 - [ ] send STScI the repo URL + chosen regex list + any notes on expected execution order
 
 ## Nexus
@@ -44,6 +46,8 @@
 - [-] edit the tools notebook to match the Colab/RRN freindly style
 - [ ] add Citations sections in all notebooks (is this already done?)
 - [ ] delete the `roman_notebooks` PR + fork
+- [ ] test that the notebooks run using the provided yaml
+- [ ] tag long cells to avoid in quick tests
 
 ## Microlensing Tools Notebook
 - [ ] table formatting
@@ -59,4 +63,10 @@
 - [ ] proof read
 - [ ] test run
 - [ ] check data import functions
-- [ ] Add to nexus export list
+- [ ] Add to nexus export list
+
+## Website
+- [ ] remove mention of workshop sign-up
+- [ ] update with real data links
+- [x] change PR from fork to PR from branch
+- [ ] delete website fork (dc-copy)