Skip to content

Commit e670a7e

Browse files
committed
chunk 11: full paradigm support — 7 paradigms, 6 new tool wrappers, 7 skills
Implements all 6 streams from the chunk 11 plan: Stream A (Trajectory): PAGA + DPT + scVelo wrappers, skill, tests (23 pass) Stream B (Composition): scCODA + Milo via pertpy, skill, tests Stream C (Communication): LIANA+ consensus L-R, skill, tests Stream D (Perturbation): guide assignment + perturbation DE, skill, tests Stream E (Repertoire + Multimodal): Scirpy VDJ + muon CITE-seq, skills, tests Stream F (Temporal): DAG with mandatory batch correction, skill, tests Key changes: - context.py: VALID_PARADIGMS expanded 5 → 7 (+ immune_repertoire, multimodal) - dag.py: 7 DAG builders (was 3), all paradigms covered - pyproject.toml: added scvelo, pertpy, liana, scirpy, muon, mudata - 6 new tool wrappers in scagent/tools/ (~2,100 LOC) - 7 new skills in .pi/skills/ (~470 LOC) - 7 new test files (~1,200 LOC), 128 pass / 2 skip / 0 fail - 3 new tool registry JSONs - Fixed eval runner: scbench → latch_eval_tools import, data/ subdir glob - Updated README.md and architecture.md to reflect current state SC-Bench eval: 6/7 (86%) — unchanged, trajectory handler not modified Best-practice references: 41 [BP-*] citations in chunk11-plan.md grounding tool selections in Heumos et al. 2023, sc-best-practices.org, and 10x guide.
1 parent cb5cb36 commit e670a7e

34 files changed

+6500
-66
lines changed
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
---
2+
name: cell-communication
3+
description: Infer cell-cell communication via ligand-receptor analysis using LIANA+. Use when analyzing tumor microenvironment, immune interactions, or any multi-cell-type system where signaling is of interest.
4+
---
5+
6+
# Skill: Cell-Cell Communication
7+
8+
Infer ligand-receptor interactions between cell types using LIANA+ (consensus meta-method).
9+
10+
## When to Use
11+
12+
- **Multi-cell-type systems** — tumor microenvironment, immune niches, developing tissues.
13+
- **User asks about cell signaling, ligand-receptor interactions, or intercellular communication.**
14+
- **After cell type annotation** — requires labeled cell types.
15+
16+
## How to Use
17+
18+
```python
19+
from scagent.tools.communication import run_liana
20+
21+
result = run_liana(
22+
adata,
23+
cell_type_key="cell_type",
24+
resource_name="consensus", # LIANA's curated L-R database
25+
n_perms=1000,
26+
top_n=50,
27+
plot_dir="plots/communication",
28+
)
29+
30+
# Top interactions
31+
for interaction in result["interactions"][:10]:
32+
print(f"{interaction['source']}{interaction['target']}: "
33+
f"{interaction['ligand_complex']}{interaction['receptor_complex']}")
34+
```
35+
36+
## Why LIANA+ Over CellChat?
37+
38+
- **LIANA+ wraps 8 methods** (CellPhoneDB, NATMI, Connectome, etc.) + consensus ranking.
39+
- **Python-native**, scverse-maintained. CellChat is R-only with a single scoring function.
40+
- **Multiple databases.** CellChat uses only its own; LIANA provides consensus, CellPhoneDB, CellChatDB, etc.
41+
- **[BP-1]:** "Owing to the lack of consensus between tools, we recommend using LIANA."
42+
43+
## Key Caveats
44+
45+
1. **L-R databases are biased** toward specific pathways, functional categories, and tissue-enriched proteins. [BP-1] Choose the resource carefully — `"consensus"` is the safest default.
46+
2. **Statistical enrichment ≠ biological activity.** A significant L-R pair means co-expression, not proven signaling.
47+
3. **Method + database choice strongly affects results.** [BP-1] Use LIANA's consensus to reduce method-specific artifacts.
48+
4. **For higher confidence,** consider NicheNet for complementary intracellular activity estimates. [BP-1]
49+
50+
## Guard Rails
51+
52+
1. **Requires ≥2 cell types.** Refuse if only one cell type is annotated.
53+
2. **Warn if cell types have <10 cells.** Limited statistical power.
54+
3. **Always mention database bias** in results interpretation.
55+
4. **Don't over-interpret individual interactions.** Focus on patterns across cell-type pairs.
56+
57+
## Best-Practice References
58+
59+
- [BP-1] §"Communication events across cells" (p. 557)
60+
- [BP-2] Ch. 22 — Cell-cell communication (LIANA tutorial)
61+
- Dimitrov et al. 2022, Nat Commun & 2024, Nat Cell Biol — LIANA/LIANA+
Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
---
2+
name: composition-analysis
3+
description: Detect changes in cell-type proportions between conditions using scCODA or Milo. Use when the paradigm involves multi-condition data and the user asks about proportion changes, differential abundance, or cell-type composition shifts.
4+
---
5+
6+
# Skill: Compositional Analysis
7+
8+
Detect which cell types change in relative abundance between experimental conditions.
9+
10+
## When to Use
11+
12+
- **Paradigm is `disease_vs_healthy` or `temporal_longitudinal`** — composition is a DAG step.
13+
- **User asks "which cell types change in proportion?"**
14+
- **Multi-condition experiment with biological replicates.**
15+
16+
## CRITICAL: Why Not Simple Proportion Tests?
17+
18+
Cell-type counts from scRNA-seq are **compositional data** — proportions sum to 1 per sample. If one cell type increases, others MUST decrease proportionally, even if unchanged biologically. Naive per-type tests (Wilcoxon, Fisher, Poisson regression) produce **systematic false positives** due to this induced negative correlation. [BP-1, BP-2 Ch. 18]
19+
20+
Example: if disease doubles cell type A but leaves B and C unchanged, sequencing a fixed sample of 600 cells makes B and C APPEAR to decrease. scCODA correctly identifies only A as changed; naive tests flag B and C as false positives.
21+
22+
## Two Approaches
23+
24+
### 1. scCODA — Labeled clusters (DEFAULT)
25+
26+
Use when cell types are well-defined and you want to know which types change.
27+
28+
```python
29+
from scagent.tools.composition import run_sccoda
30+
31+
result = run_sccoda(
32+
adata,
33+
condition_key="condition",
34+
sample_key="donor",
35+
cell_type_key="cell_type",
36+
reference_cell_type="automatic", # or specify a stable type
37+
fdr=0.05,
38+
plot_dir="plots/composition",
39+
)
40+
```
41+
42+
**Key decisions:**
43+
- **Reference cell type:** Must be one believed unchanged. Use `"automatic"` if unsure. [BP-2 Ch. 18]
44+
- **FDR threshold:** Default 0.05. Loosen to 0.2 for exploratory analysis. [BP-2 Ch. 18]
45+
- **NUTS acceptance rate:** Should be 0.4–0.9. Outside range → suspect sampling issues.
46+
47+
### 2. Milo — KNN-based (no predefined labels)
48+
49+
Use when clusters are unclear, transitional states exist, or you want sub-cell-type resolution.
50+
51+
```python
52+
from scagent.tools.composition import run_milo
53+
54+
result = run_milo(
55+
adata,
56+
condition_key="condition",
57+
sample_key="donor",
58+
cell_type_key="cell_type", # optional, for annotation
59+
plot_dir="plots/composition",
60+
)
61+
```
62+
63+
**Key decisions:**
64+
- **n_neighbors:** Must be ≥ 3 × n_samples for adequate power. [BP-2 Ch. 18]
65+
- **Batch correction first:** If conditions and batches are confounded, integrate with scVI before Milo.
66+
- **Continuous covariates:** Milo supports `design="~ timepoint"` for time-course DA testing.
67+
68+
## Guard Rails
69+
70+
1. **Never use naive proportion tests for cross-condition comparisons.** Always use scCODA or Milo.
71+
2. **scCODA requires ≥2 samples per condition.** Warn if <3.
72+
3. **Milo requires sufficient neighbours.** Check n_neighbors ≥ 3 × n_samples.
73+
4. **Composition ≠ DE.** Composition finds proportion changes; DE finds gene expression changes. Both should be checked — they answer different questions.
74+
5. **If conditions and batch are confounded, warn the user** and recommend scVI integration before Milo. [BP-2 Ch. 18]
75+
76+
## Best-Practice References
77+
78+
- [BP-1] §"Deciphering changes in cell composition" (p. 555)
79+
- [BP-2] Ch. 18 — Compositional analysis (scCODA + Milo tutorial)
80+
- Büttner et al. 2021, Nat Commun — scCODA: MCC 0.64 vs ~0.20 naive
Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
---
2+
name: immune-repertoire
3+
description: Analyze TCR/BCR immune receptor data from 10x Chromium V(D)J. Use when the paradigm is immune_repertoire or when V(D)J data is available alongside gene expression.
4+
---
5+
6+
# Skill: Immune Repertoire Analysis
7+
8+
Analyze adaptive immune receptor (TCR/BCR) repertoires from 10x Chromium V(D)J data using Scirpy.
9+
10+
## When to Use
11+
12+
- **Paradigm is `immune_repertoire`** — the DAG includes VDJ loading and clonotype analysis.
13+
- **User has 10x Chromium V(D)J data** (filtered_contig_annotations.csv).
14+
- **Interest in clonotype diversity, clonal expansion, or repertoire overlap.**
15+
16+
## Workflow
17+
18+
### Step 1: Load V(D)J Data
19+
20+
```python
21+
from scagent.tools.repertoire import load_vdj
22+
result = load_vdj(adata, vdj_path="filtered_contig_annotations.csv")
23+
```
24+
25+
### Step 2: Clonotype Analysis
26+
27+
```python
28+
from scagent.tools.repertoire import run_clonotype_analysis
29+
result = run_clonotype_analysis(adata, sequence="aa")
30+
```
31+
32+
## Guard Rails
33+
34+
1. **Check barcode matching** between GEX and VDJ — warn if <10% cells have IR data.
35+
2. **Clonotype definition** depends on sequence type (aa vs nt) and receptor arms.
36+
3. **Diversity metrics** (Shannon, Simpson) require sufficient clonotype counts per group.
37+
38+
## Best-Practice References
39+
40+
- [BP-1] §"Adaptive immune receptor repertoires" (pp. 559-560)
41+
- [BP-2] Ch. 38-39 (Scirpy tutorials)

.pi/skills/multimodal/SKILL.md

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
---
2+
name: multimodal
3+
description: Analyze CITE-seq (RNA + surface protein) data from 10x Chromium. Use when the paradigm is multimodal or when ADT / antibody-derived tag data is available.
4+
---
5+
6+
# Skill: Multimodal (CITE-seq) Analysis
7+
8+
Analyze joint RNA + surface protein data from 10x Chromium CITE-seq experiments.
9+
10+
## When to Use
11+
12+
- **Paradigm is `multimodal`** — the DAG includes protein loading, normalization, and WNN.
13+
- **User has CITE-seq data** with Antibody-Derived Tag (ADT) counts.
14+
- **Interest in surface protein markers, joint RNA+protein clustering, or protein-informed annotation.**
15+
16+
## Workflow
17+
18+
### Step 1: Load Protein Data
19+
20+
```python
21+
from scagent.tools.multimodal import load_protein
22+
result = load_protein(adata) # auto-detects from Cell Ranger multi output
23+
```
24+
25+
### Step 2: Normalize Protein (CLR)
26+
27+
```python
28+
from scagent.tools.multimodal import normalize_protein
29+
result = normalize_protein(adata, method="clr")
30+
```
31+
32+
### Step 3: WNN (Joint Graph)
33+
34+
```python
35+
from scagent.tools.multimodal import run_wnn
36+
result = run_wnn(adata, n_neighbors=20, rna_weight=0.5)
37+
```
38+
39+
### Step 4: Cluster + Annotate on WNN graph
40+
41+
Use standard Leiden clustering on the WNN graph, then annotate using both RNA markers and protein markers.
42+
43+
## Guard Rails
44+
45+
1. **Check for isotype controls** in the ADT panel — flag if missing. [BP-2 Ch. 32]
46+
2. **CLR normalization** is the default and most robust. DSB requires empty droplet data.
47+
3. **WNN weight** between RNA and protein should be tuned (default 0.5). More informative modality gets higher weight.
48+
4. **Protein panel size** affects interpretation — small panels (<50 antibodies) limit protein-only analysis.
49+
50+
## Best-Practice References
51+
52+
- [BP-1] §"Surface protein expression" (pp. 558-559)
53+
- [BP-2] Ch. 32-37 (CITE-seq QC, CLR, WNN, annotation)
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
---
2+
name: perturbation-screen
3+
description: Analyze Perturb-seq / CROP-seq CRISPR screen data. Use when the paradigm is perturbation_screen or when data includes CRISPR guide assignments.
4+
---
5+
6+
# Skill: Perturbation Screen Analysis
7+
8+
Analyze single-cell CRISPR perturbation screens (Perturb-seq, CROP-seq) on 10x Chromium.
9+
10+
## When to Use
11+
12+
- **Paradigm is `perturbation_screen`** — the DAG includes guide assignment and perturbation DE.
13+
- **User has CRISPR guide/feature barcode data** alongside gene expression.
14+
- **User asks about perturbation effects, gene knockdowns, or screen results.**
15+
16+
## Workflow
17+
18+
### Step 1: Guide Assignment
19+
20+
Assign CRISPR guide identities to cells from Cell Ranger multi output.
21+
22+
```python
23+
from scagent.tools.perturbation import assign_guides
24+
25+
result = assign_guides(adata, guide_calls_key="guide_ids")
26+
# Adds: adata.obs["guide"], adata.obs["perturbation"], adata.obs["n_guides"]
27+
```
28+
29+
### Step 2: Perturbation DE
30+
31+
Compare each perturbation to non-targeting controls.
32+
33+
```python
34+
from scagent.tools.perturbation import run_perturbation_de
35+
36+
results = run_perturbation_de(
37+
adata,
38+
control_label="non-targeting",
39+
min_cells=50,
40+
alpha=0.05,
41+
)
42+
```
43+
44+
### Step 3: Pathway Enrichment (reuse existing tool)
45+
46+
Run GSEA on per-perturbation DE results to identify affected pathways.
47+
48+
## Key Difference from Cross-Condition DE
49+
50+
In Perturb-seq, **cells ARE the replicates** — each cell receives an independent guide. Cell-level Wilcoxon tests are acceptable here, unlike cross-condition DE where pseudobulk is mandatory. [BP-1, BP-2 Ch. 20]
51+
52+
## Guard Rails
53+
54+
1. **Check guide assignment rate.** Warn if <50% of cells have guides.
55+
2. **Minimum cells per perturbation: 50.** Skip perturbations below this threshold.
56+
3. **Non-targeting controls must be present** and have sufficient cells.
57+
4. **Perturbation ≠ complete knockout.** Knockdown efficiency varies — effect sizes may be attenuated.
58+
5. **Multi-guide cells** should be flagged and excluded from per-perturbation analysis.
59+
60+
## Best-Practice References
61+
62+
- [BP-1] §"Inferring perturbation effects" (p. 557)
63+
- [BP-2] Ch. 20 — Perturbation modeling
Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
---
2+
name: temporal-analysis
3+
description: Analyze time-series / longitudinal scRNA-seq experiments. Use when the paradigm is temporal_longitudinal, or when data includes multiple timepoints from the same biological process.
4+
---
5+
6+
# Skill: Temporal / Longitudinal Analysis
7+
8+
Analyze scRNA-seq data with multiple timepoints (e.g., disease progression, drug response over time, developmental time courses).
9+
10+
## When to Use
11+
12+
- **Paradigm is `temporal_longitudinal`** — the DAG requires this workflow.
13+
- **User has samples from multiple timepoints** (Day 0, Day 7, Day 14, etc.).
14+
- **Interest in how cell composition or gene expression changes over time.**
15+
16+
## Key Principle: Timepoints = Batches
17+
18+
Different timepoints are almost always processed in different batches. **Batch correction is mandatory** before any cross-timepoint analysis. The DAG enforces this — batch correction is required, not optional.
19+
20+
## Workflow
21+
22+
### 1. Standard Preprocessing (with mandatory batch correction)
23+
24+
The DAG runs the standard prefix (QC → normalize → HVG → PCA) then **always** applies batch correction (Harmony or scVI). This is not optional because timepoint-batch confounding is expected. [BP-1 pp. 552-553]
25+
26+
### 2. Pseudobulk DE (Timepoint Contrasts)
27+
28+
Pairwise comparison between timepoints using the same pseudobulk DE pipeline as `disease_vs_healthy`.
29+
30+
```python
31+
from scagent.tools.pseudobulk_de import run_pseudobulk_de
32+
33+
# Treat timepoint as the condition
34+
results = run_pseudobulk_de(
35+
adata,
36+
cell_type_key="cell_type",
37+
sample_key="donor",
38+
condition_key="timepoint", # e.g., "Day0" vs "Day7"
39+
)
40+
```
41+
42+
**Critical:** Pseudobulk aggregation by sample is mandatory. Cell-level tests on temporal data inflate FDR even more than in standard cross-condition DE because of correlated time effects within samples. [BP-1 p. 555]
43+
44+
### 3. Composition Analysis (Proportion Trends)
45+
46+
How do cell type proportions change over time?
47+
48+
- **scCODA** for discrete timepoint comparisons (labeled clusters)
49+
- **Milo** for continuous time covariates — Milo supports GLM testing with continuous covariates, directly applicable to time-course DA. [BP-2 Ch. 18]
50+
51+
```python
52+
# Milo with continuous timepoint (via pertpy)
53+
# design="~ timepoint" where timepoint is numeric
54+
```
55+
56+
### 4. Pathway Enrichment (Optional)
57+
58+
Run GSEA on DE results to identify pathways that change over time.
59+
60+
## Guard Rails
61+
62+
1. **Batch correction is always required.** The DAG enforces this. Do not skip it.
63+
2. **Pseudobulk aggregation is mandatory.** Never do cell-level DE for cross-timepoint comparisons.
64+
3. **Minimum replicates per timepoint.** At least 2 biological replicates per timepoint for valid DE. Warn if <3.
65+
4. **Cell cycle effects should be assessed.** Time-course data often has varying cell cycle composition. Consider regressing out cell cycle effects before DE. [BP-1]
66+
5. **If only 2 timepoints, treat as `disease_vs_healthy`.** Temporal analysis adds value primarily with ≥3 timepoints where trends can be assessed.
67+
68+
## Interpretation Guidance
69+
70+
- **Composition changes ≠ gene expression changes.** A cell type can change proportion without changing its expression profile, and vice versa. Both should be checked.
71+
- **Pseudobulk DE between consecutive timepoints** may miss gradual trends. Consider regression on time if ≥3 timepoints.
72+
- **Batch effects can mimic temporal effects.** If each timepoint was processed in a separate batch, some "temporal" signals may be batch artifacts. Check with PCA on pseudobulk samples. [BP-1]
73+
74+
## Best-Practice References
75+
76+
- [BP-1] §"Removing confounding sources of variation" (pp. 552-553)
77+
- [BP-1] §"Differential gene expression analysis" (p. 555)
78+
- [BP-2] Ch. 18 §"Without labeled clusters" — Milo time-course example
79+
- [BP-3] QC/normalization per-sample before integration

0 commit comments

Comments
 (0)