Skip to content

Commit aa16469

Browse files
feat: Introduce review commands for diff analysis and reporting
- Added `review_cmds.py` to handle various review commands including: - `review_diff_cmd`: Review standard diff results. - `review_stats_cmd`: Review column-level statistics for a diff. - `review_missing_cmd`: Review rows that exist only on one side. - `review_columns_cmd`: List available columns in the diff data. - `review_export_cmd`: Export diff results to an XLSX report. - `review_meta_cmd`: Emit JSON metadata for AI/LLM analysis. - Created `run.py` to manage different run commands for comparing tables, files, and SQL queries, including: - `run_file_cmd`: Compare two CSV/XLSX files. - `run_query_cmd`: Compare two SQL query results. - `run_unified_cmd`: A concise command to run comparisons based on mode. - `run_dataset_cmd`: Compare datasets defined in YAML configuration. - Implemented `run_auto.py` for automatic detection of input types (tables, files, SQL) and comparison execution. - Added `run_stats.py` for comparing table statistics without row-by-row comparison. - Developed `run_table.py` for detailed comparison of two tables or CSV/XLSX files. - Updated tests to reflect changes in command structure from `inspect` to `review`, ensuring all functionalities are covered. - Refactored CLI commands to improve usability and maintainability.
1 parent ef2948a commit aa16469

25 files changed

Lines changed: 546 additions & 164 deletions

README.md

Lines changed: 29 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ When you modify logic, filters, or inputs, it lets you compare the previous and
55

66
You can compare datasets in two complementary ways:
77

8-
1) Row-by-row comparison (with an ID): detect missing rows on either side, identify the columns with the most changes, and inspect before/after values for any record.
8+
1) Row-by-row comparison (with an ID): detect missing rows on either side, identify the columns with the most changes, and review before/after values for any record.
99
2) Statistical comparison: compare column-level statistics such as null counts, distinct counts, and other aggregates to quickly understand overall impact.
1010

1111
---
@@ -49,56 +49,55 @@ Compare two tables on a key:
4949

5050
```bash
5151
export SQLCOMPARE_CONN_DEFAULT="postgresql://<user>:<pass>@<host>/<db>"
52-
sqlcompare run analytics.fact_sales analytics.fact_sales_new id
52+
sqlcompare run table analytics.fact_sales analytics.fact_sales_new id
5353
```
5454

5555
That command prints a **diff_id**. Use it for follow-up analysis:
5656

5757
```bash
58-
sqlcompare inspect <diff_id> --stats
59-
sqlcompare inspect <diff_id> --column revenue --limit 100
60-
sqlcompare inspect <diff_id> --missing-current
61-
sqlcompare diff-queries <diff_id>
62-
sqlcompare inspect <diff_id> --save summary
63-
sqlcompare inspect <diff_id> --save complete --file-path ./reports/full_diff.xlsx
58+
sqlcompare review stats <diff_id>
59+
sqlcompare review diff <diff_id> --column revenue --limit 100
60+
sqlcompare review missing <diff_id> --side current
61+
sqlcompare review meta <diff_id>
62+
sqlcompare review export <diff_id> --mode summary
63+
sqlcompare review export <diff_id> --mode complete --output ./reports/full_diff.xlsx
6464
```
6565

66-
## Inspect report export (XLSX)
66+
## Review report export (XLSX)
6767

68-
You can export inspect results as a multi-tab Excel report using `--save`.
68+
You can export review results as a multi-tab Excel report using `review export`.
6969

7070
Modes:
7171

72-
* `--save none` (default): no file output
73-
* `--save summary`: creates `Overview` + per-column tabs (top 200 rows each) + `SQL Reference`
74-
* `--save complete`: same tabs, but without the 200-row cap (limited only by XLSX limits)
72+
* `--mode summary` (default): creates `Overview` + per-column tabs (top 200 rows each) + `SQL Reference`
73+
* `--mode complete`: same tabs, but without the 200-row cap (limited only by XLSX limits)
7574

7675
Examples:
7776

7877
```bash
7978
# Save summary report in current directory with generated timestamped filename
80-
sqlcompare inspect <diff_id> --save summary
79+
sqlcompare review export <diff_id> --mode summary
8180

8281
# Save full report to a specific location
83-
sqlcompare inspect <diff_id> --save complete --file-path ./reports/full_diff.xlsx
82+
sqlcompare review export <diff_id> --mode complete --output ./reports/full_diff.xlsx
8483

8584
# Save a single-column summary report
86-
sqlcompare inspect <diff_id> --column revenue --save summary --file-path ./reports/revenue_diff.xlsx
85+
sqlcompare review export <diff_id> --mode summary --column revenue --output ./reports/revenue_diff.xlsx
8786
```
8887

8988
Notes:
9089

91-
* `--file-path` is optional; if omitted, SQLCompare generates a readable timestamped filename.
92-
* `--save summary|complete` is for the standard diff view and should not be combined with `--stats`, `--missing-current`, `--missing-previous`, or `--list-columns`.
90+
* `--output` is optional; if omitted, SQLCompare generates a readable timestamped filename.
91+
* `review export` is for the standard diff view and should not be combined with `review stats`, `review missing`, or `review columns`.
9392

9493
---
9594

96-
## diff-queries (AI-friendly metadata)
95+
## review meta (AI-friendly metadata)
9796

98-
Use `diff-queries` to get a JSON payload describing the queryable tables and ready-to-run SQL templates for a given `diff_id`. This is especially useful for AI agents that need structured context before running analysis queries.
97+
Use `review meta` to get a JSON payload describing the queryable tables and ready-to-run SQL templates for a given `diff_id`. This is especially useful for AI agents that need structured context before running analysis queries.
9998

10099
```bash
101-
sqlcompare diff-queries <diff_id>
100+
sqlcompare review meta <diff_id>
102101
```
103102

104103
Output (JSON):
@@ -137,7 +136,7 @@ SQLCompare does two things:
137136
Best for production validation and regression checks across supported connectors.
138137

139138
```bash
140-
sqlcompare run analytics.users analytics.users_new user_id,tenant_id
139+
sqlcompare run table analytics.users analytics.users_new user_id,tenant_id
141140
```
142141

143142
Why it’s useful:
@@ -154,17 +153,17 @@ Use this when tables aren’t materialized yet or you want a filtered slice.
154153
Inline SQL:
155154

156155
```bash
157-
sqlcompare run \
158-
"SELECT * FROM analytics.orders WHERE order_date < '2024-01-01'" \
159-
"SELECT * FROM analytics.orders WHERE order_date >= '2024-01-01'" \
160-
order_id \
156+
sqlcompare run query \
157+
--previous "SELECT * FROM analytics.orders WHERE order_date < '2024-01-01'" \
158+
--current "SELECT * FROM analytics.orders WHERE order_date >= '2024-01-01'" \
159+
--index order_id \
161160
-c snowflake_prod
162161
```
163162

164163
SQL files:
165164

166165
```bash
167-
sqlcompare run queries/previous.sql queries/current.sql order_id -c snowflake_prod
166+
sqlcompare run query --previous queries/previous.sql --current queries/current.sql --index order_id -c snowflake_prod
168167
```
169168

170169
Or create a dataset config:
@@ -184,7 +183,7 @@ new:
184183
Run the compare:
185184
186185
```bash
187-
sqlcompare dataset path/to/dataset.yaml
186+
sqlcompare run dataset path/to/dataset.yaml
188187
```
189188

190189
Why it’s useful:
@@ -216,7 +215,7 @@ new:
216215
Set a local default connector and run:
217216
218217
```bash
219-
sqlcompare run path/to/previous.csv path/to/current.xlsx id
218+
sqlcompare run file path/to/previous.csv path/to/current.xlsx id
220219
```
221220

222221
Why it’s useful:
@@ -232,7 +231,7 @@ Use this when your data lives in a local `.duckdb` file.
232231

233232
```bash
234233
export SQLCOMPARE_CONN_LOCAL="duckdb:////absolute/path/to/warehouse.duckdb"
235-
sqlcompare run raw.customers staged.customers id -c local
234+
sqlcompare run table raw.customers staged.customers id -c local
236235
```
237236

238237
Why it’s useful:

examples/row_compare.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ Compare rows using a dataset YAML with file inputs
55
## Command
66

77
```bash
8-
sqlcompare dataset tests/datasets/row_compare/dataset.yaml
8+
sqlcompare run dataset tests/datasets/row_compare/dataset.yaml
99
```
1010

1111
## Output
@@ -45,6 +45,6 @@ Rows only in previous dataset: 1
4545
| 1 | name | alpha | alfa |
4646
| 1 | amount | 10 | 11 |
4747
+----+--------+--------+---------+
48-
🔎 To review the diff, run: sqlcompare inspect compare_sqlcompare_dataset_dataset_998aa249_previous_sqlcompare_dataset_dataset_998aa249_new_20260104_025829_b28946b5
49-
💡 Tips: --stats for per-column counts, --missing-current/--missing-previous for row-only, --column <name> to filter, --list-columns to inspect available fields.
48+
🔎 To review the diff, run: sqlcompare review compare_sqlcompare_dataset_dataset_998aa249_previous_sqlcompare_dataset_dataset_998aa249_new_20260104_025829_b28946b5
49+
💡 Tips: sqlcompare review stats <diff_id> for per-column counts, sqlcompare review missing <diff_id> --side current|previous for row-only, sqlcompare review diff <diff_id> --column <name> to filter, sqlcompare review columns <diff_id> to inspect available fields.
5050
```

examples/stats_compare.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ Compare statistics between two CSV files
55
## Command
66

77
```bash
8-
sqlcompare stats tests/datasets/stats_compare/previous.csv tests/datasets/stats_compare/current.csv
8+
sqlcompare run stats tests/datasets/stats_compare/previous.csv tests/datasets/stats_compare/current.csv
99
```
1010

1111
## Output

skills/sqlcompare-connection-debug/SKILL.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ Use these to isolate authentication vs query issues.
6262
1. Simple query against the target connector.
6363
`sqlcompare query "SELECT 1" -c <name>`
6464
2. List available diffs to confirm local metadata access.
65-
`sqlcompare list-diffs`
65+
`sqlcompare history`
6666

6767
## Common Fixes
6868
1. Use `-c <name>` explicitly to avoid accidental default selection.

skills/sqlcompare-core-run/SKILL.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
name: sqlcompare-core-run
3-
description: Run SQLCompare core comparisons across tables, SQL queries, and files to produce a diff_id. Use when the user wants to compare two datasets, choose index keys, set connectors, or run dataset configs via sqlcompare run/table/dataset/stats/query.
3+
description: Run SQLCompare core comparisons across tables, SQL queries, and files to produce a diff_id. Use when the user wants to compare two datasets, choose index keys, set connectors, or run dataset configs via sqlcompare run subcommands or sqlcompare query.
44
---
55

66
# SQLCompare Core Run
@@ -17,23 +17,23 @@ Run the primary SQLCompare commands that create comparison artifacts and return
1717
Use these patterns to pick the right entry point.
1818

1919
1. Compare two tables or views.
20-
`sqlcompare run analytics.fact_sales analytics.fact_sales_new id`
20+
`sqlcompare run table analytics.fact_sales analytics.fact_sales_new id`
2121
2. Compare two SQL queries (inline or .sql files).
22-
`sqlcompare run "SELECT ..." "SELECT ..." id -c snowflake_prod`
23-
`sqlcompare run queries/previous.sql queries/current.sql id -c snowflake_prod`
22+
`sqlcompare run query --previous "SELECT ..." --current "SELECT ..." --index id -c snowflake_prod`
23+
`sqlcompare run query --previous queries/previous.sql --current queries/current.sql --index id -c snowflake_prod`
2424
3. Compare local files (CSV/XLSX) with DuckDB.
25-
`sqlcompare run path/to/previous.csv path/to/current.xlsx id`
25+
`sqlcompare run file path/to/previous.csv path/to/current.xlsx id`
2626
4. Use a dataset config for repeatable runs.
27-
`sqlcompare dataset path/to/dataset.yaml`
27+
`sqlcompare run dataset path/to/dataset.yaml`
2828
5. Run a fast statistical comparison (no diff_id needed for row-level drilldown).
29-
`sqlcompare stats analytics.users analytics.users_new -c snowflake_prod`
29+
`sqlcompare run stats analytics.users analytics.users_new -c snowflake_prod`
3030
6. Run a quick sanity query against a connector.
3131
`sqlcompare query "SELECT COUNT(*) FROM analytics.users" -c snowflake_prod`
3232

3333
## Inputs And Keys
3434
1. Provide index columns that exist in both datasets.
3535
2. Use comma-separated keys for composite indexes.
36-
`sqlcompare run analytics.users analytics.users_new user_id,tenant_id`
36+
`sqlcompare run table analytics.users analytics.users_new user_id,tenant_id`
3737
3. Fully qualify or quote identifiers when required by your database.
3838

3939
## Connections And Defaults
Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,48 +1,48 @@
11
---
22
name: sqlcompare-output-analysis
3-
description: Analyze SQLCompare outputs after a core run. Use when the user has a diff_id and needs to inspect stats, missing rows, column-level diffs, exports, or AI-friendly diff-queries.
3+
description: Analyze SQLCompare outputs after a core run. Use when the user has a diff_id and needs to review stats, missing rows, column-level diffs, exports, or AI-friendly meta output.
44
---
55

66
# SQLCompare Output Analysis
77

88
## Overview
9-
Interpret and drill into a SQLCompare diff_id using inspect, list-diffs, and diff-queries. Produce summaries, targeted column reviews, and exportable reports.
9+
Interpret and drill into a SQLCompare diff_id using review, history, and review meta. Produce summaries, targeted column reviews, and exportable reports.
1010

1111
## Start From A diff_id
1212
1. If you already have a diff_id, proceed to inspection commands.
1313
2. If you do not, list recent diffs and pick the correct one.
14-
`sqlcompare list-diffs`
15-
`sqlcompare list-diffs users`
14+
`sqlcompare history`
15+
`sqlcompare history users`
1616

1717
## Core Inspection Commands
1818
1. Overall statistics summary.
19-
`sqlcompare inspect <diff_id> --stats`
20-
2. Inspect a specific column with row samples.
21-
`sqlcompare inspect <diff_id> --column revenue --limit 100`
19+
`sqlcompare review stats <diff_id>`
20+
2. Review a specific column with row samples.
21+
`sqlcompare review diff <diff_id> --column revenue --limit 100`
2222
3. Show rows missing on either side.
23-
`sqlcompare inspect <diff_id> --missing-current`
24-
`sqlcompare inspect <diff_id> --missing-previous`
23+
`sqlcompare review missing <diff_id> --side current`
24+
`sqlcompare review missing <diff_id> --side previous`
2525
4. List available columns.
26-
`sqlcompare inspect <diff_id> --list-columns`
26+
`sqlcompare review columns <diff_id>`
2727

2828
## Export Reports (XLSX)
2929
1. Summary report with capped row samples per column.
30-
`sqlcompare inspect <diff_id> --save summary`
30+
`sqlcompare review export <diff_id> --mode summary`
3131
2. Full report without per-column row caps.
32-
`sqlcompare inspect <diff_id> --save complete --file-path ./reports/full_diff.xlsx`
32+
`sqlcompare review export <diff_id> --mode complete --output ./reports/full_diff.xlsx`
3333
3. Single-column summary report.
34-
`sqlcompare inspect <diff_id> --column revenue --save summary --file-path ./reports/revenue_diff.xlsx`
34+
`sqlcompare review export <diff_id> --column revenue --mode summary --output ./reports/revenue_diff.xlsx`
3535

3636
Notes:
37-
1. `--save summary|complete` is for the standard diff view.
38-
2. Do not combine `--save summary|complete` with `--stats`, `--missing-current`, `--missing-previous`, or `--list-columns`.
37+
1. `review export --mode summary|complete` is for the standard diff view.
38+
2. Do not combine `review export` with `review stats`, `review missing`, or `review columns`.
3939

4040
## AI-Friendly Diff Queries
41-
Use diff-queries to get structured metadata and SQL templates.
42-
`sqlcompare diff-queries <diff_id>`
41+
Use review meta to get structured metadata and SQL templates.
42+
`sqlcompare review meta <diff_id>`
4343

4444
## Interpretation Hints
4545
1. Start with `--stats` to see which columns change most.
4646
2. Use `--missing-current` and `--missing-previous` to find dropped or new rows.
47-
3. Use `--column` to inspect high-impact fields.
47+
3. Use `--column` to review high-impact fields.
4848
4. Export a summary report for sharing, then refine analysis with targeted column checks.

sqlcompare/cli.py

Lines changed: 25 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -2,27 +2,34 @@
22

33
import typer
44

5-
from sqlcompare.dataset import dataset_cmd
6-
from sqlcompare.diff_queries import diff_queries_cmd
7-
from sqlcompare.run_cmd import run_cmd
8-
from sqlcompare.inspect import inspect_cmd
9-
from sqlcompare.list_diffs import list_diffs_cmd
105
from sqlcompare.query import query_cmd
11-
from sqlcompare.stats import stats_cmd
12-
from sqlcompare.table import table_cmd
13-
14-
app = typer.Typer(help="Compare database tables and inspect diffs.")
15-
16-
# Register all commands
17-
app.command("table")(table_cmd)
18-
app.command("run")(run_cmd)
19-
app.command("inspect")(inspect_cmd)
20-
app.command("diff-queries")(diff_queries_cmd)
21-
app.command("stats")(stats_cmd)
22-
app.command("list-diffs")(list_diffs_cmd)
6+
from sqlcompare.history import history_cmd
7+
from sqlcompare.review_cmds import (
8+
review_columns_cmd,
9+
review_diff_cmd,
10+
review_export_cmd,
11+
review_meta_cmd,
12+
review_missing_cmd,
13+
review_stats_cmd,
14+
)
15+
from sqlcompare.run import run_unified_cmd
16+
17+
app = typer.Typer(help="Compare datasets and review diffs.")
18+
19+
review_app = typer.Typer(help="Review diff results and export reports.")
20+
21+
app.command("run")(run_unified_cmd)
22+
app.add_typer(review_app, name="review")
23+
app.command("history")(history_cmd)
24+
2325
app.command("query")(query_cmd)
24-
app.command("dataset")(dataset_cmd)
2526

27+
review_app.command("diff")(review_diff_cmd)
28+
review_app.command("stats")(review_stats_cmd)
29+
review_app.command("missing")(review_missing_cmd)
30+
review_app.command("columns")(review_columns_cmd)
31+
review_app.command("export")(review_export_cmd)
32+
review_app.command("meta")(review_meta_cmd)
2633

2734
def main() -> None:
2835
app()

sqlcompare/compare/comparator.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -455,6 +455,6 @@ def compare(
455455
save_test_runs(runs)
456456

457457
log.debug(f"\U0001f4c1 Analysis data saved with ID: {diff_id}")
458-
log.debug(f"Use 'sqlcompare inspect {diff_id}' to review differences")
458+
log.debug(f"Use 'sqlcompare review {diff_id}' to review differences")
459459

460460
return diff_id
Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
from sqlcompare.log import log
1111

1212

13-
def list_diffs(pattern: str | None, test: str | None) -> None:
13+
def list_history(pattern: str | None, test: str | None) -> None:
1414
"""List available diff data files and runs."""
1515
tests_folder = get_tests_folder()
1616
search_pattern = f"{tests_folder}/*/diffs/*.pkl"
@@ -59,7 +59,7 @@ def list_diffs(pattern: str | None, test: str | None) -> None:
5959
)
6060

6161

62-
def list_diffs_cmd(
62+
def history_cmd(
6363
pattern: str | None = typer.Argument(None, help="Match diff IDs"),
6464
test: str | None = typer.Option(None, "--test", help="Filter by test name"),
6565
) -> None:
@@ -69,15 +69,15 @@ def list_diffs_cmd(
6969
7070
Examples:
7171
# List all diffs
72-
sqlcompare list-diffs
72+
sqlcompare history
7373
7474
# Filter by pattern
75-
sqlcompare list-diffs users
75+
sqlcompare history users
7676
7777
# Filter by test name
78-
sqlcompare list-diffs --test migration_check
78+
sqlcompare history --test migration_check
7979
8080
Output:
8181
Displays: diff_id, test name, file size, creation date
8282
"""
83-
list_diffs(pattern, test)
83+
list_history(pattern, test)

0 commit comments

Comments
 (0)