Skip to content

Commit 5d5bd83

Browse files
authored
Update Sarif reporter to provide region info based on validator findings (#442)
* feat: add SARIF region with startLine/startColumn for PR annotations Add ValidationError type with optional Line/Column fields to enable GitHub Actions inline annotations on affected lines. - 10 validators now return structured position info (JSON, YAML, TOML, XML, HCL, CSV, ENV, HOCON, TOON, PList) - 4 validators without position info (INI, EditorConfig, Properties, SARIF) fall back to file-level annotations - SARIF reporter emits region only when line info is available - Report struct carries StartLine/StartColumn from ValidationError - Remove dead getCustomErr function (logic moved into JSON ValidateSyntax) - Add GitHub Action section to README.md and index.md * docs: update CHANGELOG * fix: improve XSD validation error messages with detailed diagnostics Use helium ErrorCollector to capture individual validation errors instead of the generic 'xsd: validation failed' message. Errors now include the line number and specific schema violation details. * feat: separate multiple validation errors across all reporters - Standard: each error on its own indented line - JSON: errors as an array instead of a single joined string - SARIF: each error as its own result entry for per-error annotations - JUnit: each error on its own line within the failure message Add SchemaErrors type to carry individual error messages from JSONSchemaValidate, ValidateXSD, and SARIF schema validation. The CLI populates ValidationErrors slice on Report for reporters to consume. * fix: clean up XSD validation error format Reformat helium XSD errors from (string):5: Schemas validity error : Element 'port': ... to line 5: Element 'port': ... Consistent with the error format used by other validators. * feat: add syntax/schema error prefixes and error-type groupby - Prefix each validation error with 'syntax:' or 'schema:' to distinguish error types across all reporters - Add ErrorType field to Report struct - Add 'error-type' as a new groupby option that groups results into syntax, schema, and Passed categories - Update flag help text, README, and index * docs: update CHANGELOG for 2.1.0
1 parent 6bf3dc5 commit 5d5bd83

37 files changed

+454
-151
lines changed

CHANGELOG.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
## [2.1.0] - 2026-04-09
11+
12+
### Added
13+
14+
- SARIF reporter now includes `region` with `startLine`/`startColumn` for inline PR annotations in GitHub Actions
15+
- `ValidationError` type with optional `Line`/`Column` fields for structured error positions
16+
- Multiple validation errors are now separated across all reporters: each error on its own line (standard), array of errors (JSON), individual result entries (SARIF), newline-separated in failure message (JUnit)
17+
- `SchemaErrors` type to carry individual schema validation error messages
18+
- Validation errors are prefixed with `syntax:` or `schema:` to distinguish error types
19+
- `error-type` groupby option to group output by syntax errors, schema errors, and passed files
20+
- GitHub Action section in README and index referencing `Boeing/validate-configs-action@v2.0.0`
21+
22+
### Fixed
23+
24+
- XSD validation errors now show detailed diagnostics instead of generic "xsd: validation failed"
25+
- XSD error format cleaned up from `(string):5: Schemas validity error : ...` to `line 5: ...`
26+
27+
## [2.0.0] - 2026-04-08
28+
1029
### Added
1130

1231
- SARIF syntax and schema validation using the go-sarif library

PKGBUILD

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
# Maintainer: Clayton Kehoe <clayton.j.kehoe at boeing dot com>
22
# Contributor : wiz64 <wiz64 dot com>
33
pkgname=config-file-validator
4-
pkgver=1.11.0
4+
pkgver=2.1.0
55
pkgrel=1
6-
pkgdesc="A tool to validate the syntax of configuration files"
6+
pkgdesc="A tool to validate the syntax and schema of configuration files"
77
arch=('x86_64')
88
url="https://github.com/Boeing/config-file-validator"
99
license=('Apache 2.0')
1010
depends=('glibc')
11-
makedepends=('go>=1.25' 'git' 'sed')
11+
makedepends=('go>=1.26.2' 'git' 'sed')
1212
source=("git+https://github.com/Boeing/config-file-validator.git")
1313
sha256sums=('SKIP')
1414
md5sums=('SKIP')

README.md

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,16 @@ If you have a go environment on your desktop you can use [go install](https://go
124124
go install github.com/Boeing/config-file-validator/cmd/validator@latest
125125
```
126126

127+
## GitHub Action
128+
129+
A GitHub Action is available to run the config-file-validator as part of your CI/CD pipeline. It posts validation results as PR comments with inline annotations on the affected files and lines.
130+
131+
```yaml
132+
- uses: Boeing/validate-configs-action@v2.0.0
133+
```
134+
135+
See the [validate-configs-action](https://github.com/Boeing/validate-configs-action) repository for full usage and configuration options.
136+
127137
## Usage
128138
129139
```
@@ -152,7 +162,7 @@ optional flags:
152162
-globbing
153163
If globbing flag is set, check for glob patterns in the arguments.
154164
-groupby string
155-
Group output by filetype, directory, pass-fail. Supported for Standard and JSON reports
165+
Group output by filetype, directory, pass-fail, error-type. Supported for Standard and JSON reports
156166
-no-schema
157167
Disable all schema validation. Only syntax is checked.
158168
Cannot be used with --require-schema, --schema-map, or --schemastore.
@@ -279,7 +289,7 @@ validator --reporter=json:output.json --reporter=standard /path/to/search
279289

280290
### Group report output
281291

282-
Group the report output by file type, directory, or pass-fail. Supports one or more groupings.
292+
Group the report output by file type, directory, pass-fail, or error-type. Supports one or more groupings.
283293

284294
```shell
285295
validator -groupby filetype

cmd/validator/testdata/groupby.txtar

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,15 @@ stdout 'Summary'
1111
exec validator -groupby=pass-fail files
1212
stdout 'Passed'
1313

14+
# Group by error-type (all valid files → Passed group)
15+
exec validator -groupby=error-type files
16+
stdout 'Passed'
17+
18+
# Group by error-type with failures
19+
! exec validator -groupby=error-type mixed
20+
stdout 'syntax'
21+
stdout 'Passed'
22+
1423
# Double grouping
1524
exec validator -groupby=filetype,directory files
1625
stdout 'json'
@@ -39,3 +48,7 @@ stdout 'totalPassed'
3948
{"key": "value"}
4049
-- files/good.yaml --
4150
key: value
51+
-- mixed/good.json --
52+
{"key": "value"}
53+
-- mixed/bad.json --
54+
{"key": value}

cmd/validator/testdata/json_schema.txtar

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ stdout '✓'
1111
# Invalid package.json (version must be string)
1212
! exec validator invalid_package.json
1313
stdout '×'
14-
stdout 'schema validation failed'
14+
stdout 'error:'
1515

1616
# --- tsconfig.json schema ---
1717

@@ -22,7 +22,7 @@ stdout '✓'
2222
# Invalid tsconfig.json (target must be string)
2323
! exec validator invalid_tsconfig.json
2424
stdout '×'
25-
stdout 'schema validation failed'
25+
stdout 'error:'
2626

2727
# --- eslintrc schema ---
2828

@@ -39,7 +39,7 @@ stdout '✓'
3939
# Invalid workflow (missing required jobs)
4040
! exec validator invalid_workflow.json
4141
stdout '×'
42-
stdout 'schema validation failed'
42+
stdout 'error:'
4343

4444
# --- Strict config schema ---
4545

@@ -50,27 +50,27 @@ stdout '✓'
5050
# Missing required field
5151
! exec validator missing_required.json
5252
stdout '×'
53-
stdout 'schema validation failed'
53+
stdout 'error:'
5454

5555
# Wrong type
5656
! exec validator wrong_type.json
5757
stdout '×'
58-
stdout 'schema validation failed'
58+
stdout 'error:'
5959

6060
# Extra property not allowed
6161
! exec validator extra_property.json
6262
stdout '×'
63-
stdout 'schema validation failed'
63+
stdout 'error:'
6464

6565
# Value out of range
6666
! exec validator out_of_range.json
6767
stdout '×'
68-
stdout 'schema validation failed'
68+
stdout 'error:'
6969

7070
# Enum violation
7171
! exec validator bad_enum.json
7272
stdout '×'
73-
stdout 'schema validation failed'
73+
stdout 'error:'
7474

7575
# --- Array schema ---
7676

@@ -81,12 +81,12 @@ stdout '✓'
8181
# Too few items
8282
! exec validator too_few_items.json
8383
stdout '×'
84-
stdout 'schema validation failed'
84+
stdout 'error:'
8585

8686
# Wrong item type
8787
! exec validator wrong_item_type.json
8888
stdout '×'
89-
stdout 'schema validation failed'
89+
stdout 'error:'
9090

9191
# --- Mixed: files with and without $schema ---
9292

cmd/validator/testdata/no_schema.txtar

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ stdout '✓'
66
# Without --no-schema it fails
77
! exec validator bad_with_schema.json
88
stdout '×'
9-
stdout 'schema validation failed'
9+
stdout 'error:'
1010

1111
# --no-schema with YAML schema comment — skips validation
1212
exec validator --no-schema bad_with_schema.yaml

cmd/validator/testdata/schema_map.txtar

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ stdout '✓'
55
# --schema-map with invalid data fails
66
! exec validator --schema-map=bad.json:schema.json bad.json
77
stdout '×'
8-
stdout 'schema validation failed'
8+
stdout 'error:'
99

1010
# --schema-map with glob pattern
1111
exec validator --schema-map=**/*.json:schema.json subdir/config.json

cmd/validator/testdata/schemastore.txtar

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ stdout '✓'
1414
# Invalid package.json (version must be string)
1515
! exec validator --schemastore schemastore invalid/package.json
1616
stdout '×'
17-
stdout 'schema validation failed'
17+
stdout 'error:'
1818

1919
# Valid nodemon.json
2020
exec validator --schemastore schemastore valid/nodemon.json
@@ -23,7 +23,7 @@ stdout '✓'
2323
# Invalid nodemon.json (unknown field, additionalProperties: false)
2424
! exec validator --schemastore schemastore invalid/nodemon.json
2525
stdout '×'
26-
stdout 'schema validation failed'
26+
stdout 'error:'
2727

2828
# ============================================================
2929
# PART 2: --schemastore automatic matching (YAML)
@@ -36,7 +36,7 @@ stdout '✓'
3636
# Invalid GitHub workflow (missing required 'jobs')
3737
! exec validator --schemastore schemastore invalid/.github/workflows/ci.yml
3838
stdout '×'
39-
stdout 'schema validation failed'
39+
stdout 'error:'
4040

4141
# Valid artifacthub-repo.yml
4242
exec validator --schemastore schemastore valid/artifacthub-repo.yml
@@ -45,7 +45,7 @@ stdout '✓'
4545
# Invalid artifacthub-repo.yml (additionalProperties: false)
4646
! exec validator --schemastore schemastore invalid/artifacthub-repo.yml
4747
stdout '×'
48-
stdout 'schema validation failed'
48+
stdout 'error:'
4949

5050
# ============================================================
5151
# PART 3: --schemastore automatic matching (TOML)
@@ -58,7 +58,7 @@ stdout '✓'
5858
# Invalid stylua.toml (column_width must be integer)
5959
! exec validator --schemastore schemastore invalid/stylua.toml
6060
stdout '×'
61-
stdout 'schema validation failed'
61+
stdout 'error:'
6262

6363
# ============================================================
6464
# PART 4: --schemastore automatic matching (TOON)
@@ -71,7 +71,7 @@ stdout '✓'
7171
# Invalid app.toon (port out of range)
7272
! exec validator --schemastore schemastore invalid/app.toon
7373
stdout '×'
74-
stdout 'schema validation failed'
74+
stdout 'error:'
7575

7676
# ============================================================
7777
# PART 5: Unmatched files pass syntax-only
@@ -115,7 +115,7 @@ stdout '✓'
115115
# schema-map points to strict schema that rejects the file
116116
! exec validator --schema-map=package.json:strict_schema.json --schemastore schemastore valid/package.json
117117
stdout '×'
118-
stdout 'schema validation failed'
118+
stdout 'error:'
119119

120120
# ============================================================
121121
# PART 9: --schema-map with YAML
@@ -126,7 +126,7 @@ stdout '✓'
126126

127127
! exec validator --schema-map=config.yaml:server_schema.json invalid/config.yaml
128128
stdout '×'
129-
stdout 'schema validation failed'
129+
stdout 'error:'
130130

131131
# ============================================================
132132
# PART 10: --schema-map with TOML
@@ -137,7 +137,7 @@ stdout '✓'
137137

138138
! exec validator --schema-map=config.toml:server_schema.json invalid/config.toml
139139
stdout '×'
140-
stdout 'schema validation failed'
140+
stdout 'error:'
141141

142142
# ============================================================
143143
# PART 11: --schema-map with TOON
@@ -148,7 +148,7 @@ stdout '✓'
148148

149149
! exec validator --schema-map=config.toon:server_schema.json invalid/config.toon
150150
stdout '×'
151-
stdout 'schema validation failed'
151+
stdout 'error:'
152152

153153
# ============================================================
154154
# PART 12: --schema-map with glob patterns
@@ -159,7 +159,7 @@ stdout '✓'
159159

160160
! exec validator --schema-map=**/configs/*.json:server_schema.json invalid/configs/db.json
161161
stdout '×'
162-
stdout 'schema validation failed'
162+
stdout 'error:'
163163

164164
# ============================================================
165165
# PART 13: Multiple files, mixed results

cmd/validator/testdata/toml_schema.txtar

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ stdout '✓'
55
# TOML with invalid data fails schema validation
66
! exec validator invalid.toml
77
stdout '×'
8-
stdout 'schema validation failed'
8+
stdout 'error:'
99

1010
# TOML without $schema passes (syntax only)
1111
exec validator plain.toml

cmd/validator/testdata/toon_schema.txtar

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ stdout '✓'
55
# TOON with invalid data fails schema validation
66
! exec validator invalid.toon
77
stdout '×'
8-
stdout 'schema validation failed'
8+
stdout 'error:'
99

1010
# TOON without $schema passes (syntax only)
1111
exec validator plain.toon

0 commit comments

Comments
 (0)