Skip to content

Simplify policy import model: replace entriesAdditions and importsAliases with entry-level references#2424

Open
thjaeckle wants to merge 8 commits intoeclipse-ditto:masterfrom
beyonnex-io:feature/simplify-policy-import-references
Open

Simplify policy import model: replace entriesAdditions and importsAliases with entry-level references#2424
thjaeckle wants to merge 8 commits intoeclipse-ditto:masterfrom
beyonnex-io:feature/simplify-policy-import-references

Conversation

@thjaeckle
Copy link
Copy Markdown
Member

@thjaeckle thjaeckle commented Apr 22, 2026

Summary

Resolves #2423

Replace entriesAdditions and importsAliases concepts with a single references array on policy entries. Each reference is either:

  • Import reference ({"import": "policyId", "entry": "label"}) — inherits resources and namespaces from the imported entry
  • Local reference ({"entry": "label"}) — inherits subjects, resources, and namespaces from a local entry

What's new

  • EntryReference model with import and local reference support
  • REST API: GET/PUT/DELETE /entries/{label}/references
  • Commands, responses, events for references with full Ditto protocol adapter coverage
  • Referential integrity validation:
    • Local references must target existing entries (400)
    • Import references must target declared imports (400)
    • Cannot delete an import still referenced by entries (409)
    • Cannot delete an entry locally referenced by another entry (409)
    • Validated on CreatePolicy, ModifyPolicy, ModifyPolicyEntries, ModifyPolicyEntry, ModifyPolicyEntryReferences

What's removed

  • entriesAdditions on imports (model, signals, strategies, routes, tests)
  • importsAliases on policy (model, signals, strategies, routes, tests)
  • ~17,700 lines of code deleted

What's preserved (backward compat)

  • entries filter on import
  • importable (implicit/never)
  • transitiveImports
  • allowedImportAdditions

…ases with entry-level references

Resolves eclipse-ditto#2423

Replace the `entriesAdditions`, `importsAliases`, `importReference`, and `alias`
concepts with a single `references` array on policy entries. Each reference is either
an import reference (`{"import": "policyId", "entry": "label"}`) inheriting resources
and namespaces from the imported entry, or a local reference (`{"entry": "label"}`)
inheriting subjects, resources, and namespaces from a local entry.

Key changes:
- Add EntryReference model with import and local reference support
- Add references CRUD: GET/PUT/DELETE /entries/{label}/references
- Add new signals (commands, responses, events) for references
- Wire signals into Ditto protocol adapter with full mapping coverage
- Add referential integrity validation:
  - Local references must target existing entries
  - Import references must target declared imports
  - Cannot delete an import still referenced by entries (409)
  - Cannot delete an entry locally referenced by another (409)
  - Validated on CreatePolicy, ModifyPolicy, ModifyPolicyEntries,
    ModifyPolicyEntry, and ModifyPolicyEntryReferences
- Fix all 6-arg newPolicyEntry calls to use 7-arg overload preserving references
  (ImmutablePolicy, event strategies, placeholder substitution, command strategies)
- Remove entriesAdditions model, signals, strategies, routes, and tests
- Remove importsAliases model, signals, strategies, routes, and tests
- Update OpenAPI spec, JSON schema, documentation, and protocol examples
- Keep entries filter on import and importable types for backward compatibility

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@thjaeckle thjaeckle requested a review from hu-ahmed April 22, 2026 09:51
thjaeckle and others added 2 commits April 22, 2026 12:21
An entry must not reference itself via a local reference. This is
validated at write time in both ModifyPolicyEntryReferencesStrategy
and the shared validateReferencesIntegrity check (covering CreatePolicy,
ModifyPolicy, ModifyPolicyEntries, ModifyPolicyEntry).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The newEffectedImportedLabels call used the old 3-arg signature with
entriesAdditions as the middle parameter. Updated to the 2-arg
signature (labels, transitiveImports).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

@hu-ahmed hu-ahmed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @thjaeckle — the 4 design answers all verified against code and runtime:

  • A1 (one-level resolution, no cycles) — confirmed at PolicyImporter.java:307-324
  • A2 (409 on delete of referenced entry) — verified live: "policies:entry.referenceconflict" with clear message naming both entries
  • A3 (self-reference rejection at write time) — verified on PUT /references, PUT /entries/{label}, and PUT /policies/{id}
  • A4 (allowedImportAdditions only for import refs) — confirmed at PolicyImporter.java:366-368

Four follow-up items from live testing against the branch:

1. Local-ref-only policies silently lose authorization

This is the one I'd most like to address before merge. Reproduced live:

Same policy structure; only difference is top-level imports{} being empty vs populated.
  imports: {}                      → Alice GET /features/temp → HTTP 404                                                                                                                                                        
  imports: {ns:template-empty: {}} → Alice GET /features/temp → HTTP 200

Root cause: Policy.java:470 gates resolveReferences behind !imports.isEmpty(). A policy that uses only local references — exactly the shape in the power-plant example in this issue's body — never triggers reference resolution. Suggest calling resolveReferences whenever any entry has non-empty references, regardless of whether imports is empty.

2. Status code / exception type inconsistency

The issue body describes rule 1 as "400 Bad Request." Actual behavior:

Scenario Status Error code
PUT /entries/{label}/references with undeclared import 400 policies:import.invalid
CreatePolicy / ModifyPolicy / ModifyPolicyEntry with invalid reference 403 policies:entry.modificationinvalid
Self-reference (any path) 403 policies:entry.modificationinvalid
Ref to non-existent local entry 403 policies:entry.modificationinvalid

PolicyEntryModificationInvalidException is hardcoded HttpStatus.FORBIDDEN at line 57 with a DEFAULT_DESCRIPTION about WRITE-on-policy:/ — it's the "no-writer-left" exception. Reusing it for reference validation produces a 403 where 400 is more appropriate, and a cross-command inconsistency (400 via one endpoint, 403 via others). Suggest PolicyEntryInvalidException (400) or a reference-specific exception.

3. Silent drop of non-object elements in references arrays

PUT /entries/operator/references                          
Body:     [{"entry":"shared"}, 42, "bogus"]
Response: 204                                                                                                                                                                                                                   
Stored:   [{"entry":"shared"}]

Six call sites using .filter(JsonValue::isObject)ImmutablePolicyEntry.readReferences, ModifyPolicyEntryReferences.fromJson/setEntity, PolicyEntryReferencesModified.fromJson/setEntity,
RetrievePolicyEntryReferencesResponse, AbstractPolicyMappingStrategies.entryReferencesFrom. Same antipattern class flagged before — worth an explicit validating loop that throws PolicyEntryInvalidException naming the bad element.

4. Cross-command READ authorization for references

PolicyImportsPreEnforcer.apply (lines 94-108) routes only ModifyPolicyEntryReferences to validateReferencesModification. For CreatePolicy, ModifyPolicy, ModifyPolicyEntries, and ModifyPolicyEntry — only top-level imports are READ-checked, per-entry reference targets are not. A caller can add an import reference to a template entry they don't have READ on by going through ModifyPolicyEntry rather than /references. Impact scope: information disclosure at resolution time rather than privilege escalation (import refs don't merge subjects), but a cross-command discipline gap worth closing.

Smaller notes:

  • PolicyImporter.rewriteLabel drops references from imported entries (6-param factory) while the merge path preserves them (7-param). Either reject references on non-NEVER entries at write time, or extract a
    cloneEntry helper so no path reconstructs a PolicyEntry with fewer than 7 fields.
  • Duplicate references ([{entry:"x"},{entry:"x"}]) are accepted as-is; resolution merges twice.
  • No ETag support on /references sub-resource.
  • ImmutablePolicyBuilder.setPolicyEntry only puts entryReferences when non-empty — re-setting with an empty list doesn't clear prior value (latent).

Fixes 4 issues and several smaller notes raised in PR review:

1. Fix local-ref-only policies not resolving references: call
   resolveReferences even when imports are empty if any entry has
   non-empty references (Policy.withResolvedImports)

2. Fix HTTP status 403→400 for reference validation errors: add
   policyEntryReferenceInvalid() helper using PolicyEntryInvalidException
   (400 Bad Request) instead of PolicyEntryModificationInvalidException
   (403 Forbidden) for reference validation

3. Reject non-object elements in references arrays: add shared
   PoliciesModelFactory.parseEntryReferences(JsonArray) that validates
   all elements are objects, replacing silent .filter(JsonValue::isObject)
   drops in 8 call sites

4. Close cross-command READ authorization gap: extend
   PolicyImportsPreEnforcer to validate READ access on import reference
   targets for CreatePolicy, ModifyPolicy, ModifyPolicyEntry, and
   ModifyPolicyEntries (previously only ModifyPolicyEntryReferences)

Smaller notes:
- Reject duplicate references (same import+entry pair) with 400
- Add ETag support on /references sub-resource
- Fix builder empty list handling: setPolicyEntry always stores
  references (even empty) to clear previous values

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@thjaeckle thjaeckle force-pushed the feature/simplify-policy-import-references branch from 9986928 to 8f4f507 Compare April 22, 2026 13:13
Java 8 compatibility fixes (policy model is public API):
- Replace orElseThrow() with orElseThrow(NoSuchElementException::new)
- Replace Collectors.toUnmodifiableList() with collectingAndThen
- Replace List.toArray(IntFunction) with List.toArray(new T[0])
- Replace Optional.isEmpty() with !Optional.isPresent()

Other fixes:
- Remove accidentally committed .claude review file
- Support 201 Created response for PUT /references on first creation
  (previously always returned 204; now returns 201 with body when
  setting references on an entry that had none)
- Updated OpenAPI spec to document 201 response

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@thjaeckle thjaeckle force-pushed the feature/simplify-policy-import-references branch 2 times, most recently from 7bac7be to 07d4562 Compare April 22, 2026 13:40
With entry-level references, an entry can inherit subjects and resources
from referenced entries, making explicit declaration unnecessary. Both
fields now default to empty when absent from JSON.

This enables patterns like:
- Entry with only references (inherits everything)
- Entry with only subjects (shared-subjects for local references)

Updated ImmutablePolicyEntry.fromJson, OpenAPI schema, and JSON schema.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@thjaeckle thjaeckle force-pushed the feature/simplify-policy-import-references branch from 07d4562 to 02aa727 Compare April 22, 2026 13:46
Three interconnected fixes for multi-level import reference resolution:

1. Resolve imported policy's references before importing: when importing
   a policy whose entries use references (e.g. intermediate's driver
   references template's driver for resources), resolveReferences is
   called BEFORE entries are imported and references stripped. This
   materializes the inherited values.

2. Apply import prefix to transitive entries: transitive entries now
   get their import prefix so they don't collide with the loaded
   policy's own entries (e.g. both having "driver"). This allows
   resolveReferences to find transitive entries by their prefixed labels.

3. Merge subjects for import references: import references now merge
   subjects additively (same as local references). Without this, the
   leaf in a 3-level hierarchy could never inherit subjects from the
   intermediate level.

New tests:
- testThreeLevelReferenceResolution: full 3-level hierarchy (template
  → intermediate → leaf) verifying resources and subjects propagate
- testLocalRefOnlyPolicyResolvesViaWithResolvedImports: verifies the
  fix from the previous commit (no-imports path)
- testImportedPolicyReferencesAreResolvedBeforeImport: 2-level scenario
  verifying resources inherited via references survive import

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@thjaeckle thjaeckle force-pushed the feature/simplify-policy-import-references branch from 78fc0c7 to ce0ecd0 Compare April 22, 2026 14:17
When an entry has import references, the pre-enforcer now checks
the referenced template entry's allowedImportAdditions against the
referencing entry's own subjects and resources:

- If the referencing entry has subjects but the template doesn't
  allow subject additions, the request is rejected (400)
- If the referencing entry has resources but the template doesn't
  allow resource additions, the request is rejected (400)

This validation applies to all commands that carry entries with
import references: PUT /references, CreatePolicy, ModifyPolicy,
ModifyPolicyEntry, ModifyPolicyEntries.

Fixes two bugs found during integration testing:
1. PUT /references accepted references to templates without
   allowedImportAdditions permitting the entry's content
2. Entries with own resources were not constrained by the
   referenced template's allowedImportAdditions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Simplify policy import model: replace entriesAdditions and importsAliases with entry-level references

2 participants