Hi @isc-tleavitt
I've drafted the YAML schema for the zpm test LLM-friendly output. As we know this structure aims to be clean and minimal while ensuring we capture all the failures, line number context, and overall status indicators as requested.

I would greatly appreciate your input on whether this structure aligns with the output needed for LLM consumption and integration tools.

Proposed YAML Structure:

unitTest:
  # Overall Summary Data
  id: 37
  namespace: "USER"
  overallStatus: "failed"       # Explicitly states the result (crucial for LLMs/Scripts)
  duration: 53.064875
  testDateTime: "2025-11-26 05:50:28"
  
  # Detailed Failure Context (only includes failed tests by default)
  results:
    - suiteName: "ResourceProcessor"
      testcases:
        - testcaseName: "Test.PM.Unit.ResourceProcessor.AttributeMetadata"
          methods:
            - methodName: "TestAnswer"
              status: "failed"  # Explicit status at the granular method level
              # testMethod (Full unique ID) can be included if required for traceability
              assertAction: "AssertStatusOK"
              assertCounter: 1
              assertDescription: >  # Uses YAML block scalar for clean multiline error text
                An exception occurred. => ERROR #5540: SQLCODE: -150 Message:
                Optimistic concurrency locking for class definition '%IPM.Utils.Module'
                failed. Unable to acquire Class-Changed_Timestamp...
              assertLocation: "UtilGetRow+16^Test.PM.Unit.ResourceProcessor.AttributeMetadata.cls"
              
            # Add more methods here if multiple methods in the TestCase failed

Key Design Decisions for Review:

Explicitness: We added overallStatus at the top level and status at the method level.
Context: The assertDescription and assertLocation fields directly satisfy the requirement for "failures with their associated line numbers" and context.
Hierarchy: The structure (suiteName > testcases > methods) accurately mirrors the IRIS test environment, which should simplify debugging.

Please share your thoughts

Thank you!

LLM-friendly output mode for zpm "test" #971

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions