Background
Multi-modal agents built on Vertex AI (and other platforms) commonly exhibit a number of reliability and reproducibility issues, such as:
- Identity drift across generations (same subject → different face)
- Body-shape or pose inconsistencies (especially for solid/stocky builds)
- Scene or lighting changes that violate user constraints
- Execution paths that vary between runs for the same input
- Lack of traceable execution metadata or replay mode for debugging
Proposal
Introduce an optional “deterministic / constraint-driven execution mode” for the Vertex AI agents framework.
The key features of this mode would include:
- A config-driven constraint layer (YAML/JSON) defining anchors such as facial_identity, body_ratio, pose_structure, fabric_tension_model, scene_context, lighting_signature.
- Deterministic routing / execution path design: same input → same prompts → same pipeline → same output (where possible).
- Audit-friendly metadata output: trace IDs, execution logs, geometry_score, semantic_drift_score, aesthetic_score.
- A minimal stub or example demonstrating the mode (e.g., a “stocky body type portrait reconstruction” experiment) to illustrate feasibility.
Benefits
- Improves reliability and consistency in production multi-modal agent workflows
- Enables enterprise use cases (fashion, virtual avatars, identity-preserving rendering) where reproducibility matters
- Provides better traceability and debugging capability for agent execution
- Fills a known gap in multi-modal agent architecture currently noted by users
Example Implementation
A proof-of-concept repository exists demonstrating such a mode:
https://github.com/yuer-dsl/deterministic-rag-poc
It uses a constraint-driven YAML specification and builds both positive and negative prompts, along with stub scoring logic for geometry/semantic drift/aesthetics.
Request
Would the Vertex AI samples team consider adding a new example or extension under this repository (or companion repo) demonstrating the deterministic / constraint-driven agent mode?
It could be structured as:
examples/deterministic_agent_constraints.yamlsamples/vertex_agent_deterministic/- Documentation describing the concept and rationale
Thank you for all your work on Vertex AI samples.
I believe this capability could serve a wide range of users and enterprise workflows.
Kind regards,
Yuer
Background
Multi-modal agents built on Vertex AI (and other platforms) commonly exhibit a number of reliability and reproducibility issues, such as:
Proposal
Introduce an optional “deterministic / constraint-driven execution mode” for the Vertex AI agents framework.
The key features of this mode would include:
Benefits
Example Implementation
A proof-of-concept repository exists demonstrating such a mode:
https://github.com/yuer-dsl/deterministic-rag-poc
It uses a constraint-driven YAML specification and builds both positive and negative prompts, along with stub scoring logic for geometry/semantic drift/aesthetics.
Request
Would the Vertex AI samples team consider adding a new example or extension under this repository (or companion repo) demonstrating the deterministic / constraint-driven agent mode?
It could be structured as:
examples/deterministic_agent_constraints.yamlsamples/vertex_agent_deterministic/Thank you for all your work on Vertex AI samples.
I believe this capability could serve a wide range of users and enterprise workflows.
Kind regards,
Yuer