Environment Progression should consider Verification Status for success rate calculation

[jsbroks]

Problem

When evaluating environment progression policies, the success rate calculation only considers JobStatus (e.g., successful, failure) but does not consider JobVerificationStatus (e.g., passed, failed, running).

This means a job could have JobStatus: successful but JobVerificationStatus: failed, and the environment progression policy would still count it as a successful deployment for calculating the success rate percentage.

Current Behavior

In apps/workspace-engine/pkg/workspace/releasemanager/policy/evaluator/environmentprogression/jobtracker.go:

if t.SuccessStatuses[job.Status] && job.CompletedAt != nil {
    targetKey := rt.Key()
    // Store the oldest successful completion time for this release target
    if existingTime, exists := t.successfulReleaseTargets[targetKey]; !exists || job.CompletedAt.Before(existingTime) {
        t.successfulReleaseTargets[targetKey] = *job.CompletedAt
    }
}

The code only checks job.Status against SuccessStatuses, with no consideration of verification outcomes.

Expected Behavior

The EnvironmentProgressionRule should have an option to require verification to pass before counting a job as successful for progression purposes. Possible approaches:

Option 1: Add requireVerificationPassed flag

Add a boolean flag to EnvironmentProgressionRule:

requireVerificationPassed: {
  type: 'boolean',
  default: false,
  description: 'If true, jobs must also have passed verification to count toward the success percentage',
},

Option 2: Add verification statuses to success criteria

Allow specifying required verification statuses alongside job statuses:

requiredVerificationStatuses: {
  type: 'array',
  items: { $ref: '#/components/schemas/JobVerificationStatus' },
  description: 'Verification statuses that must be met for a job to count as successful',
},

Impact

Without this fix, users cannot safely use environment progression policies when they also have verification rules, as a deployment with failed verification could still trigger progression to subsequent environments.

Related Files

• apps/workspace-engine/pkg/workspace/releasemanager/policy/evaluator/environmentprogression/jobtracker.go
• apps/workspace-engine/pkg/workspace/releasemanager/policy/evaluator/environmentprogression/environmentprogression.go
• apps/workspace-engine/pkg/workspace/releasemanager/policy/evaluator/environmentprogression/passrate.go
• apps/workspace-engine/oapi/spec/schemas/policy.jsonnet (EnvironmentProgressionRule schema)