rototo
How-to: Validation

How to Test a Config Change Before Merge

Use this when a workspace change should be proven before review or before a production promotion ref moves.

The goal is to test the same behavior reviewers care about: the workspace loads, lint passes, and representative runtime requests resolve to the expected value keys and values.

Expected outcome

After this change:

Start with lint

Run workspace lint for every change:

rototo workspace lint config/

Lint catches structural problems: invalid manifests, missing schemas, unknown environments, unknown value keys, invalid values, invalid qualifier references, and custom lint diagnostics.

Add representative context fixtures

Create fixtures for the cases the change is meant to affect:

config/tests/
  prod-enterprise.context.json
  prod-default.context.json

Example matching context:

{
  "account": {
    "plan": "enterprise",
    "seats": 250
  }
}

Example default context:

{
  "account": {
    "plan": "team",
    "seats": 25
  }
}

These fixtures should be small and intentional. They document the behavior the change is supposed to preserve.

Resolve the changed variable

Run the changed variable against each fixture:

rototo variable resolve llm-agent-config \
  --workspace config/ \
  --env prod \
  --context @config/tests/prod-enterprise.context.json \
  --json

rototo variable resolve llm-agent-config \
  --workspace config/ \
  --env prod \
  --context @config/tests/prod-default.context.json \
  --json

Check both value_key and value. The key explains the branch that matched; the value is what the application will receive.

Automate the checks

Put the commands in the config repository's test script or CI job:

rototo workspace lint config/
rototo variable resolve llm-agent-config --workspace config/ --env prod \
  --context @config/tests/prod-enterprise.context.json --json
rototo variable resolve llm-agent-config --workspace config/ --env prod \
  --context @config/tests/prod-default.context.json --json

If your repository uses expected JSON files, compare the selected value_key and important fields in value. Avoid snapshots that make harmless metadata changes hard to review.

Common mistakes

Do not test only the happy path. Include the default or non-matching context.

Do not rely on lint alone for rollout changes. Lint can prove the workspace is valid, but representative resolution proves the behavior.

Do not use production secrets or full request payloads as fixtures. Keep fixtures minimal and safe to review.

Related docs