amplihack Recipe Runner

A code-enforced workflow execution engine that reads declarative YAML recipe files and executes them step-by-step using AI agents. Unlike prompt-based workflow instructions that models can interpret loosely or skip, the Recipe Runner controls the execution loop in compiled Rust code — making it physically impossible to skip steps.

Feature Highlights

• Comprehensive test suite covering unit, integration, recipe, example, and property-based testing
• Parallel step execution, tag filtering, JSONL audit logs
• Recipe composition via extends, pre/post/on_error hooks
• Safe condition language with recursive descent parser

Quick Start

# Build
cargo build --release

# Run a recipe
recipe-runner-rs path/to/recipe.yaml

# With context overrides
recipe-runner-rs recipe.yaml --set task_description="Add auth" --set repo_path="."

# Dry run
recipe-runner-rs recipe.yaml --dry-run

See the Quick Start guide for a more detailed walkthrough.

Quick Start

Get up and running with the amplihack Recipe Runner in minutes.

Install

# Clone the repository
git clone https://github.com/rysweet/amplihack-recipe-runner.git
cd amplihack-recipe-runner

# Build in release mode
cargo build --release

# The binary is at target/release/recipe-runner-rs
# Optionally copy it to your PATH:
cp target/release/recipe-runner-rs ~/.local/bin/

Your First Recipe

Create a file called hello.yaml:

name: "hello-world"
description: "A minimal recipe to verify your setup"
version: "1.0.0"
context:
  greeting: "Hello from the Recipe Runner!"
steps:
  - id: "greet"
    command: "echo '{{greeting}}'"

Run It

recipe-runner-rs hello.yaml

You should see the greeting printed to stdout.

Override Context

Pass --set to override context variables at runtime:

recipe-runner-rs hello.yaml --set greeting="Howdy, partner!"

Dry Run

Use --dry-run to see what would execute without actually running anything:

recipe-runner-rs hello.yaml --dry-run

Using Agent Steps

Agent steps invoke an AI agent instead of a shell command:

name: "analyze-project"
description: "Analyze a codebase with an AI agent"
version: "1.0.0"
context:
  repo_path: "."
steps:
  - id: "analyze"
    agent: "amplihack:core:architect"
    prompt: "Analyze the project at {{repo_path}} and summarize its structure"
    output: "analysis"
    parse_json: true

  - id: "report"
    command: "echo 'Analysis complete'"
    condition: "analysis"

Next Steps

• YAML Recipe Format — Full schema reference
• CLI Reference — All flags and subcommands
• Condition Language — Conditional step execution
• Architecture — How it works under the hood

YAML Recipe Format Reference

Complete schema reference for amplihack recipe runner YAML files.

Top-Level Fields

RecursionConfig

Controls sub-recipe nesting limits.

RecipeHooks

Shell commands executed at lifecycle boundaries.

Hook commands receive context variables via template substitution. Each runs as a bash step with a fixed 30-second timeout, under the same resolved bash interpreter as ordinary command: steps. A failing hook is logged as a warning and does not fail the step.

Step Fields

Note: The context field provides step-specific variables that are passed as overrides to sub-recipes. In YAML you write context:.

Unknown and retired step fields

A step key that isn’t in the table above is ignored, with a warning. The recipe still parses and runs, so an existing recipe never breaks because of an unknown key, but a typo or an unsupported field can’t silently do nothing.

The runner prints one warning per unknown key when it parses the recipe:

recipe 'deploy': step 'build' has unknown field 'comand' (ignored)

A field that used to mean something but is now retired gets a specific message that says what to do. The only retired field is max_env_value_bytes:

recipe 'workflow-tdd': step 'checkpoint-3': field 'max_env_value_bytes' is retired and ignored — the runner now spills oversized env values to private files automatically; remove it from the recipe

max_env_value_bytes was a per-step setting to avoid Argument list too long errors. The runner now handles oversized values automatically without truncating them (see Per-value env spill), so you can remove the field.

Details:

• A step without an id is shown as <index N> (zero-based). A recipe without a name is shown as <unnamed>. A non-string key is shown as <non-string key>.
• Recipe names, step ids and keys are escaped (\n, \u{1b}, …) and cut to 128 characters, so a warning is always a single line.
• A recipe gives at most 100 field warnings, followed by … and N more unknown step fields.
• Warnings go through the logger at warn level when it’s enabled (for example RUST_LOG=warn). Otherwise they go to stderr prefixed with warning: .
• Each distinct warning is printed once per process, so a sub-recipe that runs several times in one recipe run is reported once.
• --validate-only lists the same messages in its own report and doesn’t print them twice. Unknown fields are warnings, so it still exits 0. See --validate-only.

Library users who parse recipes themselves can turn the parse-time warnings off:

#![allow(unused)]
fn main() {
use std::path::Path;
use recipe_runner_rs::parser::RecipeParser;

let parser = RecipeParser::new().without_field_warnings();
let recipe = parser.parse_file(Path::new("recipes/deploy.yaml"))?;
}

RecipeParser::new() (and RecipeParser::default()) print the warnings. RecipeParser is a struct with a private field, so build it with new() or default(); the unit-struct expression RecipeParser doesn’t compile.

Type Inference Rules

When type is omitted, the effective step type is inferred in this order:

1. recipe field present → recipe type
2. agent field present → agent type
3. prompt present without command → agent type
4. Otherwise → bash type (default)

An explicit type value always takes precedence.

# Inferred as bash (has command, no agent/recipe/prompt)
- id: build
  command: cargo build --release

# Inferred as agent (agent field present)
- id: review
  agent: code-reviewer
  prompt: "Review {{file}}"

# Inferred as agent (prompt without command)
- id: summarise
  prompt: "Summarise the changes in {{diff}}"

# Inferred as recipe (recipe field present)
- id: deploy
  recipe: deploy-production
  context:
    env: staging

# Explicit type overrides inference
- id: special
  type: bash
  prompt: "This prompt is ignored because type is bash"
  command: echo "explicit wins"

Template Syntax

Variables are substituted using {{variable_name}} syntax. Variable names may contain letters, digits, underscores, hyphens, and dots.

context:
  project: my-app
  branch: main

steps:
  - id: greet
    command: echo "Building {{project}} on {{branch}}"

Dot Notation

Nested context values are accessed with dot notation:

context:
  deploy:
    target: production
    region: us-east-1

steps:
  - id: deploy
    command: ./deploy.sh --target {{deploy.target}} --region {{deploy.region}}

Shell Escaping

When templates are rendered for shell commands, values are shell-escaped automatically via shell_escape to prevent injection. Undefined variables resolve to an empty string.

Condition Syntax

The condition field accepts an expression that is evaluated against the current context. Steps with a falsy condition are skipped.

See conditions.md for the full reference. Supported operators and built-in functions include:

• Comparisons: ==, !=, <, <=, >, >=
• Logical: and, or, not
• Membership: in, not in
• Functions: int(), str(), len(), bool(), float(), min(), max()
• Methods: strip(), lower(), upper(), startswith(), endswith(), replace(), split(), join(), count(), find()

- id: deploy
  condition: "branch == 'main' and tests_passed == 'true'"
  command: ./deploy.sh

JSON Extraction (parse_json)

When parse_json: true, the runner attempts to extract structured JSON from step output using three strategies in order:

1. Direct parse — the entire trimmed output is valid JSON.
2. Markdown fence extraction — JSON inside ```json ... ``` fences.
3. Balanced bracket detection — locates the first {…} or […] block with proper depth tracking, string awareness, and escape handling.

If all strategies fail a warning is logged and the raw output is stored.

- id: get-config
  command: curl -s https://api.example.com/config
  output: api_config
  parse_json: true

- id: use-config
  command: echo "Region is {{api_config.region}}"

Strict Mode (parse_json_required)

By default, JSON extraction failure degrades the step (status becomes degraded) but the recipe continues. Set parse_json_required: true to make extraction failure a hard error that stops the recipe.

- id: must-be-json
  command: curl -s https://api.example.com/data
  parse_json: true
  parse_json_required: true  # fails the recipe if output isn't valid JSON
  output: api_data

Sub-Recipe Recovery (recovery_on_failure)

When a sub-recipe step fails, set recovery_on_failure: true to trigger an agentic recovery attempt. The runner sends the failure details to an agent, which attempts to complete the remaining work.

- id: deploy
  recipe: deploy-to-staging
  recovery_on_failure: true  # agent attempts recovery if deploy fails

If the agent’s recovery output contains “STATUS: COMPLETE” or “recovered”, the step is marked as recovered and the recipe continues. Otherwise, the original failure propagates.

Model Override (model)

Agent steps can override the default model using the model field. The value is passed to the selected provider adapter. For Codex, it is forwarded explicitly as --model; omission leaves selection to native Codex configuration. Codex never uses an implicit auto fallback. Choose a provider-supported identifier; the Claude examples below are not portable Codex model names. See Codex agent steps.

- id: quick-check
  agent: reviewer
  prompt: "Quick lint check on {{file_path}}"
  model: haiku  # fast, cheap model for simple tasks

- id: deep-review
  agent: reviewer
  prompt: "Thorough security review of {{file_path}}"
  model: sonnet  # more capable model for complex analysis

Complete Examples

1. Simple Bash-Only Recipe

name: build-and-test
version: "1.0"
description: Build the project and run tests
author: dev-team
tags: [ci, build]

context:
  build_mode: release

steps:
  - id: clean
    command: cargo clean

  - id: build
    command: cargo build --{{build_mode}}

  - id: test
    command: cargo test --{{build_mode}}
    output: test_results

  - id: report
    command: echo "Tests complete. Results {{test_results}}"

2. Agent-Based Workflow

name: code-review-workflow
version: "1.0"
description: Automated code review with AI agents

context:
  target_branch: main

steps:
  - id: get-diff
    command: git diff {{target_branch}} --stat
    output: diff_summary

  - id: review
    agent: code-reviewer
    prompt: |
      Review the following changes against {{target_branch}}:
      {{diff_summary}}
      Focus on correctness, security, and performance.
    output: review_result
    parse_json: true

  - id: check-approved
    condition: "review_result.approved == true"
    command: echo "Review passed"

  - id: request-changes
    condition: "review_result.approved != true"
    command: echo "Changes requested — see review_result.comments"

3. Sub-Recipe Composition

name: full-pipeline
version: "2.0"
description: End-to-end pipeline composing smaller recipes

context:
  environment: staging

steps:
  - id: lint
    recipe: lint-check

  - id: build
    recipe: build-project
    context:
      build_mode: release
      target: "{{environment}}"

  - id: deploy
    recipe: deploy-service
    context:
      env: "{{environment}}"
      version: "{{build.version}}"
    condition: "environment != 'local'"

4. Recipe with Hooks, Tags, and Recursion Limits

name: guarded-pipeline
version: "1.0"
description: Pipeline with lifecycle hooks and safety limits
author: platform-team
tags: [production, safe]

recursion:
  max_depth: 3
  max_total_steps: 50

hooks:
  pre_step: echo "[$(date -Iseconds)] Starting step"
  post_step: echo "[$(date -Iseconds)] Finished step"
  on_error: |
    echo "FAILED — sending alert"
    curl -s -X POST https://alerts.example.com/hook \
      -d '{"step": "failed", "recipe": "guarded-pipeline"}'

context:
  notify: true

steps:
  - id: preflight
    command: ./scripts/preflight-check.sh

  - id: migrate
    command: ./scripts/migrate.sh
    when_tags: [database]

  - id: deploy
    command: ./scripts/deploy.sh
    when_tags: [deploy]

  - id: smoke-test
    command: ./scripts/smoke-test.sh
    timeout: 120
    when_tags: [deploy]

  - id: notify
    condition: "notify == 'true'"
    command: echo "Pipeline complete"

5. Recipe with continue_on_error and Conditions

name: resilient-checks
version: "1.0"
description: Run multiple checks, collecting results even on failures

context:
  strict: false

steps:
  - id: lint
    command: cargo clippy -- -D warnings
    output: lint_result
    continue_on_error: true

  - id: test
    command: cargo test 2>&1
    output: test_result
    continue_on_error: true

  - id: audit
    command: cargo audit
    output: audit_result
    continue_on_error: true

  - id: gate
    condition: "strict == 'true'"
    command: |
      echo "Lint: {{lint_result}}"
      echo "Test: {{test_result}}"
      echo "Audit: {{audit_result}}"
      # Fail the pipeline in strict mode if any check failed
      exit 1

  - id: summary
    condition: "strict != 'true'"
    command: |
      echo "=== Check Summary ==="
      echo "Lint:  {{lint_result}}"
      echo "Test:  {{test_result}}"
      echo "Audit: {{audit_result}}"
      echo "Non-strict mode — pipeline continues"

CLI Reference

Complete reference for the recipe-runner-rs command-line interface.

Synopsis

recipe-runner-rs [OPTIONS] [RECIPE] [COMMAND]
recipe-runner-rs list [OPTIONS]
recipe-runner-rs --capabilities

Subcommands

list

Discover and display all available recipes found in the configured search directories.

recipe-runner-rs list
recipe-runner-rs list --recipe-dir ./custom-recipes
recipe-runner-rs list --recipe-dir ./team-recipes --recipe-dir ./personal-recipes

Global Options

--capabilities

Standalone compatibility probe. Emits one JSON object with schema_version: 1, the compiled package version, and capabilities: ["codex_exec"] on Unix (an empty capability list on non-Unix), then exits zero. It performs no update, cache-write, network, or agent-startup work. Do not combine it with other arguments. See the capability API.

--agent-binary <PROVIDER>

Select the agent provider, for example claude, copilot, or codex. Takes precedence over AMPLIHACK_AGENT_BINARY; the default is claude.

recipe-runner-rs review.yaml --agent-binary codex

Codex uses noninteractive exec with complete stdin instructions and private final-message capture. See Codex agent steps for configuration, model, timeout, and result behavior. Codex exit zero alone is insufficient: stdin delivery, lifecycle cleanup, and final-file validation must succeed. Successful final text is verbatim within MAX_STEP_OUTPUT_BYTES; oversized files fail instead of being truncated.

-C, --working-dir <DIR>

Set the working directory for recipe execution.

Default: . (current directory)

# Run a recipe from a different directory
recipe-runner-rs deploy.yaml --working-dir /home/user/my-project

# Short form
recipe-runner-rs deploy.yaml -C /home/user/my-project

# Combine with other options
recipe-runner-rs build.yaml -C ../other-repo --dry-run

-R, --recipe-dir <DIR>

Add a directory to the recipe search path. Can be specified multiple times to search across several directories.

# Single directory
recipe-runner-rs my-recipe --recipe-dir ./recipes

# Multiple directories (searched in order)
recipe-runner-rs my-recipe \
  --recipe-dir ./project-recipes \
  --recipe-dir ~/.config/recipes \
  --recipe-dir /opt/shared-recipes

# Short form
recipe-runner-rs my-recipe -R ./recipes -R ../shared

# Combine with list to discover recipes across directories
recipe-runner-rs list -R ./recipes -R /opt/shared-recipes

--set <KEY=VALUE>

Override a context variable. Can be specified multiple times to set several variables. Values are automatically typed using smart parsing (see Smart Context Value Parsing).

# String value
recipe-runner-rs deploy.yaml --set environment=production

# Integer value (auto-detected)
recipe-runner-rs scale.yaml --set replicas=5

# Float value (auto-detected)
recipe-runner-rs tune.yaml --set ratio=0.75

# Boolean value (auto-detected)
recipe-runner-rs build.yaml --set verbose=true

# JSON value (auto-detected)
recipe-runner-rs config.yaml --set data='{"host": "localhost", "port": 8080}'

# Multiple overrides
recipe-runner-rs deploy.yaml \
  --set environment=production \
  --set replicas=3 \
  --set debug=false \
  --set version=2.1.0

--dry-run

Parse and validate the recipe without executing any steps. Useful for checking recipe correctness before committing to a run.

recipe-runner-rs deploy.yaml --dry-run

# Combine with --set to validate context overrides
recipe-runner-rs deploy.yaml --dry-run --set environment=staging

# Combine with --progress to see what steps would run
recipe-runner-rs deploy.yaml --dry-run --progress

--no-auto-stage

Disable automatic git staging of file changes made during recipe execution.

recipe-runner-rs codegen.yaml --no-auto-stage

# Useful when you want to review changes before staging
recipe-runner-rs refactor.yaml --no-auto-stage -C /path/to/repo

--validate-only

Parse and validate the recipe, print any warnings, then exit. Does not execute any steps. More thorough than --dry-run as it focuses on surfacing validation warnings.

Unknown step fields are listed once, in the same wording a normal run uses, and the command still exits 0 because they are warnings. A normal run reports the same unknown fields as parse-time warnings and keeps going (see Unknown and retired step fields). --validate-only doesn’t repeat them as parse-time warnings.

recipe-runner-rs deploy.yaml --validate-only

# Validate a recipe in a specific directory
recipe-runner-rs my-recipe --validate-only -R ./recipes

# Validate with context overrides to check for missing variables
recipe-runner-rs deploy.yaml --validate-only --set environment=production

--explain

Show the structure of a recipe without executing it. Displays the recipe name, version, and each step with its conditions, agents, and commands.

recipe-runner-rs deploy.yaml --explain

# Explain a recipe found via search path
recipe-runner-rs my-recipe --explain -R ./recipes

Example output:

Recipe: deploy
Version: 1.2.0

Steps:
  1. build
     Agent: builder
     Command: cargo build --release
  2. test
     Condition: when context.run_tests == true
     Agent: tester
     Command: cargo test
  3. deploy
     Agent: deployer
     Command: ./scripts/deploy.sh

--progress

Print step progress events to stderr. Emits events when each step starts and completes, useful for monitoring long-running recipes.

recipe-runner-rs deploy.yaml --progress

# Capture progress separately from output
recipe-runner-rs deploy.yaml --progress 2>progress.log

# Combine with JSON output for machine-readable progress + results
recipe-runner-rs deploy.yaml --progress --output-format json

Example stderr output:

[step:start] build (1/3)
[step:complete] build (1/3) — ok
[step:start] test (2/3)
[step:complete] test (2/3) — ok
[step:start] deploy (3/3)
[step:complete] deploy (3/3) — ok

--include-tags <TAGS>

Comma-separated list of tags. Only steps whose when_tags match at least one of the specified tags will run. All other steps are skipped.

# Run only steps tagged "frontend"
recipe-runner-rs build.yaml --include-tags frontend

# Run steps tagged "test" or "lint"
recipe-runner-rs ci.yaml --include-tags test,lint

# Combine with --explain to preview filtered steps
recipe-runner-rs ci.yaml --include-tags test --explain

--exclude-tags <TAGS>

Comma-separated list of tags. Steps whose when_tags match any of the specified tags will be skipped.

# Skip slow integration tests
recipe-runner-rs ci.yaml --exclude-tags slow

# Skip multiple categories
recipe-runner-rs full-pipeline.yaml --exclude-tags slow,experimental,deprecated

# Include some, exclude others
recipe-runner-rs ci.yaml --include-tags test --exclude-tags slow

--audit-dir <DIR>

Directory where JSONL audit log files are written. Each recipe run produces one audit log file.

# Write audit logs to a directory
recipe-runner-rs deploy.yaml --audit-dir ./audit-logs

# Combine with other options for a fully audited production run
recipe-runner-rs deploy.yaml \
  --audit-dir /var/log/recipe-runner \
  --set environment=production \
  --progress

--output-format <FORMAT>

Control the output format. Available formats:

# Default text output
recipe-runner-rs deploy.yaml

# JSON output for scripting / CI pipelines
recipe-runner-rs deploy.yaml --output-format json

# Pipe JSON output to jq
recipe-runner-rs deploy.yaml --output-format json | jq '.steps[] | select(.status == "failed")'

# JSON output with progress on stderr
recipe-runner-rs deploy.yaml --output-format json --progress 2>/dev/null

Exit Codes

# Check exit code in scripts
recipe-runner-rs deploy.yaml
if [ $? -eq 0 ]; then
  echo "Deploy succeeded"
elif [ $? -eq 1 ]; then
  echo "Deploy failed — check step output"
elif [ $? -eq 2 ]; then
  echo "Recipe is invalid — check YAML syntax"
fi

# Use && / || for simple chaining
recipe-runner-rs build.yaml && recipe-runner-rs deploy.yaml

# Validate before running
recipe-runner-rs deploy.yaml --validate-only && recipe-runner-rs deploy.yaml

Smart Context Value Parsing (--set)

When using --set KEY=VALUE, the runner automatically determines the value type by attempting each parse strategy in order:

# JSON — parsed as a structured object
recipe-runner-rs setup.yaml --set config='{"host": "localhost", "port": 8080}'
recipe-runner-rs setup.yaml --set tags='["web", "api"]'

# Boolean — parsed as bool
recipe-runner-rs build.yaml --set release=true
recipe-runner-rs build.yaml --set skip_tests=false

# Integer — parsed as i64
recipe-runner-rs scale.yaml --set workers=8
recipe-runner-rs scale.yaml --set retries=0

# Float — parsed as f64
recipe-runner-rs tune.yaml --set threshold=0.95
recipe-runner-rs tune.yaml --set learning_rate=0.001

# String — fallback for everything else
recipe-runner-rs deploy.yaml --set branch=main
recipe-runner-rs deploy.yaml --set message="deploy to production"

Environment Variables

RECIPE_RUNNER_RECIPE_DIRS

Additional recipe search directories, separated by colons. These directories are searched in addition to any specified via --recipe-dir.

# Set via environment
export RECIPE_RUNNER_RECIPE_DIRS="/opt/recipes:/home/user/.config/recipes"
recipe-runner-rs my-recipe

# Inline for a single invocation
RECIPE_RUNNER_RECIPE_DIRS=./recipes recipe-runner-rs list

# Combine with --recipe-dir (both are searched)
export RECIPE_RUNNER_RECIPE_DIRS="/opt/shared-recipes"
recipe-runner-rs my-recipe --recipe-dir ./local-recipes

AMPLIHACK_BASH

Overrides the interpreter used to run bash steps. Must be an absolute path to an executable file.

Bash steps do not hardcode /bin/bash. The runner resolves one interpreter in the parent process and uses that same absolute path for every bash step — inline (-c) or file-backed, with or without a timeout wrapper.

What it covers

Resolution order

An empty AMPLIHACK_BASH (AMPLIHACK_BASH=) is treated as unset and falls through to rule 2. There is no trimming: a whitespace-only value is a path, and it fails loudly like any other bad one. A value containing bytes that are not valid UTF-8 is not treated as unset — you named an interpreter, so running a different one silently is the one outcome rule 1 exists to prevent.

Non-absolute and empty PATH entries are skipped during the rule-2 scan. An empty entry means “current directory” to POSIX, and the current directory of a bash step is the recipe’s working_dir — recipe-controlled data has no business selecting an interpreter.

Rule 2 honours PATH order exactly. There is no version detection: if an older bash comes first on your PATH, it wins, because PATH order is your stated preference. Use AMPLIHACK_BASH when you want a specific build regardless of PATH.

Resolution reads the parent process environment only. A step’s env: block or a RECIPE_VAR_* value named AMPLIHACK_BASH is merged into the child environment and cannot change which interpreter the runner picks.

Invalid values fail loudly

A set-but-unusable AMPLIHACK_BASH is a hard error. The runner does not silently fall back to PATH or /bin/bash, because a step that quietly runs under a different interpreter than the operator asked for is worse than a step that refuses to run.

One message covers all three rejections:

AMPLIHACK_BASH is set to "<path>" but <reason>. Unset AMPLIHACK_BASH to resolve
bash from PATH, or point it at an absolute path to an executable bash.

<reason> is one of a fixed set of strings:

The permission-denied case is called out separately on purpose: an executable bash under an unreadable parent directory exists, and reporting it as missing would send you looking for a file that is already there.

Two properties of that message are worth knowing before you paste it into a bug report:

• It echoes your AMPLIHACK_BASH value and nothing else. It never contains PATH, the directories searched, or the candidates tried — PATH routinely leaks project, customer, and username identifiers through directory names.
• Symlinks are followed when validating, so /bin/bash pointing at /usr/bin/bash is accepted. The path you supplied is the path that gets executed; it is never rewritten to its symlink target.

The absolute-path requirement is not style. In a timeout-wrapped step the interpreter is passed as an argument to timeout, so a value beginning with - would be swallowed by timeout’s option parser instead of being treated as the command. Requiring a leading / makes that unrepresentable.

Conversely, shell metacharacters in the value are inert and are not rejected. There is no shell between the resolved path and exec, so ;, spaces, $(…) and newlines are ordinary filename bytes. AMPLIHACK_BASH=/tmp/x;evil names a file literally called x;evil.

Resolution is logged

Each bash step logs the interpreter it resolved and which rule chose it, at info. Resolution is per step and uncached, so a recipe with N bash steps emits N lines.

The binary initialises env_logger with no default filter, so nothing below error is shown until you set RUST_LOG:

$ RUST_LOG=info recipe-runner-rs build
[2026-09-22T17:04:11Z INFO  recipe_runner_rs::adapters::cli_subprocess] bash interpreter: /opt/homebrew/bin/bash (source: PATH)

The message body — the part after env_logger’s timestamp/level/target prefix — takes one of three shapes:

bash interpreter: /usr/local/bin/bash (source: AMPLIHACK_BASH)
bash interpreter: /opt/homebrew/bin/bash (source: PATH)
bash interpreter: /bin/bash (source: default)

Like the error text, the log line carries the resolved path and the rule tag only. It never prints PATH or the candidates that were tried.

It propagates to child processes

AMPLIHACK_BASH matches the protected AMPLIHACK_ prefix, so it is never dropped by env-budget trimming and is inherited by every subprocess the runner spawns — bash steps and agent steps alike. A bash step or an agent that goes on to invoke recipe-runner-rs itself therefore sees the same pin. Exporting it once covers a whole tree of runs, not just one process.

Examples

# Pin the exact interpreter, ignoring PATH entirely.
AMPLIHACK_BASH=/opt/homebrew/bin/bash recipe-runner-rs build

# Restore the pre-resolution behaviour (always /bin/bash) for one run.
AMPLIHACK_BASH=/bin/bash recipe-runner-rs build

# Pin it for a whole session, including nested sub-recipes.
export AMPLIHACK_BASH=/usr/local/bin/bash
recipe-runner-rs deploy.yaml --set environment=production

# Confirm which bash a recipe will use, without running it.
command -v bash          # what rule 2 would pick
bash --version | head -1

On macOS, /bin/bash is bash 3.2 and lacks mapfile, declare -A, and ${x@Q}. A recipe that uses bash 4+ syntax works as long as a newer bash is on PATH ahead of /bin/bash (Homebrew installs one), or AMPLIHACK_BASH names it.

Behaviour note. Earlier versions always ran /bin/bash. Bash steps now prefer the first bash on PATH. Set AMPLIHACK_BASH=/bin/bash to get the old behaviour back exactly.

See Bash interpreter resolution for why the parent process resolves the path instead of letting execvp search for it.

Rate-Limit Retry Knobs

Agent steps automatically retry on transient provider rate limits with bounded exponential backoff. Tune the behavior with these variables (see Rate-Limit Handling for full details):

Each attempt receives its own step timeout. Backoff and repeated attempts increase total duration, and retries may repeat side effects. Codex retains its explicit model or native selection on every attempt.

Usage Examples

Basic Usage

# Run a recipe by file path
recipe-runner-rs ./recipes/build.yaml

# Run a recipe by name (searched in recipe directories)
recipe-runner-rs build

# List all discoverable recipes
recipe-runner-rs list

CI/CD Pipeline

# Validate, then run with JSON output and auditing
recipe-runner-rs deploy.yaml --validate-only \
  && recipe-runner-rs deploy.yaml \
    --set environment=production \
    --set version="$(git describe --tags)" \
    --output-format json \
    --audit-dir /var/log/deploys \
    --progress

Development Workflow

# Preview what a recipe will do
recipe-runner-rs refactor.yaml --explain

# Dry-run with overrides to test logic
recipe-runner-rs refactor.yaml --dry-run \
  --set target_module=auth \
  --set aggressive=true

# Run without auto-staging to review changes manually
recipe-runner-rs refactor.yaml \
  --set target_module=auth \
  --no-auto-stage

Selective Step Execution

# Run only unit tests
recipe-runner-rs ci.yaml --include-tags unit

# Run everything except slow tests
recipe-runner-rs ci.yaml --exclude-tags slow,integration

# Explain which steps match the filter
recipe-runner-rs ci.yaml --include-tags unit --explain

Multi-Directory Recipe Management

# Search across project, team, and global recipes
recipe-runner-rs list \
  -R ./recipes \
  -R ~/team-recipes \
  -R /opt/global-recipes

# Or use the environment variable
export RECIPE_RUNNER_RECIPE_DIRS="./recipes:~/team-recipes:/opt/global-recipes"
recipe-runner-rs list

Scripting and Automation

# Capture JSON output for downstream processing
output=$(recipe-runner-rs analyze.yaml --output-format json)
echo "$output" | jq '.summary'

# Run with full observability
recipe-runner-rs deploy.yaml \
  --output-format json \
  --progress \
  --audit-dir ./audit \
  --set environment=production \
  2>progress.log \
  1>result.json

Codex Agent Steps

title: Codex agent steps description: Codex execution, configuration, results, and terminal failure contracts. doc_type: reference last_updated: 2026-10-07

Codex agent steps

The runner executes Codex agent steps through Amplihack using modern noninteractive codex exec. Claude remains the default provider; selecting Codex leaves Claude and Copilot invocation and result contracts unchanged.

Contents

• Run a recipe
• Provider and configuration
• Instructions, results, and failures
• Cancellation and ownership
• Terminal cleanup failures
• Compatibility probe API
• Rust adapter API

Run a recipe

Install the runner, a production Amplihack launcher with Codex exec stdin support, and Codex CLI. Configure authentication using Codex’s native configuration. The runner uses that existing authentication; it does not provision credentials or change Codex configuration. Codex CLI 0.160.0 is the compatibility baseline.

Save this as codex-summary.yaml:

name: codex-summary
version: "1.0"
context:
  subject: "the purpose of this repository"
steps:
  - id: summarize
    type: agent
    prompt: |
      Explain {{subject}} in three sentences.
      Return only the explanation.
    timeout: 120
    auto_stage: false
    output: summary

Validate, then run in the repository you want Codex to inspect:

recipe-runner-rs codex-summary.yaml --validate-only
# Output: ✓ Recipe 'codex-summary' is valid (1 steps)
recipe-runner-rs codex-summary.yaml --agent-binary codex -C . --progress

The final response is stored in summary. --output-format json serializes the recipe result; it does not enable Codex JSON streaming. Progress and diagnostics are separate from the stored response.

To apply a packaged persona, add an existing resolvable agent reference, for example agent: "amplihack:core:architect". Resolved persona text joins the effective system instructions. Existing agent-resolution warnings still apply when a reference cannot be resolved.

Provider and configuration

--agent-binary codex takes precedence over AMPLIHACK_AGENT_BINARY=codex. With neither set, the runner uses Claude. Provider selection is separate from the recipe’s agent persona reference.

The runner launches this argument layout, with an optional explicit model:

amplihack codex -- exec --output-last-message UNIQUE_FINAL_PATH [--model EXPLICIT_MODEL] -

The task is delivered through stdin. Codex -p selects a profile and is not used for prompt delivery. The child runs in the effective step working directory, falling back to the recipe working directory. The runner adds no implicit --add-dir scope.

Omitting the recipe step’s model leaves model selection to native Codex configuration. If supplied, the value is forwarded as an explicit --model value; choose an identifier supported by your Codex setup. The runner never selects auto for Codex, including when AMPLIHACK_RATELIMIT_FALLBACK_AUTO_MODEL is enabled. Claude/Copilot retain their existing opt-in fallback behavior.

Temporary resources use portable tempfile handling and the platform’s temporary-directory configuration, including TMPDIR on Unix. To use a chosen writable runtime directory:

runtime_dir=$(mktemp -d)
TMPDIR="$runtime_dir" recipe-runner-rs codex-summary.yaml --agent-binary codex
rmdir "$runtime_dir"

The command creates an owner-only runtime directory and removes it after successful attempt-resource cleanup. Each attempt owns a unique private directory and a final pathname that is absent before launch. On Unix, attempt directories are owner-only (0700), with owner-only final-file access (0600). Resources remain owned through process completion and extraction, then are removed. Allocation and cleanup failures are reported.

Instructions, results, and failures

Every prompt size uses the same deterministic UTF-8 stdin envelope: effective system instructions, resolved persona, recipe leaf/no-reentry instructions, task, and autonomy footer. The full envelope is written and stdin closed. Leading dashes, newlines, Unicode, and large prompts do not change transport. Instructions are not placed in AGENTS.md.

Successful raw output is only the content of the final-message file. Whitespace, trailing newlines, and empty content are preserved verbatim. An existing zero-byte file is a valid empty response; a missing file is an error. Progress stdout/stderr never substitutes for a final response. Explicit recipe JSON parsing retains its documented extraction/transformation semantics.

The existing MAX_STEP_OUTPUT_BYTES limit is 10,000,000 bytes. Codex final output at or below the limit is preserved; larger output fails explicitly before unbounded allocation. Successful Codex output is never silently truncated. There is no runtime output-limit setting.

Spawn errors, incomplete or failed stdin writes, timeout, nonzero exit, missing or nonregular final files, symlinks, unsafe permissions, read failures, invalid UTF-8, and cleanup failures fail the attempt. A final file does not turn a nonzero exit or incomplete instruction delivery into success. Stdout and stderr are drained concurrently, retaining the latest 64 KiB per stream in memory while continuing to drain. After shutdown, each reader drains queued bytes until EOF or the pipe would block, bounded by 1 MiB of additional reads or 50 ms, so detached writers cannot prevent completion. No diagnostic spool files are created. Reader failures fail the attempt.

A step’s timeout applies separately to each attempt. Its absolute deadline starts immediately after successful launcher spawn and covers group-anchor establishment (including the helper acknowledgment), reader setup, stdin delivery and execution. timeout: 0 expires immediately. With no timeout, helper startup has no arbitrary short cap, but still checks for cancellation while awaiting acknowledgment; cleanup retains its own bounds. Synchronous OS spawn is outside this polling deadline.

Cleanup stops stdin delivery, retires the owned launcher within two seconds, and terminates the original group while a private anchor retains its identity. Group TERM has at most 100 ms grace; the final group KILL is attempted even after TERM or probe errors when authority remains valid. Signaling is then sealed. Helper retirement and observational confirmation share a two-second deadline measured from completion of that KILL attempt. Process polling uses at most 4.1 seconds, plus the existing reader and resource cleanup bounds. Failed signals or reaping remain errors even after eventual absence. See the lifecycle reference for startup, ownership assumptions and failure paths.

Linux process metadata distinguishes zombies from live members; other Unix platforms require group absence. A proven transient pre-reap zombie-only observation resolved by bounded successful reaping and final absence does not add a false terminal cleanup error. EPERM or final absence alone cannot establish that condition. Genuine permission, signal, probe, reap, reader, resource and surviving-descendant failures remain terminal. Unix process groups contain ordinary inherited descendants; deliberately detached sessions are outside that containment. Codex execution is Unix-only; non-Unix execution fails explicitly. Unix capability advertisement does not establish native runtime validation on every Unix platform.

Bounded rate-limit retries and JSON repair retries retain the effective instructions, task, persona, no-reentry context, provenance, working directory, explicit model, and timeout. A JSON repair attempt adds repair instructions. Every attempt gets fresh private output resources. Backoff waits and repeated attempts increase total step duration beyond a single timeout. Retries can repeat tool calls and other side effects; use idempotent operations where possible. Stdin, file, decoding, and cleanup errors are not made retryable merely by rate-limit text in diagnostics.

The runner has no generic Codex argument passthrough API. Use recipe fields for task, model, working directory, and timeout.

Cancellation and ownership

SIGINT and SIGTERM observed during active Codex ownership stop stdin delivery and use bounded process, reader and resource cleanup. Cancellation remains effective during last-owner signal restoration, including SIGTERM delivered while its product handler remains installed after SIGINT restoration. The adapter captures the immutable interruption and restoration result under the ownership mutex before an independent owner can begin a new epoch.

Concurrent calls share signal ownership and the same observation baseline. Closing a nonlast owner samples cancellation without restoring handlers. The last owner attempts restoration of both complete prior dispositions. Registration, rollback or restoration failures retain diagnostics and pending restoration metadata; a later installation reconciles them within bounds or refuses before spawning.

The guarantee starts at the handler’s observable atomic entry publication. A published event remains observable even if that handler is still in flight. Persistent per-signal observations are never reset. A kernel-selected handler that has not reached publication at the retirement snapshot lies outside this guarantee; atomics, the mutex and sigaction do not establish a universal kernel dispatch fence.

Cancellation stops the entire recipe, including nested recovery and pending parallel steps, even with continue_on_error: true or fatal: false. JSON repair cannot accept degraded success after cancellation. Already admitted work is joined under its existing execution contract. Ordinary Claude/Copilot failures retain their existing policies. No recipe field configures the private anchor, observation counters, handler retirement or cleanup budgets.

Terminal cleanup failures

Cleanup failure stops the entire recipe, even without SIGINT or SIGTERM. Process shutdown, diagnostic-reader completion, and owned-resource deletion failures remain terminal through agent calls, Bash calls, lifecycle hooks, nested recipes, recovery, parallel execution, and JSON repair.

Already admitted work may finish. A parallel failure remains associated with its original step, and successful siblings cannot turn the recipe into success. Step results, listener notifications, audit entries, and checkpoints reflect the effective failure, including failure in a final post-hook. Terminal state persists through nesting and aggregation and resets only for an independent top-level execution using the runner again.

Ordinary errors retain their existing policies: nonfatal continuation, nested recovery, optional JSON degradation, and warning-only hook failures still apply. Terminality comes from typed errors, not diagnostic text containing words such as “cleanup” or “interrupted”. There is no configuration switch to bypass it.

For example, a nested Codex step whose final message is valid but whose owned resource deletion fails causes its parent recipe to fail. Setting recovery_on_failure: true and continue_on_error: true on the parent step does not launch recovery or the next step. An ordinary child error remains eligible for the configured recovery policy.

Failure diagnostics retain the unsuccessful child exit status, safe failure classification, and cleanup details. If interruption and cleanup failure occur together, interruption takes precedence while cleanup diagnostics remain visible. Both Interruption and CleanupFailure remain discoverable through typed error composition, together with the primary ordinary error or unsuccessful exit status. A cleanup error does not establish that every owned resource was removed. See Codex verification for regression coverage of these boundaries.

Compatibility probe API

recipe-runner-rs --capabilities

The standalone probe exits zero and writes exactly one JSON object, optionally followed by a newline, with empty stderr. For a Unix build with package version 0.5.0:

{"capabilities":["codex_exec"],"schema_version":1,"version":"0.5.0"}

version is the compiled Cargo package version. The producer emits exactly these three keys. Unix builds advertise codex_exec; non-Unix builds emit an empty capability list because execution is unsupported. The probe runs before logging initialization, update checks, update-cache writes, network access, or agent startup, regardless of ambient logging/update settings. Mixed recipe/subcommand/option invocations with --capabilities are rejected before execution or update effects. Failed stdout writes report failure.

Consumers require numeric schema_version: 1 and membership of codex_exec. Compatible newer/custom builds may pass; an exact source SHA is a separate delivery pin, not part of this JSON. codex_exec promises full stdin instructions, final-message-only results, explicit failure propagation, and no implicit model.

A stale runner that rejects the probe is incompatible. Managed Amplihack installations can upgrade to their corrected immutable runner revision and re-probe. An incompatible explicit RECIPE_RUNNER_RS_PATH override must be replaced or unset by the operator; it is not silently replaced. Check the runner capability and launcher exec support separately.

Rust adapter API

The public Adapter: Sync interface and StepResult types are unchanged. execute_agent_step accepts prompt, optional agent name, effective system prompt, mode, working directory, optional model, and optional timeout, returning Result<String, anyhow::Error>.

For a Codex CLISubprocessAdapter, Ok(String) contains the verbatim final message within the output bound, including Ok(String::new()) for an intentionally empty file. Execution/resource failures return contextual Err; ordinary failures follow existing failure and continue_on_error policies. SIGINT/SIGTERM cancellation and cleanup failure are terminal and stop the recipe regardless of those policies, including during nested recovery. Optional JSON parsing happens after successful raw extraction. No new public error enum, output-path parameter, or JSONL protocol is required.

The runner classifies private typed CleanupFailure and Interruption markers in the error chain before converting an error to text. Classification survives execution boundaries without changing the public result schema. Cleanup failures never enter rate-limit retries; the terminal cleanup contract also governs recovery, nonfatal policies, hooks, and JSON repair.

Condition Language Reference

The recipe runner’s condition evaluator is a hand-rolled tokenizer + recursive-descent parser implemented in src/context.rs. Conditions are expressions evaluated to determine if a step should execute. If the condition evaluates to truthy, the step runs; otherwise it’s skipped.

If evaluation itself fails (e.g., a syntax error), the step is marked Failed — not skipped.

Truthiness

Operators

Listed by precedence, lowest to highest:

Type coercion in comparisons

• Equality (==, !=): Same types compare directly. Mixed types fall back to comparing string representations (so 5 == "5" is true).
• Ordering (<, <=, >, >=): Number–Number is numeric. String–String is lexicographic. String–Number attempts to parse the string as f64 then compares numerically. All other combinations are incomparable (condition evaluates as falsy).
• Membership (in, not in): Against a string, checks substring containment. Against an array, checks element equality via values_equal. Against any other type, evaluates as falsy.

Literals

Identifiers

Identifiers are alphanumeric names (plus underscores) that look up values in the recipe context.

Dot-notation in identifiers is resolved during parsing — each segment walks one level deeper into nested JSON values. If any segment is missing, the whole expression resolves to Null.

Function Calls

Only whitelisted function names are allowed. Calling an unknown function is an error.

Method Calls

Methods use .method(args) syntax and can only be called on string values. Calling a method on a non-string is an error. Only whitelisted method names are allowed.

Methods can be chained: name.strip().lower().

Safety Features

1. Whitelist-only execution — Only the functions and methods listed above are allowed. Unknown names produce an error, not silent null.
2. Dunder blocking — Any expression containing __ (e.g., __class__, __import__) is rejected before parsing even begins.
3. No assignment, no side effects — The expression language is pure; it can only read context values and compute results.
4. Unknown identifiers are null — Referencing a variable that doesn’t exist returns Null (falsy) rather than raising an error. This is intentional for optional-variable patterns.

Important Gotchas

1. All numbers are f64

Numbers are parsed and stored as f64 internally. This means str(42) produces "42.0", not "42". If you need the integer string representation, store it as a string in the context instead of using str() on a numeric literal.

2. shell_escape::escape() wraps values in single quotes

When using render_shell() for template expansion, empty strings become '' (two single quotes), not the empty string. This is correct for shell safety but may surprise you in conditions that check the rendered result.

3. Unknown identifiers are null by design

This is a feature, not a bug. It allows patterns like condition: "optional_var" to work — if optional_var isn’t set, the condition is falsy and the step is skipped without error.

4. not in is a single operator

The tokenizer uses lookahead to parse not in as one token (NotIn), distinct from a standalone not followed by in. This means not in always means “not contained in”, never “negation of the result of in” — though the result is the same.

5. Boolean keywords are case-sensitive

Only true/True and false/False are recognized. TRUE, FALSE, tRue, etc. are treated as regular identifiers and will resolve to Null.

6. none is not a keyword

There is no none or None literal. Writing none creates an identifier lookup that (typically) resolves to Null because no context variable named none exists. This works in practice but is not guaranteed if someone sets a context variable called none.

Examples

Basic truthiness

# Truthy if 'analysis' is set and non-empty in context
condition: "analysis"

# Always true
condition: "true"

# Always false
condition: "false"

String comparison

# Exact match
condition: "status == 'success'"

# Not equal
condition: "status != 'error'"

# Case-insensitive comparison via method
condition: "status.lower() == 'success'"

Numeric comparison

# Greater than
condition: "count > 0"

# Compound range check
condition: "count > 0 and count < 10"

# With function conversion
condition: "int(exit_code) == 0"

Logical operators

# Negation
condition: "not skip_tests"

# AND
condition: "has_tests and not skip_tests"

# OR
condition: "use_cache or force_rebuild"

# Combined with parentheses
condition: "(status == 'success' or status == 'partial') and not skip"

Membership tests

# Substring containment
condition: "'error' in output"

# Negated containment
condition: "'error' not in output"

# Array membership (items is an array in context)
condition: "'admin' in roles"

Function calls

# Length check
condition: "len(items) > 0"

# Type conversion
condition: "int(retry_count) < 3"

# Boolean conversion
condition: "bool(result)"

# Min/max
condition: "max(score_a, score_b) >= 80"

Method calls

# String prefix check
condition: "name.startswith('test_')"

# String suffix check
condition: "filename.endswith('.py')"

# Chained methods
condition: "input.strip().lower() == 'yes'"

# Replace and check
condition: "path.replace('\\', '/').startswith('/home')"

# Split and check length
condition: "len(csv_line.split(',')) > 3"

# Find (returns index or -1)
condition: "message.find('WARNING') >= 0"

# Count occurrences
condition: "log_output.count('ERROR') == 0"

Nested context access

# Dot-notation for nested values
condition: "result.status == 'ok'"

# Deep nesting
condition: "response.data.count > 0"

Optional variable patterns

# Skip step if variable isn't set (resolves to null → falsy)
condition: "optional_feature"

# Guard with default-like logic
condition: "config.verbose and len(debug_output) > 0"

Cross-type equality

# Number-string coercion: this is true if exit_code is 0 (the number)
condition: "exit_code == '0'"

# But be careful: str(42) gives "42.0", not "42"
# So this does NOT work as expected:
#   condition: "str(count) == '42'"    # produces "42.0" == "42" → false

Rate-Limit Handling

When the Recipe Runner executes an agent step, it spawns the amplihack launcher to run an AI agent (copilot, claude, or codex). Hosted agents enforce transient rate limits: under load, the underlying provider can reject a request with a short cooldown window — for example, a Copilot enterprise limit exits with status 1 and the message:

You’ve hit your rate limit. Please wait for your limit to reset in under a minute or switch to auto model to continue.

This is not a logic failure. It is temporary back-pressure that clears on its own. Rather than aborting the whole workflow run — and discarding every step that already succeeded — the runner detects rate-limit signals, waits, and retries the same step with bounded exponential backoff before giving up with a clear error.

Scope. Rate-limit handling applies only to agent steps (amplihack <agent> subprocesses). Bash steps (execute_bash_step) are unaffected. Claude/Copilot retain their existing output-capture/truncation behavior. Codex requires a validated final-message file and preserves successful output verbatim within MAX_STEP_OUTPUT_BYTES; oversize output fails.

How it works

1. Run the step. The agent subprocess is spawned, monitored against the per-attempt timeout. Claude/Copilot retain their existing stdout/stderr capture. Codex captures bounded diagnostics separately and validates its private final-message file; progress output never becomes the result.
2. Inspect failures. On a non-zero exit, the runner checks the captured output for rate-limit signals (case-insensitive substring match):
   • hit your rate limit
   • reset in
   • rate limit
   • 429
   • too many requests
3. Decide.
   • No rate-limit signal → fail fast with the existing amplihack <cli> failed (exit N) error. Authentication errors, bad arguments, and genuine agent failures are never retried or swallowed.
   • Rate-limit signal and retries remain → log a loud banner, sleep the backoff delay, then retry the same step.
4. Bound the work. After the configured number of retries is exhausted, the runner stops and fails explicitly — it never loops forever.

Rate-limit handling bounds executions to 1 + max_retries per adapter call; recipe JSON repair can invoke the adapter again. The timeout applies separately to each attempt, not to the entire retry loop. Backoff and repeated attempts increase total step duration. Retries can repeat tool calls and other side effects; use idempotent operations where possible.

Backoff policy

The wait before retry number n is:

delay = min(base_delay * 2^(n-1), max_delay)

n is a retry counter, not the overall execution count. It starts at 1 for the first retry (the second total execution of the step) and increments by one per retry. The initial attempt is execution 0 and has no preceding wait. Implementations must call the backoff function with this retry index — passing the overall execution count would shift every wait by one doubling (the first wait would become 120s) and contradict the table below.

• The first retry (n = 1) waits one base_delay window (default ~60s, matching the provider’s “under a minute” hint).
• Each subsequent retry doubles the wait, capped at max_delay (default 600s).
• The delay arithmetic is saturating: extreme attempt counts can never overflow, and the result is always ≤ max_delay.
• Setting base_delay = 0 makes every wait instant — useful for fast tests.

Example with defaults (base=60, cap=600, max_retries=5):

Loud, never silent

Every detected rate-limit and every backoff is reported to stderr — the runner never waits silently. Each retry emits a banner naming the matched signal, the wait duration, and the attempt counter:

================================================================================
⏳ RATE LIMIT detected (signal: "hit your rate limit")
   Agent step throttled by the provider. This is transient — backing off.
   Waiting 60s before retry 2/5 ...
================================================================================

When retries are exhausted, the step fails with an explicit, actionable message (this exact wording is the canonical target the implementation’s bail! must emit):

Error: amplihack copilot rate limit persisted after 5 retries (6 attempts total).
The provider kept throttling this step. Increase AMPLIHACK_RATELIMIT_MAX_RETRIES
or AMPLIHACK_RATELIMIT_BASE_DELAY_SECS, or retry later.

Optional --model auto fallback

The provider message suggests switching to the auto model to keep working. The runner can apply this automatically for Claude/Copilot as a last resort. Codex is excluded: it preserves the explicit model or native model selection on every attempt, even when this variable is enabled. When AMPLIHACK_RATELIMIT_FALLBACK_AUTO_MODEL is set to a non-empty value, the final retry attempt is run with --model auto appended, overriding any explicit model configured for that step. This behavior is off by default so that normal runs never silently change models.

Configuration

All knobs are environment variables, parsed leniently — an unset or unparseable value falls back to its default rather than failing the run. The variable names and defaults below are the published contract: RateLimitConfig::from_env() must read exactly these keys and apply exactly these defaults.

Examples

# More patient: up to 8 retries with a longer base window
AMPLIHACK_RATELIMIT_MAX_RETRIES=8 \
AMPLIHACK_RATELIMIT_BASE_DELAY_SECS=90 \
recipe-runner-rs build.yaml

# Aggressive cap so no single wait exceeds 2 minutes
AMPLIHACK_RATELIMIT_MAX_DELAY_SECS=120 \
recipe-runner-rs deploy.yaml

# Let Claude/Copilot fall back to --model auto on the last attempt
AMPLIHACK_RATELIMIT_FALLBACK_AUTO_MODEL=1 \
recipe-runner-rs long-task.yaml

# Disable backoff waits entirely (fast feedback in CI / tests)
AMPLIHACK_RATELIMIT_BASE_DELAY_SECS=0 \
recipe-runner-rs smoke.yaml

Behavior summary

FAQ

Does this retry every failing step? No. Only nonzero-exit failures whose captured diagnostics match a rate-limit signal are eligible. Codex stdin, final-file, decoding, and cleanup failures are not retried merely because diagnostics contain rate-limit text. All other non-zero exits fail immediately, exactly as before.

Can it loop forever? No. Executions are bounded to 1 + max_retries, and max_retries is clamped to a hard ceiling. A persistent rate limit always terminates with a clear error.

Does it parse the exact reset time from the message? No. The message only hints “under a minute,” so the runner uses the configured base_delay with exponential backoff instead of fragile text parsing.

Will it change my model without asking? Only for Claude/Copilot if you opt in with AMPLIHACK_RATELIMIT_FALLBACK_AUTO_MODEL, and only on the final retry attempt. Codex never receives an implicit auto model.

Tutorial Examples

Progressive tutorials that teach one recipe runner feature at a time. Each tutorial is a self-contained YAML recipe you can run directly.

Source: examples/tutorials/

Tutorials

Recommended Order

Start with 01-hello-world and work through sequentially. Each tutorial builds on concepts from previous ones.

Workflow Pattern Examples

Real-world workflow patterns that show how to compose recipe runner features for common development scenarios.

Source: examples/patterns/

Patterns

Combining Patterns

Patterns compose via sub-recipe steps, hooks, tags, and parallel groups. Here’s a full deployment recipe that chains three patterns together — CI first, then review, then deploy — with quality audit as a gate between stages:

name: "ship-release"
description: "CI → Review → Quality Gate → Deploy"
version: "1.0"

context:
  repo_path: "."
  environment: "staging"

hooks:
  on_error: "echo 'Pipeline failed at step: $STEP_ID' >> pipeline.log"

steps:
  # ── Stage 1: Build & Test (sub-recipe) ──
  - id: "ci"
    recipe: "ci-pipeline"
    context:
      repo_path: "{{repo_path}}"
    output: "ci_result"

  # ── Stage 2: Parallel code reviews ──
  - id: "security-review"
    agent: "amplihack:security"
    parallel_group: "reviews"
    prompt: "Review {{repo_path}} for security vulnerabilities."
    output: "security_findings"

  - id: "architecture-review"
    agent: "amplihack:architect"
    parallel_group: "reviews"
    prompt: "Review {{repo_path}} for architectural issues."
    output: "arch_findings"

  # ── Stage 3: Quality gate (sub-recipe, conditional) ──
  - id: "quality-gate"
    recipe: "quality-audit"
    condition: "ci_result and 'PASS' in ci_result"
    context:
      repo_path: "{{repo_path}}"
    output: "audit_result"

  # ── Stage 4: Deploy (tagged — only runs with --include-tags release) ──
  - id: "deploy"
    recipe: "deploy-pipeline"
    when_tags: ["release"]
    condition: "'PASS' in audit_result"
    context:
      repo_path: "{{repo_path}}"
      environment: "{{environment}}"
    output: "deploy_result"

  # ── Notification ──
  - id: "notify"
    command: |
      echo "Release pipeline complete."
      echo "CI: {{ci_result}}"
      echo "Audit: {{audit_result}}"
      echo "Deploy: {{deploy_result}}"

This recipe demonstrates:

• Sub-recipes (recipe:) — CI, quality audit, and deploy each run as self-contained workflows
• Parallel groups (parallel_group:) — security and architecture reviews run concurrently
• Conditional gates (condition:) — quality audit only runs if CI passed; deploy only if audit passed
• Tag filtering (when_tags:) — deploy step only executes when --include-tags release is passed
• Error hooks (hooks.on_error:) — logs which step failed for post-mortem
• Output chaining — each stage’s result flows into the next stage’s conditions

Testing & Edge Cases

title: Testing and edge-case recipes description: Regression recipe catalog and Codex verification coverage. doc_type: reference last_updated: 2026-10-07

Testing & Edge-Case Recipes

Recipes designed to exercise specific recipe runner features and edge cases. Useful as regression tests and as references for condition syntax.

Source: recipes/testing/

Recipes

Codex verification

The Codex agent-step contract requires both deterministic regression tests and production integration evidence. The following checks guide acceptance; deterministic fixtures alone do not establish production completion.

Use an isolated fake launcher via AMPLIHACK_LAUNCHER_BINARY for deterministic coverage. Have it record argv, stdin and cwd, emit distinct progress diagnostics, and write controlled final files. Keep fixtures and output under a private mktemp -d directory and clean it up after the checks.

Cleanup-only fixtures compose the production cleanup error and assert that its chain contains CleanupFailure and excludes Interruption. Call counters prove that later steps, recovery, hooks, and repair never execute. Combined interruption/cleanup coverage alone cannot establish this contract. Parallel fixtures acknowledge the worker’s classification boundary before releasing an in-flight agent; bounded channel waits detect deadlocks without timing sleeps.

Test locations and discovery

tests/codex_exec_tests.rs remains the integration-test target and retains the root adapter_worker subprocess entrypoint selected by --exact adapter_worker. Its scenarios share infrastructure under tests/codex_exec/:

The target and every extracted module, including fixtures, contain at most 300 physical lines. Extraction retains every case, assertion, macro expansion, platform gate, and worker selector. Discovery comparison maps changed module qualifications; equal test counts alone do not establish preservation.

Private cleanup-only runner regressions live in src/adapters/codex_exec/tests/runner_cleanup/: shared fixtures.rs, sequencing_hooks.rs, nested_recovery.rs, json.rs, and parallel.rs. Private lifecycle tests remain in src/adapters/codex_exec/tests/. tests/codex_capabilities_tests.rs covers the standalone probe, and tests/codex_retry_context_tests.rs covers repair context. Existing adapter and runner suites retain ordinary provider/error-policy controls. No model API call is needed for deterministic checks.

Lifecycle discrimination and evidence

The lifecycle reference defines group authority and observable cancellation retirement. Acceptance uses identical production-path semantic assertions on an immutable baseline and corrected source. Both epochs must compile successfully before the selected assertion is evaluated. Compile failure, zero selection, timeout, cached artifacts or an older defect reproduction cannot substitute for semantic RED/GREEN.

Private lifecycle fixtures are organized under src/adapters/codex_exec/tests/: launcher_retirement.rs, group_authority.rs, anchor_lifecycle.rs, anchor_descriptors.rs, retirement_signals.rs, retirement_concurrency.rs, retirement_failures.rs, retirement_fixtures.rs and lifecycle_model.rs. Observational adaptations are transparent and identical between source epochs; they gate real syscall, entry-publication and retirement boundaries without changing causal control flow or classification. Tests never simulate signal receipt by calling the handler or setting cancellation state. Kernel selection before observable entry is separately qualified, not advertised as synchronized.

Preservation inventory

Retain and execute all 835 baseline controls, at least 51 cleanup regressions, the 29 original integration cases and 58 original assertion sites, 29-case discovery, the unchanged root --exact adapter_worker selector/body and both existing eight-variant SIGINT/SIGTERM matrices. Preserve extracted modules and test targets. Every new source, test and helper module has at most 300 physical lines. An aggregate pass count cannot replace identity/assertion comparison.

The 50-worker oracle checks each admitted-0 through admitted-47 completion and its attributed result exactly once, plus the held and cleanup identities. A deterministic two-worker barrier proves distinct admission. Pure cleanup terminal publication occurs before pending work and joins; already admitted workers finish and are joined. Ordinary-error controls retain Claude/Copilot, recovery, hook and JSON policies.

Descriptor capacity and endpoint privacy

The authority fixture chooses min(actual_hard_capacity, 65537) - 1, requires that descriptor to exceed the deliberately lowered 4096 limit, and raises only its isolated worker’s soft capacity when needed. It verifies the descriptor is open and inheritable before spawn and unchanged after lowering the soft limit. Both authority contracts run under child-only initial soft 65536 and under child-only soft/hard 65536; parent limits remain identical. Capable hosts retain FD 65536 stress; hard 65536 uses FD 65535. Allocation/capacity failures are test failures, never skips. Isolation, retained membership after launcher reap, successful delivery/cleanup and final group absence remain required in both.

Portable-endpoint controls exec a real concurrent launcher after each open, verify private descriptor identities do not survive exec, and observe EOF while that launcher remains alive. A separate actual parent-SIGKILL control requires an untimed helper to exit through EOF and verifies the unrelated launcher survives; owned orphans are adopted and reaped. Partial construction must close all descriptors and remove temporary paths. Linux execution of this constructor is mechanism evidence; native Darwin remains a separate validation obligation.

Fixture stdout/stderr and case-required events must exist and decode as UTF-8. Only explicitly optional files may be missing (NotFound), with absence printed separately from empty contents. Other read/hash/decode errors include operation and path and fail visibly alongside the worker exit. When evidence preservation is requested, keep and identify the directory before reading, hashing or reporting failures. Invalid UTF-8 and directory-as-file controls verify this without relying on permission errors that disappear under root.

Evidence bindings

Each oracle records the actual command, allowlisted nonsecret environment, resolved compiler/tool paths, executable SHA256 and versions, genuine compile exit, selected-test count, semantic result and process exit. Bind immutable source, fixtures, compiled binaries and logs by path and SHA256. Preserve failed attempts. Historical missing compiler identity remains qualified; measuring a current compiler does not repair earlier records.

Separate modeled policy coverage, native Linux fixture execution, target compilation and native platform runtime. Conditional Apple UNIX03 source inference is not a deployed Darwin ABI or macOS runtime result. Ubuntu-only runner CI and another repository’s Darwin CI do not verify new runner Darwin bindings or helper behavior. Controlled adapter-worker evidence does not prove live model execution or actual recipe continuation.

Focused and full suites, formatting, warnings-denied all-target Clippy and pre-commit are acceptance checks. Their results belong in external logs/CI, not this reference. Finding closure additionally requires independent review and complete workflow/publication evidence; passing tests alone is insufficient.

Production verification

Later production verification must use the updated production Amplihack launcher and an absolute real Codex executable, without a bootstrap adapter. First inspect codex --version, codex exec --help, and recipe-runner-rs --capabilities. Then run a minimal recipe with auto_stage: false in a disposable workspace; verify stdin passthrough, final-response-only completion, failure propagation, and launcher process containment. Record sanitized commands, resolved executable paths, versions and the tested immutable runner revision. A help/probe check alone does not prove production task completion. Use idempotent tasks: retries may repeat side effects, and each attempt’s timeout plus backoff increases total time.

Production Recipes

These recipes ship with amplihack and demonstrate real-world workflow patterns at scale.

Source: amplifier-bundle/recipes/

Development Workflows

Quality & Reliability

Multi-Agent Decision Making

Orchestration

Migration

Architecture

title: Recipe runner architecture description: Module boundaries, execution flow and subprocess ownership. doc_type: explanation last_updated: 2026-10-07

Architecture — amplihack-recipe-runner

Rust implementation of the amplihack recipe runner. Parses YAML recipe files, evaluates conditions in a sandboxed expression language, and executes steps (bash commands, AI agent prompts, or nested sub-recipes) through a pluggable adapter layer.

Contents

• Module dependency diagram
• Data flow
• Core types
• CLI interface
• Adapter pattern
• Execution flow
• Safety model
• Interior mutability
• Recipe discovery
• JSONL audit log
• JSON output extraction

Module Dependency Diagram

┌─────────────────────────────────────────────────────────────────────┐
│                           main.rs (CLI)                            │
│  clap args → parse → build runner → execute → format output        │
└──────┬──────────┬───────────┬──────────┬───────────┬───────────────┘
       │          │           │          │           │
       ▼          ▼           ▼          ▼           ▼
   parser.rs  runner.rs  discovery.rs  adapters/  models.rs
       │       │  │  │       │        cli_subprocess.rs
       │       │  │  │       │              │
       │       │  │  └───────┘              │
       │       │  │                         │
       │       ▼  ▼                         │
       │  context.rs  agent_resolver.rs     │
       │       │                            │
       └───────┴────────────────────────────┘
                  models.rs (shared types)

graph TD
    main[main.rs — CLI] --> parser[parser.rs]
    main --> runner[runner.rs]
    main --> discovery[discovery.rs]
    main --> cli_sub[cli_subprocess.rs]

    lib[lib.rs — Public API] --> parser
    lib --> runner
    lib --> discovery

    runner --> context[context.rs]
    runner --> agent_resolver[agent_resolver.rs]
    runner --> discovery
    runner --> adapters[adapters/mod.rs — Adapter trait]

    cli_sub --> adapters
    cli_sub --> codex_exec[codex_exec/mod.rs — private attempt coordinator]

    parser --> models[models.rs]
    runner --> models
    context --> models
    discovery --> models
    main --> models
    lib --> models

Module Roles at a Glance

Data Flow

  YAML file
      │
      ▼
 ┌──────────┐   file size check    ┌────────────┐
 │ parser.rs │ ──────────────────► │ serde_norway │
 └──────────┘   MAX_YAML_SIZE 1MB  │ deserialize │
      │                            └─────┬──────┘
      │  validate: name, steps,          │
      │  unique IDs, field typos         ▼
      │                           Recipe (models.rs)
      ▼
 ┌──────────┐   merge recipe.context
 │ runner.rs │   + user overrides (--set)
 └──────────┘
      │
      │  for each step:
      │    1. Tag filter (when_tags vs active/exclude)
      │    2. Condition evaluation (context.evaluate)
      │    3. Template rendering (context.render / render_shell)
      │    4. Dispatch: Bash │ Agent │ Sub-Recipe
      │    5. Optional JSON parse of output
      │    6. Store output in context
      │    7. Write JSONL audit entry
      │    8. Run post_step / on_error hook
      │
      ▼
 RecipeResult
   ├── success: bool
   ├── step_results: Vec<StepResult>
   ├── context: final variable state
   └── duration: wall-clock time

Early Capability Dispatch

main.rs handles exact standalone --capabilities before logger initialization and update checks. It emits the frozen schema/version/capability JSON; mixed invocations fail before execution or update effects. See the probe contract.

Parse Phase

1. RecipeParser::parse_file reads the file and rejects anything over 1 MB (YAML bomb protection).
2. serde_norway (imported as serde_yaml for source compatibility) deserializes into Recipe. Step fields like command, agent, prompt, and recipe determine the implicit StepType via Step::effective_type().
3. Structural validation: name must be non-empty, at least one step required, step IDs must be unique.
4. Unknown step fields are reported, never rejected. Once the size, anchor and structural checks pass, parse() re-reads the YAML as a serde_yaml::Value and checks each step’s keys against KNOWN_STEP_FIELDS. It emits one warning per unknown key (recipe '<name>': step '<id>' has unknown field '<field>' (ignored)) and a specific “retired” message for max_env_value_bytes. validate_with_yaml (used by --validate-only) uses the same step_field_warnings helper, so both paths produce the same text. --validate-only and lib::validate_recipe build the parser with without_field_warnings(), so nothing is reported twice. See Unknown and retired step fields.

Execute Phase

RecipeRunner::execute merges the recipe’s context map with any user-supplied --set KEY=VALUE overrides, then iterates steps sequentially:

1. Tag filter — should_skip_by_tags checks when_tags against active_tags / exclude_tags.
2. Condition — RecipeContext::evaluate runs a sandboxed boolean expression (see Safety Model).
3. Dispatch — routes to execute_bash_step, execute_agent_step, or execute_sub_recipe on the adapter.
4. Output capture — if parse_json is set, the runner tries three extraction strategies (direct parse → markdown fence → balanced brackets), with an optional retry that re-prompts the agent for JSON-only output.
5. Context update — step output is stored under step.output (or step.id) in the context for downstream templates.
6. Hooks — pre_step runs before dispatch, post_step after success, on_error after failure. Hook commands are rendered through the context.
7. Audit — each step result is appended to a JSONL file (<audit_dir>/<recipe>_<timestamp>.jsonl).

Core Types (models.rs)

Step

#![allow(unused)]
fn main() {
struct Step {
    id:                String,
    command:           Option<String>,       // Bash step
    agent:             Option<String>,       // Agent reference
    prompt:            Option<String>,       // Agent prompt
    recipe:            Option<String>,       // Sub-recipe name
    output:            Option<String>,       // Context variable for result
    condition:         Option<String>,       // Boolean expression
    parse_json:        Option<bool>,         // Auto-parse output as JSON
    mode:              Option<String>,       // Execution mode
    working_dir:       Option<String>,       // Override cwd
    timeout:           Option<u64>,          // Seconds
    auto_stage:        Option<bool>,         // git add -A after agent steps
    continue_on_error: Option<bool>,         // Don't fail-fast
    when_tags:         Option<Vec<String>>,  // Tag-based filtering
    parallel_group:    Option<String>,       // Concurrent step grouping (fully implemented)
    sub_context:       Option<HashMap<…>>,   // Context overrides for sub-recipe
}
}

Step::effective_type() infers the step type from which fields are present: recipe → Recipe, agent/prompt → Agent, command → Bash.

Recipe

#![allow(unused)]
fn main() {
struct Recipe {
    name:        String,
    version:     Option<String>,
    description: Option<String>,
    author:      Option<String>,
    tags:        Option<Vec<String>>,
    context:     Option<HashMap<String, Value>>,
    steps:       Vec<Step>,
    recursion:   Option<RecursionConfig>,   // max_depth (6), max_total_steps (200)
    hooks:       Option<RecipeHooks>,       // pre_step, post_step, on_error
    extends:     Option<String>,            // Parent recipe (inheritance)
}
}

Result Types

#![allow(unused)]
fn main() {
struct StepResult {
    step_id:  String,
    status:   StepStatus,   // Pending | Running | Completed | Skipped | Failed
    output:   Option<String>,
    error:    Option<String>,
    duration: Duration,
}

struct RecipeResult {
    recipe_name:  String,
    success:      bool,
    step_results: Vec<StepResult>,
    context:      HashMap<String, Value>,   // Final state (skipped in JSON serialization)
    duration:     Duration,
}
}

CLI Interface (main.rs)

recipe-runner-rs [OPTIONS] [RECIPE] [COMMAND]

Commands:
  list   List discovered recipes

Arguments:
  [RECIPE]   Path to a .yaml recipe file

Options:
  -C, --working-dir <DIR>        Working directory (default: ".")
  -R, --recipe-dir <DIR>         Additional recipe search directories (repeatable)
      --set <KEY=VALUE>           Context variable overrides (repeatable)
      --dry-run                   Log steps without executing
      --validate-only             Parse and validate, then exit
      --explain                   Print step plan without executing
      --progress                  Emit progress to stderr (StderrListener)
      --include-tags <TAGS>       Only run steps matching these tags (comma-separated)
      --exclude-tags <TAGS>       Skip steps matching these tags (comma-separated)
      --audit-dir <DIR>           Directory for JSONL audit logs
      --output-format <FMT>      Output format: text (default) or json

--set values are auto-typed: JSON objects/arrays are parsed as-is, true/false become booleans, numeric strings become numbers, everything else stays a string.

Adapter Pattern

The Adapter trait decouples the runner from any specific execution backend:

#![allow(unused)]
fn main() {
pub trait Adapter: Sync {
    fn execute_agent_step(
        &self,
        prompt: &str,
        agent_name: Option<&str>,
        system_prompt: Option<&str>,
        mode: Option<&str>,
        working_dir: &str,
        model: Option<&str>,
        timeout: Option<u64>,
    ) -> Result<String, anyhow::Error>;

    fn execute_bash_step(
        &self,
        command: &str,
        working_dir: &str,
        timeout: Option<u64>,
        extra_env: &std::collections::HashMap<String, String>,
    ) -> Result<String, anyhow::Error>;

    fn is_available(&self) -> bool;
    fn name(&self) -> &str;
}
}

CLISubprocessAdapter

The production adapter spawns subprocesses:

• Bash steps — <bash> -c <command>, or <bash> <script-file> when the command exceeds 64 KiB, optionally wrapped with timeout. <bash> is a single absolute path resolved in the parent process (see Bash interpreter resolution). Lifecycle hooks (pre_step, post_step, on_error) take the same path, with a fixed 30 s timeout.
• Agent steps — dispatch by selected provider through amplihack. Claude/Copilot retain their existing transport and result contracts. Codex uses amplihack codex -- exec --output-last-message UNIQUE_FINAL_PATH [--model EXPLICIT_MODEL] - in the effective workspace, without implicit --add-dir or model selection. Full system/persona, leaf/no-reentry, task and autonomy instructions are delivered through stdin for every prompt size.

Private Codex execution ownership

The private lifecycle modules implement the ownership and retirement contract described below.

cli_subprocess.rs constructs the full stdin envelope and typed command arguments, allocates a fresh private temporary directory for each attempt, and owns retry policy and resource removal. Temporary resources use the platform temporary location, honoring TMPDIR; the directory is private and the final pathname is absent before launch. There is no additional public adapter API.

The private adapters/codex_exec/ modules divide the attempt by responsibility:

• mod.rs coordinates spawn, delivery/wait, teardown and result classification. It composes primary, cleanup and immutable cancellation-retirement results, retaining both downcastable Interruption and CleanupFailure markers.
• cancellation.rs owns reference-counted SIGINT/SIGTERM registration. The first active owner saves full prior dispositions and captures a baseline before installation; concurrent owners share it. Nonlast close snapshots under the ownership mutex. Last close attempts both restorations and constructs an immutable observation/restoration result before unlocking. Failed restoration retains original actions and pending metadata for bounded reconciliation or refusal before a later spawn. The scope spans retries and backoff.
• signal_observations.rs publishes observable handler entry through persistent, target-proven lock-free per-signal saturating sequences, never reset between owners. Handlers perform no blocking, allocation or epoch-dependent later writes. Published entered/in-flight events during restoration are covered; a kernel-selected old handler first publishing after the retirement snapshot is outside demonstrated epoch attribution. Saturation is a typed terminal fault.
• process.rs owns the launcher, nonblocking complete stdin delivery and one absolute attempt deadline computed immediately after successful spawn. Anchor and reader setup, delivery and execution consume that same deadline. Owned anchor and launcher inspection errors immediately acquire typed CleanupFailure context, retaining their original error chains. Successful later cleanup cannot make these failures nonfatal or admit pending work.
• group_anchor.rs, anchor_io.rs, anchor_fifo.rs and anchor_child.rs establish private retained membership in the original Unix group before any launcher polling can reap it. Readiness requires descriptor isolation and child-local signal policy. The non-cloneable anchor holds group authority through the final destructive call; a numeric PGID alone never authorizes signaling.
• launcher_cleanup.rs retires the direct launcher before anchored group TERM and KILL. Initial reap, direct TERM with at most 100 ms grace, direct KILL if still owned/live and PID-specific reap share cleanup start plus two seconds. Group TERM has at most 100 ms grace; TERM/probe errors do not skip the final KILL while authority remains. That KILL attempt seals destructive group access, including on failure. Helper fallback/reap, consuming launcher observation and final group confirmation share the deadline computed immediately before the KILL attempt (plus two seconds). After helper retirement, PID-specific launcher polling consumes any exit caused by late group termination and retains earlier direct-signal or timeout errors. Exhausted budgets still permit one nonblocking consuming observation without a new wait window. This observation never signals. EINTR, fallback and Drop never renew these deadlines.
• diagnostics.rs drains both pipes concurrently, retaining 64 KiB tails. Stopped readers finish within a 50 ms or 1 MiB additional-read bound. Exit classification uses fixed actionable categories, including late stderr rate-limit evidence, without exposing raw provider output or credentials.
• final_output.rs validates the opened descriptor using no-follow and nonblocking flags, regular-file type, ownership and permissions. Metadata checks and a limit-plus-one read enforce the 10,000,000-byte output bound before unbounded allocation. Strict UTF-8 decoding preserves all content, including an empty message and whitespace.

After spawn, every outcome attempts bounded cleanup: close stdin, retire the launcher, tear down the original group while authority remains, seal signaling, retire/reap the helper, observe the group and consume launcher status within the same final deadline, then stop and join diagnostic readers. Lost authority prohibits uncertain direct/group signals; cleanup continues for positively owned resources and aggregates genuine signal, probe, reap, reader and resource failures. Complete reaping and group absence are success conditions, not guarantees after cleanup failure. Later absence cannot erase an earlier genuine failure. Final-file extraction requires complete stdin delivery, successful process exit and successful lifecycle cleanup. The adapter explicitly closes temporary resources before returning; removal failure is terminal. Exit zero or an existing final file alone cannot establish success; progress output never substitutes for a missing final message, and oversized output fails rather than being truncated.

Typed cancellation takes precedence while retaining both terminal markers and primary diagnostics when cleanup also fails. Cancellation and genuine cleanup failure stop pending dispatch before joins; already admitted work is joined. Both stop the enclosing recipe despite continue_on_error or fatal: false, including nested execution, parallel scheduling and JSON repair. Recorded interruption prevents agentic recovery from starting; interruption during recovery aborts the enclosing recipe. No rate-limit retry, automatic model fallback or JSON repair runs after cancellation. Already running parallel Bash steps retain their existing join behavior. Ordinary failures retain the established recovery and nonfatal policies.

Timeout enforcement: Codex’s per-attempt deadline includes anchor and reader setup, stdin delivery and execution. Establishment is capped at 100 ms and the remaining attempt time. Cleanup uses shared absolute deadlines with at most 4.1 seconds of process polling plus existing reader/resource bounds; synchronous spawn, syscall scheduling and resource I/O have no hard real-time guarantee. Timeout attempts owned-group and I/O cleanup before returning; authority loss or incomplete cleanup remains typed CleanupFailure. Deliberately detached sessions are outside Unix process-group containment. Timed execution without equivalent platform tree cleanup fails explicitly. Backoff and repeated attempts extend total duration and may repeat side effects. Rate-limit and JSON repair retries retain the full execution context and use fresh output resources; Codex is excluded from implicit --model auto fallback. Genuine cleanup failures permit no retry, JSON repair, recovery or nonfatal continuation. See Codex agent steps and the lifecycle reference for helper isolation, reaper assumptions and the unverified Darwin/other Unix scope.

Environment propagation: build_child_env() forwards session-tracking variables (AMPLIHACK_SESSION_DEPTH, AMPLIHACK_TREE_ID, AMPLIHACK_MAX_DEPTH, AMPLIHACK_MAX_SESSIONS) and strips CLAUDECODE to prevent nested session confusion. It also drops any inherited variable whose whole KEY=value\0 string exceeds MAX_ARG_STRLEN (131072 bytes), the kernel’s per-string limit, and logs a warning for each one, naming it and giving its pair size. For bash steps, env_spill::spill_for_bash() then moves oversized and over-budget recipe values into private files and prepends a preamble that reads them back. See Per-value env spill.

Bash interpreter resolution

Bash steps do not hardcode an interpreter. resolve_bash_interpreter() picks one absolute path at the top of execute_bash_step, before the step’s child environment is built and before any temporary script file is created:

1. AMPLIHACK_BASH, if set and non-empty — validated, never silently bypassed.
2. Otherwise the first executable bash under an absolute PATH entry, in PATH order.
3. Otherwise /bin/bash.

Validation is metadata() (symlinks followed) plus is_file() and the owner/group/other execute bits. A set-but-unusable AMPLIHACK_BASH — not valid UTF-8, relative, missing, a directory, unreadable, or not executable — aborts the step with a single error string naming the variable, the rejected value, a fixed reason, and the remedy. Note that “set” is decided on NotPresent alone: a value that is present but not decodable is a rejection, never an absence. It does not fall through to rules 2 or 3; see CLI Reference for the operator-facing contract.

The resolved path and the rule tag are logged once per bash-step execution, including lifecycle hooks — resolution is per-call and uncached, and hooks are themselves bash steps, so a recipe with both pre_step and post_step emits up to three lines per step. It is one line per bash-step execution, not one per process. The line is emitted at info. (main.rs calls env_logger::init() with no default filter, so it needs RUST_LOG set to be shown.) The line and the error text carry the resolved or rejected path plus the rule/reason and nothing else — never PATH, never the candidate list, which routinely carry project and username identifiers in directory names.

There is deliberately no version probing. PATH order is the operator’s stated preference, and AMPLIHACK_BASH covers the override case.

Why it resolves first

That ordering is load-bearing, not an optimisation. Resolution reads the parent process environment (std::env::var) and must never read the child map. build_child_env() runs further down and then merges extra_env — step env: blocks and RECIPE_VAR_* values — over the top, so a recipe can put AMPLIHACK_BASH and PATH into the child environment. Resolving above that merge makes it structurally impossible for recipe YAML to choose the interpreter. The cheap side benefits follow from the same placement: a misconfigured AMPLIHACK_BASH aborts before any env-budget computation and before the >64 KiB tempfile spill, so it costs nothing and leaves nothing behind, and the interpreter log line lands next to the step-entry line an operator is reading.

The validation predicate is a fail-fast diagnostic, not a security control. Nothing prevents the file from changing between metadata() and execvp; the check exists to turn a confusing exec failure into a message that names the cause. Absoluteness is the one rule that is a hardening measure: in the timeout arms the interpreter is argv[2], an argument to timeout, so a leading - would be parsed as a timeout option. Requiring / makes that unrepresentable. There is no lexical or metacharacter validation beyond that — Command execs a real argv vector with no shell in between, so ;, spaces and $(…) in a path are inert bytes, and a whitelist would reject legitimate paths (Nix store hashes, spaces) while adding nothing.

Why the parent resolves it

Rust’s Command installs the child environment into environ before calling execvp, so a bare program name would be searched against the child’s PATH — the env_clear()ed, bounded_env-filtered one. A timeout-wrapped step is worse still: timeout runs its own execvp on its first argument, a second search mechanism against a second PATH. Resolving to an absolute path in the parent makes the question vacuous, because execvp never searches PATH for an argument containing /.

That guarantee only holds if there is exactly one place the interpreter can enter the argument vector, so all four shapes are produced by one pure builder and spawned from one call site:

argv is built as OsStrings, so a script path that is not valid UTF-8 is passed through intact rather than being lossily collapsed.

Resolution runs per step, not once per process: a handful of stat calls is noise next to fork/exec, and caching would make AMPLIHACK_BASH read-once in a way that is harder to operate and to test.

timeout itself is still located by name against the child’s PATH. The absolute-interpreter guarantee therefore describes what argv contains, not necessarily what executes, in the two timeout arms. build_child_env forces a non-empty PATH and is_env_protected keeps it undroppable, but extra_env is merged afterwards and can override it — non-empty is not the same as trustworthy. Net risk is nil, since the step body is already arbitrary bash. Widening the fix to the timeout binary is out of scope here and tracked as #145.

Execution Flow

Lifecycle of a Recipe Run

CLI args
  │
  ├─ standalone --capabilities ──► frozen JSON probe ──► exit
  │   (before logging, updates, cache writes, network or agent startup)
  ├─ --validate-only ──► parse + validate ──► print warnings ──► exit
  ├─ --explain ─────────► parse ──► print step plan ──► exit
  │
  ▼
RecipeRunner::execute(recipe, user_context)
  │
  ├─ Check recursion limits (depth ≤ max_depth, total_steps ≤ max_total_steps)
  ├─ Merge recipe.context + user_context
  ├─ Open JSONL audit log (if --audit-dir set)
  │
  │  ┌─── for each step ──────────────────────────────────────────┐
  │  │                                                             │
  │  │  1. should_skip_by_tags(step) ──► skip if filtered out      │
  │  │  2. run_hook(pre_step)                                      │
  │  │  3. evaluate condition ──► Skipped if false                 │
  │  │  4. render templates in command/prompt                      │
  │  │  5. dispatch:                                               │
  │  │     ├─ Bash  → adapter.execute_bash_step()                  │
  │  │     ├─ Agent → resolve agent, adapter.execute_agent_step()  │
  │  │     └─ Recipe → execute_sub_recipe() (recursive)            │
  │  │  6. parse JSON output (if parse_json, with retry)           │
  │  │  7. store output in context                                 │
  │  │  8. maybe_auto_stage (git add -A for agent steps)           │
  │  │  9. run_hook(post_step) or run_hook(on_error)               │
  │  │ 10. write JSONL audit entry                                 │
  │  │ 11. fail-fast unless continue_on_error                      │
  │  │                                                             │
  │  └─────────────────────────────────────────────────────────────┘
  │
  ▼
RecipeResult { success, step_results, context, duration }

Sub-Recipe Execution

When a step has step_type: Recipe:

1. The runner searches for the recipe file using discovery::find_recipe across recipe_search_dirs, then falls back to a direct path relative to working_dir.
2. Recursion depth is checked against RecursionConfig::max_depth (default 6). total_steps is checked against max_total_steps (default 200).
3. The sub-recipe’s context inherits from the parent context, merged with any sub_context overrides defined on the step.
4. A new execute_with_depth(recipe, context, depth + 1) call runs the sub-recipe. Depth and total-step counters are tracked via Cell<u32>.
5. After execution, the sub-recipe’s final context is propagated back into the parent context.

Hooks

Defined in RecipeHooks:

hooks:
  pre_step: "echo 'Starting step {{step_id}}'"
  post_step: "echo 'Completed step {{step_id}}'"
  on_error: "notify-send 'Step {{step_id}} failed'"

Hooks are shell commands rendered through the context. pre_step runs before every step dispatch. post_step runs after a successful step. on_error runs after a failed step. Hook failures are logged but do not abort the recipe.

Execution Listeners

The ExecutionListener trait provides real-time progress callbacks:

#![allow(unused)]
fn main() {
trait ExecutionListener {
    fn on_step_start(&self, step_id: &str, step_type: &str);
    fn on_step_complete(&self, result: &StepResult);
    fn on_output(&self, step_id: &str, line: &str);
}
}

Activated with --progress.

Safety Model

Condition Evaluator (context.rs)

The condition evaluator is a hand-written recursive descent parser that evaluates boolean expressions over recipe context variables. It does not call eval() or execute arbitrary code.

Supported syntax:

status == "ok" and (retries < 3 or force == true)
len(items) > 0
name.startswith("test_")
value not in "blocked,disabled"

Operator precedence (low → high): or, and, not, comparison (==, !=, <, <=, >, >=, in, not in).

Security constraints:

The tokenizer produces typed tokens (String, Number, Ident, Eq, And, Or, …) and the parser consumes them with lookahead. Unrecognized tokens produce a parse error rather than silent misbehavior.

Template Rendering

RecipeContext::render replaces {{var}} placeholders with values from the context. Dot-notation ({{obj.nested.key}}) traverses into JSON objects. Missing variables render as empty strings.

RecipeContext::render_shell does the same but shell-escapes every substituted value to prevent injection in bash commands.

Agent Resolver Path Safety (agent_resolver.rs)

Agent references use a namespaced format (namespace:category:name or namespace:name). Each segment is validated against:

#![allow(unused)]
fn main() {
static SAFE_NAME_RE: Regex = Regex::new(r"^[a-zA-Z0-9_-]+$");
}

This rejects /, .., and any characters that could enable path traversal.

As defense-in-depth, after resolving the file path, the resolver canonicalizes both the candidate path and the search base directory, then verifies the resolved path is a child of the search base. This defends against symlink attacks.

Parser Protections (parser.rs)

• File size limit: 1 MB (MAX_YAML_SIZE_BYTES). Prevents YAML bombs and memory exhaustion.
• Structural validation: Rejects empty names, zero-step recipes, and duplicate step IDs.
• Unknown field warnings: Unknown top-level and step-level fields trigger warnings and are otherwise ignored. No correction is suggested.

Subprocess Isolation (cli_subprocess.rs)

• Codex executes in the effective workspace; its private per-attempt output directory is owned through execution, extraction and explicit cleanup. Claude/Copilot retain their existing temporary-resource behavior.
• CLAUDECODE is stripped from the child environment to prevent the nested Claude process from attaching to the parent’s session.
• Session depth tracking (AMPLIHACK_SESSION_DEPTH) prevents runaway recursive spawning.

Codex holds concrete original-group identity through a private anchor before launcher reaping can release it. Teardown retires the launcher, terminates the anchored group, seals signaling, then reaps the helper before observational confirmation. This ordering avoids launcher-only zombie observations while preventing destructive access to an unrelated reused PGID. Genuine cleanup failures remain terminal even when the group eventually disappears.

Cancellation ownership and group authority are separate lifetimes. Persistent signal-entry publications survive shared-owner retirement; the last owner composes its immutable interruption/restoration result before unlocking. This covers observable entered handlers during restoration, with an explicit limit for kernel-selected handlers that have not published entry. Both typed terminal causes survive adapter composition, so worker classification stops pending dispatch before joins while ordinary provider failures keep existing policies.

The lifecycle reference defines deadlines, helper isolation, reaper assumptions and platform limitations; verification contracts define the discriminating evidence for those boundaries.

Interior Mutability Pattern

The runner tracks recursion state with std::cell::Cell<u32>:

#![allow(unused)]
fn main() {
struct RecipeRunner<A: Adapter> {
    // ...
    depth:       Cell<u32>,
    total_steps: Cell<u32>,
    // ...
}
}

Why Cell?

RecipeRunner::execute takes &self (shared reference) because the runner is logically immutable during a run — the adapter, working directory, tag filters, and listener never change. But recursion tracking requires mutating two counters.

Cell<u32> provides interior mutability for Copy types without runtime borrow-checking overhead (no RefCell needed). The runner is single-threaded, so Cell is sufficient and zero-cost.

Usage in Recursion

execute(&self, recipe, context)
    │
    ├─ self.depth.get() checked against max_depth
    ├─ self.total_steps.get() checked against max_total_steps
    │
    └─ execute_sub_recipe(&self, step, ctx)
         │
         ├─ self.depth.set(self.depth.get() + 1)
         ├─ execute_with_depth(&self, sub_recipe, ctx, new_depth)
         └─ self.total_steps.set(self.total_steps.get() + sub_step_count)

The RecursionConfig defaults (max_depth: 6, max_total_steps: 200) can be overridden per-recipe in the YAML:

recursion:
  max_depth: 3
  max_total_steps: 50

Recipe Discovery (discovery.rs)

Search Directories (default order)

1. ~/.amplihack/.claude/recipes
2. ./amplifier-bundle/recipes
3. ./src/amplihack/amplifier-bundle/recipes
4. ./.claude/recipes

Additional directories can be added with -R <dir> (repeatable).

Discovery Functions

Manifest & Upstream Sync

update_manifest writes _recipe_manifest.json — a map of filenames to their SHA-256 hashes (first 16 hex chars). check_upstream_changes diffs the current directory state against the manifest and reports new, modified, or deleted files.

sync_upstream adds a git remote, fetches, and diffs local recipes against the upstream branch, returning a summary of changes.

JSONL Audit Log

When --audit-dir is set, each recipe run produces a file:

<audit-dir>/<recipe-name>_<ISO-timestamp>.jsonl

Each line is a JSON object:

{"step_id": "build", "status": "Completed", "duration_ms": 1423, "error": null, "output_len": 256}

Audit logs enable post-hoc analysis of recipe execution without cluttering stdout.

JSON Output Extraction

When parse_json: true is set on a step, the runner extracts structured JSON from potentially noisy output using three strategies (tried in order):

1. Direct parse — serde_json::from_str(output). Works when the output is pure JSON.
2. Markdown fence — extracts content between ```json and ``` delimiters. Common in LLM output.
3. Balanced brackets — finds the first { or [ and matches it to its closing counterpart, counting nesting depth.

If all three fail, the runner optionally retries the agent step with a JSON-only reminder appended to the prompt, then re-applies the extraction pipeline.

Migration to 0.5.0

Version 0.5.0 switches recipe parsing from the archived serde_yaml package to the maintained serde_norway fork. YAML recipe syntax, CLI flags, and command behavior remain unchanged.

The Rust library’s public parser::ParseError::Yaml variant now contains serde_norway::Error. Most callers that handle the variant without naming its payload need no changes. If your code names or matches the old concrete serde_yaml::Error type, add serde_norway as a direct dependency and update that type path to serde_norway::Error.

The crate retains the serde_yaml dependency alias internally so existing parser and serializer call sites remain stable. The alias does not preserve the old external error type identity.

Migration to 0.6.0 (upcoming)

These changes ship in the next minor release, 0.6.0.

Version 0.6.0 changes how bash steps receive large recipe values, and how the parser reports unknown step fields. Most recipes need no changes. This page lists every breaking change and how to update.

For background, see Environment Budget & Safe Spawning.

AMPLIHACK_CONTEXT_FILE is no longer set

Before 0.6.0, when a step’s recipe values were too large in total for the environment, the runner wrote the whole context to a JSON file, set AMPLIHACK_CONTEXT_FILE to its path, and passed only four variables inline. Every other RECIPE_VAR_* was empty in that step.

From 0.6.0, every recipe variable is always available as $RECIPE_VAR_<name>. Oversized values are read back from private spill files before your command runs. Step execution never sets AMPLIHACK_CONTEXT_FILE.

A script that still reads the file breaks:

• With set -u, "$AMPLIHACK_CONTEXT_FILE" fails with AMPLIHACK_CONTEXT_FILE: unbound variable.
• Without set -u, the path is empty, so jq fails to open '', or the script quietly gets an empty value.

Before

- id: summarise
  command: |
    set -euo pipefail
    summary=$(jq -r '.report' "$AMPLIHACK_CONTEXT_FILE")
    printf '%s\n' "$summary" | head -n 20

After

Read the variable directly:

- id: summarise
  command: |
    set -euo pipefail
    summary=$RECIPE_VAR_report
    printf '%s\n' "$summary" | head -n 20

Nested values use __ as the separator, so .build.log becomes $RECIPE_VAR_build__log.

If a script parses a JSON-valued variable, pipe it in on stdin instead of exporting it. A spilled variable is shell-local, so it is not passed to child processes through the environment:

printf '%s' "$RECIPE_VAR_report" | jq -r '.summary'

Scripts that must run on both versions

if [ -n "${AMPLIHACK_CONTEXT_FILE:-}" ]; then
  summary=$(jq -r '.report' "$AMPLIHACK_CONTEXT_FILE")   # before 0.6.0
else
  summary=$RECIPE_VAR_report                              # 0.6.0 and later
fi

Unknown step field warnings have new wording

The parser now reports unknown step fields whenever it parses a recipe, not only under --validate-only. Each distinct warning is printed once per process. Unknown fields are still warnings, and --validate-only still exits 0. The step-field message changed:

Update any CI check or log filter that matches (possible typo) or index N in step-field warnings. The full rules (escaping, the 128-character limit, the 100-warning cap) are in Unknown and retired step fields.

The top-level field warning, Unrecognized top-level field '<field>' (possible typo), is unchanged.

If a recipe sets max_env_value_bytes, delete the field. The runner now keeps every environment string under the kernel limit without truncating values.

shell_env_for_step returns a HashMap

RecipeContext::shell_env_for_step() no longer returns a temp-file path, because no context file is written.

#![allow(unused)]
fn main() {
// Before 0.6.0
let (env, context_file): (HashMap<String, String>, Option<PathBuf>) =
    ctx.shell_env_for_step();
// ... run the step, then delete `context_file` if it is Some

// 0.6.0
let env: HashMap<String, String> = ctx.shell_env_for_step();
}

Remove any cleanup code for the returned path. If you need the context as a JSON file for your own purposes, RecipeContext::write_context_file() is still public.

RecipeParser is no longer a unit struct

RecipeParser now has a private field that controls the parse-time warnings. The bare expression RecipeParser no longer compiles:

#![allow(unused)]
fn main() {
// Before 0.6.0
let parser = RecipeParser;

// 0.6.0
let parser = RecipeParser::new();          // or RecipeParser::default()
}

To parse without printing step-field warnings, for example when you report them yourself:

#![allow(unused)]
fn main() {
let parser = RecipeParser::new().without_field_warnings();
}

Related

• Environment Budget & Safe Spawning: how spilling works.
• YAML Recipe Format: the warning rules.
• Migration to 0.5.0: the previous migration guide.

Codex Lifecycle

title: Codex lifecycle reference description: Private FIFO permissions, group authority, cleanup budgets and observable signal retirement contracts. doc_type: reference last_updated: 2026-10-07

Codex lifecycle reference

Each Codex attempt owns its launcher, original process group, private anchor, diagnostic readers, output resources and scoped cancellation registration. The Codex agent-step reference describes usage and public APIs. These lifecycle interfaces are private and add no recipe settings.

Contents

• Private interfaces
• Group authority
• Helper protocol and isolation
• Portable FIFO permissions
• Deadlines and cleanup order
• Failure classification
• Signal observation and retirement
• Platform and evidence scope

Private interfaces

Explicit close reports failures; close and Drop release each owner exactly once. Emergency Drop cannot return an error and logs retirement failures; ordinary completion explicitly closes resources and composes failures into the result. The public Adapter: Sync, result types, capability schema and provider/model policies follow the existing adapter contract.

Group authority

The launcher creates its own process group with launcher PID equal to PGID. Before any try_wait or other launcher reap, a private helper joins that same group and acknowledges readiness. Its retained membership keeps group identity owned through TERM and the final group KILL attempt, including when the launcher has already exited. Destructive operations consume phase-appropriate authority.

A numeric PGID, same UID, successful earlier reap, cached PID or unprotected membership snapshot is insufficient authority. Positive identities and checked signed conversions exclude zero, broadcast and caller-group targets. A reused numeric PGID cannot authorize signaling an unrelated group.

If helper establishment fails, the retained, positively owned launcher supplies authority for bounded startup unwind only while its identity remains held. If neither launcher nor anchor supplies authority, uncertain direct/group signals stop and incomplete cleanup remains typed CleanupFailure. Ordinary inherited descendants remain cleanup responsibilities; deliberately detached sessions remain outside process-group containment.

The mechanism assumes cooperative exclusive reaping of owned children. Explicit SIGCHLD SIG_IGN, SA_NOCLDWAIT and unknown competing custom reapers are rejected without changing global signal policy. Reaping uses PID-specific waits. Unexpected helper exit, control EOF, changed reaping policy, stolen child or ECHILD invalidates authority when retention cannot be established. Concurrent external signal-policy mutation or undetectable competing reapers are outside the cooperating-process guarantee.

Helper protocol and isolation

Readiness and control descriptors are allocated before fork. A fixed bounded acknowledgment confirms group join, descriptor isolation, child-local signal policy and lifetime setup before launcher polling or reap. Startup guards own every child and endpoint throughout unwind. The helper performs no model work.

Every endpoint is created close-on-exec, so a concurrent launcher cannot retain private protocol handles across exec. Linux uses pipe2(O_CLOEXEC | O_NONBLOCK). Other Unix targets use a directional FIFO. Its intended directory creation contract is specified in portable FIFO permissions. Create the FIFO with mode 0600, open read first and write second with O_CLOEXEC | O_NONBLOCK | O_NOFOLLOW, and verify type, owner, mode and descriptor identity. The constructor unlinks the FIFO, explicitly closes the directory descriptor and removes the directory before returning. Partial construction closes every acquired endpoint and composes filesystem/close failures with the original error. No keeper writer or read/write endpoint masks EOF on parent loss. Filesystem work finishes before fork; the helper still performs only raw operations afterward.

Apple’s open semantics and FIFO implementation support this construction. It uses mkfifo, rather than mkfifoat, which Apple’s SDK declarations restrict to macOS 13+. The other directory-relative operations are available within Rust’s macOS deployment floors (10.12 x86, 11.0 ARM); native SDK linking and runtime remain unverified. Linux exercises the identical FIFO constructor, including exec during construction and actual parent death on an untimed attempt. Those controls do not establish native Darwin behavior.

Only the two protocol endpoints remain open in the helper. Every other inherited descriptor is closed before readiness, including concurrent attempts’ pipe writers and credential-bearing handles. CLOEXEC alone is insufficient for a helper that does not exec. On Linux, checked raw close_range calls use flags zero over ranges excluding the retained endpoints; unavailable syscalls or closure errors refuse readiness. Darwin queries its own post-fork kernel descriptor-table bound through proc_pidinfo(PROC_PIDLISTFDS) and closes the complete bounded range, independent of lowered resource limits. The child opens no descriptors during that closure. A table exceeding 262,144 entries refuses readiness. This derives from the bound XNU table query and its direct syscall wrapper. Other Unix close-all paths remain unverified. An unavailable isolation mechanism is an establishment failure.

The complete post-fork child path uses validated async-signal-safe raw operations and _exit, with no allocation, mutexes, logging, unwinding, standard I/O, directory enumeration or destructors. Child-local masks/dispositions cover SIGINT, SIGTERM and SIGPIPE without changing parent policy; the helper installs local SIGINT/SIGTERM ignore dispositions before readiness. The helper survives TERM while authority is needed. A timed attempt supplies a finite helper lifetime ceiling derived from its attempt deadline plus maximum cleanup allowance. Untimed attempts retain bounded establishment/retirement and parent-loss control EOF; they have no fixed execution-duration ceiling. Expiry or EOF outside intentional retirement is a typed fault when detected. The inherited parent memory is not a memory sandbox.

Portable FIFO permissions

The private anchor_fifo::new() -> anyhow::Result<Pipe> factory configures tempfile::Builder::permissions(Permissions::from_mode(0o700)) before tempdir(). Both fixture roots passed to construct() use the same permission request before tempdir_in(). Directory privacy holds at creation, without a later chmod repair. Construction does not change the process umask.

The selected temporary parent follows Unix TMPDIR configuration, as described in the temporary-resource usage reference. The parent must be trusted and writable. Mode 0700 does not isolate malicious same-UID or privileged processes or establish hostile-TMPDIR protection. There is no recipe option, public FIFO API or provider-policy change. Ordinary Linux endpoint selection retains pipe2; Linux mechanism controls explicitly exercise the shared portable constructor.

Umask can remove requested permission bits but cannot add permissions. The required behavior is:

Exact directory/FIFO modes, types and effective-UID ownership remain mandatory, including for effective root. Owner-bit-restricted objects are rejected rather than widened. Other owner-bit-removing masks have no universal success guarantee. Each opened endpoint must match the FIFO device/inode observed through AT_SYMLINK_NOFOLLOW; directional, NONBLOCK, CLOEXEC and nofollow gates remain mandatory. The private construct(root, opened) callback observes endpoint-open boundaries; production supplies a no-op and performs filesystem work before fork.

Successful construction checks unlink, directory-descriptor close and directory removal before returning endpoints. Failure closes acquired endpoints and retains cleanup errors alongside the original diagnostic. A privacy refusal with clean teardown preserves its primary error; genuine cleanup failures remain typed CleanupFailure. Failure after launcher creation still invokes bounded owned startup unwind. Cancellation combined with cleanup retains both terminal causes.

Verification requires independent production-factory and fixture observations under all four common masks in disposable children, leaving the parent mask unchanged. Observations must record requested and actual creation modes, ownership and identity before names disappear. Controls must preserve readiness delivery, both partial-open boundaries, unlink-error composition, concurrent-exec isolation and actual parent-SIGKILL EOF on an untimed attempt. Internal integration covers FIFO-to-anchor-to-launcher ownership, natural helper exit/reaping, unrelated-launcher survival and final group absence. These contracts do not assert completed runtime or platform acceptance; see verification requirements. Step 13 acceptance, independent finding closure, current-head CI and publication remain pending. Native Darwin runtime and deployed-ABI verification remain unverified; Linux shared-constructor checks do not establish those guarantees.

Deadlines and cleanup order

The absolute attempt deadline is computed immediately after successful launcher spawn. Anchor setup, reader setup, delivery and execution consume that same deadline. Overflow and zero timeout refuse before launcher entry; representable budgets are measured again immediately after spawn. Synchronous spawn, syscall scheduling and resource I/O have no hard real-time guarantee.

Process polling totals at most 4.1 seconds plus existing reader/resource bounds. EINTR, fallback and Drop do not renew budget windows. Unlimited execution removes the attempt deadline, not helper startup/retirement bounds. Loss of authority overrides the usual final-KILL obligation: uncertain targets are never signaled.

Failure classification

Cleanup retains every genuine permission, direct/group TERM or KILL, probe, launcher/helper reap, reader, resource and surviving-descendant failure. A failed direct kill remains diagnostic even if the launcher is subsequently reaped. Successful final absence cannot erase an earlier genuine failure.

No blanket EPERM suppression exists. Neither errno nor absence alone establishes the transient case. Linux distinguishes live members from zombies through process metadata; other Unix targets require group absence after owned reaping.

Errors exclude prompts, credentials and raw model diagnostics. Genuine terminal failures permit no rate-limit retry, JSON repair, recovery, nonfatal continuation or degraded success. Ordinary errors retain their existing provider policies. The runner publishes a worker’s terminal classification before joins and pending dispatch; already admitted work is joined and retains its original attribution.

Signal observation and retirement

Static, target-proven lock-free per-signal sequences publish observable handler entry using saturating atomics. Publication and snapshots use SeqCst ordering. The handler preserves errno as required and performs no blocking, allocation, locking or later epoch-dependent writes. The sequences are never reset. Saturation is a permanent typed fault and refuses later installs before spawn.

The first owner captures immutable baselines before installation and saves both complete prior sigaction dispositions. Concurrent owners share those baselines. An observed change identifies cancellation and retains both signal identities when both occurred; the mechanism is not an exact accounting of kernel deliveries. An existing primary Interruption retains its representative signal; otherwise SIGINT represents a both-signal result, with both identities kept in diagnostics.

Registration rollback and partial restoration follow the same retained-metadata rule. A faulted epoch retains its last reported snapshot: a new publication before or during reconciliation refuses independent admission rather than being discarded with previously returned observations. Ownership overflow, poisoning and unsupported atomics fail with typed diagnostics; they do not manufacture an idle successful state.

The guarantee includes entries published while product handlers remain owned during restoration, even when the handler is paused in flight. SIGTERM remains covered after SIGINT restoration while its product handler is still installed. An independent next owner cannot clear an earlier owner’s immutable result. A post-close-only sample is insufficient.

A kernel-selected old handler that first publishes after the retirement snapshot is outside demonstrated epoch attribution. It can publish after a new owner has captured its baseline. Persistent atomics, the ownership mutex and sigaction are not a universal pre-user-entry kernel synchronization guarantee. This qualification does not exclude published entered/in-flight handler events.

Platform and evidence scope

The conditional Darwin zombie-only case derives from primary Apple source: explicit-group filtering in XNU, group departure during reaping and the UNIX03 kill wrapper. That source supports the conditional inference; it verifies neither a shipped Darwin ABI nor native macOS runtime behavior. Modeled tests establish policy semantics, not native Darwin execution.

Runner CI is Ubuntu-only. The private lifecycle modules have been type-checked for x86_64 Darwin with real Rust 1.97 target libraries. The full Darwin crate check is blocked by the available C compiler’s unsupported Darwin flags in ring. No native Darwin runtime, deployed ABI, descriptor-closure execution or retained-membership runtime has been verified. Other Unix helper paths remain unverified. Capability advertisement and another repository’s Darwin CI do not supply that evidence. Unsupported proof obligations require mechanism review rather than silent descriptor leaks or narrowed containment.

Controlled adapter-worker signal fixtures establish their recorded process and typed-result behavior. They do not establish live Codex/model execution or recipe continuation unless those routes are actually exercised. Compiler identity absent from historical evidence remains absent; a current executable hash cannot repair that history. See verification contracts for causal assertions and evidence requirements.

Environment Budget & Safe Subprocess Spawning

The recipe runner starts bash and agent subprocesses for many steps. On Linux the kernel puts two limits on the environment of a new process. If either is exceeded, execve(2) fails with E2BIG (Argument list too long (os error 7)) and the child never starts:

The second limit applies even when the total is well under ARG_MAX. One 200 KiB step output stored in RECIPE_VAR_report is enough to make execve fail.

This page explains how the runner keeps every spawn under both limits:

1. Sanitized environment. Before every spawn the inherited parent environment is cleared (env_clear) and rebuilt from a bounded set. Nothing leaks in from the parent implicitly.
2. Per-value env spill. For bash steps, any recipe value whose KEY=value string is too large for one env string, and the largest remaining values when the total is over budget, are written to private files. A short preamble reads them back into shell variables of the same name before your script runs. Recipe scripts need no changes.
3. A measured budget. The total-size ceiling comes from the host’s real kernel limits at runtime. It isn’t hardcoded.
4. Fail-loud enforcement. If a variable that can’t be spilled can’t fit either, the runner refuses to spawn and gives a named error. It never lets a raw E2BIG reach you.

As a result, a recipe never dies with an opaque os error 7, and a script never silently sees an empty value where its recipe set one.

The measured budget

The environment byte budget is computed once per process from the host’s real limits and cached in a OnceLock.

Formula

budget = min( sysconf(_SC_ARG_MAX), RLIMIT_STACK / 4 )
         - argv_reserve

Every spawn calls Command::env_clear() and installs only the curated environment, so that curated set is the child’s whole environment block. There’s no separately inherited baseline to subtract.

The computation uses saturating arithmetic, and the result is clamped to a floor of 256 KiB.

Conservative fallback

If sysconf(_SC_ARG_MAX) or getrlimit(RLIMIT_STACK) is unavailable or returns a non-positive value, the budget falls back to the 256 KiB floor.

How sizes are measured

Every size check (the per-value cap, the budget planner and bounded_env) counts a variable the way the kernel does, including the = and the terminating NUL:

env_pair_bytes(KEY, value) = len(KEY) + 1 + len(value) + 1     // "KEY=value\0"

There is exactly one definition, in adapters::env_spill, so the checks always agree.

Single source of truth

#![allow(unused)]
fn main() {
/// Returns the measured environment byte budget for child processes.
///
/// Computed once from `sysconf(_SC_ARG_MAX)` and `RLIMIT_STACK`, cached in a
/// `OnceLock`.
pub(crate) fn env_byte_budget() -> usize
}

The bash-step spill planner and bounded_env both call env_byte_budget(). No second copy of the limit exists.

Per-value env spill (bash steps)

When a value is spilled

Each bash step gets its environment by merging the recipe values (RECIPE_VAR_<name>, nested RECIPE_VAR_<a>__<b>, and the legacy uppercase alias <NAME>) over the sanitized inherited environment. The runner then plans a spill in two phases:

1. Per-value cap. Every recipe value with env_pair_bytes(KEY, value) > PER_VALUE_ENV_CAP is spilled, whatever the total is.

   MAX_ARG_STRLEN    = 131072
   PER_VALUE_ENV_CAP = MAX_ARG_STRLEN - 4096 = 126976
   

2. Total budget. If the whole merged environment (inherited plus remaining recipe values) is still over env_byte_budget(), the remaining recipe values are spilled largest first (ties go to the name that sorts first) until the total fits.

The plan is deterministic. Only recipe values (the step’s extra_env) are candidates; inherited variables are never spilled. Empty values are never spilled. Because the spill runs before bounded_env, the runner moves recipe values to files rather than dropping ambient variables: a non-protected inherited variable is trimmed only if the environment is still over budget after every spillable recipe value has been moved.

RECIPE_VAR_x and its legacy alias X hold the same value but are planned independently, each with its own file. Each is judged on its own KEY=value size, and that size includes the key. A large value usually spills under both names. Near the cap, though, RECIPE_VAR_x=value (11 bytes of name) can be over while X=value is still under, so only the longer name spills. Budget-only spilling can also leave the shorter alias inline. Either way, both names give the script the same value.

When nothing exceeds the cap or the budget, nothing changes: no spill directory is created, and the script and the environment are byte-identical to what they would be without this feature.

What the script sees

The runner removes each spilled variable from the child environment and puts one preamble line per variable in front of your command::

{ IFS= read -r -d '' RECIPE_VAR_report || :; } < '/tmp/amplihack-env-spill-Xa81Qz/v0' || { printf 'amplihack: failed to load spilled env value RECIPE_VAR_report\n' >&2; exit 125; }
{ IFS= read -r -d '' REPORT || :; } < '/tmp/amplihack-env-spill-Xa81Qz/v1' || { printf 'amplihack: failed to load spilled env value REPORT\n' >&2; exit 125; }
# ... your command: follows unchanged

IFS= read -r -d '' reads the whole file into the variable without splitting it, without interpreting backslashes, and without dropping leading or trailing whitespace. The value is byte-identical, trailing newlines included. Quotes, $(), backticks, *, ; and embedded newlines come back exactly as they went in, because value bytes only ever live in the file and never in shell text. The only shell text in the preamble is the fixed read syntax, a validated variable name, and a single-quoted path (with ' written as '\'').

The preamble fails closed. If a spill file can’t be opened, the step exits with status 125, prints a message naming only the variable, and your command does not run. The step never continues with an empty value.

A recipe that writes a large report and reads it in a later step needs no changes:

name: big-report
steps:
  - id: gen
    command: python3 scripts/build_report.py   # prints ~200 KiB
    output: report
  - id: check
    command: |
      printf '%s' "$RECIPE_VAR_report" | sha256sum
      printf '%s' "$REPORT" | wc -c          # legacy alias, same bytes

Note: The runner trims captured step output before storing it. A value that came from a step’s output therefore has no trailing whitespace. Values set some other way, such as --set or a sub-recipe context:, keep their trailing newlines.

The preamble applies to every path that goes through execute_bash_step: sequential bash steps, parallel_group bash steps, and pre_step/post_step hooks.

Spilled variables are shell-local, not exported

A spilled variable is an ordinary shell variable inside the step script. It isn’t exported, so child processes of your script (jq, git, python3) don’t inherit it. Variables that weren’t spilled are exported exactly as before.

This is deliberate:

No working recipe can depend on the export. Before this feature these values either crashed execve or were silently replaced by an empty string, so no child ever saw them.

To pass a spilled value to a child process, use stdin or a file instead of the environment:

printf '%s' "$RECIPE_VAR_report" | jq -r '.summary'
printf '%s' "$RECIPE_VAR_report" > "$tmp/report.json" && python3 check.py "$tmp/report.json"

export RECIPE_VAR_report is still possible. A child started after that fails with E2BIG if RECIPE_VAR_report=value\0 is over MAX_ARG_STRLEN (131072 bytes, name included), or if the export pushes the total over ARG_MAX. PER_VALUE_ENV_CAP is the runner’s own safety margin and doesn’t apply here. A value spilled under the cap but still under MAX_ARG_STRLEN exports without error. That’s under your script’s control and outside the runner’s guarantees.

Names that can’t be spilled

A recipe value can only be read back into a shell variable of the same name if the name is a valid identifier that bash allows you to assign. A name is spillable when:

• it matches ^[A-Za-z_][A-Za-z0-9_]*$ (ASCII only), and
• it isn’t one of IFS PATH HOME EUID PPID SHELLOPTS UID GROUPS FUNCNAME LINENO RANDOM SRANDOM SECONDS EPOCHSECONDS EPOCHREALTIME HISTCMD DIRSTACK OPTIND OPTARG PS4 ENV POSIXLY_CORRECT CDPATH GLOBIGNORE, and
• it doesn’t start with BASH (which covers BASHOPTS, BASHPID and BASH_*) or COMP_.

A recipe value whose name fails these rules and is over the per-value cap is dropped from the child environment, with one warning naming the variable and its size. The step still starts. Such a value couldn’t have reached the script before this feature either, because the spawn would have failed.

Values containing NUL bytes

read -d '' stops at the first NUL byte, so a NUL can’t round-trip through a spill file. If a value that has to be spilled contains a NUL, the step fails before spawning with an error naming the variable and its length. The value is never silently truncated.

Spill files

Parallel steps each get their own directory. Spill files are never written into the step’s working_dir, and spilled values are never copied into the temporary script file used for large scripts (below).

If the runner process itself is killed with SIGKILL, it can’t clean up, and the directory is left in TMPDIR. Its contents are still private (0600 files in a 0700 directory). If TMPDIR points at a world-writable directory without the sticky bit, other local users can rename or delete entries in it. Use a private or sticky TMPDIR on shared hosts.

Interaction with large scripts

Commands over 64 KiB are already written to a private temporary script file instead of being passed as bash -c <script>. The 64 KiB check is applied to the combined script (preamble plus your command), so the preamble always runs first, whether the script is inline or file-backed.

$LINENO and shebangs

When something is spilled, the preamble adds one line per spilled variable in front of your command:

• $LINENO and bash error messages (line 12: ...) are offset by that many lines.
• A #! line at the top of command: stops being the first line and becomes an ordinary comment. Bash steps always run under the resolved bash interpreter, so it never selected the interpreter anyway.

Neither applies when nothing is spilled.

No more context-file fallback

Earlier versions handled an over-budget context differently. They wrote the whole context to an AMPLIHACK_CONTEXT_FILE JSON file and kept only four variables inline, so every other RECIPE_VAR_* silently became empty. That fallback has been removed:

• Every recipe variable is available to every bash step, either inline or through the preamble.
• Step execution no longer sets AMPLIHACK_CONTEXT_FILE.
• RecipeContext::shell_env_for_step() returns the full HashMap<String, String> of shell variables. It used to return a (HashMap<String, String>, Option<PathBuf>) tuple.
• RecipeContext::write_context_file() remains public API for library users. Step execution doesn’t call it.

Scripts that read $AMPLIHACK_CONTEXT_FILE must switch to $RECIPE_VAR_<name>. See Migration to 0.6.0.

Agent steps

Agent subprocesses don’t receive RECIPE_VAR_* values. Recipe data reaches an agent through its prompt, and prompts over MAX_INLINE_AGENT_PROMPT_BYTES already go through a file. An agent child’s environment is the sanitized inherited environment plus AMPLIHACK_AGENT_BINARY.

That inherited environment is filtered before any step-specific variable is added. A variable is dropped, with a warning naming it and giving its size, when its whole KEY=value\0 string is longer than MAX_ARG_STRLEN:

env_pair_bytes(KEY, value) = len(KEY) + 1 + len(value) + 1 > 131072

This is the same check the kernel applies in execve, so the runner drops only inherited variables that no child could receive. The rule counts the key, = and the NUL, so a 1,000-byte name with a 130,071-byte value (131,073 bytes in all) is dropped, while the same name with a 130,070-byte value (exactly 131,072 bytes) is passed through. PER_VALUE_ENV_CAP does not apply to inherited variables. That 4 KiB margin governs only the recipe values the runner spills. The filter applies to bash and agent children alike. For a bash step, the interpreter is resolved first, from the unfiltered parent environment, and the filter and the extra_env merge follow (see Order of operations for a bash step).

Protected variables

Some variables are protected. They’re needed for correct nested execution, and bounded_env never drops them to fit the budget.

A variable is protected if its name is one of:

• PATH
• HOME
• TASK_DESCRIPTION
• REPO_PATH
• TASK_TYPE
• WORKSTREAM_COUNT

…or if its name starts with one of these prefixes:

• AMPLIHACK_ (e.g. AMPLIHACK_SESSION_DEPTH, AMPLIHACK_TREE_ID, AMPLIHACK_BASH)
• RECIPE_VAR_

PATH is protected so that budget trimming can never change binary resolution in the child. For bash steps the one live dependency is the timeout binary: a step with timeout: spawns timeout <secs> <bash> …, and timeout is looked up by name against the child’s PATH.

The bash interpreter itself does not depend on this. It’s resolved to an absolute path against the parent’s PATH before the child environment is built (see Bash interpreter resolution). That’s also why the spill happens after interpreter resolution: a recipe value can never choose the interpreter.

AMPLIHACK_BASH is protected by the AMPLIHACK_ prefix rule, so an operator’s interpreter pin survives trimming.

Everything else is non-protected and can be trimmed.

Order of operations for a bash step

resolve_bash_interpreter          (parent env only)
build_child_env                   (sanitized inherited env, per-string filter)
merge extra_env                   (RECIPE_VAR_*, nested, legacy aliases)
plan + write spill                (per-value cap, then budget; 0600 files)
script = preamble + command
bounded_env                       (unchanged: trims non-protected inherited vars)
64 KiB inline-vs-file decision    (on the combined script)
spawn, wait
drop script file, drop spill dir

bounded_env runs after the spill and behaves exactly as before. Because the recipe values that would overflow have already been moved to files, it only has to deal with inherited variables:

1. Drop non-protected inherited variables largest-first until the total fits. Each drop is logged by name and size only. These are parent-process variables, not recipe values.
2. Fail loud if protected variables still overflow. If the protected variables alone exceed the budget after every non-protected variable has been dropped, the runner doesn’t spawn and returns EnvBudgetError.

Fail-loud error

The message below is illustrative. Its content is guaranteed (the offending names, their sizes and the budget, plus advice to move large inherited values out of the environment); its exact wording isn’t.

Error: environment budget exceeded before subprocess spawn.

The following required variable(s) do not fit within the measured
environment budget (262144 bytes):
  - AMPLIHACK_SESSION_STATE (198734 bytes)

These variables are protected and were not dropped. Recipe values are
spilled to private files automatically; large inherited or protected
variables must be moved out of the environment.

Guarantees:

• The error names the offending variables and their byte sizes, never their values.
• A raw execve os error 7 never reaches you, because the guard runs first.
• Protected variables are never silently dropped to make room.
• The message doesn’t point to AMPLIHACK_CONTEXT_FILE, which no longer exists.

Logging & privacy

The environment code never logs a variable’s value. Diagnostics only ever contain variable names, byte sizes, and (at debug) the spill directory.

The examples below are illustrative. The events, levels and the fields they carry (names and byte sizes, never values) are guaranteed; the exact wording isn’t, so don’t match on it in scripts.

Spill errors name the variable and its byte length, plus the underlying I/O error, for example failed to spill NAME (LEN bytes): <io error>. Types that hold values don’t derive Debug. A sentinel test checks that no value bytes show up in logs or in the error chain.

Retired: max_env_value_bytes

Some recipes set a per-step max_env_value_bytes: 65536. An older in-process executor honoured it to avoid E2BIG at checkpoint steps. This runner ignores the field, and the parser warns about it once per step:

recipe 'workflow-tdd': step 'checkpoint-3': field 'max_env_value_bytes' is retired and ignored — the runner now spills oversized env values to private files automatically; remove it from the recipe

It’s retired, not implemented, because automatic per-value spilling already keeps every env string under MAX_ARG_STRLEN with no per-step setting. Implementing the field would only bring back lossy truncation of values. You can remove it from your recipes. See Unknown and retired step fields.

Configuration

The budget and the spill are fully automatic, with nothing to configure in normal use. Operators who want a larger total budget can raise the stack limit before starting the runner:

# Inspect the host's kernel argument-list limit:
getconf ARG_MAX

# Raise the stack rlimit (raises RLIMIT_STACK / 4, which may raise the budget):
ulimit -s unlimited

Spill files go to TMPDIR, so set TMPDIR to put them somewhere else.

There’s intentionally no environment variable or config-file setting to override the budget or the per-value cap. MAX_ARG_STRLEN is a fixed kernel constant.

Test-only seams

Two #[cfg(test)]-only seams make the spill paths deterministic in unit tests:

#![allow(unused)]
fn main() {
#[cfg(test)]
fn set_env_byte_budget_for_test(bytes: usize);   // hold ENV_BUDGET_TEST_LOCK

#[cfg(test)]
struct SpillRootGuard;  // thread-local spill-root override, restored on Drop
}

Neither is an environment variable, so neither can leak into a child process or be triggered in production.

Behavior summary

Related

• Architecture: where the subprocess adapter and context layer sit in the overall design.
• YAML Recipe Format: parse-time warnings for unknown and retired step fields.
• CLI Reference: running recipes that spawn subprocesses.
• AMPLIHACK_BASH: choosing the interpreter those bash subprocesses run under.