orion-server test
Runs a directory of offline workflow test cases. Each *.case.json file names a workflow, an input, optional request metadata and connector stubs, and what it expects. Prints a per-case diff and exits non-zero on any failure.
Synopsis
orion-server test <path>
Description
A case expects output values (expect), task-error codes (expect_errors), connector calls (expect_calls) and executed task ids (expect_tasks). See Test a workflow offline for the case format.
Options
| Argument | Description |
|---|---|
path | A directory of *.case.json files, or a single case file. Paths inside a case resolve relative to the case file. |
--definitions | Directory holding the set’s shared definitions, resolved before each case’s workflow is validated and run. |
--plugin-dir | Directory of plugin manifests and their components, loaded and compiled once for the whole suite. Plugin functions run for real, never stubbed; a case naming one whose component is absent fails as PLUGIN_ARTIFACT_UNAVAILABLE, never as a silently passing stub. Repeatable. |
--model-dir | Directory of model manifests and their artifacts, loaded once for the whole suite — each model is loaded into the runtime once, however many cases call it. model_infer runs the model for real, as dry-run does with the flag; a case naming a model the directory does not hold fails as MODEL_ARTIFACT_UNAVAILABLE. Without the flag a case’s stubs answer the function by name. Repeatable. |
Examples
orion-server test examples/workflow-tests
Related
- Test a workflow offline: the case format, with every
expectroot. orion-server dry-run: one execution, printed in full.- CI/CD with packages: the suite as a pipeline stage.
orion-servercommands: everyorion-serversubcommand.
Last verified 14 September 2026