Skip to content

dokimi tasks

usage: dokimi tasks [-h] {list,harnesses,run} ...

positional arguments:
  {list,harnesses,run}
    list                List terminal-bench task metadata
    harnesses           List known coding harness aliases
    run                 Run selected tasks through one or more harnesses

options:
  -h, --help            show this help message and exit

dokimi tasks list

usage: dokimi tasks list [-h] [--tasks-root TASKS_ROOT]

options:
  -h, --help            show this help message and exit
  --tasks-root TASKS_ROOT
                        Directory holding one sub-directory per task (each with task.toml or
                        task.md). Required for bare task names and for --all; the tasks live in a
                        separate evaluation-content checkout, e.g. --tasks-root /path/to/content-
                        repo/evaluation/harbor/tasks

dokimi tasks harnesses

usage: dokimi tasks harnesses [-h]

options:
  -h, --help  show this help message and exit

dokimi tasks run

usage: dokimi tasks run [-h] (--tasks TASKS | --all) --harnesses HARNESSES
                        [--baseline-harness BASELINE_HARNESS] [--model MODEL]
                        [--tasks-root TASKS_ROOT] [--output-dir OUTPUT_DIR] [--out-json OUT_JSON]
                        [--out-md OUT_MD] [--repeats REPEATS]
                        [--command-template COMMAND_TEMPLATE] [--dry-run]

options:
  -h, --help            show this help message and exit
  --tasks TASKS         Comma-separated task names or paths
  --all                 Run all discovered tasks
  --harnesses HARNESSES
                        Comma-separated harness ids: codex, claude-code, terminal-bench:<agent>
  --baseline-harness BASELINE_HARNESS
  --model MODEL         Model ref passed to the harness
  --tasks-root TASKS_ROOT
                        Directory holding one sub-directory per task (each with task.toml or
                        task.md). Required for bare task names and for --all; the tasks live in a
                        separate evaluation-content checkout, e.g. --tasks-root /path/to/content-
                        repo/evaluation/harbor/tasks
  --output-dir OUTPUT_DIR
  --out-json OUT_JSON
  --out-md OUT_MD       Markdown report path (default: alongside --out-json)
  --repeats REPEATS     Run each (task, harness) cell N times for pass-rate/cost variance
                        (default: 1)
  --command-template COMMAND_TEMPLATE
                        Override command for every harness. Placeholders: {task_path}, {task},
                        {model}, {output_dir}, {prompt}, {prompt_quoted}.
  --dry-run             Build artifact without invoking agents