Benchmarks

ferroplan is differentially tested against reference planners over a curated subset of the IPC contest suites — classical (gripper, blocks, logistics, …), numeric (rovers, satellite, depots, …), ADL, and IPC-5 simple-preferences. Coverage, plan validity, and speed are checked against the C Metric-FF; IPC-5 preference quality is scored against SGPlan5, the contest winner (per-instance metrics from the official archive — see the scoreboard). A locally-built SGPlan6 is also used for solvability/validity agreement.

The harness lives in benchmarks/: solvability agreement, plan validity, plan length, and (for preferences) metric comparison, plus criterion micro-benchmarks for parse/ground/search.

Full-corpus scoreboards over IPC-2008/2011 (all VAL-validated) live alongside: seq-sat, tempo-sat, and net-benefit — regenerated per release by benchmarks/ipc67.py. The game track has its own scoreboard: bazaar-thinks.md (think latency/quality curves and the live multi-mind loop, generated by the bazaar_thinks / bazaar_live examples).

Curated-subset results tables are committed and published here as-is (they are machine-dependent; regenerate locally per benchmarks/COMPARING.md).