pybench · pytxo bench
What lands on main, measured.
Pybench builds a throwaway repository for each scenario and gives two policies the same agent branches. Per-branch CI tests each branch alone and merges every branch that passed. pytxo land tests the combination and refuses what moved. No model is called. The scenarios show the mechanism; they do not say how often each situation happens in real projects.
Reference run
pytxo 1.3.0 · windows x86_64 · 16 cpus · 2026-10-10
- Broken mains refused
- 4/4
- Decided as expected
- 12/12
- Good changes refused
- 0
- Median gate time
- 3.1 s
| Scenario | Per-branch CI | pytxo land | Gate |
|---|---|---|---|
| Two agents, separate files | lands, fine | ready | 4.1 s |
| Same file, different functions | lands, fine | ready | 3.3 s |
| Same function, compatible edits (reported, not blocked) | lands, fine | ready · same function | 4.1 s |
| Both agents rewrite the same line | stops on a conflict | blocked (conflict) · same function | 4.0 s |
| One agent renames a function, another adds a caller of the old name | lands a broken main | blocked (combined check) | 2.5 s |
| One agent adds a required parameter, another adds a call without it | lands a broken main | blocked (combined check) | 2.6 s |
| One agent deletes a helper it thinks is unused, another starts using it | lands a broken main | blocked (combined check) | 2.8 s |
| One agent changes a default, another adds a test that relies on the old one | lands a broken main | blocked (combined check) | 2.9 s |
| The target branch moves while the gate is checking | not modeled | stale | 6.2 s |
| Three agents, separate files | lands, fine | ready | 3.1 s |
| Both agents create the same new file differently | stops on a conflict | blocked (conflict) | 2.7 s |
| One agent deletes a file the other edits | stops on a conflict | blocked (conflict) · same function | 3.1 s |
Gate cost without checks
Median of three runs: sequential git merge of every input against pytxo land in shadow mode, which also maps file and function ownership and writes the receipt. Git process start-up dominates both on Windows.
| Inputs | Files | git merge | pytxo land |
|---|---|---|---|
| 2 | 10 | 1.7 s | 4.0 s |
| 5 | 10 | 4.2 s | 6.9 s |
| 10 | 10 | 5.5 s | 8.4 s |
| 2 | 100 | 1.6 s | 2.4 s |
| 5 | 100 | 3.5 s | 6.0 s |
| 10 | 100 | 9.5 s | 11.5 s |
Signal Core on the Pytxo repository: 374 source files, 5,030,538 bytes reduced to a 716,736-byte structural scaffold (85.8% smaller). Bytes, not tokens or cost.
Board
Summaries published by signed-in users with pytxo bench --submit: machine type and numbers only.
No published results yet. Be the first:
pytxo bench --submit
Needs Pytxo 1.3, git and node. --submit asks you to sign in; without it, nothing leaves your machine.