Session 3 optimizations, measured

This file gives the serial confirmation runs of 2026-10-29 for the session 3 optimizations of all three designs (branch head 3e74450). The Docker setup is the same as in VERSIONS.md: Postgres 16.15, 17.11 and 18.6 with pg_tle 1.5.3. It uses one server at a time, cpus: 4, mem_limit: 6g, 4 pgbench clients, and 10 s for each workload. The 27 configurations are the same as in the matrix before the optimizations. They are large default, medium deep and medium wide, with 3 versions and 3 designs. They ran with --tag opt.

The before numbers are the Docker runs before the optimizations (pg<NN>-docker-<design>-<scale>-<shape>.md, from 08:12-09:35 the same morning). The after numbers are pg<NN>-docker-<design>-<scale>-<shape>-opt.md. The JSON for both is in data/. The reports are in ../../bench/baselines/. The “Optimizations (session 3)” sections of ../designs/ give what each design changed.

The design documents claim optimization gains. We measured those claims on local Postgres 16.4 with 1 client (C used 4 clients for write_nest_team), interleaved, while other agents were busy. This file measures them again in the setup of the benchmark. Before the runs, the test suite passed on all three versions: A 18/18 files (199 assertions), B 18/18 (189), C 21/21 (229).

Summary

• Correctness: every checksum is unchanged. For all 27 configurations, the per-policy checksums of each design (rbac, sharing, parents_proj, parents) equal the checksums before the optimizations. The 10 repeat runs are also identical. The checksums still agree across designs and versions.
• Largest gains (PG 18, p50): write_nest_team of B 107 → 0.62 ms (172x). The large single-branch parents page of B 102 → 13.7 ms. The medium wide sharing page of A 125 → 14.4 ms, and the large single-branch page of A 240 → 45.6 ms. write_nest_team of C 300 → 57.7 ms, and the wide sharing seed of C 176 → 24.7 s. nest_check of B 1.14 → 0.32 ms. The storage of B after the parents seed is half (762.8 → 377.9 MB).
• Every claimed “after” value was reached within 1.3x. Some speedup factors are smaller than claimed. These are the large parents of A and B, nest_check of B and the wide page of B. The cause is the “before” value. Docker ran those PL/pgSQL-heavy paths about 2x faster than local 16.4 before the optimizations too. See Calibration in VERSIONS.md.
• Ranking changes: nest_check is now a three-way tie (0.30-0.32 ms). On wide sharing, A now beats B (C is still fastest). The nesting write of B is now in the range of A (0.6 against 0.15 ms; C is at 58 ms). At large, the storage of B is now below A (378 against 404 MB; C is at 182 MB).
• What did not change or got slightly worse: rbac_check of A is 10-20% slower on every version and configuration (PG 18 large: 0.26 → 0.30 ms). The wide share_list_objects of B improved on 16 (8.96 → 4.47 ms), but not on 17 (9.02 → 9.26, reproduced by a repeat run). It improved only partly on 18 (11.4 → 8.4). parents_proj_select_count at large did not change for any design (about 8 ms, the heap-fetch floor of minor releases that VERSIONS.md describes).

Checksums

The checksums are identical before and after, for each configuration and policy, on every version:

configuration	rbac	sharing	parents_proj	parents
large default	2e1dd348	58a66d1d	bffa5242	bffa5242
medium deep	d394663f	7cf80111	3261b092	3261b092
medium wide	d394663f	6f7c008d	f2867b7f	f2867b7f

Claims

The “Claim” column comes from the design documents (local 16.4, mostly 1 client). The “Measured” column gives the Docker p50 at 4 clients, before → after, on 16 / 17 / 18. We confirm a claim when the measured after value is within 2x of the claimed after value. The table also lists the speedup factor. The before values in Docker are lower than the local values for the PL/pgSQL paths of A and B.

Design	Claim	Measured before → after (16 / 17 / 18)	Speedup (16 / 17 / 18)	Verdict
A	medium wide share_select_page 207 → 18 ms	98.6 → 14.7 / 103.6 → 20.2 / 124.5 → 14.4	6.7x / 5.1x / 8.6x	Confirmed
A	large single-branch parents (parents_select_page) 505 → 35 ms	241.7 → 46.0 / 246.1 → 47.0 / 239.8 → 45.6	5.3x / 5.2x / 5.3x	After value confirmed (1.3x off). The speedup is 2.7x below the claim; see below
A	large parents_select_count 491 → 52 ms (design document)	261.4 → 79.6 / 265.6 → 68.2 / 268.6 → 75.1	3.3x / 3.9x / 3.6x	After value 1.3-1.5x off; confirmed within 2x
B	write_nest_team 307 → 0.6 ms (4 clients)	107.3 → 0.72 / 111.6 → 0.70 / 106.6 → 0.62	149x / 159x / 172x	Confirmed
B	nest_check 3.3 → 0.37 ms	large: 1.30 → 0.32 / 1.31 → 0.32 / 1.14 → 0.32	4.1x / 4.1x / 3.6x	After value confirmed. The speedup is below the claim (the Docker before value was 1.1-1.3 ms)
B	medium wide page (share_select_page) 70 → 15 ms	33.6 → 11.9 / 33.8 → 18.9 / 36.9 → 17.8	2.8x / 1.8x / 2.1x	After value confirmed (0.8-1.3x). The speedup is below the claim (the Docker before value was 34-37 ms)
B	large single-branch parents 190 → 13 ms	95.4 → 15.3 / 94.7 → 20.1 / 102.2 → 13.7	6.3x / 4.7x / 7.5x	Confirmed (the 17 value is a repeat run with a slow parents phase; the original 17 run had 14.8)
B	per-document parents storage halved	762.8 → 377.9 MB after the large parents seed, every version. Seed 118-121 → 45-49 s		Confirmed
C	write_nest_team 367 → 58 ms (4 clients)	298.5 → 55.0 / 302.8 → 56.9 / 299.7 → 57.7	5.4x / 5.3x / 5.2x	Confirmed
C	wide sharing seed 185 → 25 s	181.5 → 23.6 / 183.8 → 22.6 / 176.4 → 24.7	7.7x / 8.1x / 7.1x	Confirmed
C	large single-branch parents 9.4 → 5.0 ms	10.7 → 5.26 / 10.5 → 5.04 / 9.26 → 4.94	2.0x / 2.1x / 1.9x	Confirmed (parents_select_count: 23.1 → 18.6 / 18.1 → 12.3 / 16.9 → 11.9)

We checked why the speedups of A and B are smaller than the claims. We ran each speedup that was more than 2x below its claim again once. Each result reproduced. The values are: A large pg16 46.6 ms and pg18 45.65 ms.

B large pg16 15.6 ms. B wide pg17 18.6 ms and pg18 17.9 ms. The next table shows EXPLAIN (ANALYZE, BUFFERS) of the single-branch page of A for user 1, from the reports:

run	planning	execution	buffers
local 16.4, before	213 ms	209 ms	180k (execution) + 181k (planning)
Docker 18.6, before	113 ms	103 ms	180k + 181k
Docker 18.6, after	0.46 ms	41 ms	21k + 0.1k

The optimization removed the plan-time evaluation (half the cost) and cut the walk from 180k to 21k buffers. The Docker before value is half the local value for the same plan and buffers. So the same after value gives a smaller factor. The single-branch page of B shows the same pattern: 32.7 ms planning + 25.5 ms execution before, 0.15 + 19.4 after.

Before and after on PG 18

The values are p50 ms. “Before” is the Docker run before the optimizations. “After” is the -opt run, with the same version and configuration.

Large default

workload	A before	A after	A speedup	B before	B after	B speedup	C before	C after	C speedup
rbac_check	0.26	0.30	0.9x	0.26	0.27	1.0x	0.25	0.24	1.0x
rbac_select_page	0.76	0.76	1.0x	0.77	0.73	1.1x	0.73	0.67	1.1x
rbac_select_count	0.85	0.78	1.1x	0.85	0.78	1.1x	0.82	0.73	1.1x
rbac_update_one	0.53	0.47	1.1x	0.52	0.47	1.1x	0.45	0.36	1.2x
write_assign_role	0.16	0.15	1.1x	0.14	0.14	1.0x	0.18	0.17	1.1x
share_list_objects	0.34	0.29	1.2x	0.29	0.29	1.0x	0.26	0.25	1.0x
share_select_page	1.45	1.03	1.4x	1.16	1.00	1.2x	1.00	0.93	1.1x
write_share	0.15	0.15	1.0x	0.14	0.14	1.0x	0.17	0.17	1.0x
parents_proj_select_page	2.09	1.33	1.6x	1.30	1.15	1.1x	1.05	1.05	1.0x
parents_proj_select_count	8.56	8.42	1.0x	7.79	8.20	1.0x	7.48	7.88	0.9x
parents_select_page	240	45.6	5.3x	102	13.7	7.5x	9.26	4.94	1.9x
parents_select_count	269	75.1	3.6x	122	21.6	5.7x	16.9	11.9	1.4x
write_nest_team	0.15	0.15	1.0x	107	0.62	172x	300	57.7	5.2x
nest_check	0.52	0.31	1.7x	1.14	0.32	3.6x	0.30	0.30	1.0x

Medium deep

workload	A before	A after	A speedup	B before	B after	B speedup	C before	C after	C speedup
rbac_check	0.25	0.29	0.9x	0.26	0.27	1.0x	0.25	0.24	1.0x
rbac_select_page	0.47	0.44	1.1x	0.50	0.46	1.1x	0.48	0.42	1.1x
rbac_select_count	0.39	0.35	1.1x	0.39	0.37	1.1x	0.37	0.32	1.2x
rbac_update_one	0.45	0.48	0.9x	0.51	0.46	1.1x	0.43	0.35	1.2x
write_assign_role	0.15	0.15	1.0x	0.14	0.14	1.0x	0.18	0.18	1.0x
share_list_objects	0.42	0.31	1.4x	0.48	0.32	1.5x	0.28	0.28	1.0x
share_select_page	1.42	0.76	1.9x	1.68	0.72	2.3x	0.77	0.66	1.2x
write_share	0.15	0.15	1.0x	0.14	0.14	1.0x	0.17	0.17	1.0x
parents_proj_select_page	2.01	1.00	2.0x	1.05	0.83	1.3x	0.71	0.73	1.0x
parents_proj_select_count	2.25	1.25	1.8x	1.31	1.14	1.1x	0.97	0.99	1.0x
parents_select_page	47.8	7.41	6.5x	10.9	4.52	2.4x	3.74	2.83	1.3x
parents_select_count	46.2	6.40	7.2x	9.89	3.74	2.6x	3.04	2.10	1.4x
write_nest_team	0.15	0.15	1.0x	124	0.63	196x	307	75.1	4.1x
nest_check	0.63	0.34	1.9x	1.18	0.34	3.5x	0.30	0.30	1.0x

Medium wide

workload	A before	A after	A speedup	B before	B after	B speedup	C before	C after	C speedup
rbac_check	0.26	0.29	0.9x	0.29	0.28	1.0x	0.24	0.24	1.0x
rbac_select_page	0.50	0.44	1.1x	0.57	0.46	1.2x	0.48	0.44	1.1x
rbac_select_count	0.41	0.36	1.1x	0.42	0.36	1.2x	0.38	0.33	1.2x
rbac_update_one	0.53	0.49	1.1x	0.57	0.48	1.2x	0.46	0.38	1.2x
write_assign_role	0.16	0.15	1.1x	0.15	0.14	1.1x	0.17	0.18	0.9x
share_list_objects	29.1	4.44	6.5x	11.4	8.41	1.4x	4.25	3.41	1.2x
share_select_page	125	14.4	8.6x	36.9	17.8	2.1x	10.2	8.20	1.2x
write_share	0.16	0.15	1.1x	0.15	0.16	0.9x	0.18	0.17	1.1x
parents_proj_select_page	123	15.0	8.2x	40.7	15.1	2.7x	11.1	8.73	1.3x
parents_proj_select_count	112	13.2	8.4x	38.5	13.4	2.9x	8.97	6.33	1.4x
parents_select_page	118	14.7	8.0x	39.3	13.7	2.9x	11.8	8.06	1.5x
parents_select_count	121	13.3	9.1x	41.2	12.2	3.4x	9.40	6.61	1.4x
write_nest_team	0.15	0.15	1.0x	133	1.47	91x	181	61.2	2.9x
nest_check	1.34	0.37	3.6x	1.22	0.34	3.6x	0.31	0.31	1.0x

All versions

The tables show the p50 speedup (before ÷ after) on 16 / 17 / 18. A value above 1x means that the run after the optimization is faster.

Large default:

workload	A	B	C
rbac_check	0.9x / 0.9x / 0.9x	1.3x / 1.3x / 1.0x	1.0x / 1.0x / 1.0x
rbac_select_page	1.1x / 1.1x / 1.0x	1.1x / 1.1x / 1.1x	1.1x / 1.2x / 1.1x
rbac_select_count	1.1x / 1.1x / 1.1x	1.1x / 1.1x / 1.1x	1.1x / 1.1x / 1.1x
rbac_update_one	1.1x / 1.2x / 1.1x	1.7x / 1.6x / 1.1x	1.3x / 1.4x / 1.2x
write_assign_role	1.0x / 1.0x / 1.1x	1.1x / 1.1x / 1.0x	1.0x / 1.0x / 1.1x
share_list_objects	1.1x / 1.1x / 1.2x	1.1x / 1.1x / 1.0x	1.0x / 1.0x / 1.0x
share_select_page	1.4x / 1.4x / 1.4x	1.2x / 1.2x / 1.2x	1.1x / 1.1x / 1.1x
write_share	0.9x / 1.0x / 1.0x	1.1x / 0.9x / 1.0x	1.0x / 1.0x / 1.0x
parents_proj_select_page	1.6x / 1.7x / 1.6x	1.2x / 1.2x / 1.1x	1.0x / 1.0x / 1.0x
parents_proj_select_count	1.0x / 1.0x / 1.0x	1.0x / 0.9x / 1.0x	1.0x / 0.9x / 0.9x
parents_select_page	5.3x / 5.2x / 5.3x	6.3x / 4.7x / 7.5x	2.0x / 2.1x / 1.9x
parents_select_count	3.3x / 3.9x / 3.6x	4.1x / 4.1x / 5.7x	1.2x / 1.5x / 1.4x
write_nest_team	1.0x / 1.0x / 1.0x	149x / 159x / 172x	5.4x / 5.3x / 5.2x
nest_check	2.2x / 2.2x / 1.7x	4.1x / 4.1x / 3.6x	1.0x / 1.0x / 1.0x

Medium deep:

workload	A	B	C
rbac_check	0.9x / 0.9x / 0.9x	1.3x / 1.3x / 1.0x	1.0x / 1.0x / 1.0x
rbac_select_page	1.1x / 1.1x / 1.1x	1.2x / 1.2x / 1.1x	1.2x / 1.2x / 1.1x
rbac_select_count	1.1x / 1.2x / 1.1x	1.2x / 1.2x / 1.1x	1.2x / 1.2x / 1.2x
rbac_update_one	1.2x / 1.1x / 0.9x	1.7x / 1.6x / 1.1x	1.5x / 1.4x / 1.2x
write_assign_role	1.0x / 1.0x / 1.0x	1.1x / 1.0x / 1.0x	1.0x / 1.0x / 1.0x
share_list_objects	1.2x / 1.3x / 1.4x	1.6x / 1.7x / 1.5x	1.0x / 1.0x / 1.0x
share_select_page	1.8x / 1.9x / 1.9x	2.2x / 2.5x / 2.3x	1.1x / 1.2x / 1.2x
write_share	1.1x / 1.1x / 1.0x	0.9x / 1.0x / 1.0x	1.0x / 1.0x / 1.0x
parents_proj_select_page	2.0x / 2.0x / 2.0x	1.4x / 1.4x / 1.3x	0.9x / 1.0x / 1.0x
parents_proj_select_count	1.7x / 1.8x / 1.8x	1.2x / 1.2x / 1.1x	1.0x / 1.0x / 1.0x
parents_select_page	5.0x / 6.3x / 6.5x	2.0x / 2.4x / 2.4x	1.3x / 1.4x / 1.3x
parents_select_count	6.0x / 7.0x / 7.2x	2.3x / 2.7x / 2.6x	1.3x / 1.5x / 1.4x
write_nest_team	1.1x / 1.0x / 1.0x	213x / 219x / 196x	4.9x / 4.9x / 4.1x
nest_check	1.6x / 2.2x / 1.9x	4.0x / 4.1x / 3.5x	1.0x / 1.0x / 1.0x

Medium wide:

workload	A	B	C
rbac_check	0.8x / 0.9x / 0.9x	1.3x / 1.2x / 1.0x	1.0x / 1.0x / 1.0x
rbac_select_page	1.1x / 1.1x / 1.1x	1.2x / 1.2x / 1.2x	1.2x / 1.2x / 1.1x
rbac_select_count	1.2x / 1.2x / 1.1x	1.2x / 1.2x / 1.2x	1.3x / 1.2x / 1.2x
rbac_update_one	1.3x / 1.2x / 1.1x	1.7x / 1.7x / 1.2x	1.4x / 1.3x / 1.2x
write_assign_role	1.0x / 1.0x / 1.1x	1.1x / 1.0x / 1.1x	1.0x / 1.0x / 0.9x
share_list_objects	5.4x / 3.6x / 6.5x	2.0x / 1.0x / 1.4x	1.2x / 1.2x / 1.2x
share_select_page	6.7x / 5.1x / 8.6x	2.8x / 1.8x / 2.1x	1.1x / 1.2x / 1.2x
write_share	1.0x / 1.0x / 1.1x	1.1x / 1.0x / 0.9x	1.0x / 1.0x / 1.1x
parents_proj_select_page	6.6x / 6.9x / 8.2x	2.6x / 2.7x / 2.7x	1.1x / 1.2x / 1.3x
parents_proj_select_count	6.1x / 7.7x / 8.4x	2.5x / 2.8x / 2.9x	1.2x / 1.3x / 1.4x
parents_select_page	6.0x / 7.9x / 8.0x	2.2x / 2.5x / 2.9x	1.2x / 1.3x / 1.5x
parents_select_count	6.6x / 8.7x / 9.1x	2.4x / 2.7x / 3.4x	1.3x / 1.4x / 1.4x
write_nest_team	1.0x / 1.0x / 1.0x	89x / 109x / 91x	2.4x / 2.8x / 2.9x
nest_check	4.0x / 4.2x / 3.6x	4.1x / 4.3x / 3.6x	1.0x / 1.0x / 1.0x

Seed time and storage, before and after (PG 18)

The tables show the seed seconds and the cumulative size of the permissions tables (MB) after each phase. The sizes are identical on 16, 17 and 18. The seed times on 16 and 17 are within the usual ±5% of 18. The exceptions are one-off outliers in the RBAC seed of the first run.

Large default:

phase	A s	B s	C s	A MB	B MB	C MB
rbac	4.38 → 5.14	4.06 → 4.31	4.86 → 4.95	7 → 7	5.3 → 7.4	5.3 → 5.3
sharing	0.72 → 0.65	1.05 → 1	4.08 → 1.93	18.6 → 18.7	25.4 → 19.6	21.3 → 21.3
parents_proj	0.07 → 0.08	0.14 → 0.11	0.3 → 0.34	19.6 → 19.6	26.4 → 20.6	22.3 → 22.3
parents	27.29 → 31.02	121.15 → 44.82	17.13 → 17.63	397.6 → 404.3	762.8 → 377.9	182.4 → 182.4
nesting	0.07 → 0.05	0.05 → 0.02	0.06 → 0.04	397.6 → 404.3	763.9 → 377.9	183.2 → 183.2

Medium deep:

phase	A s	B s	C s	A MB	B MB	C MB
rbac	0.6 → 0.6	0.49 → 0.48	0.83 → 0.81	3.6 → 3.6	2.9 → 3.9	2.9 → 2.9
sharing	0.39 → 0.36	0.6 → 0.48	2.13 → 1.08	9.5 → 9.5	18 → 10.1	15 → 15
parents_proj	0.04 → 0.04	0.08 → 0.06	0.16 → 0.15	10 → 10	18.5 → 10.6	15.5 → 15.5
parents	2.29 → 2.14	11.26 → 3.97	1.58 → 1.57	47.9 → 48.6	92.2 → 46.5	31.5 → 31.5
nesting	0.02 → 0.02	0.08 → 0.02	0.09 → 0.06	47.9 → 48.6	94.2 → 46.5	33.1 → 33.1

Medium wide:

phase	A s	B s	C s	A MB	B MB	C MB
rbac	0.85 → 0.58	0.86 → 0.51	0.91 → 0.84	3.6 → 3.6	2.9 → 3.9	2.9 → 2.9
sharing	7.75 → 5.88	26.06 → 12.56	176.36 → 24.71	97.1 → 97.4	225.9 → 105.3	191.6 → 191.6
parents_proj	0.06 → 0.04	0.11 → 0.35	0.19 → 0.2	97.6 → 97.9	226.4 → 105.8	192.2 → 192.2
parents	2.71 → 2.32	13.98 → 12.99	1.95 → 1.65	135.4 → 136.4	300 → 141.6	208.2 → 208.2
nesting	0.04 → 0.04	0.06 → 0.02	0.1 → 0.04	135.4 → 136.4	301.1 → 141.6	209.1 → 209.1

What changed in storage and in the seeds:

• B: The storage is about half wherever per-document closure rows existed. Large parents phase: 762.8 → 377.9 MB. Medium wide after sharing: 225.9 → 105.3 MB. Medium deep: 92.2 → 46.5 MB. The large single-branch parents seed changed from 121 to 45 s, and the wide sharing seed from 26 to 12.6 s. The RBAC phase of B is slightly bigger (5.3 → 7.4 MB at large) because the primary-key index of role_grants is wider.
• C: The wide sharing seed changed from 176 to 24.7 s. The default sharing seed changed from 4.1 to 1.9 s (large) and from 2.1 to 1.1 s (medium deep). The storage did not change.
• A: The storage is 1.7% bigger at large (397.6 → 404.3 MB, because of the new partial index tuples_edge_kinds_idx). It did not change at medium. The seed times are the same.

Ranking after the optimizations

The table shows PG 18, -opt runs, p50 ms. The order is the same on 16 and 17, unless the text says otherwise. The symbol “≈” means within 15%.

Workload class	A	B	C	Order	Before (PG 18)
RBAC reads (rbac_check, rbac_select_page, rbac_select_count, rbac_update_one, large)	0.30, 0.76, 0.78, 0.47	0.27, 0.73, 0.78, 0.47	0.24, 0.67, 0.73, 0.36	C ≈ B ≈ A, all under 1 ms; C is 10-25% ahead on rbac_check and rbac_update_one	same, all within 15%
RBAC write (write_assign_role)	0.15	0.14	0.17	B ≈ A ≈ C	same
Sharing, default (large share_list_objects, share_select_page)	0.29, 1.03	0.29, 1.00	0.25, 0.93	C ≈ B ≈ A	C ≈ B < A (A’s page was 1.45)
Sharing, deep (medium deep)	0.31, 0.76	0.32, 0.72	0.28, 0.66	C ≈ B ≈ A	C < A < B (page 0.77 / 1.42 / 1.68)
Sharing, wide (medium wide)	4.44, 14.4	8.41, 17.8	3.41, 8.20	C < A < B (on 16: C < A ≈ B, B 4.47 / 11.9)	C < B < A (page 10.2 / 36.9 / 125)
Parents, projection (large page / count; medium wide page / count)	1.33 / 8.42; 15.0 / 13.2	1.15 / 8.20; 15.1 / 13.4	1.05 / 7.88; 8.73 / 6.33	C < B ≈ A. Large count is the same floor for all (about 8 ms)	C < B < A (wide page 11.1 / 40.7 / 123)
Parents, single branch (large page / count)	45.6 / 75.1	13.7 / 21.6	4.94 / 11.9	C < B < A, gaps narrowed: 9x and 2.8x on the page (were 26x and 11x)	C < B < A (9.26 / 102 / 240)
Parents, single branch (medium wide page / count)	14.7 / 13.3	13.7 / 12.2	8.06 / 6.61	C < B ≈ A	C < B < A (11.8 / 39.3 / 118)
Nesting write (write_nest_team, large / deep / wide)	0.15 / 0.15 / 0.15	0.62 / 0.63 / 1.47	57.7 / 75.1 / 61.2	A < B ≪ C	A ≪ B < C (0.15 / 107 / 300)
nest_check (large / deep / wide)	0.31 / 0.34 / 0.37	0.32 / 0.34 / 0.34	0.30 / 0.30 / 0.31	C ≈ A ≈ B	C < A < B (0.30 / 0.52 / 1.14 at large)
Storage (large, after parents)	404.3 MB	377.9 MB	182.4 MB	C < B < A	C < A < B (182 / 398 / 763)
Storage (medium wide, after sharing)	97.4 MB	105.3 MB	191.6 MB	A < B < C	A < C < B (97 / 192 / 226)
Biggest seed (large parents / wide sharing)	28.2 / 5.9 s	44.8 / 12.6 s	17.6 / 24.7 s	parents C < A < B; wide sharing A < B < C	parents C < A < B (17 / 27 / 121); wide sharing A < B < C (7.8 / 26 / 176)

Budgets (--check semantics)

--check fails a workload when its p95 is over the budget for that scale in bench/budgets.tsv. The shape also matters (medium-deep, medium-wide). A workload without a budget never fails. The table shows PG 18 -opt runs, p95 ms. The mark ❌ means over budget:

workload	large budget	A	B	C	medium deep budget	A	B	C	medium wide budget	A	B	C
rbac_check	1	0.56	0.41	0.33	1	0.4	0.36	0.32	1	0.4	0.37	0.35
rbac_select_page	3	1.44	1.1	0.91	2	0.56	0.57	0.57	2	0.56	0.59	0.61
rbac_select_count	3	1.07	0.99	0.93	2	0.48	0.48	0.41	2	0.48	0.45	0.43
rbac_update_one	3	0.65	0.68	0.5	3	0.66	0.58	0.46	3	0.63	0.95	0.49
write_assign_role	1	0.23	0.22	0.24	1	0.23	0.21	0.24	1	0.23	0.24	0.26
share_list_objects	2	0.38	0.43	0.36	1	0.41	0.4	0.37	13	5.57	15.8 ❌	3.87
share_select_page	6	1.35	1.29	1.27	3	1	0.89	0.9	42	16.96	24.18	16.47
write_share	1	0.23	0.21	0.25	1	0.22	0.2	0.23	1	0.22	0.4	0.24
parents_proj_select_page	4	2.62	1.55	1.57	4	1.26	1.14	1	51	18.27	21.68	18.83
parents_proj_select_count	9	14.66 ❌	12.36 ❌	9.97 ❌	4	1.47	1.4	1.18	31	16.48	18.05	7.17
parents_select_page	32	66.27 ❌	16.82	6.55	25	9.76	5.54	3.8	46	19.23	19.98	10.65
parents_select_count	70	92.44 ❌	27.61	15.53	23	8.06	6.01	2.69	40	16.64	17.51	7.48
write_nest_team	1	0.27	0.78	77.18 ❌	1	0.23	0.73	115.12 ❌	1	0.21	1.81 ❌	73.27 ❌
nest_check	2	0.48	0.42	0.45	2	0.47	0.44	0.41	2	0.48	0.61	0.44

The next table shows the failing budgets for each design, PG 18 -opt (the values for 16 and 17 are in parentheses where they differ):

Design	Large	Medium deep	Medium wide	Before (PG 18)
A	parents_proj_select_count, parents_select_page, parents_select_count (3)	none	none	large 3 (same), deep 2, wide 7
B	parents_proj_select_count (1). 16 also misses write_nest_team (p95 1.29 ms against 1). The repeat run of 17 also misses both single-branch parents budgets and write_nest_team (its original run passed them)	none	share_list_objects (15.8 against 13; 16 and 17 pass), write_nest_team (1.4-1.8 against 1)	large 3, deep 1, wide 7
C	parents_proj_select_count, write_nest_team (2)	write_nest_team	write_nest_team	large 1, deep 1, wide 1

• parents_proj_select_count at large (9 ms) fails for every design. Its p50 is about 8 ms for all three, because current minor releases read every heap page in that bitmap count (VERSIONS.md, Calibration). The budget was set on 16.4, which skips those reads. So it is stricter than any current Postgres release can meet for this query. Phase 6 later changed this budget to 20 ms (see D.md).
• A still misses the large single-branch parents budgets (32 and 70 ms), with p95 66 and 92 ms. A now passes every medium deep and medium wide budget.
• B now passes the large single-branch parents budgets (p95 16.8 and 27.6 ms on 18) and nest_check everywhere. write_nest_team passes at large and medium deep (p95 0.73-0.78 ms on 18) and misses at medium wide (1.4-1.8 ms).
• The only miss of C besides parents_proj_select_count is write_nest_team: p95 64-115 ms against a budget of 1 ms, down from 340-490 ms.
• We did not run small and medium default again after the optimizations. So the small --check result in ../goals.md is still the result before the optimizations.

Other differences

• rbac_check of A is 10-20% slower in every cell. On PG 18: large 0.26 → 0.30 ms, deep 0.25 → 0.29, wide 0.26 → 0.29. 16 and 17 are the same. It is the only consistent slowdown. Every other workload of A is the same or faster.
• The RBAC reads of B got faster on 16 and 17, but not on 18: rbac_check 1.2-1.3x and rbac_update_one 1.3-1.7x on 16 and 17. On 18: 1.0-1.2x. B made its hot wrappers PL/pgSQL. The reason is that Postgres 16 plans SQL function bodies again on every call (design B, step 3 in ../designs/b-policy-first.md). Postgres 18 already caches those plans, so there was less to gain there.
• The wide share_list_objects of B depends on the version. 8.96 → 4.47 ms on 16, 9.02 → 9.26 on 17, 11.4 → 8.41 on 18. Repeat runs reproduced the values for 17 and 18 (8.84 and 8.51). The wide page follows: 11.9 on 16 against 17.8-18.9 on 17 and 18. We did not investigate this.
• The p50 of parents_proj_select_count at large for C is 0.9x on 17 and 18 (7.48 → 7.88 ms on 18). It is about 1.0x on 16 (8.24 → 8.06, repeat run). This is within the noise of the heap-fetch floor.

Repeatability and replaced runs

We treated a run as disturbed by the host when 5 or more workloads had a p95 more than 2x their p50. This includes the RBAC workloads, which the optimizations barely touched. The load average stayed at 3-5 from desktop apps, even with no other agents. We ran each disturbed run again once, and replaced it when the rerun was cleaner. We also replaced the pg18 A large run. Its share_select_page (p50 3.08 ms, p95 31 ms, 214 tps, apparently a 2x regression) did not reproduce, and the repeat measured 1.03 / 1.35 ms.

run	original -opt run (workloads with p95 > 2× p50)	repeat	published
pg16 C large	10 (for example rbac_update_one p95 4.38 ms, parents_proj_select_count 23.6)	0	repeat
pg17 A large	9	3	repeat
pg17 B large	11	3 (but its parents phase was slower: page p50 20.1 against 14.8 ms)	repeat
pg18 A large	3, share_select_page 3.08 / 31.0 ms	0, 1.03 / 1.35 ms	repeat
pg16 A large (claim check)	1	5	original
pg16 B large (claim check)	1	1 (page 15.6 against 15.3 ms; count 36.0 against 28.4)	original
pg18 B large (check)	0	0 (page 13.67 against 13.65 ms)	original
pg17 B wide, pg18 B wide (check)	0, 1	1, 1 (share_list_objects 8.84, 8.51 against 9.26, 8.41)	original

The JSON of the repeats is in bench/results/optrerun-data/ (git-ignored), and their reports are in bench/results/. Between two runs of the same configuration, the p50 typically agrees within 5% for sub-millisecond workloads. It agrees within about 20% for the heavy workloads (parents_select_count of B on pg16 large: 28.4 against 36.0 ms).

Caveats

• One run for each configuration, 10 s for each workload, on a laptop that was not idle (load average 3-5).
• We measured the claims locally (16.4) with 1 client. These runs are Docker with 4 clients, and Docker runs the PL/pgSQL paths of A and B about 2x faster than local (VERSIONS.md). Compare the after values, not the factors.
• The budgets come from the local 16.4 runs. No design can meet the large parents_proj_select_count budget on current minor releases.
• We did not run the small and medium default configurations again.
• fsync=off, and every write rolled back: the write numbers are optimistic.