Postgres versions
Do the designs behave in the same way on different Postgres major versions? This page shows every version that exists in
the benchmark data. Each run file has a pg_version, an environment and an optional tag. A series is one major version on one
environment with one tag. The page compares versions only within the same environment and tag. The optimized designs (tag opt) come first,
and the runs before the optimizations follow.
The Postgres versions report has the Docker analysis. Optimizations shows the effect of the optimization pass. The benchmarks dashboard describes the workloads and the charts.
3 Postgres versions available: PG 16, PG 17, PG 18. A series is one Postgres major version on one environment with one tag (the optimized designs have the tag opt). The page compares only series with the same environment and tag, so a version gap never includes a machine gap or an optimization gap. The optimized designs come first, and the runs before the optimizations follow. The page uses the latest run for each design, scale and shape.
| Series | Postgres | Runs | Designs | Configurations | Host |
|---|---|---|---|---|---|
| PG 16 (docker) | 16.15 | 9 | A, B, C | large default, medium deep, medium wide | Apple M4, 10 CPU, 24 GB; container cpus=4 mem=6g |
| PG 16 (docker, optimized) | 16.15 | 9 | A, B, C | large default, medium deep, medium wide | Apple M4, 10 CPU, 24 GB; container cpus=4 mem=6g |
| PG 17 (docker) | 17.11 | 9 | A, B, C | large default, medium deep, medium wide | Apple M4, 10 CPU, 24 GB; container cpus=4 mem=6g |
| PG 17 (docker, optimized) | 17.11 | 9 | A, B, C | large default, medium deep, medium wide | Apple M4, 10 CPU, 24 GB; container cpus=4 mem=6g |
| PG 18 (docker) | 18.6 | 9 | A, B, C | large default, medium deep, medium wide | Apple M4, 10 CPU, 24 GB; container cpus=4 mem=6g |
| PG 18 (docker, optimized) | 18.6 | 12 | A, B, C, D | large default, medium deep, medium wide | Apple M4, 10 CPU, 24 GB; container cpus=4 mem=6g |
| PG 18 (docker, opt2) | 18.6 | 3 | D | large default, medium deep, medium wide | Apple M4, 10 CPU, 24 GB; container cpus=4 mem=6g |
| PG 18 (docker, opt3) | 18.6 | 3 | D | large default, medium deep, medium wide | Apple M4, 10 CPU, 24 GB; container cpus=4 mem=6g |
| PG 18 (docker, p5) | 18.6 | 3 | E | large default, medium default, small default | Apple M4, 10 CPU, 24 GB; container cpus=4 mem=6g |
| PG 18 (docker, p6) | 18.6 | 12 | A, B, C, D | large default, medium default, small default | Apple M4, 10 CPU, 24 GB; container cpus=4 mem=6g |
| PG 18 (docker, p6check) | 18.6 | 1 | D | large default | Apple M4, 10 CPU, 24 GB; container cpus=4 mem=6g |
| PG 18 (docker, p9) | 18.6 | 2 | E | medium default, small default | Apple M4, 10 CPU, 24 GB; container cpus=4 mem=6g |
| PG 16 (local) | 16.4 | 15 | A, B, C | large default, medium default, small default, medium deep, medium wide | Apple M4, 10 CPU, 24 GB |
Docker, optimized designs (tag opt)
p50 latency by Postgres version
Key workloads, one panel for each workload. The columns are designs, and the groups are Postgres versions. Log axis, milliseconds.
- A Tuple core
- B Policy first
- C Compiled model
- D Per-model tables
- E Cedar
large default
RBAC: single check
rbac_check
Sharing: first page
share_select_page
Parents, projection: first page
parents_proj_select_page
Parents, single branch: first page
parents_select_page
Nesting: nest a team
write_nest_team
Nesting: check through a nested team
nest_check
Table view: large default, all workloads
| Workload (p50 ms) | PG 16 A | PG 16 B | PG 16 C | PG 17 A | PG 17 B | PG 17 C | PG 18 A | PG 18 B | PG 18 C |
|---|---|---|---|---|---|---|---|---|---|
| RBAC: single check | 0.32 | 0.27 | 0.27 | 0.34 | 0.27 | 0.27 | 0.30 | 0.27 | 0.24 |
| RBAC: first page | 0.75 | 0.75 | 0.75 | 0.77 | 0.74 | 0.73 | 0.76 | 0.73 | 0.67 |
| RBAC: count visible | 0.85 | 0.86 | 0.86 | 0.83 | 0.81 | 0.80 | 0.78 | 0.78 | 0.73 |
| RBAC: update one row | 0.60 | 0.49 | 0.51 | 0.57 | 0.47 | 0.51 | 0.47 | 0.47 | 0.36 |
| RBAC: assign a role | 0.17 | 0.16 | 0.22 | 0.16 | 0.14 | 0.20 | 0.15 | 0.14 | 0.17 |
| Sharing: list visible ids | 0.32 | 0.29 | 0.27 | 0.34 | 0.29 | 0.27 | 0.29 | 0.29 | 0.25 |
| Sharing: first page | 1.13 | 1.01 | 1.01 | 1.14 | 1.02 | 1.00 | 1.03 | 1.00 | 0.93 |
| Sharing: share one document | 0.17 | 0.15 | 0.19 | 0.15 | 0.15 | 0.17 | 0.15 | 0.14 | 0.17 |
| Parents, projection: first page | 1.40 | 1.17 | 1.09 | 1.36 | 1.15 | 1.06 | 1.33 | 1.15 | 1.05 |
| Parents, projection: count visible | 9.43 | 8.60 | 8.06 | 8.69 | 8.96 | 8.46 | 8.42 | 8.20 | 7.88 |
| Parents, single branch: first page | 46.0 | 15.3 | 5.26 | 47.0 | 20.1 | 5.04 | 45.6 | 13.7 | 4.94 |
| Parents, single branch: count visible | 79.6 | 28.4 | 18.6 | 68.2 | 27.3 | 12.3 | 75.1 | 21.6 | 11.9 |
| Nesting: nest a team | 0.16 | 0.72 | 55.0 | 0.15 | 0.70 | 56.9 | 0.15 | 0.62 | 57.7 |
| Nesting: check through a nested team | 0.30 | 0.32 | 0.32 | 0.31 | 0.32 | 0.32 | 0.31 | 0.32 | 0.30 |
medium deep
RBAC: single check
rbac_check
Sharing: first page
share_select_page
Parents, projection: first page
parents_proj_select_page
Parents, single branch: first page
parents_select_page
Nesting: nest a team
write_nest_team
Nesting: check through a nested team
nest_check
Table view: medium deep, all workloads
| Workload (p50 ms) | PG 16 A | PG 16 B | PG 16 C | PG 17 A | PG 17 B | PG 17 C | PG 18 A | PG 18 B | PG 18 C |
|---|---|---|---|---|---|---|---|---|---|
| RBAC: single check | 0.32 | 0.26 | 0.28 | 0.31 | 0.27 | 0.27 | 0.29 | 0.27 | 0.24 |
| RBAC: first page | 0.48 | 0.46 | 0.47 | 0.47 | 0.45 | 0.48 | 0.44 | 0.46 | 0.42 |
| RBAC: count visible | 0.41 | 0.36 | 0.39 | 0.38 | 0.36 | 0.38 | 0.35 | 0.37 | 0.32 |
| RBAC: update one row | 0.59 | 0.46 | 0.44 | 0.56 | 0.45 | 0.46 | 0.48 | 0.46 | 0.35 |
| RBAC: assign a role | 0.17 | 0.15 | 0.22 | 0.16 | 0.15 | 0.20 | 0.15 | 0.14 | 0.18 |
| Sharing: list visible ids | 0.34 | 0.30 | 0.28 | 0.34 | 0.30 | 0.29 | 0.31 | 0.32 | 0.28 |
| Sharing: first page | 0.83 | 0.76 | 0.79 | 0.83 | 0.71 | 0.69 | 0.76 | 0.72 | 0.66 |
| Sharing: share one document | 0.16 | 0.16 | 0.19 | 0.14 | 0.14 | 0.17 | 0.15 | 0.14 | 0.17 |
| Parents, projection: first page | 1.06 | 0.83 | 0.85 | 1.06 | 0.82 | 0.73 | 1.00 | 0.83 | 0.73 |
| Parents, projection: count visible | 1.43 | 1.22 | 1.10 | 1.40 | 1.21 | 1.10 | 1.25 | 1.14 | 0.99 |
| Parents, single branch: first page | 9.68 | 6.37 | 4.16 | 7.84 | 4.63 | 2.92 | 7.41 | 4.52 | 2.83 |
| Parents, single branch: count visible | 8.01 | 4.89 | 3.31 | 6.89 | 3.74 | 2.16 | 6.40 | 3.74 | 2.10 |
| Nesting: nest a team | 0.15 | 0.64 | 72.0 | 0.15 | 0.61 | 68.6 | 0.15 | 0.63 | 75.1 |
| Nesting: check through a nested team | 0.30 | 0.33 | 0.31 | 0.30 | 0.33 | 0.32 | 0.34 | 0.34 | 0.30 |
medium wide
RBAC: single check
rbac_check
Sharing: first page
share_select_page
Parents, projection: first page
parents_proj_select_page
Parents, single branch: first page
parents_select_page
Nesting: nest a team
write_nest_team
Nesting: check through a nested team
nest_check
Table view: medium wide, all workloads
| Workload (p50 ms) | PG 16 A | PG 16 B | PG 16 C | PG 17 A | PG 17 B | PG 17 C | PG 18 A | PG 18 B | PG 18 C |
|---|---|---|---|---|---|---|---|---|---|
| RBAC: single check | 0.35 | 0.27 | 0.27 | 0.31 | 0.28 | 0.28 | 0.29 | 0.28 | 0.24 |
| RBAC: first page | 0.49 | 0.46 | 0.48 | 0.48 | 0.47 | 0.48 | 0.44 | 0.46 | 0.44 |
| RBAC: count visible | 0.38 | 0.37 | 0.38 | 0.38 | 0.37 | 0.38 | 0.36 | 0.36 | 0.33 |
| RBAC: update one row | 0.53 | 0.47 | 0.47 | 0.56 | 0.46 | 0.51 | 0.49 | 0.48 | 0.38 |
| RBAC: assign a role | 0.17 | 0.16 | 0.21 | 0.16 | 0.15 | 0.20 | 0.15 | 0.14 | 0.18 |
| Sharing: list visible ids | 4.57 | 4.47 | 3.40 | 7.18 | 9.26 | 3.39 | 4.44 | 8.41 | 3.41 |
| Sharing: first page | 14.7 | 11.9 | 8.89 | 20.2 | 18.9 | 8.27 | 14.4 | 17.8 | 8.20 |
| Sharing: share one document | 0.16 | 0.15 | 0.19 | 0.15 | 0.14 | 0.17 | 0.15 | 0.16 | 0.17 |
| Parents, projection: first page | 15.3 | 13.9 | 8.62 | 15.4 | 13.8 | 8.28 | 15.0 | 15.1 | 8.73 |
| Parents, projection: count visible | 16.2 | 14.5 | 8.30 | 13.8 | 12.4 | 6.30 | 13.2 | 13.4 | 6.33 |
| Parents, single branch: first page | 18.8 | 17.0 | 12.2 | 14.9 | 13.2 | 8.16 | 14.7 | 13.7 | 8.06 |
| Parents, single branch: count visible | 17.1 | 15.1 | 10.2 | 13.3 | 11.4 | 6.78 | 13.3 | 12.2 | 6.61 |
| Nesting: nest a team | 0.16 | 1.40 | 60.8 | 0.15 | 1.23 | 61.9 | 0.15 | 1.47 | 61.2 |
| Nesting: check through a nested team | 0.33 | 0.33 | 0.33 | 0.32 | 0.33 | 0.33 | 0.37 | 0.34 | 0.31 |
Takeaway. In this group the largest gap between versions is Sharing: first page on B (medium wide): 11.9 ms on PG 16 against 18.9 ms on PG 17 (1.6x). Gaps under about 20% are noise.
Raw report: bench/baselines/data
Docker, opt2 designs (tag opt2)
This group has only PG 18 (docker, opt2), so nothing is available to compare yet.
Docker, opt3 designs (tag opt3)
This group has only PG 18 (docker, opt3), so nothing is available to compare yet.
Docker, p5 designs (tag p5)
This group has only PG 18 (docker, p5), so nothing is available to compare yet.
Docker, p6 designs (tag p6)
This group has only PG 18 (docker, p6), so nothing is available to compare yet.
Docker, p6check designs (tag p6check)
This group has only PG 18 (docker, p6check), so nothing is available to compare yet.
Docker, p9 designs (tag p9)
This group has only PG 18 (docker, p9), so nothing is available to compare yet.
Docker, untagged runs (before the optimization pass)
p50 latency by Postgres version
Key workloads, one panel for each workload. The columns are designs, and the groups are Postgres versions. Log axis, milliseconds.
- A Tuple core
- B Policy first
- C Compiled model
- D Per-model tables
- E Cedar
large default
RBAC: single check
rbac_check
Sharing: first page
share_select_page
Parents, projection: first page
parents_proj_select_page
Parents, single branch: first page
parents_select_page
Nesting: nest a team
write_nest_team
Nesting: check through a nested team
nest_check
Table view: large default, all workloads
| Workload (p50 ms) | PG 16 A | PG 16 B | PG 16 C | PG 17 A | PG 17 B | PG 17 C | PG 18 A | PG 18 B | PG 18 C |
|---|---|---|---|---|---|---|---|---|---|
| RBAC: single check | 0.30 | 0.35 | 0.28 | 0.29 | 0.35 | 0.28 | 0.26 | 0.26 | 0.25 |
| RBAC: first page | 0.84 | 0.84 | 0.85 | 0.83 | 0.83 | 0.86 | 0.76 | 0.77 | 0.73 |
| RBAC: count visible | 0.93 | 0.94 | 0.94 | 0.90 | 0.89 | 0.92 | 0.85 | 0.85 | 0.82 |
| RBAC: update one row | 0.67 | 0.81 | 0.66 | 0.69 | 0.75 | 0.71 | 0.53 | 0.52 | 0.45 |
| RBAC: assign a role | 0.17 | 0.17 | 0.22 | 0.16 | 0.15 | 0.20 | 0.16 | 0.14 | 0.18 |
| Sharing: list visible ids | 0.35 | 0.31 | 0.28 | 0.36 | 0.31 | 0.28 | 0.34 | 0.29 | 0.26 |
| Sharing: first page | 1.55 | 1.25 | 1.11 | 1.56 | 1.24 | 1.11 | 1.45 | 1.16 | 1.00 |
| Sharing: share one document | 0.16 | 0.16 | 0.19 | 0.15 | 0.14 | 0.17 | 0.15 | 0.14 | 0.17 |
| Parents, projection: first page | 2.21 | 1.41 | 1.12 | 2.27 | 1.38 | 1.09 | 2.09 | 1.30 | 1.05 |
| Parents, projection: count visible | 9.12 | 8.37 | 8.24 | 8.98 | 8.19 | 7.96 | 8.56 | 7.79 | 7.48 |
| Parents, single branch: first page | 242 | 95.4 | 10.7 | 246 | 94.7 | 10.5 | 240 | 102 | 9.26 |
| Parents, single branch: count visible | 261 | 116 | 23.1 | 266 | 112 | 18.1 | 269 | 122 | 16.9 |
| Nesting: nest a team | 0.16 | 107 | 299 | 0.15 | 112 | 303 | 0.15 | 107 | 300 |
| Nesting: check through a nested team | 0.67 | 1.30 | 0.33 | 0.69 | 1.31 | 0.33 | 0.52 | 1.14 | 0.30 |
medium deep
RBAC: single check
rbac_check
Sharing: first page
share_select_page
Parents, projection: first page
parents_proj_select_page
Parents, single branch: first page
parents_select_page
Nesting: nest a team
write_nest_team
Nesting: check through a nested team
nest_check
Table view: medium deep, all workloads
| Workload (p50 ms) | PG 16 A | PG 16 B | PG 16 C | PG 17 A | PG 17 B | PG 17 C | PG 18 A | PG 18 B | PG 18 C |
|---|---|---|---|---|---|---|---|---|---|
| RBAC: single check | 0.29 | 0.35 | 0.28 | 0.29 | 0.34 | 0.28 | 0.25 | 0.26 | 0.25 |
| RBAC: first page | 0.54 | 0.56 | 0.57 | 0.53 | 0.54 | 0.56 | 0.47 | 0.50 | 0.48 |
| RBAC: count visible | 0.46 | 0.45 | 0.47 | 0.45 | 0.44 | 0.46 | 0.39 | 0.39 | 0.37 |
| RBAC: update one row | 0.68 | 0.79 | 0.68 | 0.63 | 0.74 | 0.63 | 0.45 | 0.51 | 0.43 |
| RBAC: assign a role | 0.17 | 0.16 | 0.22 | 0.16 | 0.15 | 0.20 | 0.15 | 0.14 | 0.18 |
| Sharing: list visible ids | 0.42 | 0.48 | 0.29 | 0.43 | 0.50 | 0.29 | 0.42 | 0.48 | 0.28 |
| Sharing: first page | 1.52 | 1.65 | 0.89 | 1.57 | 1.74 | 0.82 | 1.42 | 1.68 | 0.77 |
| Sharing: share one document | 0.17 | 0.15 | 0.19 | 0.15 | 0.14 | 0.17 | 0.15 | 0.14 | 0.17 |
| Parents, projection: first page | 2.13 | 1.15 | 0.80 | 2.16 | 1.13 | 0.75 | 2.01 | 1.05 | 0.71 |
| Parents, projection: count visible | 2.46 | 1.49 | 1.11 | 2.48 | 1.47 | 1.11 | 2.25 | 1.31 | 0.97 |
| Parents, single branch: first page | 48.8 | 12.6 | 5.24 | 49.8 | 11.3 | 4.08 | 47.8 | 10.9 | 3.74 |
| Parents, single branch: count visible | 48.3 | 11.1 | 4.22 | 48.4 | 10.2 | 3.31 | 46.2 | 9.89 | 3.04 |
| Nesting: nest a team | 0.16 | 136 | 353 | 0.15 | 134 | 334 | 0.15 | 124 | 307 |
| Nesting: check through a nested team | 0.49 | 1.31 | 0.32 | 0.66 | 1.34 | 0.32 | 0.63 | 1.18 | 0.30 |
medium wide
RBAC: single check
rbac_check
Sharing: first page
share_select_page
Parents, projection: first page
parents_proj_select_page
Parents, single branch: first page
parents_select_page
Nesting: nest a team
write_nest_team
Nesting: check through a nested team
nest_check
Table view: medium wide, all workloads
| Workload (p50 ms) | PG 16 A | PG 16 B | PG 16 C | PG 17 A | PG 17 B | PG 17 C | PG 18 A | PG 18 B | PG 18 C |
|---|---|---|---|---|---|---|---|---|---|
| RBAC: single check | 0.29 | 0.35 | 0.28 | 0.29 | 0.35 | 0.28 | 0.26 | 0.29 | 0.24 |
| RBAC: first page | 0.54 | 0.57 | 0.57 | 0.54 | 0.55 | 0.57 | 0.50 | 0.57 | 0.48 |
| RBAC: count visible | 0.46 | 0.46 | 0.48 | 0.45 | 0.44 | 0.47 | 0.41 | 0.42 | 0.38 |
| RBAC: update one row | 0.67 | 0.81 | 0.67 | 0.67 | 0.79 | 0.67 | 0.53 | 0.57 | 0.46 |
| RBAC: assign a role | 0.17 | 0.17 | 0.22 | 0.16 | 0.15 | 0.20 | 0.16 | 0.15 | 0.17 |
| Sharing: list visible ids | 24.5 | 8.96 | 4.08 | 26.1 | 9.02 | 4.23 | 29.1 | 11.4 | 4.25 |
| Sharing: first page | 98.6 | 33.6 | 9.40 | 104 | 33.8 | 9.59 | 125 | 36.9 | 10.3 |
| Sharing: share one document | 0.16 | 0.16 | 0.19 | 0.15 | 0.14 | 0.17 | 0.16 | 0.15 | 0.18 |
| Parents, projection: first page | 100 | 36.2 | 9.61 | 107 | 36.7 | 9.75 | 123 | 40.7 | 11.1 |
| Parents, projection: count visible | 99.5 | 36.3 | 10.2 | 106 | 34.6 | 8.04 | 112 | 38.5 | 8.97 |
| Parents, single branch: first page | 112 | 36.9 | 14.3 | 118 | 32.9 | 10.8 | 118 | 39.3 | 11.8 |
| Parents, single branch: count visible | 113 | 35.5 | 12.8 | 115 | 31.3 | 9.32 | 121 | 41.2 | 9.40 |
| Nesting: nest a team | 0.16 | 124 | 148 | 0.15 | 134 | 174 | 0.15 | 133 | 181 |
| Nesting: check through a nested team | 1.33 | 1.34 | 0.34 | 1.35 | 1.41 | 0.34 | 1.34 | 1.22 | 0.31 |
Takeaway. In this group the largest gap between versions is Parents, single branch: first page on C (medium deep): 3.74 ms on PG 18 against 5.24 ms on PG 16 (1.4x). Gaps under about 20% are noise.
Raw report: bench/baselines/data
Local, untagged runs (before the optimization pass)
This group has only PG 16 (local), so nothing is available to compare yet.