changegroup: restore default node ordering (issue6001)
Changeset db5501d9 changed the default node ordering from "storage" to
"linearize".
While the new API is more explicit and cleaner, the "linearize" order is
problematic on certain repositories like netbeans where it makes bundling
slower the more nodes we bundle.
Pushing and pulling 100 changesets was ~20% slower and pushing and pulling
1000 changesets was ~600% slower.
A very quick analysis of profile traces showed that the pull operation was
taking more time creating the delta.
Putting back the old default order seems to be the safe option. With more time
during the next cycle, we can understand better the impact of sorting with the
DAG order by default, the source of the regression and how to mitigate it.
/!\ We are still waiting for the full performance impact but with this patch,
bundling and pulling locally (not on the performance workstation) 1000
changesets on the netbeans repository is as fast as before the regression.
Differential Revision: https://phab.mercurial-scm.org/D5196
--- a/mercurial/utils/storageutil.py Mon Oct 29 17:26:25 2018 +0100
+++ b/mercurial/utils/storageutil.py Wed Oct 31 12:08:37 2018 -0700
@@ -337,11 +337,11 @@
if nodesorder == 'nodes':
revs = [frev(n) for n in nodes]
- elif nodesorder == 'storage':
- revs = sorted(frev(n) for n in nodes)
- else:
+ elif nodesorder == 'linear':
revs = set(frev(n) for n in nodes)
revs = dagop.linearize(revs, store.parentrevs)
+ else: # storage and default
+ revs = sorted(frev(n) for n in nodes)
prevrev = None
--- a/tests/test-generaldelta.t Mon Oct 29 17:26:25 2018 +0100
+++ b/tests/test-generaldelta.t Wed Oct 31 12:08:37 2018 -0700
@@ -279,61 +279,61 @@
14 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg -R relax-chain debugdeltachain -m
rev chain# chainlen prev delta size rawsize chainsize ratio lindist extradist extraratio
- 0 1 1 -1 base 47 46 47 1.02174 47 0 0.00000
- 1 1 2 0 p1 58 92 105 1.14130 105 0 0.00000
- 2 1 3 1 p1 58 138 163 1.18116 163 0 0.00000
- 3 1 4 2 p1 58 184 221 1.20109 221 0 0.00000
- 4 1 5 3 p1 58 230 279 1.21304 279 0 0.00000
- 5 1 6 4 p1 58 276 337 1.22101 337 0 0.00000
- 6 1 7 5 p1 58 322 395 1.22671 395 0 0.00000
- 7 1 8 6 p1 58 368 453 1.23098 453 0 0.00000
- 8 1 9 7 p1 58 414 511 1.23430 511 0 0.00000
- 9 1 10 8 p1 58 460 569 1.23696 569 0 0.00000
- 10 1 11 9 p1 58 506 627 1.23913 627 0 0.00000
- 11 1 12 10 p1 58 552 685 1.24094 685 0 0.00000
- 12 1 13 11 p1 58 598 743 1.24247 743 0 0.00000
- 13 1 14 12 p1 58 644 801 1.24379 801 0 0.00000
- 14 1 15 13 p1 58 690 859 1.24493 859 0 0.00000
- 15 1 16 14 p1 58 736 917 1.24592 917 0 0.00000
- 16 1 17 15 p1 58 782 975 1.24680 975 0 0.00000
- 17 1 18 16 p1 58 828 1033 1.24758 1033 0 0.00000
- 18 1 19 17 p1 58 874 1091 1.24828 1091 0 0.00000
- 19 1 20 18 p1 58 920 1149 1.24891 1149 0 0.00000
- 20 1 21 19 p1 58 966 1207 1.24948 1207 0 0.00000
- 21 1 22 20 p1 58 1012 1265 1.25000 1265 0 0.00000
- 22 1 23 21 p1 58 1058 1323 1.25047 1323 0 0.00000
- 23 1 24 22 p1 58 1104 1381 1.25091 1381 0 0.00000
- 24 1 25 23 p1 58 1150 1439 1.25130 1439 0 0.00000
- 25 1 26 24 p1 58 1196 1497 1.25167 1497 0 0.00000
- 26 1 27 25 p1 58 1242 1555 1.25201 1555 0 0.00000
- 27 1 28 26 p1 58 1288 1613 1.25233 1613 0 0.00000
- 28 1 29 27 p1 58 1334 1671 1.25262 1671 0 0.00000
- 29 1 30 28 p1 58 1380 1729 1.25290 1729 0 0.00000
- 30 1 31 29 p1 58 1426 1787 1.25316 1787 0 0.00000
- 31 2 1 -1 base 46 45 46 1.02222 46 0 0.00000
- 32 2 2 31 p1 57 90 103 1.14444 103 0 0.00000
- 33 2 3 32 p1 57 135 160 1.18519 160 0 0.00000
- 34 2 4 33 p1 57 180 217 1.20556 217 0 0.00000
- 35 2 5 34 p1 57 225 274 1.21778 274 0 0.00000
- 36 2 6 35 p1 57 270 331 1.22593 331 0 0.00000
- 37 2 7 36 p1 58 316 389 1.23101 389 0 0.00000
- 38 2 8 37 p1 58 362 447 1.23481 447 0 0.00000
- 39 3 1 -1 base 46 45 46 1.02222 46 0 0.00000
- 40 3 2 39 p1 57 90 103 1.14444 103 0 0.00000
- 41 3 3 40 p1 57 135 160 1.18519 160 0 0.00000
- 42 3 4 41 p1 57 180 217 1.20556 217 0 0.00000
- 43 3 5 42 p1 58 226 275 1.21681 275 0 0.00000
- 44 3 6 43 p1 58 272 333 1.22426 333 0 0.00000
- 45 3 7 44 p1 58 318 391 1.22956 391 0 0.00000
- 46 3 8 45 p1 58 364 449 1.23352 449 0 0.00000
- 47 3 9 46 p1 58 410 507 1.23659 507 0 0.00000
- 48 3 10 47 p1 58 456 565 1.23904 565 0 0.00000
- 49 3 11 48 p1 58 502 623 1.24104 623 0 0.00000
- 50 3 12 49 p1 58 548 681 1.24270 681 0 0.00000
- 51 3 13 50 p1 58 594 739 1.24411 739 0 0.00000
- 52 3 14 51 p1 58 640 797 1.24531 797 0 0.00000
- 53 4 1 -1 base 0 0 0 0.00000 0 0 0.00000
- 54 5 1 -1 base 369 640 369 0.57656 369 0 0.00000
+ 0 1 1 -1 base 46 45 46 1.02222 46 0 0.00000
+ 1 1 2 0 p1 57 90 103 1.14444 103 0 0.00000
+ 2 1 3 1 p1 57 135 160 1.18519 160 0 0.00000
+ 3 1 4 2 p1 57 180 217 1.20556 217 0 0.00000
+ 4 1 5 3 p1 57 225 274 1.21778 274 0 0.00000
+ 5 1 6 4 p1 57 270 331 1.22593 331 0 0.00000
+ 6 2 1 -1 base 46 45 46 1.02222 46 0 0.00000
+ 7 2 2 6 p1 57 90 103 1.14444 103 0 0.00000
+ 8 2 3 7 p1 57 135 160 1.18519 160 0 0.00000
+ 9 2 4 8 p1 57 180 217 1.20556 217 0 0.00000
+ 10 2 5 9 p1 58 226 275 1.21681 275 0 0.00000
+ 11 2 6 10 p1 58 272 333 1.22426 333 0 0.00000
+ 12 2 7 11 p1 58 318 391 1.22956 391 0 0.00000
+ 13 2 8 12 p1 58 364 449 1.23352 449 0 0.00000
+ 14 2 9 13 p1 58 410 507 1.23659 507 0 0.00000
+ 15 2 10 14 p1 58 456 565 1.23904 565 0 0.00000
+ 16 2 11 15 p1 58 502 623 1.24104 623 0 0.00000
+ 17 2 12 16 p1 58 548 681 1.24270 681 0 0.00000
+ 18 3 1 -1 base 47 46 47 1.02174 47 0 0.00000
+ 19 3 2 18 p1 58 92 105 1.14130 105 0 0.00000
+ 20 3 3 19 p1 58 138 163 1.18116 163 0 0.00000
+ 21 3 4 20 p1 58 184 221 1.20109 221 0 0.00000
+ 22 3 5 21 p1 58 230 279 1.21304 279 0 0.00000
+ 23 3 6 22 p1 58 276 337 1.22101 337 0 0.00000
+ 24 3 7 23 p1 58 322 395 1.22671 395 0 0.00000
+ 25 3 8 24 p1 58 368 453 1.23098 453 0 0.00000
+ 26 3 9 25 p1 58 414 511 1.23430 511 0 0.00000
+ 27 3 10 26 p1 58 460 569 1.23696 569 0 0.00000
+ 28 3 11 27 p1 58 506 627 1.23913 627 0 0.00000
+ 29 3 12 28 p1 58 552 685 1.24094 685 0 0.00000
+ 30 3 13 29 p1 58 598 743 1.24247 743 0 0.00000
+ 31 3 14 30 p1 58 644 801 1.24379 801 0 0.00000
+ 32 3 15 31 p1 58 690 859 1.24493 859 0 0.00000
+ 33 3 16 32 p1 58 736 917 1.24592 917 0 0.00000
+ 34 3 17 33 p1 58 782 975 1.24680 975 0 0.00000
+ 35 3 18 34 p1 58 828 1033 1.24758 1033 0 0.00000
+ 36 3 19 35 p1 58 874 1091 1.24828 1091 0 0.00000
+ 37 3 20 36 p1 58 920 1149 1.24891 1149 0 0.00000
+ 38 3 21 37 p1 58 966 1207 1.24948 1207 0 0.00000
+ 39 3 22 38 p1 58 1012 1265 1.25000 1265 0 0.00000
+ 40 3 23 39 p1 58 1058 1323 1.25047 1323 0 0.00000
+ 41 3 24 40 p1 58 1104 1381 1.25091 1381 0 0.00000
+ 42 3 25 41 p1 58 1150 1439 1.25130 1439 0 0.00000
+ 43 3 26 42 p1 58 1196 1497 1.25167 1497 0 0.00000
+ 44 3 27 43 p1 58 1242 1555 1.25201 1555 0 0.00000
+ 45 3 28 44 p1 58 1288 1613 1.25233 1613 0 0.00000
+ 46 3 29 45 p1 58 1334 1671 1.25262 1671 0 0.00000
+ 47 3 30 46 p1 58 1380 1729 1.25290 1729 0 0.00000
+ 48 3 31 47 p1 58 1426 1787 1.25316 1787 0 0.00000
+ 49 4 1 -1 base 197 316 197 0.62342 197 0 0.00000
+ 50 4 2 49 p1 58 362 255 0.70442 255 0 0.00000
+ 51 2 13 17 p1 58 594 739 1.24411 2781 2042 2.76319
+ 52 5 1 -1 base 369 640 369 0.57656 369 0 0.00000
+ 53 6 1 -1 base 0 0 0 0.00000 0 0 0.00000
+ 54 7 1 -1 base 369 640 369 0.57656 369 0 0.00000
$ hg clone --pull source-repo --config experimental.maxdeltachainspan=0 noconst-chain --config format.generaldelta=yes
requesting all changes
adding changesets
@@ -345,58 +345,58 @@
14 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg -R noconst-chain debugdeltachain -m
rev chain# chainlen prev delta size rawsize chainsize ratio lindist extradist extraratio
- 0 1 1 -1 base 47 46 47 1.02174 47 0 0.00000
- 1 1 2 0 p1 58 92 105 1.14130 105 0 0.00000
- 2 1 3 1 p1 58 138 163 1.18116 163 0 0.00000
- 3 1 4 2 p1 58 184 221 1.20109 221 0 0.00000
- 4 1 5 3 p1 58 230 279 1.21304 279 0 0.00000
- 5 1 6 4 p1 58 276 337 1.22101 337 0 0.00000
- 6 1 7 5 p1 58 322 395 1.22671 395 0 0.00000
- 7 1 8 6 p1 58 368 453 1.23098 453 0 0.00000
- 8 1 9 7 p1 58 414 511 1.23430 511 0 0.00000
- 9 1 10 8 p1 58 460 569 1.23696 569 0 0.00000
- 10 1 11 9 p1 58 506 627 1.23913 627 0 0.00000
- 11 1 12 10 p1 58 552 685 1.24094 685 0 0.00000
- 12 1 13 11 p1 58 598 743 1.24247 743 0 0.00000
- 13 1 14 12 p1 58 644 801 1.24379 801 0 0.00000
- 14 1 15 13 p1 58 690 859 1.24493 859 0 0.00000
- 15 1 16 14 p1 58 736 917 1.24592 917 0 0.00000
- 16 1 17 15 p1 58 782 975 1.24680 975 0 0.00000
- 17 1 18 16 p1 58 828 1033 1.24758 1033 0 0.00000
- 18 1 19 17 p1 58 874 1091 1.24828 1091 0 0.00000
- 19 1 20 18 p1 58 920 1149 1.24891 1149 0 0.00000
- 20 1 21 19 p1 58 966 1207 1.24948 1207 0 0.00000
- 21 1 22 20 p1 58 1012 1265 1.25000 1265 0 0.00000
- 22 1 23 21 p1 58 1058 1323 1.25047 1323 0 0.00000
- 23 1 24 22 p1 58 1104 1381 1.25091 1381 0 0.00000
- 24 1 25 23 p1 58 1150 1439 1.25130 1439 0 0.00000
- 25 1 26 24 p1 58 1196 1497 1.25167 1497 0 0.00000
- 26 1 27 25 p1 58 1242 1555 1.25201 1555 0 0.00000
- 27 1 28 26 p1 58 1288 1613 1.25233 1613 0 0.00000
- 28 1 29 27 p1 58 1334 1671 1.25262 1671 0 0.00000
- 29 1 30 28 p1 58 1380 1729 1.25290 1729 0 0.00000
- 30 1 31 29 p1 58 1426 1787 1.25316 1787 0 0.00000
- 31 2 1 -1 base 46 45 46 1.02222 46 0 0.00000
- 32 2 2 31 p1 57 90 103 1.14444 103 0 0.00000
- 33 2 3 32 p1 57 135 160 1.18519 160 0 0.00000
- 34 2 4 33 p1 57 180 217 1.20556 217 0 0.00000
- 35 2 5 34 p1 57 225 274 1.21778 274 0 0.00000
- 36 2 6 35 p1 57 270 331 1.22593 331 0 0.00000
- 37 2 7 36 p1 58 316 389 1.23101 389 0 0.00000
- 38 2 8 37 p1 58 362 447 1.23481 447 0 0.00000
- 39 3 1 -1 base 46 45 46 1.02222 46 0 0.00000
- 40 3 2 39 p1 57 90 103 1.14444 103 0 0.00000
- 41 3 3 40 p1 57 135 160 1.18519 160 0 0.00000
- 42 3 4 41 p1 57 180 217 1.20556 217 0 0.00000
- 43 3 5 42 p1 58 226 275 1.21681 275 0 0.00000
- 44 3 6 43 p1 58 272 333 1.22426 333 0 0.00000
- 45 3 7 44 p1 58 318 391 1.22956 391 0 0.00000
- 46 3 8 45 p1 58 364 449 1.23352 449 0 0.00000
- 47 3 9 46 p1 58 410 507 1.23659 507 0 0.00000
- 48 3 10 47 p1 58 456 565 1.23904 565 0 0.00000
- 49 3 11 48 p1 58 502 623 1.24104 623 0 0.00000
- 50 3 12 49 p1 58 548 681 1.24270 681 0 0.00000
- 51 3 13 50 p1 58 594 739 1.24411 739 0 0.00000
- 52 3 14 51 p1 58 640 797 1.24531 797 0 0.00000
+ 0 1 1 -1 base 46 45 46 1.02222 46 0 0.00000
+ 1 1 2 0 p1 57 90 103 1.14444 103 0 0.00000
+ 2 1 3 1 p1 57 135 160 1.18519 160 0 0.00000
+ 3 1 4 2 p1 57 180 217 1.20556 217 0 0.00000
+ 4 1 5 3 p1 57 225 274 1.21778 274 0 0.00000
+ 5 1 6 4 p1 57 270 331 1.22593 331 0 0.00000
+ 6 2 1 -1 base 46 45 46 1.02222 46 0 0.00000
+ 7 2 2 6 p1 57 90 103 1.14444 103 0 0.00000
+ 8 2 3 7 p1 57 135 160 1.18519 160 0 0.00000
+ 9 2 4 8 p1 57 180 217 1.20556 217 0 0.00000
+ 10 2 5 9 p1 58 226 275 1.21681 275 0 0.00000
+ 11 2 6 10 p1 58 272 333 1.22426 333 0 0.00000
+ 12 2 7 11 p1 58 318 391 1.22956 391 0 0.00000
+ 13 2 8 12 p1 58 364 449 1.23352 449 0 0.00000
+ 14 2 9 13 p1 58 410 507 1.23659 507 0 0.00000
+ 15 2 10 14 p1 58 456 565 1.23904 565 0 0.00000
+ 16 2 11 15 p1 58 502 623 1.24104 623 0 0.00000
+ 17 2 12 16 p1 58 548 681 1.24270 681 0 0.00000
+ 18 3 1 -1 base 47 46 47 1.02174 47 0 0.00000
+ 19 3 2 18 p1 58 92 105 1.14130 105 0 0.00000
+ 20 3 3 19 p1 58 138 163 1.18116 163 0 0.00000
+ 21 3 4 20 p1 58 184 221 1.20109 221 0 0.00000
+ 22 3 5 21 p1 58 230 279 1.21304 279 0 0.00000
+ 23 3 6 22 p1 58 276 337 1.22101 337 0 0.00000
+ 24 3 7 23 p1 58 322 395 1.22671 395 0 0.00000
+ 25 3 8 24 p1 58 368 453 1.23098 453 0 0.00000
+ 26 3 9 25 p1 58 414 511 1.23430 511 0 0.00000
+ 27 3 10 26 p1 58 460 569 1.23696 569 0 0.00000
+ 28 3 11 27 p1 58 506 627 1.23913 627 0 0.00000
+ 29 3 12 28 p1 58 552 685 1.24094 685 0 0.00000
+ 30 3 13 29 p1 58 598 743 1.24247 743 0 0.00000
+ 31 3 14 30 p1 58 644 801 1.24379 801 0 0.00000
+ 32 3 15 31 p1 58 690 859 1.24493 859 0 0.00000
+ 33 3 16 32 p1 58 736 917 1.24592 917 0 0.00000
+ 34 3 17 33 p1 58 782 975 1.24680 975 0 0.00000
+ 35 3 18 34 p1 58 828 1033 1.24758 1033 0 0.00000
+ 36 3 19 35 p1 58 874 1091 1.24828 1091 0 0.00000
+ 37 3 20 36 p1 58 920 1149 1.24891 1149 0 0.00000
+ 38 3 21 37 p1 58 966 1207 1.24948 1207 0 0.00000
+ 39 3 22 38 p1 58 1012 1265 1.25000 1265 0 0.00000
+ 40 3 23 39 p1 58 1058 1323 1.25047 1323 0 0.00000
+ 41 3 24 40 p1 58 1104 1381 1.25091 1381 0 0.00000
+ 42 3 25 41 p1 58 1150 1439 1.25130 1439 0 0.00000
+ 43 3 26 42 p1 58 1196 1497 1.25167 1497 0 0.00000
+ 44 3 27 43 p1 58 1242 1555 1.25201 1555 0 0.00000
+ 45 3 28 44 p1 58 1288 1613 1.25233 1613 0 0.00000
+ 46 3 29 45 p1 58 1334 1671 1.25262 1671 0 0.00000
+ 47 3 30 46 p1 58 1380 1729 1.25290 1729 0 0.00000
+ 48 3 31 47 p1 58 1426 1787 1.25316 1787 0 0.00000
+ 49 1 7 5 p1 58 316 389 1.23101 2857 2468 6.34447
+ 50 1 8 49 p1 58 362 447 1.23481 2915 2468 5.52125
+ 51 2 13 17 p1 58 594 739 1.24411 2642 1903 2.57510
+ 52 2 14 51 p1 58 640 797 1.24531 2700 1903 2.38770
53 4 1 -1 base 0 0 0 0.00000 0 0 0.00000
54 5 1 -1 base 369 640 369 0.57656 369 0 0.00000