aggressivemergedeltas: enabled the option by default
authorPaul Morelle <paul.morelle@octobus.net>
Fri, 22 Jun 2018 01:42:38 +0200
changeset 38602 44f5acfb9ad2
parent 38601 af8d8513d7de
child 38603 4df549d0f1fd
aggressivemergedeltas: enabled the option by default The option has been around for a while (August 2015) but was never turned on by default. In-depth testing shows large wins for having that on with no significant drawbacks. When enabled, revlog consider delta against both p1 and p2 at the same time when storing a revision. Selecting a delta against "p2" can produce better deltas and chain. This raise large benefit for all repositories, especially if they have a lot of merges. Comparison of `.hg/store/` size: mercurial (6.74% merges): before: 54,225,348 bytes after: 47,279,959 bytes -13% pypy (8.30% merges): before: 459,041,759 bytes after: 346,090,067 bytes -25% netbeans (34.21% merges): before: 2,468,041,333 bytes after: 1,364,077,645 bytes -45% mozilla-central (4.84% merges): before: 2,731,799,546 bytes after: 2,157,718,019 bytes -21% Comparison of `00manifest.d` size: mercurial (6.74% merges): before: 11,682,516 bytes after: 6,143,044 bytes -47% pypy (8.30% merges): before: 156,447,163 bytes after: 52,941,780 bytes -66% netbeans (34.21% merges): before: 1,250,363,851 bytes after: 130,088,982 bytes -90% mozilla-central (4.84% merges): before: 468,202,733 bytes after: 215,096,339 bytes -54% In addition, the better deltas help with the performance of multiple core operations. However, better chains mean longer chains, which can affect performance negatively (mostly manifest revision retrieval time). Chains length is a deeper problem that also affects linear repository too. Overall we think the benefits of using p2 as a diff target are bigger than the downsizes. In addition, we are also working on ways to improve the performance impact of chain length, so theses downsizes get fixed in the future. Below are interesting items from the full benchmark run: bundling 100 revisions from pypy: before: 670ms after: 480ms -28% bundle 10000 revisions from pypy: before: 1.38s after: 1.10s -54% bundle 10000 revisions from pypy: before: 16.1s after: 7.81s -52% bundle 10000 revisions from netbeans: before: 19.3s after: 15.5s -19% unbundle 1000 revisions to pypy: before: 641ms after: 315ms - 51% clone mercurial (http): before: 26.0s after: 22.6s -23% pulling 1000 revisions from pypy (shh): before: 2.07s after: 1.36s -44% pushing 1000 revision through http (pypy repository) before: 2.18s after: 1.35s -48% diff time in mozilla-central: before: 1.420s after: 0.983s -31% status time in mozilla-central: before: 1.260s after: 0.828s -34% Impact in other cases seems minimal (within a couple of percent in worse cases) and can be seen in both direction: Timing for a simple `hg commit`: mozilla-central: before: 3.37s after: 3.22s -4% pypy: before: 194ms after: 197ms +2% Timing for status (from tip to parent of tip): mercurial: before: 52.4ms after: 52.4ms (same) pypy: before: 55.2 after: 56.9 +3% Timing for `hg update` mozilla-central, across 10 revisions: before: 4.82s after: 4.59s -5% mozilla-central, across 10000 revisions: before: 49.1s after: 49.9s +2% pypy, across 10 revisions: before: 213ms after: 216ms +1% pypy, across 10000 revisions: before: 5.31ms after: 5.24ms -1% The negative consequences are related to manifest fetch time: (timing for the tip revision tested by the benchmark) pypy-2018: before: 2.60ms after: 3.88ms +50% mozilla-central-2018: before: 565ms after: 652ms +15% (~+100ms) netbeans-2018: before: 101ms after: 250ms +48% (~+150ms) This shows up as a fixed overhead on some command we benchmarked: no-op push of mozilla-central: before: 945ms after: 1040ms +10% (~+100ms) pushing 10 changeset in netbeabs over ssh: before: 557ms after: 712ms +28% (+155ms) pushing 100 changeset in netbeabs over ssh: before: 592ms after: 771ms +30% (+179ms)
mercurial/configitems.py
mercurial/revlog.py
tests/test-generaldelta.t
--- a/mercurial/configitems.py	Mon Jul 09 09:50:23 2018 -0400
+++ b/mercurial/configitems.py	Fri Jun 22 01:42:38 2018 +0200
@@ -629,7 +629,7 @@
     generic=True,
 )
 coreconfigitem('format', 'aggressivemergedeltas',
-    default=False,
+    default=True,
 )
 coreconfigitem('format', 'chunkcachesize',
     default=None,
--- a/mercurial/revlog.py	Mon Jul 09 09:50:23 2018 -0400
+++ b/mercurial/revlog.py	Fri Jun 22 01:42:38 2018 +0200
@@ -615,7 +615,7 @@
         # How much data to read and cache into the raw revlog data cache.
         self._chunkcachesize = 65536
         self._maxchainlen = None
-        self._aggressivemergedeltas = False
+        self._aggressivemergedeltas = True
         self.index = []
         # Mapping of partial identifiers to full nodes.
         self._pcache = {}
--- a/tests/test-generaldelta.t	Mon Jul 09 09:50:23 2018 -0400
+++ b/tests/test-generaldelta.t	Fri Jun 22 01:42:38 2018 +0200
@@ -139,7 +139,7 @@
       rev  chain# chainlen     prev   delta       size    rawsize  chainsize     ratio   lindist extradist extraratio
         0       1        1       -1    base         59        215         59   0.27442        59         0    0.00000
         1       1        2        0    prev         61         86        120   1.39535       120         0    0.00000
-        2       1        3        1      p1         65        301        185   0.61462       185         0    0.00000
+        2       1        2        0      p2         62        301        121   0.40199       182        61    0.50413
 
   $ hg strip -q -r . --config extensions.strip=