stream-clone: disable gc for the entry listing section for the v2 format
This is similar to the change we did for the v3 format in 6e4c8366c5ce.
The benchmark bellow show this gives us a notable gains, especially on larger
repositories.
### benchmark.name = hg.perf.stream-locked-section
# benchmark.name = hg.perf.stream-locked-section
# bin-env-vars.hg.flavor = default
# bin-env-vars.hg.py-re2-module = default
# benchmark.variants.version = v2
## data-env-vars.name = pypy-2018-08-01-zstd-sparse-revlog
5e931bf8707c: 0.503820 ~~~~~
1106d1bf695e: 0.470078 (-6.70%, -0.03)
## data-env-vars.name = pypy-2024-03-22-zstd-sparse-revlog
5e931bf8707c: 0.535756 ~~~~~
1106d1bf695e: 0.490249 (-8.49%, -0.05)
## data-env-vars.name = heptapod-public-2024-03-25-zstd-sparse-revlog
5e931bf8707c: 1.327041 ~~~~~
1106d1bf695e: 1.174636 (-11.48%, -0.15)
## data-env-vars.name = netbeans-2018-08-01-zstd-sparse-revlog
5e931bf8707c: 2.439158 ~~~~~
1106d1bf695e: 2.220515 (-8.96%, -0.22)
## data-env-vars.name = netbeans-2019-11-07-zstd-sparse-revlog
5e931bf8707c: 2.630794 ~~~~~
1106d1bf695e: 2.261473 (-14.04%, -0.37)
## data-env-vars.name = mozilla-central-2018-08-01-zstd-sparse-revlog
5e931bf8707c: 5.769002 ~~~~~
1106d1bf695e: 5.062000 (-12.26%, -0.71)
## data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog
5e931bf8707c: 13.351750 ~~~~~
1106d1bf695e: 12.346655 (-7.53%, -1.01)
## data-env-vars.name = mozilla-central-2024-03-22-zstd-sparse-revlog
5e931bf8707c: 10.772939 ~~~~~
1106d1bf695e: 9.495407 (-11.86%, -1.28)
## data-env-vars.name = mozilla-unified-2024-03-22-zstd-sparse-revlog
5e931bf8707c: 10.864297 ~~~~~
1106d1bf695e: 9.475597 (-12.78%, -1.39)
## data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog
5e931bf8707c: 17.448335 ~~~~~
1106d1bf695e: 16.027474 (-8.14%, -1.42)
#require no-windows
$ . "$TESTDIR/remotefilelog-library.sh"
$ hg init repo
$ cd repo
$ cat >> .hg/hgrc <<EOF
> [remotefilelog]
> server=True
> EOF
$ echo x > x
$ echo y > y
$ echo z > z
$ hg commit -qAm xy
$ cd ..
$ cat > cacheprocess-logger.py <<EOF
> import os
> import shutil
> import sys
> f = open('$TESTTMP/cachelog.log', 'w')
> srccache = os.path.join('$TESTTMP', 'oldhgcache')
> def log(message):
> f.write(message)
> f.flush()
> destcache = sys.argv[-1]
> try:
> while True:
> cmd = sys.stdin.readline().strip()
> log('got command %r\n' % cmd)
> if cmd == 'exit':
> sys.exit(0)
> elif cmd == 'get':
> count = int(sys.stdin.readline())
> log('client wants %r blobs\n' % count)
> wants = []
> for _ in range(count):
> key = sys.stdin.readline()[:-1]
> wants.append(key)
> if '\0' in key:
> _, key = key.split('\0')
> srcpath = os.path.join(srccache, key)
> if os.path.exists(srcpath):
> dest = os.path.join(destcache, key)
> destdir = os.path.dirname(dest)
> if not os.path.exists(destdir):
> os.makedirs(destdir)
> shutil.copyfile(srcpath, dest)
> else:
> # report a cache miss
> sys.stdout.write(key + '\n')
> sys.stdout.write('0\n')
> for key in sorted(wants):
> log('requested %r\n' % key)
> sys.stdout.flush()
> elif cmd == 'set':
> raise Exception('todo writing')
> else:
> raise Exception('unknown command! %r' % cmd)
> except Exception as e:
> log('Exception! %s\n' % e)
> raise
> EOF
$ cat >> $HGRCPATH <<EOF
> [remotefilelog]
> cacheprocess = "$PYTHON" $TESTTMP/cacheprocess-logger.py
> EOF
Test cache keys and cache misses.
$ hgcloneshallow ssh://user@dummy/repo clone -q
3 files fetched over 1 fetches - (3 misses, 0.00% hit ratio) over *s (glob)
$ cat cachelog.log
got command 'get'
client wants 3 blobs
requested 'master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/1406e74118627694268417491f018a4a883152f0'
requested 'master/39/5df8f7c51f007019cb30201c49e884b46b92fa/69a1b67522704ec122181c0890bd16e9d3e7516a'
requested 'master/95/cb0bfd2977c761298d9624e4b4d4c72a39974a/076f5e2225b3ff0400b98c92aa6cdf403ee24cca'
got command 'set'
Exception! todo writing
Test cache hits.
$ mv hgcache oldhgcache
$ rm cachelog.log
$ hgcloneshallow ssh://user@dummy/repo clone-cachehit -q
3 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over *s (glob)
$ cat cachelog.log | grep -v exit
got command 'get'
client wants 3 blobs
requested 'master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/1406e74118627694268417491f018a4a883152f0'
requested 'master/39/5df8f7c51f007019cb30201c49e884b46b92fa/69a1b67522704ec122181c0890bd16e9d3e7516a'
requested 'master/95/cb0bfd2977c761298d9624e4b4d4c72a39974a/076f5e2225b3ff0400b98c92aa6cdf403ee24cca'
$ cat >> $HGRCPATH <<EOF
> [remotefilelog]
> cacheprocess.includepath = yes
> EOF
Test cache keys and cache misses with includepath.
$ rm -r hgcache oldhgcache
$ rm cachelog.log
$ hgcloneshallow ssh://user@dummy/repo clone-withpath -q
3 files fetched over 1 fetches - (3 misses, 0.00% hit ratio) over *s (glob)
$ cat cachelog.log
got command 'get'
client wants 3 blobs
requested 'x\x00master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/1406e74118627694268417491f018a4a883152f0'
requested 'y\x00master/95/cb0bfd2977c761298d9624e4b4d4c72a39974a/076f5e2225b3ff0400b98c92aa6cdf403ee24cca'
requested 'z\x00master/39/5df8f7c51f007019cb30201c49e884b46b92fa/69a1b67522704ec122181c0890bd16e9d3e7516a'
got command 'set'
Exception! todo writing
Test cache hits with includepath.
$ mv hgcache oldhgcache
$ rm cachelog.log
$ hgcloneshallow ssh://user@dummy/repo clone-withpath-cachehit -q
3 files fetched over 1 fetches - (0 misses, 100.00% hit ratio) over *s (glob)
$ cat cachelog.log | grep -v exit
got command 'get'
client wants 3 blobs
requested 'x\x00master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/1406e74118627694268417491f018a4a883152f0'
requested 'y\x00master/95/cb0bfd2977c761298d9624e4b4d4c72a39974a/076f5e2225b3ff0400b98c92aa6cdf403ee24cca'
requested 'z\x00master/39/5df8f7c51f007019cb30201c49e884b46b92fa/69a1b67522704ec122181c0890bd16e9d3e7516a'