tests/test-narrow-clone-stream.t
author Pierre-Yves David <pierre-yves.david@octobus.net>
Tue, 09 Apr 2024 22:37:15 +0200
changeset 51595 3a6fae3bef35
parent 51181 dcaa2df1f688
permissions -rw-r--r--
outgoing: add a simple fastpath when there is no common This further speed up case like `hg bundle --all` for larger repository. ### data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog # benchmark.name = hg.command.bundle # bin-env-vars.hg.flavor = default # bin-env-vars.hg.py-re2-module = default # benchmark.variants.revs = all # benchmark.variants.type = none-streamv2 before: 316.749699 after: 311.165461 (-1.76%, -5.58) There is further work to be done in this area like not doing any outgoing computation in the stream case for example. however the recent changes already gives use a large win for a small amount of local work. ### benchmark.name = hg.command.bundle # bin-env-vars.hg.flavor = default # bin-env-vars.hg.py-re2-module = default # benchmark.variants.revs = all # benchmark.variants.type = none-streamv2 ## data-env-vars.name = mercurial-public-2024-03-22-zstd-sparse-revlog pre-%ln-change: 1.263859 the-%ln-change: 0.700229 (-44.60%, -0.56) prev-changeset: 0.496050 (-60.75%, -0.77) this-changeset: 0.495243 (-60.81%, -0.77) ## data-env-vars.name = tryton-public-2024-03-22-zstd-sparse-revlog pre-%ln-change: 2.975765 the-%ln-change: 1.870798 (-37.13%, -1.10) prev-changeset: 1.461583 (-50.88%, -1.51) this-changeset: 1.469185 (-50.63%, -1.51) ## data-env-vars.name = pypy-2024-03-22-zstd-sparse-revlog pre-%ln-change: 4.540080 the-%ln-change: 3.401700 (-25.07%, -1.14) prev-changeset: 2.915810 (-35.78%, -1.62) this-changeset: 2.911643 (-35.87%, -1.63) ## data-env-vars.name = heptapod-public-2024-03-25-zstd-sparse-revlog pre-%ln-change: 10.138396 the-%ln-change: 7.750458 (-23.55%, -2.39) prev-changeset: 6.665565 (-34.25%, -3.47) this-changeset: 6.672078 (-34.19%, -3.47) ## data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog pre-%ln-change: 399.484481 the-%ln-change: 346.508952 (-13.26%, -52.98) prev-changeset: 316.749699 (-20.71%, -82.73) this-changeset: 311.165461 (-22.11%, -88.32)
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
40583
186921bc8e66 test-narrow-clone-stream: include no usefncache case which is broken
Yuya Nishihara <yuya@tcha.org>
parents: 40582
diff changeset
     1
#testcases tree flat-fncache flat-nofncache
40494
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
     2
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
     3
Tests narrow stream clones
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
     4
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
     5
  $ . "$TESTDIR/narrow-library.sh"
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
     6
40494
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
     7
#if tree
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
     8
  $ cat << EOF >> $HGRCPATH
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
     9
  > [experimental]
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    10
  > treemanifest = 1
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    11
  > EOF
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    12
#endif
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    13
40583
186921bc8e66 test-narrow-clone-stream: include no usefncache case which is broken
Yuya Nishihara <yuya@tcha.org>
parents: 40582
diff changeset
    14
#if flat-nofncache
186921bc8e66 test-narrow-clone-stream: include no usefncache case which is broken
Yuya Nishihara <yuya@tcha.org>
parents: 40582
diff changeset
    15
  $ cat << EOF >> $HGRCPATH
186921bc8e66 test-narrow-clone-stream: include no usefncache case which is broken
Yuya Nishihara <yuya@tcha.org>
parents: 40582
diff changeset
    16
  > [format]
186921bc8e66 test-narrow-clone-stream: include no usefncache case which is broken
Yuya Nishihara <yuya@tcha.org>
parents: 40582
diff changeset
    17
  > usefncache = 0
186921bc8e66 test-narrow-clone-stream: include no usefncache case which is broken
Yuya Nishihara <yuya@tcha.org>
parents: 40582
diff changeset
    18
  > EOF
186921bc8e66 test-narrow-clone-stream: include no usefncache case which is broken
Yuya Nishihara <yuya@tcha.org>
parents: 40582
diff changeset
    19
#endif
186921bc8e66 test-narrow-clone-stream: include no usefncache case which is broken
Yuya Nishihara <yuya@tcha.org>
parents: 40582
diff changeset
    20
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    21
Server setup
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    22
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    23
  $ hg init master
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    24
  $ cd master
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    25
  $ mkdir dir
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    26
  $ mkdir dir/src
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    27
  $ cd dir/src
40582
a4c2788e8f48 test-narrow-clone-stream: include uppercase letter in filename
Yuya Nishihara <yuya@tcha.org>
parents: 40549
diff changeset
    28
  $ for x in `$TESTDIR/seq.py 20`; do echo $x > "F$x"; hg add "F$x"; hg commit -m "Commit src $x"; done
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    29
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    30
  $ cd ..
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    31
  $ mkdir tests
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    32
  $ cd tests
40582
a4c2788e8f48 test-narrow-clone-stream: include uppercase letter in filename
Yuya Nishihara <yuya@tcha.org>
parents: 40549
diff changeset
    33
  $ for x in `$TESTDIR/seq.py 20`; do echo $x > "F$x"; hg add "F$x"; hg commit -m "Commit src $x"; done
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    34
  $ cd ../../..
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    35
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    36
Trying to stream clone when the server does not support it
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    37
40582
a4c2788e8f48 test-narrow-clone-stream: include uppercase letter in filename
Yuya Nishihara <yuya@tcha.org>
parents: 40549
diff changeset
    38
  $ hg clone --narrow ssh://user@dummy/master narrow --noupdate --include "dir/src/F10" --stream
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    39
  streaming all changes
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    40
  remote: abort: server does not support narrow stream clones
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    41
  abort: pull failed on remote
46977
3f87d2af0bd6 errors: raise RemoteError in some places in exchange.py
Martin von Zweigbergk <martinvonz@google.com>
parents: 46884
diff changeset
    42
  [100]
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    43
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    44
Enable stream clone on the server
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    45
40549
d6ec45b79277 tests: fix config knob in test-narrow-clone-stream.t
Augie Fackler <augie@google.com>
parents: 40494
diff changeset
    46
  $ echo "[experimental]" >> master/.hg/hgrc
d6ec45b79277 tests: fix config knob in test-narrow-clone-stream.t
Augie Fackler <augie@google.com>
parents: 40494
diff changeset
    47
  $ echo "server.stream-narrow-clones=True" >> master/.hg/hgrc
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    48
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    49
Cloning a specific file when stream clone is supported
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    50
40582
a4c2788e8f48 test-narrow-clone-stream: include uppercase letter in filename
Yuya Nishihara <yuya@tcha.org>
parents: 40549
diff changeset
    51
  $ hg clone --narrow ssh://user@dummy/master narrow --noupdate --include "dir/src/F10" --stream
40338
af62936c2508 streamclone: new server config and some API changes for narrow stream clones
Pulkit Goyal <pulkit@yandex-team.ru>
parents:
diff changeset
    52
  streaming all changes
40494
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    53
  * files to transfer, * KB of data (glob)
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    54
  transferred * KB in * seconds (* */sec) (glob)
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    55
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    56
  $ cd narrow
44724
5c2a4f37eace tests: deal with "ls" vs "ls -A" difference on 2BSD derived systems
Joerg Sonnenberger <joerg@bec.de>
parents: 42505
diff changeset
    57
  $ ls -A
5c2a4f37eace tests: deal with "ls" vs "ls -A" difference on 2BSD derived systems
Joerg Sonnenberger <joerg@bec.de>
parents: 42505
diff changeset
    58
  .hg
40494
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    59
  $ hg tracked
40582
a4c2788e8f48 test-narrow-clone-stream: include uppercase letter in filename
Yuya Nishihara <yuya@tcha.org>
parents: 40549
diff changeset
    60
  I path:dir/src/F10
40494
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    61
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    62
Making sure we have the correct set of requirements
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    63
48464
5e6542143d40 test: use `hg debugrequires` instead of `cat` in some tests
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48321
diff changeset
    64
  $ hg debugrequires
42505
c1850798f995 run-tests: stop matching line for missing feature
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41226
diff changeset
    65
  dotencode (tree !)
c1850798f995 run-tests: stop matching line for missing feature
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41226
diff changeset
    66
  dotencode (flat-fncache !)
48321
f35529784079 dirstate-v2: fix confusion between requirement and format config variable
Raphaël Gomès <rgomes@octobus.net>
parents: 48295
diff changeset
    67
  dirstate-v2 (dirstate-v2 !)
42505
c1850798f995 run-tests: stop matching line for missing feature
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41226
diff changeset
    68
  fncache (tree !)
c1850798f995 run-tests: stop matching line for missing feature
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41226
diff changeset
    69
  fncache (flat-fncache !)
40494
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    70
  generaldelta
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    71
  narrowhg-experimental
46884
cc3ad5c3af3b persistent-nodemap: enable the feature by default when using Rust
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46874
diff changeset
    72
  persistent-nodemap (rust !)
46874
84a93fa7ecfd revlog-compression: use zstd by default (if available)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 44724
diff changeset
    73
  revlog-compression-zstd (zstd !)
40494
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    74
  revlogv1
48669
7ee07e1a25c0 share-safe: enable by default (BC)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48464
diff changeset
    75
  share-safe
40899
192291ca909b test: enable sparse-revlog for test-narrow-clone-stream.t
Boris Feld <boris.feld@octobus.net>
parents: 40584
diff changeset
    76
  sparserevlog
40494
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    77
  store
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    78
  treemanifest (tree !)
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    79
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    80
Making sure store has the required files
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    81
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    82
  $ ls .hg/store/
51181
dcaa2df1f688 changelog: never inline changelog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50193
diff changeset
    83
  00changelog-????????.nd (glob) (rust !)
dcaa2df1f688 changelog: never inline changelog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50193
diff changeset
    84
  00changelog.d
40494
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    85
  00changelog.i
51181
dcaa2df1f688 changelog: never inline changelog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 50193
diff changeset
    86
  00changelog.n (rust !)
40494
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    87
  00manifest.i
42505
c1850798f995 run-tests: stop matching line for missing feature
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41226
diff changeset
    88
  data
c1850798f995 run-tests: stop matching line for missing feature
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41226
diff changeset
    89
  fncache (tree !)
c1850798f995 run-tests: stop matching line for missing feature
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 41226
diff changeset
    90
  fncache (flat-fncache !)
40494
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    91
  meta (tree !)
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    92
  narrowspec
48669
7ee07e1a25c0 share-safe: enable by default (BC)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 48464
diff changeset
    93
  requires
40494
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    94
  undo
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    95
  undo.backupfiles
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    96
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    97
Checking that repository has all the required data and not broken
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    98
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
    99
  $ hg verify
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
   100
  checking changesets
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
   101
  checking manifests
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
   102
  checking directory manifests (tree !)
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
   103
  crosschecking files in changesets and manifests
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
   104
  checking files
49826
c84844cd523a verify: also check dirstate
Raphaël Gomès <rgomes@octobus.net>
parents: 48669
diff changeset
   105
  checking dirstate
40494
9aeb9e2d28a7 store: introduce _matchtrackedpath() and use it to filter store files
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40338
diff changeset
   106
  checked 40 changesets with 1 changes to 1 files