tests/test-remotefilelog-sparse.t
author Pierre-Yves David <pierre-yves.david@octobus.net>
Mon, 11 Dec 2023 22:27:59 +0100
changeset 51181 dcaa2df1f688
parent 46874 84a93fa7ecfd
permissions -rw-r--r--
changelog: never inline changelog The test suite mostly use small repositories, that implies that most changelog in the tests are inlined. As a result, non-inlined changelog are quite poorly tested. Since non-inline changelog are most common case for serious repositories, this lack of testing is a significant problem that results in high profile issue like the one recently fixed by 66417f55ea33 and 849745d7da89. Inlining the changelog does not bring much to the table, the number of total file saved is negligible, and the changelog will be read by most operation anyway. So this changeset is make it so we never inline the changelog, and de-inline the one that are still inlined whenever we touch them. By doing that, we remove the "dual code path" situation for writing new entry to the changelog and move to a "single code path" situation. Having a single code path simplify the code and make sure it is covered by test (if test cover that situation obviously) This impact all tests that care about the number of file and the exchange size, but there is nothing too complicated in them just a lot of churn. The churn is made "worse" by the fact rust will use the persistent nodemap on any changelog now. Which is overall a win as it means testing the persistent nodemap more and having less special cases. In short, having inline changelog is mostly useless and an endless source of pain. We get rid of it.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
40722
0800d9e6e216 tests: disable remotefilelog on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 40572
diff changeset
     1
#require no-windows
0800d9e6e216 tests: disable remotefilelog on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 40572
diff changeset
     2
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
     3
  $ . "$TESTDIR/remotefilelog-library.sh"
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
     4
40571
95a079ea1e19 tests: drop obsolete "hginit" alias in remotefilelog tests
Martin von Zweigbergk <martinvonz@google.com>
parents: 40495
diff changeset
     5
  $ hg init master
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
     6
  $ cd master
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
     7
  $ cat >> .hg/hgrc <<EOF
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
     8
  > [remotefilelog]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
     9
  > server=True
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    10
  > EOF
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    11
  $ echo x > x
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    12
  $ echo z > z
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    13
  $ hg commit -qAm x1
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    14
  $ echo x2 > x
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    15
  $ echo z2 > z
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    16
  $ hg commit -qAm x2
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    17
  $ hg bookmark foo
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    18
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    19
  $ cd ..
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    20
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    21
# prefetch a revision w/ a sparse checkout
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    22
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    23
  $ hgcloneshallow ssh://user@dummy/master shallow --noupdate
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    24
  streaming all changes
51181
dcaa2df1f688 changelog: never inline changelog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46874
diff changeset
    25
  3 files to transfer, 527 bytes of data (no-zstd !)
46874
84a93fa7ecfd revlog-compression: use zstd by default (if available)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46847
diff changeset
    26
  transferred 527 bytes in * seconds (* */sec) (glob) (no-zstd !)
51181
dcaa2df1f688 changelog: never inline changelog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46874
diff changeset
    27
  3 files to transfer, 534 bytes of data (zstd no-rust !)
dcaa2df1f688 changelog: never inline changelog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46874
diff changeset
    28
  transferred 534 bytes in * seconds (* */sec) (glob) (zstd no-rust !)
dcaa2df1f688 changelog: never inline changelog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46874
diff changeset
    29
  5 files to transfer, 660 bytes of data (zstd rust !)
dcaa2df1f688 changelog: never inline changelog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46874
diff changeset
    30
  transferred 660 bytes in * seconds (*/sec) (glob) (zstd rust !)
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    31
  searching for changes
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    32
  no changes found
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    33
  $ cd shallow
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    34
  $ printf "[extensions]\nsparse=\n" >> .hg/hgrc
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    35
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    36
  $ hg debugsparse -I x
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    37
  $ hg prefetch -r 0
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    38
  1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    39
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    40
  $ hg cat -r 0 x
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    41
  x
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    42
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    43
  $ hg debugsparse -I z
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    44
  $ hg prefetch -r 0
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    45
  1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    46
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    47
  $ hg cat -r 0 z
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    48
  z
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    49
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    50
# prefetch sparse only on pull when configured
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    51
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    52
  $ printf "[remotefilelog]\npullprefetch=bookmark()\n" >> .hg/hgrc
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    53
  $ hg strip tip
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    54
  saved backup bundle to $TESTTMP/shallow/.hg/strip-backup/876b1317060d-b2e91d8d-backup.hg (glob)
46847
47a9527731c3 remotefilelog: include file contents in bundles produced during strip
Kyle Lippincott <spectral@google.com>
parents: 45771
diff changeset
    55
  2 files fetched over 2 fetches - (2 misses, 0.00% hit ratio) over *s (glob)
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    56
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    57
  $ hg debugsparse --delete z
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    58
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    59
  $ clearcache
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    60
  $ hg pull
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    61
  pulling from ssh://user@dummy/master
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    62
  searching for changes
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    63
  adding changesets
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    64
  adding manifests
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    65
  adding file changes
42897
d7304434390f changegroup: move message about added changes to transaction summary
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 42223
diff changeset
    66
  updating bookmark foo
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    67
  added 1 changesets with 0 changes to 0 files
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    68
  new changesets 876b1317060d
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    69
  (run 'hg update' to get a working copy)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    70
  prefetching file contents
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    71
  1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    72
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    73
# Dont consider filtered files when doing copy tracing
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    74
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    75
## Push an unrelated commit
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    76
  $ cd ../
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    77
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    78
  $ hgcloneshallow ssh://user@dummy/master shallow2
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    79
  streaming all changes
51181
dcaa2df1f688 changelog: never inline changelog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46874
diff changeset
    80
  3 files to transfer, 527 bytes of data (no-zstd !)
46874
84a93fa7ecfd revlog-compression: use zstd by default (if available)
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46847
diff changeset
    81
  transferred 527 bytes in * seconds (*) (glob) (no-zstd !)
51181
dcaa2df1f688 changelog: never inline changelog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46874
diff changeset
    82
  3 files to transfer, 534 bytes of data (zstd no-rust !)
dcaa2df1f688 changelog: never inline changelog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46874
diff changeset
    83
  transferred 534 bytes in * seconds (* */sec) (glob) (zstd no-rust !)
dcaa2df1f688 changelog: never inline changelog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46874
diff changeset
    84
  5 files to transfer, 660 bytes of data (zstd rust !)
dcaa2df1f688 changelog: never inline changelog
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46874
diff changeset
    85
  transferred 660 bytes in * seconds (*/sec) (glob) (zstd rust !)
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    86
  searching for changes
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    87
  no changes found
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    88
  updating to branch default
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    89
  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    90
  1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    91
  $ cd shallow2
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    92
  $ printf "[extensions]\nsparse=\n" >> .hg/hgrc
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    93
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    94
  $ hg up -q 0
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    95
  2 files fetched over 1 fetches - (2 misses, 0.00% hit ratio) over *s (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    96
  $ touch a
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    97
  $ hg ci -Aqm a
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    98
  $ hg push -q -f
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    99
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   100
## Pull the unrelated commit and rebase onto it - verify unrelated file was not
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   101
pulled
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   102
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   103
  $ cd ../shallow
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   104
  $ hg up -q 1
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   105
  $ hg pull -q
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   106
  $ hg debugsparse -I z
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   107
  $ clearcache
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   108
  $ hg prefetch -r '. + .^' -I x -I z
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   109
  4 files fetched over 1 fetches - (4 misses, 0.00% hit ratio) over * (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   110
  $ hg rebase -d 2 --keep
45771
f90a5c211251 rebase: change and standarize template for rebase's one-line summary
Martin von Zweigbergk <martinvonz@google.com>
parents: 42897
diff changeset
   111
  rebasing 1:876b1317060d foo "x2"