tests/test-remotefilelog-strip.t
author Pierre-Yves David <pierre-yves.david@octobus.net>
Fri, 05 Apr 2024 11:05:54 +0200
changeset 51576 de5bf3fe0233
parent 46847 47a9527731c3
permissions -rw-r--r--
revset: stop serializing node when using "%ln" Turning hundred of thousand of node from node to hex and back can be slow… what about we stop doing it? In many case were we are using node id we should be using revision id. However this is not a good reason to have a stupidly slow implementation of "%ln". This caught my attention again because the phase discovery during push make an extensive use of "%ln" or huge set. In absolute, that phase discovery probably should use "%ld" and need to improves its algorithmic complexity, but improving "%ln" seems simple and long overdue. This greatly speeds up `hg push` on repository with many drafts. Here are some relevant poulpe benchmarks: ### data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog # benchmark.name = hg.command.push # bin-env-vars.hg.flavor = default # bin-env-vars.hg.py-re2-module = default # benchmark.variants.explicit-rev = all-out-heads # benchmark.variants.issue6528 = disabled # benchmark.variants.protocol = ssh # benchmark.variants.reuse-external-delta-parent = default ## benchmark.variants.revs = any-1-extra-rev before: 44.235070 after: 20.416329 (-53.85%, -23.82) ## benchmark.variants.revs = any-100-extra-rev before: 49.234697 after: 26.519829 (-46.14%, -22.71) ### benchmark.name = hg.command.bundle # bin-env-vars.hg.flavor = default # bin-env-vars.hg.py-re2-module = default # benchmark.variants.revs = all # benchmark.variants.type = none-streamv2 ## data-env-vars.name = heptapod-public-2024-03-25-zstd-sparse-revlog before: 10.138396 after: 7.750458 (-23.55%, -2.39) ## data-env-vars.name = mercurial-public-2024-03-22-zstd-sparse-revlog before: 1.263859 after: 0.700229 (-44.60%, -0.56) ## data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog before: 399.484481 after: 346.5089 (-13.26%, -52.98) ## data-env-vars.name = pypy-2024-03-22-zstd-sparse-revlog before: 4.540080 after: 3.401700 (-25.07%, -1.14) ## data-env-vars.name = tryton-public-2024-03-22-zstd-sparse-revlog before: 2.975765 after: 1.870798 (-37.13%, -1.10)
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
46846
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
     1
#require no-windows
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
     2
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
     3
  $ . "$TESTDIR/remotefilelog-library.sh"
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
     4
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
     5
  $ hg init master
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
     6
  $ cd master
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
     7
  $ cat >> .hg/hgrc <<EOF
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
     8
  > [remotefilelog]
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
     9
  > server=True
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    10
  > EOF
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    11
  $ echo x > x
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    12
  $ hg commit -qAm x
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    13
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    14
  $ cd ..
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    15
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    16
  $ hgcloneshallow ssh://user@dummy/master shallow -q
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    17
  1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob)
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    18
  $ cd shallow
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    19
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    20
  $ cat >> $TESTTMP/get_file_linknode.py <<EOF
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    21
  > from mercurial import node, registrar, scmutil
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    22
  > cmdtable = {}
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    23
  > command = registrar.command(cmdtable)
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    24
  > @command(b'debug-file-linknode', [(b'r', b'rev', b'.', b'rev')], b'hg debug-file-linknode FILE')
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    25
  > def debug_file_linknode(ui, repo, file, **opts):
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    26
  >   rflctx = scmutil.revsingle(repo.unfiltered(), opts['rev']).filectx(file)
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    27
  >   ui.status(b'%s\n' % node.hex(rflctx.ancestormap()[rflctx._filenode][2]))
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    28
  > EOF
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    29
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    30
  $ cat >> .hg/hgrc <<EOF
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    31
  > [ui]
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    32
  > interactive=1
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    33
  > [extensions]
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    34
  > strip=
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    35
  > get_file_linknode=$TESTTMP/get_file_linknode.py
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    36
  > [experimental]
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    37
  > evolution=createmarkers,allowunstable
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    38
  > EOF
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    39
  $ echo a > a
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    40
  $ hg commit -qAm msg1
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    41
  $ hg commit --amend 're:^$' -m msg2
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    42
  $ hg commit --amend 're:^$' -m msg3
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    43
  $ hg --hidden log -G -T '{rev} {node|short}'
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    44
  @  3 df91f74b871e
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    45
  |
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    46
  | x  2 70494d7ec5ef
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    47
  |/
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    48
  | x  1 1e423846dde0
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    49
  |/
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    50
  o  0 b292c1e3311f
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    51
  
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    52
  $ hg debug-file-linknode -r 70494d a
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    53
  df91f74b871e064c89afa1fe9e2f66afa2c125df
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    54
  $ hg --hidden strip -r 1 3
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    55
  0 files updated, 0 files merged, 1 files removed, 0 files unresolved
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    56
  saved backup bundle to $TESTTMP/shallow/.hg/strip-backup/df91f74b871e-c94d67be-backup.hg
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    57
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    58
  $ hg --hidden log -G -T '{rev} {node|short}'
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    59
  o  1 70494d7ec5ef
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    60
  |
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    61
  @  0 b292c1e3311f
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    62
  
46847
47a9527731c3 remotefilelog: include file contents in bundles produced during strip
Kyle Lippincott <spectral@google.com>
parents: 46846
diff changeset
    63
Demonstrate that the linknode points to a commit that is actually in the repo
47a9527731c3 remotefilelog: include file contents in bundles produced during strip
Kyle Lippincott <spectral@google.com>
parents: 46846
diff changeset
    64
after the strip operation. Otherwise remotefilelog has to search every commit in
47a9527731c3 remotefilelog: include file contents in bundles produced during strip
Kyle Lippincott <spectral@google.com>
parents: 46846
diff changeset
    65
the repository looking for a valid linkrev every time it's queried, such as
47a9527731c3 remotefilelog: include file contents in bundles produced during strip
Kyle Lippincott <spectral@google.com>
parents: 46846
diff changeset
    66
during push.
46846
2819df466cae tests: add test-remotefilelog-strip.t to demonstrate an issue with linknodes
Kyle Lippincott <spectral@google.com>
parents:
diff changeset
    67
  $ hg debug-file-linknode -r 70494d a
46847
47a9527731c3 remotefilelog: include file contents in bundles produced during strip
Kyle Lippincott <spectral@google.com>
parents: 46846
diff changeset
    68
  70494d7ec5ef6cd3cd6939a9fd2812f9956bf553