tests/test-lfs-pointer.py
author Pierre-Yves David <pierre-yves.david@octobus.net>
Tue, 09 Apr 2024 22:36:35 +0200
changeset 51594 e3a5ec2d236a
parent 48875 6000f5b25c9b
permissions -rw-r--r--
outgoing: rework the handling of the `missingroots` case to be faster The previous implementation was slow, to the point it was taking a significant amount of `hg bundle --type none-streamv2` call. We rework the code to compute the same value much faster, making the operation disappear from the `hg bundle --type none-streamv2` profile. Someone would remark that producing a streamclone does not requires an `outgoing` object. However that is a matter for another day. There is other user of `missingroots` (non stream `hg bundle` call for example), and they will also benefit from this rework. We implement an old TODO in the process, directly computing the missing and common attribute as we have most element at hand already. ### benchmark.name = hg.command.bundle # bin-env-vars.hg.flavor = default # bin-env-vars.hg.py-re2-module = default # benchmark.variants.revs = all # benchmark.variants.type = none-streamv2 ## data-env-vars.name = heptapod-public-2024-03-25-zstd-sparse-revlog before: 7.750458 after: 6.665565 (-14.00%, -1.08) ## data-env-vars.name = mercurial-public-2024-03-22-zstd-sparse-revlog before: 0.700229 after: 0.496050 (-29.16%, -0.20) ## data-env-vars.name = mozilla-try-2023-03-22-zstd-sparse-revlog before: 346.508952 after: 316.749699 (-8.59%, -29.76) ## data-env-vars.name = pypy-2024-03-22-zstd-sparse-revlog before: 3.401700 after: 2.915810 (-14.28%, -0.49) ## data-env-vars.name = tryton-public-2024-03-22-zstd-sparse-revlog before: 1.870798 after: 1.461583 (-21.87%, -0.41) note: this whole `missingroots` of outgoing has a limited number of callers and could likely be replace by something simpler (like taking an explicit "missing_revs" set for example). However this is a wider change and we focus on a small impact, quick rework that does not change the API for now.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
37928
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
     1
# Import something from Mercurial, so the module loader gets initialized.
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
     2
from mercurial import pycompat
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40098
diff changeset
     3
37928
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
     4
del pycompat  # unused for now
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
     5
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
     6
from hgext.lfs import pointer
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
     7
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40098
diff changeset
     8
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
     9
def tryparse(text):
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    10
    r = {}
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    11
    try:
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    12
        r = pointer.deserialize(text)
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    13
        print('ok')
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    14
    except Exception as ex:
37928
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
    15
        print((b'%s' % ex).decode('ascii'))
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    16
    if r:
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    17
        text2 = r.serialize()
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    18
        if text2 != text:
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    19
            print('reconstructed text differs')
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    20
    return r
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    21
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40098
diff changeset
    22
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40098
diff changeset
    23
t = (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40098
diff changeset
    24
    b'version https://git-lfs.github.com/spec/v1\n'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40098
diff changeset
    25
    b'oid sha256:4d7a214614ab2935c943f9e0ff69d22eadbb8f32b1'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40098
diff changeset
    26
    b'258daaa5e2ca24d17e2393\n'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40098
diff changeset
    27
    b'size 12345\n'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40098
diff changeset
    28
    b'x-foo extra-information\n'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40098
diff changeset
    29
)
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    30
37928
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
    31
tryparse(b'')
35098
66c5a8cf2868 lfs: import the Facebook git-lfs client extension
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    32
tryparse(t)
37928
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
    33
tryparse(t.replace(b'git-lfs', b'unknown'))
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
    34
tryparse(t.replace(b'v1\n', b'v1\n\n'))
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
    35
tryparse(t.replace(b'sha256', b'ahs256'))
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
    36
tryparse(t.replace(b'sha256:', b''))
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
    37
tryparse(t.replace(b'12345', b'0x12345'))
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
    38
tryparse(t.replace(b'extra-information', b'extra\0information'))
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
    39
tryparse(t.replace(b'extra-information', b'extra\ninformation'))
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
    40
tryparse(t.replace(b'x-foo', b'x_foo'))
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
    41
tryparse(t.replace(b'oid', b'blobid'))
7cd1e1adc471 tests: port test-lfs-pointer.py to Python 3
Augie Fackler <augie@google.com>
parents: 35098
diff changeset
    42
tryparse(t.replace(b'size', b'size-bytes').replace(b'oid', b'object-id'))