hgext/narrow/narrowwirepeer.py
author Martin von Zweigbergk <martinvonz@google.com>
Fri, 05 Oct 2018 11:07:34 -0700
changeset 40344 2c5835b4246b
parent 40212 885d5cf9b6a4
child 42415 c767e655ffda
permissions -rw-r--r--
narrow: when widening, don't include manifests the client already has When widening, we already don't include the changelog (since f1844a10ee19) and files that the client already has (since c73c7653dfb9). However, we still include all manifests needed for the new narrowspec. When using flat manifests, that means we resend all the manifests even though the client necessarily has all of them. For tree manifests, we unnecessarily resend the root manifests and any subdirectory manifests that the client already has. This patch makes it so we no longer resend manifests that the client already has. It does so by passing an extra matcher to the changegroup packer and it uses that for filtering out directories matching the old matcher's visitdir(). For consistency between directories and files, it also makes the filtering of files look at both old and new matcher rather than passing in a diff matcher as we did before. Differential Revision: https://phab.mercurial-scm.org/D4895
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
     1
# narrowwirepeer.py - passes narrow spec with unbundle command
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
     2
#
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
     3
# Copyright 2017 Google, Inc.
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
     4
#
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
     7
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
     8
from __future__ import absolute_import
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
     9
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
    10
from mercurial import (
40070
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    11
    bundle2,
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    12
    error,
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
    13
    extensions,
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
    14
    hg,
40070
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    15
    narrowspec,
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    16
    pycompat,
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    17
    wireprototypes,
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    18
    wireprotov1peer,
39523
c90514043eaa narrow: add narrow and ellipses as server capabilities
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 36351
diff changeset
    19
    wireprotov1server,
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
    20
)
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
    21
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
    22
def uisetup():
40070
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    23
    wireprotov1peer.wirepeer.narrow_widen = peernarrowwiden
39523
c90514043eaa narrow: add narrow and ellipses as server capabilities
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 36351
diff changeset
    24
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
    25
def reposetup(repo):
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
    26
    def wirereposetup(ui, peer):
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
    27
        def wrapped(orig, cmd, *args, **kwargs):
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
    28
            if cmd == 'unbundle':
36102
b60c577b6e03 narrowwirepeer: add TODO about how we add wireproto args to unbundle :(
Augie Fackler <augie@google.com>
parents: 36101
diff changeset
    29
                # TODO: don't blindly add include/exclude wireproto
b60c577b6e03 narrowwirepeer: add TODO about how we add wireproto args to unbundle :(
Augie Fackler <augie@google.com>
parents: 36101
diff changeset
    30
                # arguments to unbundle.
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
    31
                include, exclude = repo.narrowpats
36351
87e950a070e6 narrowwirepeer: add some strkwargs to fix a crash on py3
Augie Fackler <augie@google.com>
parents: 36160
diff changeset
    32
                kwargs[r"includepats"] = ','.join(include)
87e950a070e6 narrowwirepeer: add some strkwargs to fix a crash on py3
Augie Fackler <augie@google.com>
parents: 36160
diff changeset
    33
                kwargs[r"excludepats"] = ','.join(exclude)
36079
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
    34
            return orig(cmd, *args, **kwargs)
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
    35
        extensions.wrapfunction(peer, '_calltwowaystream', wrapped)
a2a6e724d61a narrow: import experimental extension from narrowhg revision cb51d673e9c5
Augie Fackler <augie@google.com>
parents:
diff changeset
    36
    hg.wirepeersetupfuncs.append(wirereposetup)
40070
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    37
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    38
@wireprotov1server.wireprotocommand('narrow_widen', 'oldincludes oldexcludes'
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    39
                                                    ' newincludes newexcludes'
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    40
                                                    ' commonheads cgversion'
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    41
                                                    ' known ellipses',
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    42
                                    permission='pull')
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    43
def narrow_widen(repo, proto, oldincludes, oldexcludes, newincludes,
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    44
                 newexcludes, commonheads, cgversion, known, ellipses):
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    45
    """wireprotocol command to send data when a narrow clone is widen. We will
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    46
    be sending a changegroup here.
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    47
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    48
    The current set of arguments which are required:
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    49
    oldincludes: the old includes of the narrow copy
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    50
    oldexcludes: the old excludes of the narrow copy
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    51
    newincludes: the new includes of the narrow copy
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    52
    newexcludes: the new excludes of the narrow copy
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    53
    commonheads: list of heads which are common between the server and client
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    54
    cgversion(maybe): the changegroup version to produce
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    55
    known: list of nodes which are known on the client (used in ellipses cases)
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    56
    ellipses: whether to send ellipses data or not
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    57
    """
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    58
40148
74558635dad5 narrow: don't compress the bundle2 when sending 'error:abort'
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40075
diff changeset
    59
    preferuncompressed = False
40070
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    60
    try:
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    61
        oldincludes = wireprototypes.decodelist(oldincludes)
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    62
        newincludes = wireprototypes.decodelist(newincludes)
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    63
        oldexcludes = wireprototypes.decodelist(oldexcludes)
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    64
        newexcludes = wireprototypes.decodelist(newexcludes)
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    65
        # validate the patterns
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    66
        narrowspec.validatepatterns(set(oldincludes))
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    67
        narrowspec.validatepatterns(set(newincludes))
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    68
        narrowspec.validatepatterns(set(oldexcludes))
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    69
        narrowspec.validatepatterns(set(newexcludes))
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    70
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    71
        common = wireprototypes.decodelist(commonheads)
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    72
        known = None
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    73
        if known:
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    74
            known = wireprototypes.decodelist(known)
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    75
        if ellipses == '0':
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    76
            ellipses = False
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    77
        else:
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    78
            ellipses = bool(ellipses)
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    79
        cgversion = cgversion
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    80
        newmatch = narrowspec.match(repo.root, include=newincludes,
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    81
                                    exclude=newexcludes)
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    82
        oldmatch = narrowspec.match(repo.root, include=oldincludes,
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    83
                                    exclude=oldexcludes)
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    84
40344
2c5835b4246b narrow: when widening, don't include manifests the client already has
Martin von Zweigbergk <martinvonz@google.com>
parents: 40212
diff changeset
    85
        bundler = bundle2.widen_bundle(repo, oldmatch, newmatch, common, known,
40071
e8132a8897da narrow: start returning bundle2 from widen_bundle()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40070
diff changeset
    86
                                             cgversion, ellipses)
40070
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    87
    except error.Abort as exc:
40071
e8132a8897da narrow: start returning bundle2 from widen_bundle()
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40070
diff changeset
    88
        bundler = bundle2.bundle20(repo.ui)
40070
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    89
        manargs = [('message', pycompat.bytestr(exc))]
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    90
        advargs = []
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    91
        if exc.hint is not None:
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    92
            advargs.append(('hint', exc.hint))
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    93
        bundler.addpart(bundle2.bundlepart('error:abort', manargs, advargs))
40148
74558635dad5 narrow: don't compress the bundle2 when sending 'error:abort'
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40075
diff changeset
    94
        preferuncompressed = True
40070
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    95
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    96
    chunks = bundler.getchunks()
40148
74558635dad5 narrow: don't compress the bundle2 when sending 'error:abort'
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40075
diff changeset
    97
    return wireprototypes.streamres(gen=chunks,
74558635dad5 narrow: don't compress the bundle2 when sending 'error:abort'
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40075
diff changeset
    98
                                    prefer_uncompressed=preferuncompressed)
40070
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
    99
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
   100
def peernarrowwiden(remote, **kwargs):
40212
885d5cf9b6a4 py3: add some r'' prefixes in hgext/narrow/narrowwirepeer.py
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40148
diff changeset
   101
    for ch in (r'oldincludes', r'newincludes', r'oldexcludes', r'newexcludes',
885d5cf9b6a4 py3: add some r'' prefixes in hgext/narrow/narrowwirepeer.py
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40148
diff changeset
   102
               r'commonheads', r'known'):
40070
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
   103
        kwargs[ch] = wireprototypes.encodelist(kwargs[ch])
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
   104
40212
885d5cf9b6a4 py3: add some r'' prefixes in hgext/narrow/narrowwirepeer.py
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 40148
diff changeset
   105
    kwargs[r'ellipses'] = '%i' % bool(kwargs[r'ellipses'])
40070
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
   106
    f = remote._callcompressable('narrow_widen', **kwargs)
8feae5b989bc narrow: the first version of narrow_widen wireprotocol command
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 39934
diff changeset
   107
    return bundle2.getunbundler(remote.ui, f)