mercurial/wireprotov1server.py
author Pierre-Yves David <pierre-yves.david@octobus.net>
Thu, 15 Oct 2020 15:57:36 +0200
changeset 45784 74271829ddc0
parent 45682 d2e1dcd4490d
child 45785 80f32ec8653a
permissions -rw-r--r--
clonebundles: move a bundle of clone bundle related code to a new module In the process on general clone bundle automatically, we need to make some function available more widely. This is a good opportunity to extract a significant amount of code from `mercurial.exchange` into a new `mercurial.bundlecaches`. This make `mercurial.exchange` move under the 3K line range (hooray…). The module is called `bundlecaches` because I expect it to be eventually useful for more than just clone bundle (like pull bunbles). Differential Revision: https://phab.mercurial-scm.org/D9208
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
37785
b4d85bc122bd wireproto: rename wireproto to wireprotov1server (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37784
diff changeset
     1
# wireprotov1server.py - Wire protocol version 1 server functionality
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     2
#
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     3
# Copyright 2005-2010 Matt Mackall <mpm@selenic.com>
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     4
#
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     7
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
     8
from __future__ import absolute_import
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
     9
41468
9cb51e74e9ad wireprotov1server: use binascii.unhexlify
Gregory Szorc <gregory.szorc@gmail.com>
parents: 41365
diff changeset
    10
import binascii
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
    11
import os
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    12
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
    13
from .i18n import _
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
    14
from .node import (
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
    15
    hex,
32260
d0d9a4fca59b clone: add a server-side option to disable full getbundles (pull-based clones)
Siddharth Agarwal <sid0@fb.com>
parents: 31451
diff changeset
    16
    nullid,
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
    17
)
43089
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
    18
from .pycompat import getattr
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
    19
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
    20
from . import (
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
    21
    bundle2,
45784
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45682
diff changeset
    22
    bundlecaches,
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
    23
    changegroup as changegroupmod,
34097
f7d41b85bbf6 changegroup: replace changegroupsubset with makechangegroup
Durham Goode <durham@fb.com>
parents: 34062
diff changeset
    24
    discovery,
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
    25
    encoding,
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
    26
    error,
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
    27
    exchange,
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
    28
    pushkey as pushkeymod,
30924
48dea083f66d py3: convert the mode argument of os.fdopen to unicodes (1 of 2)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30914
diff changeset
    29
    pycompat,
26443
d947086d8973 streamclone: move code out of exchange.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25993
diff changeset
    30
    streamclone,
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
    31
    util,
36073
cd6ab329c5c7 wireprototypes: move wire protocol response types to new module
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36071
diff changeset
    32
    wireprototypes,
25993
0851678be71b wireproto: use absolute_import
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25913
diff changeset
    33
)
20903
8d477543882b wireproto: introduce an abstractserverproto class
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20902
diff changeset
    34
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37053
diff changeset
    35
from .utils import (
37119
d4a2e0d5d042 procutil: bulk-replace util.std* to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
    36
    procutil,
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37053
diff changeset
    37
    stringutil,
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37053
diff changeset
    38
)
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37053
diff changeset
    39
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
    40
urlerr = util.urlerr
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
    41
urlreq = util.urlreq
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
    42
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    43
bundle2requiredmain = _(b'incompatible Mercurial client; bundle2 required')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
    44
bundle2requiredhint = _(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43106
diff changeset
    45
    b'see https://www.mercurial-scm.org/wiki/IncompatibleClient'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
    46
)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    47
bundle2required = b'%s\n(%s)\n' % (bundle2requiredmain, bundle2requiredhint)
27246
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
    48
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
    49
37393
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
    50
def clientcompressionsupport(proto):
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
    51
    """Returns a list of compression methods supported by the client.
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
    52
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
    53
    Returns a list of the compression methods supported by the client
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
    54
    according to the protocol capabilities. If no such capability has
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
    55
    been announced, fallback to the default of zlib and uncompressed.
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
    56
    """
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
    57
    for cap in proto.getprotocaps():
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    58
        if cap.startswith(b'comp='):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    59
            return cap[5:].split(b',')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    60
    return [b'zlib', b'none']
37393
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
    61
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
    62
20902
1e4fda2f5cf1 wireproto: document possible return type
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20775
diff changeset
    63
# wire protocol command can either return a string or one of these classes.
15017
f4522df38c65 wireproto: add out-of-band error class to allow remote repo to report errors
Andrew Pritchard <andrewp@fogcreek.com>
parents: 14970
diff changeset
    64
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
    65
29590
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
    66
def getdispatchrepo(repo, proto, command):
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
    67
    """Obtain the repo used for processing wire protocol commands.
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
    68
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
    69
    The intent of this function is to serve as a monkeypatch point for
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
    70
    extensions that need commands to operate on different repo views under
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
    71
    specialized circumstances.
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
    72
    """
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    73
    viewconfig = repo.ui.config(b'server', b'view')
41840
d6569f1e9b37 server: allow customizing the default repo filter
Joerg Sonnenberger <joerg@bec.de>
parents: 41468
diff changeset
    74
    return repo.filtered(viewconfig)
29590
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
    75
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
    76
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    77
def dispatch(repo, proto, command):
29590
84c1a5942f1d wireproto: extract repo filtering to standalone function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29341
diff changeset
    78
    repo = getdispatchrepo(repo, proto, command)
37295
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37293
diff changeset
    79
37782
99accae4cc59 wireproto: reimplement dispatch() for version 2 server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37781
diff changeset
    80
    func, spec = commands[command]
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    81
    args = proto.getargs(spec)
37485
0b7475ea38cf wireproto: port heads command to wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37414
diff changeset
    82
37782
99accae4cc59 wireproto: reimplement dispatch() for version 2 server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37781
diff changeset
    83
    return func(repo, proto, *args)
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
    84
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
    85
13721
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
    86
def options(cmd, keys, others):
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
    87
    opts = {}
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
    88
    for k in keys:
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
    89
        if k in others:
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
    90
            opts[k] = others[k]
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
    91
            del others[k]
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
    92
    if others:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
    93
        procutil.stderr.write(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    94
            b"warning: %s ignored unexpected arguments %s\n"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    95
            % (cmd, b",".join(others))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
    96
        )
13721
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
    97
    return opts
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
    98
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
    99
27633
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   100
def bundle1allowed(repo, action):
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   101
    """Whether a bundle1 operation is allowed from the server.
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   102
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   103
    Priority is:
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   104
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   105
    1. server.bundle1gd.<action> (if generaldelta active)
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   106
    2. server.bundle1.<action>
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   107
    3. server.bundle1gd (if generaldelta active)
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   108
    4. server.bundle1
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   109
    """
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   110
    ui = repo.ui
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   111
    gd = b'generaldelta' in repo.requirements
27633
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   112
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   113
    if gd:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   114
        v = ui.configbool(b'server', b'bundle1gd.%s' % action)
27633
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   115
        if v is not None:
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   116
            return v
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   117
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   118
    v = ui.configbool(b'server', b'bundle1.%s' % action)
27246
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
   119
    if v is not None:
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
   120
        return v
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
   121
27633
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   122
    if gd:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   123
        v = ui.configbool(b'server', b'bundle1gd')
27633
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   124
        if v is not None:
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   125
            return v
37d7cf569cf3 wireproto: support disabling bundle1 only if repo is generaldelta
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27246
diff changeset
   126
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   127
    return ui.configbool(b'server', b'bundle1')
27246
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
   128
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   129
37781
352932a11905 wireproto: move command registration types to wireprototypes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37780
diff changeset
   130
commands = wireprototypes.commanddict()
20906
7a634b34fc91 wireproto: add decorator for wire protocol command
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
   131
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   132
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   133
def wireprotocommand(name, args=None, permission=b'push'):
35980
b4976912a6ef wireproto: improve docstring for @wireprotocommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35979
diff changeset
   134
    """Decorator to declare a wire protocol command.
b4976912a6ef wireproto: improve docstring for @wireprotocommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35979
diff changeset
   135
b4976912a6ef wireproto: improve docstring for @wireprotocommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35979
diff changeset
   136
    ``name`` is the name of the wire protocol command being provided.
b4976912a6ef wireproto: improve docstring for @wireprotocommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35979
diff changeset
   137
37535
69e46c1834ac wireproto: define and expose types of wire command arguments
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37534
diff changeset
   138
    ``args`` defines the named arguments accepted by the command. It is
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
   139
    a space-delimited list of argument names. ``*`` denotes a special value
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
   140
    that says to accept all named arguments.
36800
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36760
diff changeset
   141
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36760
diff changeset
   142
    ``permission`` defines the permission type needed to run this command.
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36760
diff changeset
   143
    Can be ``push`` or ``pull``. These roughly map to read-write and read-only,
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36760
diff changeset
   144
    respectively. Default is to assume command requires ``push`` permissions
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36760
diff changeset
   145
    because otherwise commands not declaring their permissions could modify
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36760
diff changeset
   146
    a repository that is supposed to be read-only.
35980
b4976912a6ef wireproto: improve docstring for @wireprotocommand
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35979
diff changeset
   147
    """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   148
    transports = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   149
        k for k, v in wireprototypes.TRANSPORTS.items() if v[b'version'] == 1
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   150
    }
36609
abc3b9801563 wireproto: allow wire protocol commands to declare transport support
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36535
diff changeset
   151
37053
cd0ca979a8b8 wireproto: nominally don't expose "batch" to version 2 wire transports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36991
diff changeset
   152
    # Because SSHv2 is a mirror of SSHv1, we allow "batch" commands through to
cd0ca979a8b8 wireproto: nominally don't expose "batch" to version 2 wire transports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36991
diff changeset
   153
    # SSHv2.
cd0ca979a8b8 wireproto: nominally don't expose "batch" to version 2 wire transports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36991
diff changeset
   154
    # TODO undo this hack when SSH is using the unified frame protocol.
cd0ca979a8b8 wireproto: nominally don't expose "batch" to version 2 wire transports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36991
diff changeset
   155
    if name == b'batch':
cd0ca979a8b8 wireproto: nominally don't expose "batch" to version 2 wire transports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36991
diff changeset
   156
        transports.add(wireprototypes.SSHV2)
cd0ca979a8b8 wireproto: nominally don't expose "batch" to version 2 wire transports
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36991
diff changeset
   157
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   158
    if permission not in (b'push', b'pull'):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   159
        raise error.ProgrammingError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   160
            b'invalid wire protocol permission; '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   161
            b'got %s; expected "push" or "pull"' % permission
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   162
        )
36800
0b18604db95e wireproto: declare permissions requirements in @wireprotocommand (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36760
diff changeset
   163
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
   164
    if args is None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   165
        args = b''
37535
69e46c1834ac wireproto: define and expose types of wire command arguments
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37534
diff changeset
   166
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
   167
    if not isinstance(args, bytes):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   168
        raise error.ProgrammingError(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43106
diff changeset
   169
            b'arguments for version 1 commands must be declared as bytes'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   170
        )
37535
69e46c1834ac wireproto: define and expose types of wire command arguments
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37534
diff changeset
   171
20906
7a634b34fc91 wireproto: add decorator for wire protocol command
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
   172
    def register(func):
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37779
diff changeset
   173
        if name in commands:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   174
            raise error.ProgrammingError(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43106
diff changeset
   175
                b'%s command already registered for version 1' % name
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   176
            )
37781
352932a11905 wireproto: move command registration types to wireprototypes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37780
diff changeset
   177
        commands[name] = wireprototypes.commandentry(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   178
            func, args=args, transports=transports, permission=permission
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   179
        )
37295
45b39c69fae0 wireproto: separate commands tables for version 1 and 2 commands
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37293
diff changeset
   180
20906
7a634b34fc91 wireproto: add decorator for wire protocol command
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
   181
        return func
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   182
20906
7a634b34fc91 wireproto: add decorator for wire protocol command
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
   183
    return register
7a634b34fc91 wireproto: add decorator for wire protocol command
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20905
diff changeset
   184
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   185
36755
ff4bc0ab6740 wireproto: check permissions when executing "batch" command (BC) (SEC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36754
diff changeset
   186
# TODO define a more appropriate permissions type to use for this.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   187
@wireprotocommand(b'batch', b'cmds *', permission=b'pull')
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   188
def batch(repo, proto, cmds, others):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
   189
    unescapearg = wireprototypes.unescapebatcharg
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   190
    res = []
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   191
    for pair in cmds.split(b';'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   192
        op, args = pair.split(b' ', 1)
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   193
        vals = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   194
        for a in args.split(b','):
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   195
            if a:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   196
                n, v = a.split(b'=')
29734
62e2e048d068 wireproto: unescape argument names in batch command (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29733
diff changeset
   197
                vals[unescapearg(n)] = unescapearg(v)
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   198
        func, spec = commands[op]
36755
ff4bc0ab6740 wireproto: check permissions when executing "batch" command (BC) (SEC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36754
diff changeset
   199
36801
66de4555cefd wireproto: formalize permissions checking as part of protocol interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36800
diff changeset
   200
        # Validate that client has permissions to perform this command.
66de4555cefd wireproto: formalize permissions checking as part of protocol interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36800
diff changeset
   201
        perm = commands[op].permission
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   202
        assert perm in (b'push', b'pull')
36801
66de4555cefd wireproto: formalize permissions checking as part of protocol interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36800
diff changeset
   203
        proto.checkperm(perm)
36755
ff4bc0ab6740 wireproto: check permissions when executing "batch" command (BC) (SEC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36754
diff changeset
   204
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   205
        if spec:
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   206
            keys = spec.split()
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   207
            data = {}
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   208
            for k in keys:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   209
                if k == b'*':
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   210
                    star = {}
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   211
                    for key in vals.keys():
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   212
                        if key not in keys:
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   213
                            star[key] = vals[key]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   214
                    data[b'*'] = star
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   215
                else:
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   216
                    data[k] = vals[k]
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   217
            result = func(repo, proto, *[data[k] for k in keys])
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   218
        else:
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   219
            result = func(repo, proto)
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   220
        if isinstance(result, wireprototypes.ooberror):
15017
f4522df38c65 wireproto: add out-of-band error class to allow remote repo to report errors
Andrew Pritchard <andrewp@fogcreek.com>
parents: 14970
diff changeset
   221
            return result
36074
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36073
diff changeset
   222
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36073
diff changeset
   223
        # For now, all batchable commands must return bytesresponse or
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36073
diff changeset
   224
        # raw bytes (for backwards compatibility).
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   225
        assert isinstance(result, (wireprototypes.bytesresponse, bytes))
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   226
        if isinstance(result, wireprototypes.bytesresponse):
36074
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36073
diff changeset
   227
            result = result.data
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
   228
        res.append(wireprototypes.escapebatcharg(result))
36074
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36073
diff changeset
   229
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   230
    return wireprototypes.bytesresponse(b';'.join(res))
14622
bd88561afb4b wireproto: add batching support to wirerepository
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14621
diff changeset
   231
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   232
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   233
@wireprotocommand(b'between', b'pairs', permission=b'pull')
11583
944c23762c3c protocol: add proto to method prototypes
Matt Mackall <mpm@selenic.com>
parents: 11581
diff changeset
   234
def between(repo, proto, pairs):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   235
    pairs = [wireprototypes.decodelist(p, b'-') for p in pairs.split(b" ")]
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   236
    r = []
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   237
    for b in repo.between(pairs):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   238
        r.append(wireprototypes.encodelist(b) + b"\n")
36074
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36073
diff changeset
   239
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   240
    return wireprototypes.bytesresponse(b''.join(r))
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   241
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   242
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   243
@wireprotocommand(b'branchmap', permission=b'pull')
11583
944c23762c3c protocol: add proto to method prototypes
Matt Mackall <mpm@selenic.com>
parents: 11581
diff changeset
   244
def branchmap(repo, proto):
18281
898c575833c9 clfilter: drop extra filtering in wireprotocol
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18280
diff changeset
   245
    branchmap = repo.branchmap()
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   246
    heads = []
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43089
diff changeset
   247
    for branch, nodes in pycompat.iteritems(branchmap):
28883
032c4c2f802a pycompat: switch to util.urlreq/util.urlerr for py3 compat
timeless <timeless@mozdev.org>
parents: 28666
diff changeset
   248
        branchname = urlreq.quote(encoding.fromlocal(branch))
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
   249
        branchnodes = wireprototypes.encodelist(nodes)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   250
        heads.append(b'%s %s' % (branchname, branchnodes))
36074
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36073
diff changeset
   251
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   252
    return wireprototypes.bytesresponse(b'\n'.join(heads))
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   253
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   254
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   255
@wireprotocommand(b'branches', b'nodes', permission=b'pull')
11583
944c23762c3c protocol: add proto to method prototypes
Matt Mackall <mpm@selenic.com>
parents: 11581
diff changeset
   256
def branches(repo, proto, nodes):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
   257
    nodes = wireprototypes.decodelist(nodes)
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   258
    r = []
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   259
    for b in repo.branches(nodes):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   260
        r.append(wireprototypes.encodelist(b) + b"\n")
36074
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36073
diff changeset
   261
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   262
    return wireprototypes.bytesresponse(b''.join(r))
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   263
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   264
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   265
@wireprotocommand(b'clonebundles', b'', permission=b'pull')
26857
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
   266
def clonebundles(repo, proto):
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
   267
    """Server command for returning info for available bundles to seed clones.
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
   268
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
   269
    Clients will parse this response and determine what bundle to fetch.
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
   270
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
   271
    Extensions may wrap this command to filter or dynamically emit data
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
   272
    depending on the request. e.g. you could advertise URLs for the closest
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
   273
    data center given the client's IP address.
e5a1df51bb25 wireproto: move clonebundles command from extension (issue4931)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26690
diff changeset
   274
    """
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   275
    return wireprototypes.bytesresponse(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   276
        repo.vfs.tryread(b'clonebundles.manifest')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   277
    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   278
20774
cdc3ac896997 wireproto: extract capabilities list in outside the wireproto function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20671
diff changeset
   279
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   280
wireprotocaps = [
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   281
    b'lookup',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   282
    b'branchmap',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   283
    b'pushkey',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   284
    b'known',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   285
    b'getbundle',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   286
    b'unbundlehash',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   287
]
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   288
20775
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
   289
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
   290
def _capabilities(repo, proto):
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
   291
    """return a list of capabilities for a repo
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
   292
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
   293
    This function exists to allow extensions to easily wrap capabilities
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
   294
    computation
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
   295
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
   296
    - returns a lists: easy to alter
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
   297
    - change done here will be propagated to both `capabilities` and `hello`
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 20969
diff changeset
   298
      command without any other action needed.
20775
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
   299
    """
20774
cdc3ac896997 wireproto: extract capabilities list in outside the wireproto function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20671
diff changeset
   300
    # copy to prevent modification of the global list
cdc3ac896997 wireproto: extract capabilities list in outside the wireproto function
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20671
diff changeset
   301
    caps = list(wireprotocaps)
36612
e89959970a08 wireproto: don't expose changegroupsubset capability if not available
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36611
diff changeset
   302
e89959970a08 wireproto: don't expose changegroupsubset capability if not available
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36611
diff changeset
   303
    # Command of same name as capability isn't exposed to version 1 of
e89959970a08 wireproto: don't expose changegroupsubset capability if not available
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36611
diff changeset
   304
    # transports. So conditionally add it.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   305
    if commands.commandavailable(b'changegroupsubset', proto):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   306
        caps.append(b'changegroupsubset')
36612
e89959970a08 wireproto: don't expose changegroupsubset capability if not available
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36611
diff changeset
   307
32744
33b7283a3828 streamclone: consider secret changesets (BC) (issue5589)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32291
diff changeset
   308
    if streamclone.allowservergeneration(repo):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   309
        if repo.ui.configbool(b'server', b'preferuncompressed'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   310
            caps.append(b'stream-preferred')
12296
d7fff529d85d clone: only use stream when we understand the revlog format
Sune Foldager <cryo@cyanite.org>
parents: 12085
diff changeset
   311
        requiredformats = repo.requirements & repo.supportedformats
d7fff529d85d clone: only use stream when we understand the revlog format
Sune Foldager <cryo@cyanite.org>
parents: 12085
diff changeset
   312
        # if our local revlogs are just revlogv1, add 'stream' cap
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   313
        if not requiredformats - {b'revlogv1'}:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   314
            caps.append(b'stream')
12296
d7fff529d85d clone: only use stream when we understand the revlog format
Sune Foldager <cryo@cyanite.org>
parents: 12085
diff changeset
   315
        # otherwise, add 'streamreqs' detailing our local revlog format
d7fff529d85d clone: only use stream when we understand the revlog format
Sune Foldager <cryo@cyanite.org>
parents: 12085
diff changeset
   316
        else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   317
            caps.append(b'streamreqs=%s' % b','.join(sorted(requiredformats)))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   318
    if repo.ui.configbool(b'experimental', b'bundle2-advertise'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   319
        capsblob = bundle2.encodecaps(bundle2.getrepocaps(repo, role=b'server'))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   320
        caps.append(b'bundle2=' + urlreq.quote(capsblob))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   321
    caps.append(b'unbundle=%s' % b','.join(bundle2.bundlepriority))
30563
e118233172fe wireproto: only advertise HTTP-specific capabilities to HTTP peers (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30473
diff changeset
   322
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   323
    if repo.ui.configbool(b'experimental', b'narrow'):
40075
ad8d8dc9be3f narrow: move adding of narrow server capabilities to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 38678
diff changeset
   324
        caps.append(wireprototypes.NARROWCAP)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   325
        if repo.ui.configbool(b'experimental', b'narrowservebrokenellipses'):
40075
ad8d8dc9be3f narrow: move adding of narrow server capabilities to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 38678
diff changeset
   326
            caps.append(wireprototypes.ELLIPSESCAP)
ad8d8dc9be3f narrow: move adding of narrow server capabilities to core
Pulkit Goyal <pulkit@yandex-team.ru>
parents: 38678
diff changeset
   327
36613
6e585bca962e wireproto: add transport specific capabilities in the transport
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36612
diff changeset
   328
    return proto.addcapabilities(repo, caps)
20775
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
   329
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   330
21024
7731a2281cf0 spelling: fixes from spell checker
Mads Kiilerich <madski@unity3d.com>
parents: 20969
diff changeset
   331
# If you are writing an extension and consider wrapping this function. Wrap
20775
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
   332
# `_capabilities` instead.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   333
@wireprotocommand(b'capabilities', permission=b'pull')
20775
982f13bef503 wireproto: move wireproto capabilities computation in a subfunction
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 20774
diff changeset
   334
def capabilities(repo, proto):
37413
33af46d639b4 wireproto: send server capabilities in canonical order
Joerg Sonnenberger <joerg@bec.de>
parents: 37412
diff changeset
   335
    caps = _capabilities(repo, proto)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   336
    return wireprototypes.bytesresponse(b' '.join(sorted(caps)))
11594
67863f9d805f protocol: unify server-side capabilities functions
Matt Mackall <mpm@selenic.com>
parents: 11593
diff changeset
   337
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   338
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   339
@wireprotocommand(b'changegroup', b'roots', permission=b'pull')
11584
1af96b090116 protocol: unify changegroup commands
Matt Mackall <mpm@selenic.com>
parents: 11583
diff changeset
   340
def changegroup(repo, proto, roots):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
   341
    nodes = wireprototypes.decodelist(roots)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   342
    outgoing = discovery.outgoing(
45144
c93dd9d9f1e6 discovery: change users of `outgoing.missingheads` to `outgoing.ancestorsof`
Manuel Jacob <me@manueljacob.de>
parents: 44782
diff changeset
   343
        repo, missingroots=nodes, ancestorsof=repo.heads()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   344
    )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   345
    cg = changegroupmod.makechangegroup(repo, outgoing, b'01', b'serve')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   346
    gen = iter(lambda: cg.read(32768), b'')
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   347
    return wireprototypes.streamres(gen=gen)
11584
1af96b090116 protocol: unify changegroup commands
Matt Mackall <mpm@selenic.com>
parents: 11583
diff changeset
   348
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   349
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   350
@wireprotocommand(b'changegroupsubset', b'bases heads', permission=b'pull')
11584
1af96b090116 protocol: unify changegroup commands
Matt Mackall <mpm@selenic.com>
parents: 11583
diff changeset
   351
def changegroupsubset(repo, proto, bases, heads):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
   352
    bases = wireprototypes.decodelist(bases)
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
   353
    heads = wireprototypes.decodelist(heads)
45144
c93dd9d9f1e6 discovery: change users of `outgoing.missingheads` to `outgoing.ancestorsof`
Manuel Jacob <me@manueljacob.de>
parents: 44782
diff changeset
   354
    outgoing = discovery.outgoing(repo, missingroots=bases, ancestorsof=heads)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   355
    cg = changegroupmod.makechangegroup(repo, outgoing, b'01', b'serve')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   356
    gen = iter(lambda: cg.read(32768), b'')
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   357
    return wireprototypes.streamres(gen=gen)
11584
1af96b090116 protocol: unify changegroup commands
Matt Mackall <mpm@selenic.com>
parents: 11583
diff changeset
   358
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   359
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   360
@wireprotocommand(b'debugwireargs', b'one two *', permission=b'pull')
13721
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
   361
def debugwireargs(repo, proto, one, two, others):
3458c15ab2f0 wireproto: fix handling of '*' args for HTTP and SSH
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13720
diff changeset
   362
    # only accept optional args from the known set
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   363
    opts = options(b'debugwireargs', [b'three', b'four'], others)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   364
    return wireprototypes.bytesresponse(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   365
        repo.debugwireargs(one, two, **pycompat.strkwargs(opts))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   366
    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   367
13720
9c4e04fe267e debug: add debugwireargs to test argument passing over the wire
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13450
diff changeset
   368
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   369
def find_pullbundle(repo, proto, opts, clheads, heads, common):
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   370
    """Return a file object for the first matching pullbundle.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   371
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   372
    Pullbundles are specified in .hg/pullbundles.manifest similar to
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   373
    clonebundles.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   374
    For each entry, the bundle specification is checked for compatibility:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   375
    - Client features vs the BUNDLESPEC.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   376
    - Revisions shared with the clients vs base revisions of the bundle.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   377
      A bundle can be applied only if all its base revisions are known by
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   378
      the client.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   379
    - At least one leaf of the bundle's DAG is missing on the client.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   380
    - Every leaf of the bundle's DAG is part of node set the client wants.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   381
      E.g. do not send a bundle of all changes if the client wants only
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   382
      one specific branch of many.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   383
    """
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   384
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   385
    def decodehexstring(s):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   386
        return {binascii.unhexlify(h) for h in s.split(b';')}
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   387
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   388
    manifest = repo.vfs.tryread(b'pullbundles.manifest')
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   389
    if not manifest:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   390
        return None
45784
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45682
diff changeset
   391
    res = bundlecaches.parseclonebundlesmanifest(repo, manifest)
74271829ddc0 clonebundles: move a bundle of clone bundle related code to a new module
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 45682
diff changeset
   392
    res = bundlecaches.filterclonebundleentries(repo, res)
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   393
    if not res:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   394
        return None
44782
3d5fb6cab832 pullbundles: use unfiltered repo for head/base matching
Joerg Sonnenberger <joerg@bec.de>
parents: 43729
diff changeset
   395
    cl = repo.unfiltered().changelog
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   396
    heads_anc = cl.ancestors([cl.rev(rev) for rev in heads], inclusive=True)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   397
    common_anc = cl.ancestors([cl.rev(rev) for rev in common], inclusive=True)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   398
    compformats = clientcompressionsupport(proto)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   399
    for entry in res:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   400
        comp = entry.get(b'COMPRESSION')
38678
7e4a856a4f05 pullbundle: fix handling of gzip bundlespecs
Joerg Sonnenberger <joerg@bec.de>
parents: 38164
diff changeset
   401
        altcomp = util.compengines._bundlenames.get(comp)
7e4a856a4f05 pullbundle: fix handling of gzip bundlespecs
Joerg Sonnenberger <joerg@bec.de>
parents: 38164
diff changeset
   402
        if comp and comp not in compformats and altcomp not in compformats:
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   403
            continue
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   404
        # No test yet for VERSION, since V2 is supported by any client
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   405
        # that advertises partial pulls
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   406
        if b'heads' in entry:
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   407
            try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   408
                bundle_heads = decodehexstring(entry[b'heads'])
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   409
            except TypeError:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   410
                # Bad heads entry
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   411
                continue
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   412
            if bundle_heads.issubset(common):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   413
                continue  # Nothing new
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   414
            if all(cl.rev(rev) in common_anc for rev in bundle_heads):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   415
                continue  # Still nothing new
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   416
            if any(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   417
                cl.rev(rev) not in heads_anc and cl.rev(rev) not in common_anc
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   418
                for rev in bundle_heads
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   419
            ):
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   420
                continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   421
        if b'bases' in entry:
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   422
            try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   423
                bundle_bases = decodehexstring(entry[b'bases'])
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   424
            except TypeError:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   425
                # Bad bases entry
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   426
                continue
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   427
            if not all(cl.rev(rev) in common_anc for rev in bundle_bases):
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   428
                continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   429
        path = entry[b'URL']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   430
        repo.ui.debug(b'sending pullbundle "%s"\n' % path)
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   431
        try:
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   432
            return repo.vfs.open(path)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   433
        except IOError:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   434
            repo.ui.debug(b'pullbundle "%s" not accessible\n' % path)
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   435
            continue
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   436
    return None
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   437
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   438
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   439
@wireprotocommand(b'getbundle', b'*', permission=b'pull')
13741
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13726
diff changeset
   440
def getbundle(repo, proto, others):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   441
    opts = options(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   442
        b'getbundle', wireprototypes.GETBUNDLE_ARGUMENTS.keys(), others
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   443
    )
43106
d783f945a701 py3: finish porting iteritems() to pycompat and remove source transformer
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43089
diff changeset
   444
    for k, v in pycompat.iteritems(opts):
37613
96d735601ca1 wireproto: move gboptsmap to wireprototypes and rename (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37612
diff changeset
   445
        keytype = wireprototypes.GETBUNDLE_ARGUMENTS[k]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   446
        if keytype == b'nodes':
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
   447
            opts[k] = wireprototypes.decodelist(v)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   448
        elif keytype == b'csv':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   449
            opts[k] = list(v.split(b','))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   450
        elif keytype == b'scsv':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   451
            opts[k] = set(v.split(b','))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   452
        elif keytype == b'boolean':
26686
3e7f675628ad wireproto: properly parse false boolean args (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26587
diff changeset
   453
            # Client should serialize False as '0', which is a non-empty string
3e7f675628ad wireproto: properly parse false boolean args (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26587
diff changeset
   454
            # so it evaluates as a True bool.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   455
            if v == b'0':
26686
3e7f675628ad wireproto: properly parse false boolean args (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26587
diff changeset
   456
                opts[k] = False
3e7f675628ad wireproto: properly parse false boolean args (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26587
diff changeset
   457
            else:
3e7f675628ad wireproto: properly parse false boolean args (BC)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 26587
diff changeset
   458
                opts[k] = bool(v)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   459
        elif keytype != b'plain':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   460
            raise KeyError(b'unknown getbundle option type %s' % keytype)
27246
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
   461
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   462
    if not bundle1allowed(repo, b'pull'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   463
        if not exchange.bundle2requested(opts.get(b'bundlecaps')):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   464
            if proto.name == b'http-v1':
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   465
                return wireprototypes.ooberror(bundle2required)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   466
            raise error.Abort(bundle2requiredmain, hint=bundle2requiredhint)
27246
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
   467
30914
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30912
diff changeset
   468
    try:
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   469
        clheads = set(repo.changelog.heads())
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   470
        heads = set(opts.get(b'heads', set()))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   471
        common = set(opts.get(b'common', set()))
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   472
        common.discard(nullid)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   473
        if (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   474
            repo.ui.configbool(b'server', b'pullbundle')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   475
            and b'partial-pull' in proto.getprotocaps()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   476
        ):
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   477
            # Check if a pre-built bundle covers this request.
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   478
            bundle = find_pullbundle(repo, proto, opts, clheads, heads, common)
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   479
            if bundle:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   480
                return wireprototypes.streamres(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   481
                    gen=util.filechunkiter(bundle), prefer_uncompressed=True
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   482
                )
37498
aacfca6f9767 wireproto: support for pullbundles
Joerg Sonnenberger <joerg@bec.de>
parents: 37490
diff changeset
   483
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   484
        if repo.ui.configbool(b'server', b'disablefullbundle'):
32260
d0d9a4fca59b clone: add a server-side option to disable full getbundles (pull-based clones)
Siddharth Agarwal <sid0@fb.com>
parents: 31451
diff changeset
   485
            # Check to see if this is a full clone.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   486
            changegroup = opts.get(b'cg', True)
35760
133a678673cb clone: allow bundle2's stream clone with 'server.disablefullbundle'
Boris Feld <boris.feld@octobus.net>
parents: 35759
diff changeset
   487
            if changegroup and not common and clheads == heads:
32260
d0d9a4fca59b clone: add a server-side option to disable full getbundles (pull-based clones)
Siddharth Agarwal <sid0@fb.com>
parents: 31451
diff changeset
   488
                raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   489
                    _(b'server has pull-based clones disabled'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   490
                    hint=_(b'remove --pull if specified or upgrade Mercurial'),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   491
                )
32260
d0d9a4fca59b clone: add a server-side option to disable full getbundles (pull-based clones)
Siddharth Agarwal <sid0@fb.com>
parents: 31451
diff changeset
   492
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   493
        info, chunks = exchange.getbundlechunks(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   494
            repo, b'serve', **pycompat.strkwargs(opts)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   495
        )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   496
        prefercompressed = info.get(b'prefercompressed', True)
30914
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30912
diff changeset
   497
    except error.Abort as exc:
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30912
diff changeset
   498
        # cleanly forward Abort error to the client
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   499
        if not exchange.bundle2requested(opts.get(b'bundlecaps')):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   500
            if proto.name == b'http-v1':
45682
d2e1dcd4490d errors: name arguments to Abort constructor
Martin von Zweigbergk <martinvonz@google.com>
parents: 45144
diff changeset
   501
                return wireprototypes.ooberror(exc.message + b'\n')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   502
            raise  # cannot do better for bundle1 + ssh
30914
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30912
diff changeset
   503
        # bundle2 request expect a bundle2 reply
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30912
diff changeset
   504
        bundler = bundle2.bundle20(repo.ui)
45682
d2e1dcd4490d errors: name arguments to Abort constructor
Martin von Zweigbergk <martinvonz@google.com>
parents: 45144
diff changeset
   505
        manargs = [(b'message', exc.message)]
30914
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30912
diff changeset
   506
        advargs = []
f3807a135e43 wireproto: properly report server Abort during 'getbundle'
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 30912
diff changeset
   507
        if exc.hint is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   508
            advargs.append((b'hint', exc.hint))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   509
        bundler.addpart(bundle2.bundlepart(b'error:abort', manargs, advargs))
35782
9d249f3de730 wireproto: don't compress errors from getbundle()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35760
diff changeset
   510
        chunks = bundler.getchunks()
35787
a84dbc87dae9 exchange: send bundle2 stream clones uncompressed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35785
diff changeset
   511
        prefercompressed = False
35782
9d249f3de730 wireproto: don't compress errors from getbundle()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35760
diff changeset
   512
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   513
    return wireprototypes.streamres(
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   514
        gen=chunks, prefer_uncompressed=not prefercompressed
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   515
    )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   516
13741
b51bf961b3cb wireproto: add getbundle() function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13726
diff changeset
   517
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   518
@wireprotocommand(b'heads', permission=b'pull')
11583
944c23762c3c protocol: add proto to method prototypes
Matt Mackall <mpm@selenic.com>
parents: 11581
diff changeset
   519
def heads(repo, proto):
18281
898c575833c9 clfilter: drop extra filtering in wireprotocol
Pierre-Yves David <pierre-yves.david@ens-lyon.org>
parents: 18280
diff changeset
   520
    h = repo.heads()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   521
    return wireprototypes.bytesresponse(wireprototypes.encodelist(h) + b'\n')
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   522
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   523
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   524
@wireprotocommand(b'hello', permission=b'pull')
11594
67863f9d805f protocol: unify server-side capabilities functions
Matt Mackall <mpm@selenic.com>
parents: 11593
diff changeset
   525
def hello(repo, proto):
36221
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36074
diff changeset
   526
    """Called as part of SSH handshake to obtain server info.
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36074
diff changeset
   527
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36074
diff changeset
   528
    Returns a list of lines describing interesting things about the
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36074
diff changeset
   529
    server, in an RFC822-like format.
11594
67863f9d805f protocol: unify server-side capabilities functions
Matt Mackall <mpm@selenic.com>
parents: 11593
diff changeset
   530
36221
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36074
diff changeset
   531
    Currently, the only one defined is ``capabilities``, which consists of a
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36074
diff changeset
   532
    line of space separated tokens describing server abilities:
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36074
diff changeset
   533
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36074
diff changeset
   534
        capabilities: <token0> <token1> <token2>
62bca1c50e96 wireproto: improve docstring for "hello"
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36074
diff changeset
   535
    """
36074
2f7290555c96 wireproto: introduce type for raw byte responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36073
diff changeset
   536
    caps = capabilities(repo, proto).data
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   537
    return wireprototypes.bytesresponse(b'capabilities: %s\n' % caps)
11594
67863f9d805f protocol: unify server-side capabilities functions
Matt Mackall <mpm@selenic.com>
parents: 11593
diff changeset
   538
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   539
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   540
@wireprotocommand(b'listkeys', b'namespace', permission=b'pull')
11583
944c23762c3c protocol: add proto to method prototypes
Matt Mackall <mpm@selenic.com>
parents: 11581
diff changeset
   541
def listkeys(repo, proto, namespace):
36529
33c6f8f0388d wireproto: sort response to listkeys
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36431
diff changeset
   542
    d = sorted(repo.listkeys(encoding.tolocal(namespace)).items())
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   543
    return wireprototypes.bytesresponse(pushkeymod.encodekeys(d))
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   544
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   545
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   546
@wireprotocommand(b'lookup', b'key', permission=b'pull')
11583
944c23762c3c protocol: add proto to method prototypes
Matt Mackall <mpm@selenic.com>
parents: 11581
diff changeset
   547
def lookup(repo, proto, key):
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   548
    try:
15925
f9fc46698352 wireproto: refuse to lookup secret csets
Matt Mackall <mpm@selenic.com>
parents: 15713
diff changeset
   549
        k = encoding.tolocal(key)
37353
ac666c5c2e0c wireproto: use repo.lookup() for lookup command
Martin von Zweigbergk <martinvonz@google.com>
parents: 37320
diff changeset
   550
        n = repo.lookup(k)
ac666c5c2e0c wireproto: use repo.lookup() for lookup command
Martin von Zweigbergk <martinvonz@google.com>
parents: 37320
diff changeset
   551
        r = hex(n)
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   552
        success = 1
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25493
diff changeset
   553
    except Exception as inst:
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37053
diff changeset
   554
        r = stringutil.forcebytestr(inst)
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   555
        success = 0
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   556
    return wireprototypes.bytesresponse(b'%d %s\n' % (success, r))
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   557
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   558
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   559
@wireprotocommand(b'known', b'nodes *', permission=b'pull')
14436
5adb52524779 wireproto: enable optional args for known() for future extensibility
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 14419
diff changeset
   560
def known(repo, proto, nodes, others):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   561
    v = b''.join(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   562
        b and b'1' or b'0' for b in repo.known(wireprototypes.decodelist(nodes))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   563
    )
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   564
    return wireprototypes.bytesresponse(v)
13723
e615765fdcc7 wireproto: add known([id]) function
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 13722
diff changeset
   565
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   566
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   567
@wireprotocommand(b'protocaps', b'caps', permission=b'pull')
37393
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
   568
def protocaps(repo, proto, caps):
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
   569
    if proto.name == wireprototypes.SSHV1:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   570
        proto._protocaps = set(caps.split(b' '))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   571
    return wireprototypes.bytesresponse(b'OK')
37393
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37353
diff changeset
   572
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   573
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   574
@wireprotocommand(b'pushkey', b'namespace key old new', permission=b'push')
11583
944c23762c3c protocol: add proto to method prototypes
Matt Mackall <mpm@selenic.com>
parents: 11581
diff changeset
   575
def pushkey(repo, proto, namespace, key, old, new):
13050
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
   576
    # compatibility with pre-1.8 clients which were accidentally
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
   577
    # sending raw binary nodes rather than utf-8-encoded hex
37084
f0b6fbea00cf stringutil: bulk-replace call sites to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37053
diff changeset
   578
    if len(new) == 20 and stringutil.escapestr(new) != new:
13050
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
   579
        # looks like it could be a binary node
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
   580
        try:
14064
e4bfb9c337f3 remove unused imports and variables
Alexander Solovyov <alexander@solovyov.net>
parents: 14048
diff changeset
   581
            new.decode('utf-8')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   582
            new = encoding.tolocal(new)  # but cleanly decodes as UTF-8
13050
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
   583
        except UnicodeDecodeError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   584
            pass  # binary, leave unmodified
13050
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
   585
    else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   586
        new = encoding.tolocal(new)  # normal path
13050
3790452d499b pushkey: use UTF-8
Matt Mackall <mpm@selenic.com>
parents: 13049
diff changeset
   587
36066
2ad145fbde54 wireprotoserver: add context manager mechanism for redirecting stdio
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35982
diff changeset
   588
    with proto.mayberedirectstdio() as output:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   589
        r = (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   590
            repo.pushkey(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   591
                encoding.tolocal(namespace),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   592
                encoding.tolocal(key),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   593
                encoding.tolocal(old),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   594
                new,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   595
            )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   596
            or False
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   597
        )
17793
8474be4412ca wireproto: fix pushkey hook failure and output on remote http repo
Wagner Bruna <wbruna@softwareexpress.com.br>
parents: 17603
diff changeset
   598
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   599
    output = output.getvalue() if output else b''
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   600
    return wireprototypes.bytesresponse(b'%d\n%s' % (int(r), output))
11581
4530b3307fb9 protocol: introduce wireproto.py
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
   601
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   602
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   603
@wireprotocommand(b'stream_out', permission=b'pull')
11585
5d907fbb9703 protocol: unify stream_out command
Matt Mackall <mpm@selenic.com>
parents: 11584
diff changeset
   604
def stream(repo, proto):
11627
04f76a954842 protocol: move the streamclone implementation into wireproto
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 11625
diff changeset
   605
    '''If the server supports streaming clone, it advertises the "stream"
04f76a954842 protocol: move the streamclone implementation into wireproto
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 11625
diff changeset
   606
    capability with a value representing the version and flags of the repo
04f76a954842 protocol: move the streamclone implementation into wireproto
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 11625
diff changeset
   607
    it is serving. Client checks to see if it understands the format.
04f76a954842 protocol: move the streamclone implementation into wireproto
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 11625
diff changeset
   608
    '''
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   609
    return wireprototypes.streamreslegacy(streamclone.generatev1wireproto(repo))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   610
11585
5d907fbb9703 protocol: unify stream_out command
Matt Mackall <mpm@selenic.com>
parents: 11584
diff changeset
   611
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   612
@wireprotocommand(b'unbundle', b'heads', permission=b'push')
11593
d054cc5c7737 protocol: unify unbundle on the server side
Matt Mackall <mpm@selenic.com>
parents: 11592
diff changeset
   613
def unbundle(repo, proto, heads):
37612
5e71dea79aae wireproto: move value encoding functions to wireprototypes (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
   614
    their_heads = wireprototypes.decodelist(heads)
11593
d054cc5c7737 protocol: unify unbundle on the server side
Matt Mackall <mpm@selenic.com>
parents: 11592
diff changeset
   615
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   616
    with proto.mayberedirectstdio() as output:
11593
d054cc5c7737 protocol: unify unbundle on the server side
Matt Mackall <mpm@selenic.com>
parents: 11592
diff changeset
   617
        try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   618
            exchange.check_heads(repo, their_heads, b'preparing changes')
37414
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   619
            cleanup = lambda: None
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   620
            try:
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   621
                payload = proto.getpayload()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   622
                if repo.ui.configbool(b'server', b'streamunbundle'):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   623
37414
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   624
                    def cleanup():
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   625
                        # Ensure that the full payload is consumed, so
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   626
                        # that the connection doesn't contain trailing garbage.
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   627
                        for p in payload:
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   628
                            pass
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   629
37414
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   630
                    fp = util.chunkbuffer(payload)
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   631
                else:
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   632
                    # write bundle data to temporary file as it can be big
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   633
                    fp, tempname = None, None
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   634
37414
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   635
                    def cleanup():
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   636
                        if fp:
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   637
                            fp.close()
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   638
                        if tempname:
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   639
                            os.unlink(tempname)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   640
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   641
                    fd, tempname = pycompat.mkstemp(prefix=b'hg-unbundle-')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   642
                    repo.ui.debug(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   643
                        b'redirecting incoming bundle to %s\n' % tempname
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   644
                    )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   645
                    fp = os.fdopen(fd, pycompat.sysstr(b'wb+'))
37414
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   646
                    for p in payload:
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   647
                        fp.write(p)
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   648
                    fp.seek(0)
27246
b288fb2724bf wireproto: config options to disable bundle1
Gregory Szorc <gregory.szorc@gmail.com>
parents: 27243
diff changeset
   649
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   650
                gen = exchange.readbundle(repo.ui, fp, None)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   651
                if isinstance(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   652
                    gen, changegroupmod.cg1unpacker
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   653
                ) and not bundle1allowed(repo, b'push'):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   654
                    if proto.name == b'http-v1':
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   655
                        # need to special case http because stderr do not get to
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   656
                        # the http client on failed push so we need to abuse
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   657
                        # some other error type to make sure the message get to
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   658
                        # the user.
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   659
                        return wireprototypes.ooberror(bundle2required)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   660
                    raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   661
                        bundle2requiredmain, hint=bundle2requiredhint
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   662
                    )
24796
61ff209fc01d bundle2: refactor error bundle creation for the wireprotocol
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24696
diff changeset
   663
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   664
                r = exchange.unbundle(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   665
                    repo, gen, their_heads, b'serve', proto.client()
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   666
                )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   667
                if util.safehasattr(r, b'addpart'):
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   668
                    # The return looks streamable, we are in the bundle2 case
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   669
                    # and should return a stream.
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   670
                    return wireprototypes.streamreslegacy(gen=r.getchunks())
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   671
                return wireprototypes.pushres(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   672
                    r, output.getvalue() if output else b''
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   673
                )
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   674
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   675
            finally:
37414
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37413
diff changeset
   676
                cleanup()
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   677
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   678
        except (error.BundleValueError, error.Abort, error.PushRaced) as exc:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   679
            # handle non-bundle2 case first
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   680
            if not getattr(exc, 'duringunbundle2', False):
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   681
                try:
25493
d8e7b0781ad7 bundle2: convey PushkeyFailed error over the wire
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25403
diff changeset
   682
                    raise
43729
1d1232c0726f wireprotov1server: capture Abort type before accessing the `hint` attribute
Matt Harbison <matt_harbison@yahoo.com>
parents: 43117
diff changeset
   683
                except error.Abort as exc:
37119
d4a2e0d5d042 procutil: bulk-replace util.std* to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
   684
                    # The old code we moved used procutil.stderr directly.
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   685
                    # We did not change it to minimise code change.
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   686
                    # This need to be moved to something proper.
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   687
                    # Feel free to do it.
45682
d2e1dcd4490d errors: name arguments to Abort constructor
Martin von Zweigbergk <martinvonz@google.com>
parents: 45144
diff changeset
   688
                    procutil.stderr.write(b"abort: %s\n" % exc.message)
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   689
                    if exc.hint is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   690
                        procutil.stderr.write(b"(%s)\n" % exc.hint)
37119
d4a2e0d5d042 procutil: bulk-replace util.std* to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 37084
diff changeset
   691
                    procutil.stderr.flush()
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   692
                    return wireprototypes.pushres(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   693
                        0, output.getvalue() if output else b''
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   694
                    )
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   695
                except error.PushRaced:
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   696
                    return wireprototypes.pusherr(
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   697
                        pycompat.bytestr(exc),
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   698
                        output.getvalue() if output else b'',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   699
                    )
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   700
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   701
            bundler = bundle2.bundle20(repo.ui)
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   702
            for out in getattr(exc, '_bundle2salvagedoutput', ()):
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   703
                bundler.addpart(out)
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   704
            try:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   705
                try:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   706
                    raise
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   707
                except error.PushkeyFailed as exc:
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   708
                    # check client caps
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   709
                    remotecaps = getattr(exc, '_replycaps', None)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   710
                    if (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   711
                        remotecaps is not None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   712
                        and b'pushkey' not in remotecaps.get(b'error', ())
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   713
                    ):
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   714
                        # no support remote side, fallback to Abort handler.
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   715
                        raise
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   716
                    part = bundler.newpart(b'error:pushkey')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   717
                    part.addparam(b'in-reply-to', exc.partid)
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   718
                    if exc.namespace is not None:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   719
                        part.addparam(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   720
                            b'namespace', exc.namespace, mandatory=False
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   721
                        )
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   722
                    if exc.key is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   723
                        part.addparam(b'key', exc.key, mandatory=False)
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   724
                    if exc.new is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   725
                        part.addparam(b'new', exc.new, mandatory=False)
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   726
                    if exc.old is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   727
                        part.addparam(b'old', exc.old, mandatory=False)
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   728
                    if exc.ret is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   729
                        part.addparam(b'ret', exc.ret, mandatory=False)
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   730
            except error.BundleValueError as exc:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   731
                errpart = bundler.newpart(b'error:unsupportedcontent')
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   732
                if exc.parttype is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   733
                    errpart.addparam(b'parttype', exc.parttype)
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   734
                if exc.params:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   735
                    errpart.addparam(b'params', b'\0'.join(exc.params))
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   736
            except error.Abort as exc:
45682
d2e1dcd4490d errors: name arguments to Abort constructor
Martin von Zweigbergk <martinvonz@google.com>
parents: 45144
diff changeset
   737
                manargs = [(b'message', exc.message)]
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   738
                advargs = []
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   739
                if exc.hint is not None:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   740
                    advargs.append((b'hint', exc.hint))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   741
                bundler.addpart(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   742
                    bundle2.bundlepart(b'error:abort', manargs, advargs)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   743
                )
36067
caca3ac2ac04 wireproto: use maybecapturestdio() for push responses (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36066
diff changeset
   744
            except error.PushRaced as exc:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   745
                bundler.newpart(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   746
                    b'error:pushraced',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   747
                    [(b'message', stringutil.forcebytestr(exc))],
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42057
diff changeset
   748
                )
37293
d5d665f6615a wireproto: stop aliasing wire protocol types (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37119
diff changeset
   749
            return wireprototypes.streamreslegacy(gen=bundler.getchunks())