mercurial/wireprotov2server.py
author Gregory Szorc <gregory.szorc@gmail.com>
Wed, 26 Sep 2018 17:16:56 -0700
changeset 40021 c537144fdbef
parent 39995 582676acaf6d
child 40023 10cf8b116dd8
permissions -rw-r--r--
wireprotov2: support response caching One of the things I've learned from managing VCS servers over the years is that they are hard to scale. It is well known that some companies have very beefy (read: very expensive) servers to power their VCS needs. It is also known that specialized servers for various VCS exist in order to facilitate scaling servers. (Mercurial is in this boat.) One of the aspects that make a VCS server hard to scale is the high CPU load incurred by constant client clone/pull operations. To alleviate the scaling pain associated with data retrieval operations, I want to integrate caching into the Mercurial wire protocol server as robustly as possible such that servers can aggressively cache responses and defer as much server load as possible. This commit represents the initial implementation of a general caching layer in wire protocol version 2. We define a new interface and behavior for a wire protocol cacher in repository.py. (This is probably where a reviewer should look first to understand what is going on.) The bulk of the added code is in wireprotov2server.py, where we define how a command can opt in to being cached and integrate caching into command dispatching. From a very high-level: * A command can declare itself as cacheable by providing a callable that can be used to derive a cache key. * At dispatch time, if a command is cacheable, we attempt to construct a cacher and use it for serving the request and/or caching the request. * The dispatch layer handles the bulk of the business logic for caching, making cachers mostly "dumb content stores." * The mechanism for invalidating cached entries (one of the harder parts about caching in general) is by varying the cache key when state changes. As such, cachers don't need to be concerned with cache invalidation. Initially, we've hooked up support for caching "manifestdata" and "filedata" commands. These are the simplest to cache, as they should be immutable over time. Caching of commands related to changeset data is a bit harder (because cache validation is impacted by changes to bookmarks, phases, etc). This will be implemented later. (Strictly speaking, censoring a file should invalidate caches. I've added an inline TODO to track this edge case.) To prove it works, this commit implements a test-only extension providing in-memory caching backed by an lrucachedict. A new test showing this extension behaving properly is added. FWIW, the cacher is ~50 lines of code, demonstrating the relative ease with which a cache can be added to a server. While the test cacher is not suitable for production workloads, just for kicks I performed a clone of just the changeset and manifest data for the mozilla-unified repository. With a fully warmed cache (of just the manifest data since changeset data is not cached), server-side CPU usage dropped from ~73s to ~28s. That's pretty significant and demonstrates the potential that response caching has on server scalability! Differential Revision: https://phab.mercurial-scm.org/D4773
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
     1
# Copyright 21 May 2005 - (c) 2005 Jake Edge <jake@edge2.net>
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
     2
# Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
     3
#
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8109
diff changeset
     4
# This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9713
diff changeset
     5
# GNU General Public License version 2 or any later version.
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
     6
27046
37fcfe52c68c hgweb: use absolute_import
Yuya Nishihara <yuya@tcha.org>
parents: 20903
diff changeset
     7
from __future__ import absolute_import
37fcfe52c68c hgweb: use absolute_import
Yuya Nishihara <yuya@tcha.org>
parents: 20903
diff changeset
     8
36066
2ad145fbde54 wireprotoserver: add context manager mechanism for redirecting stdio
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36065
diff changeset
     9
import contextlib
40021
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
    10
import hashlib
27046
37fcfe52c68c hgweb: use absolute_import
Yuya Nishihara <yuya@tcha.org>
parents: 20903
diff changeset
    11
35859
1bf5263fe5cc wireprotoserver: move sshserver into module (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35858
diff changeset
    12
from .i18n import _
39630
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
    13
from .node import (
39639
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
    14
    hex,
39630
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
    15
    nullid,
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
    16
)
35856
ef3a24a023ec wireprotoserver: rename hgweb.protocol to wireprotoserver (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35750
diff changeset
    17
from . import (
39630
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
    18
    discovery,
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
    19
    encoding,
34509
e21f274cccea hgweb: in protocol adapter, avoid control reaching end of non-void function
Augie Fackler <augie@google.com>
parents: 33821
diff changeset
    20
    error,
39811
ae20f52437e9 wireprotov2: advertise recognized path filter prefixes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39810
diff changeset
    21
    narrowspec,
34742
5a9cad0dfddb hgweb: when unpacking args from request form, convert to bytes
Augie Fackler <augie@google.com>
parents: 34740
diff changeset
    22
    pycompat,
37657
23c4ddda7bbe wireproto: expose repository formats via capabilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37653
diff changeset
    23
    streamclone,
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
    24
    util,
37052
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
    25
    wireprotoframing,
36073
cd6ab329c5c7 wireprototypes: move wire protocol response types to new module
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36072
diff changeset
    26
    wireprototypes,
27046
37fcfe52c68c hgweb: use absolute_import
Yuya Nishihara <yuya@tcha.org>
parents: 20903
diff changeset
    27
)
37810
856f381ad74b interfaceutil: module to stub out zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37784
diff changeset
    28
from .utils import (
40021
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
    29
    cborutil,
37810
856f381ad74b interfaceutil: module to stub out zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37784
diff changeset
    30
    interfaceutil,
39837
69b4a5b89dc5 py3: cast exception to bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39828
diff changeset
    31
    stringutil,
37810
856f381ad74b interfaceutil: module to stub out zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37784
diff changeset
    32
)
35856
ef3a24a023ec wireprotoserver: rename hgweb.protocol to wireprotoserver (API)
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35750
diff changeset
    33
37725
3ea8323d6f95 wireprotov2: change command response protocol to include a leading map
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37724
diff changeset
    34
FRAMINGTYPE = b'application/mercurial-exp-framing-0005'
11595
368cd5325348 protocol: move hgweb protocol support back into protocol.py
Matt Mackall <mpm@selenic.com>
parents: 11594
diff changeset
    35
37644
77c9ee77687c wireproto: rename HTTPV2 so it less like HTTP/2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37557
diff changeset
    36
HTTP_WIREPROTO_V2 = wireprototypes.HTTP_WIREPROTO_V2
35976
48a3a9283f09 sshpeer: initial definition and implementation of new SSH protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35900
diff changeset
    37
37784
ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37783
diff changeset
    38
COMMANDS = wireprototypes.commanddict()
ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37783
diff changeset
    39
40021
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
    40
# Value inserted into cache key computation function. Change the value to
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
    41
# force new cache keys for every command request. This should be done when
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
    42
# there is a change to how caching works, etc.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
    43
GLOBAL_CACHE_VERSION = 1
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
    44
37545
93397c4633f6 wireproto: extract HTTP version 2 code to own module
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37535
diff changeset
    45
def handlehttpv2request(rctx, req, res, checkperm, urlparts):
37047
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    46
    from .hgweb import common as hgwebcommon
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    47
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    48
    # URL space looks like: <permissions>/<command>, where <permission> can
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    49
    # be ``ro`` or ``rw`` to signal read-only or read-write, respectively.
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    50
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    51
    # Root URL does nothing meaningful... yet.
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    52
    if not urlparts:
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    53
        res.status = b'200 OK'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    54
        res.headers[b'Content-Type'] = b'text/plain'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    55
        res.setbodybytes(_('HTTP version 2 API handler'))
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    56
        return
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    57
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    58
    if len(urlparts) == 1:
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    59
        res.status = b'404 Not Found'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    60
        res.headers[b'Content-Type'] = b'text/plain'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    61
        res.setbodybytes(_('do not know how to process %s\n') %
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    62
                         req.dispatchpath)
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    63
        return
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    64
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    65
    permission, command = urlparts[0:2]
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    66
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    67
    if permission not in (b'ro', b'rw'):
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    68
        res.status = b'404 Not Found'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    69
        res.headers[b'Content-Type'] = b'text/plain'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    70
        res.setbodybytes(_('unknown permission: %s') % permission)
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    71
        return
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    72
37048
fc5e261915b9 wireproto: require POST for all HTTPv2 requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37047
diff changeset
    73
    if req.method != 'POST':
fc5e261915b9 wireproto: require POST for all HTTPv2 requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37047
diff changeset
    74
        res.status = b'405 Method Not Allowed'
fc5e261915b9 wireproto: require POST for all HTTPv2 requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37047
diff changeset
    75
        res.headers[b'Allow'] = b'POST'
fc5e261915b9 wireproto: require POST for all HTTPv2 requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37047
diff changeset
    76
        res.setbodybytes(_('commands require POST requests'))
fc5e261915b9 wireproto: require POST for all HTTPv2 requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37047
diff changeset
    77
        return
fc5e261915b9 wireproto: require POST for all HTTPv2 requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37047
diff changeset
    78
37047
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    79
    # At some point we'll want to use our own API instead of recycling the
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    80
    # behavior of version 1 of the wire protocol...
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    81
    # TODO return reasonable responses - not responses that overload the
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    82
    # HTTP status line message for error reporting.
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    83
    try:
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    84
        checkperm(rctx, req, 'pull' if permission == b'ro' else 'push')
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    85
    except hgwebcommon.ErrorResponse as e:
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    86
        res.status = hgwebcommon.statusmessage(e.code, pycompat.bytestr(e))
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    87
        for k, v in e.headers:
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    88
            res.headers[k] = v
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    89
        res.setbodybytes('permission denied')
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    90
        return
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
    91
37052
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
    92
    # We have a special endpoint to reflect the request back at the client.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
    93
    if command == b'debugreflect':
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
    94
        _processhttpv2reflectrequest(rctx.repo.ui, rctx.repo, req, res)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
    95
        return
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
    96
37059
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
    97
    # Extra commands that we handle that aren't really wire protocol
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
    98
    # commands. Think extra hard before making this hackery available to
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
    99
    # extension.
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   100
    extracommands = {'multirequest'}
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   101
37784
ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37783
diff changeset
   102
    if command not in COMMANDS and command not in extracommands:
37047
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   103
        res.status = b'404 Not Found'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   104
        res.headers[b'Content-Type'] = b'text/plain'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   105
        res.setbodybytes(_('unknown wire protocol command: %s\n') % command)
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   106
        return
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   107
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   108
    repo = rctx.repo
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   109
    ui = repo.ui
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   110
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   111
    proto = httpv2protocolhandler(req, ui)
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   112
37784
ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37783
diff changeset
   113
    if (not COMMANDS.commandavailable(command, proto)
37059
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   114
        and command not in extracommands):
37047
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   115
        res.status = b'404 Not Found'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   116
        res.headers[b'Content-Type'] = b'text/plain'
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   117
        res.setbodybytes(_('invalid wire protocol command: %s') % command)
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   118
        return
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   119
37129
aaabd709df72 wireproto: review fixups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37120
diff changeset
   120
    # TODO consider cases where proxies may add additional Accept headers.
37051
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37050
diff changeset
   121
    if req.headers.get(b'Accept') != FRAMINGTYPE:
37050
37d7a1d18b97 wireproto: define content negotiation for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37048
diff changeset
   122
        res.status = b'406 Not Acceptable'
37d7a1d18b97 wireproto: define content negotiation for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37048
diff changeset
   123
        res.headers[b'Content-Type'] = b'text/plain'
37d7a1d18b97 wireproto: define content negotiation for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37048
diff changeset
   124
        res.setbodybytes(_('client MUST specify Accept header with value: %s\n')
37051
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37050
diff changeset
   125
                           % FRAMINGTYPE)
37050
37d7a1d18b97 wireproto: define content negotiation for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37048
diff changeset
   126
        return
37d7a1d18b97 wireproto: define content negotiation for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37048
diff changeset
   127
37052
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   128
    if req.headers.get(b'Content-Type') != FRAMINGTYPE:
37050
37d7a1d18b97 wireproto: define content negotiation for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37048
diff changeset
   129
        res.status = b'415 Unsupported Media Type'
37d7a1d18b97 wireproto: define content negotiation for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37048
diff changeset
   130
        # TODO we should send a response with appropriate media type,
37d7a1d18b97 wireproto: define content negotiation for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37048
diff changeset
   131
        # since client does Accept it.
37d7a1d18b97 wireproto: define content negotiation for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37048
diff changeset
   132
        res.headers[b'Content-Type'] = b'text/plain'
37d7a1d18b97 wireproto: define content negotiation for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37048
diff changeset
   133
        res.setbodybytes(_('client MUST send Content-Type header with '
37051
40206e227412 wireproto: define and implement protocol for issuing requests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37050
diff changeset
   134
                           'value: %s\n') % FRAMINGTYPE)
37050
37d7a1d18b97 wireproto: define content negotiation for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37048
diff changeset
   135
        return
37d7a1d18b97 wireproto: define content negotiation for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37048
diff changeset
   136
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   137
    _processhttpv2request(ui, repo, req, res, permission, command, proto)
37046
1cfef5693203 wireproto: support /api/* URL space for exposing APIs
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36877
diff changeset
   138
37052
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   139
def _processhttpv2reflectrequest(ui, repo, req, res):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   140
    """Reads unified frame protocol request and dumps out state to client.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   141
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   142
    This special endpoint can be used to help debug the wire protocol.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   143
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   144
    Instead of routing the request through the normal dispatch mechanism,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   145
    we instead read all frames, decode them, and feed them into our state
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   146
    tracker. We then dump the log of all that activity back out to the
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   147
    client.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   148
    """
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   149
    import json
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   150
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   151
    # Reflection APIs have a history of being abused, accidentally disclosing
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   152
    # sensitive data, etc. So we have a config knob.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   153
    if not ui.configbool('experimental', 'web.api.debugreflect'):
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   154
        res.status = b'404 Not Found'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   155
        res.headers[b'Content-Type'] = b'text/plain'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   156
        res.setbodybytes(_('debugreflect service not available'))
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   157
        return
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   158
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   159
    # We assume we have a unified framing protocol request body.
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   160
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   161
    reactor = wireprotoframing.serverreactor()
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   162
    states = []
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   163
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   164
    while True:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   165
        frame = wireprotoframing.readframe(req.bodyfh)
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   166
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   167
        if not frame:
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   168
            states.append(b'received: <no frame>')
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   169
            break
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   170
37061
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
   171
        states.append(b'received: %d %d %d %s' % (frame.typeid, frame.flags,
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
   172
                                                  frame.requestid,
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
   173
                                                  frame.payload))
37052
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   174
37061
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
   175
        action, meta = reactor.onframerecv(frame)
37052
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   176
        states.append(json.dumps((action, meta), sort_keys=True,
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   177
                                 separators=(', ', ': ')))
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   178
37056
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   179
    action, meta = reactor.oninputeof()
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   180
    meta['action'] = action
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   181
    states.append(json.dumps(meta, sort_keys=True, separators=(', ',': ')))
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   182
37052
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   183
    res.status = b'200 OK'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   184
    res.headers[b'Content-Type'] = b'text/plain'
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   185
    res.setbodybytes(b'\n'.join(states))
8c3c47362934 wireproto: implement basic frame reading and processing
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37051
diff changeset
   186
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   187
def _processhttpv2request(ui, repo, req, res, authedperm, reqcommand, proto):
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   188
    """Post-validation handler for HTTPv2 requests.
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   189
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   190
    Called when the HTTP request contains unified frame-based protocol
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   191
    frames for evaluation.
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   192
    """
37056
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   193
    # TODO Some HTTP clients are full duplex and can receive data before
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   194
    # the entire request is transmitted. Figure out a way to indicate support
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   195
    # for that so we can opt into full duplex mode.
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   196
    reactor = wireprotoframing.serverreactor(deferoutput=True)
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   197
    seencommand = False
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   198
37289
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
   199
    outstream = reactor.makeoutputstream()
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
   200
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   201
    while True:
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   202
        frame = wireprotoframing.readframe(req.bodyfh)
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   203
        if not frame:
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   204
            break
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   205
37061
884a0c1604ad wireproto: define attr-based classes for representing frames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37059
diff changeset
   206
        action, meta = reactor.onframerecv(frame)
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   207
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   208
        if action == 'wantframe':
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   209
            # Need more data before we can do anything.
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   210
            continue
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   211
        elif action == 'runcommand':
37059
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   212
            sentoutput = _httpv2runcommand(ui, repo, req, res, authedperm,
37289
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
   213
                                           reqcommand, reactor, outstream,
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
   214
                                           meta, issubsequent=seencommand)
37059
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   215
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   216
            if sentoutput:
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   217
                return
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   218
37059
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   219
            seencommand = True
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   220
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   221
        elif action == 'error':
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   222
            # TODO define proper error mechanism.
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   223
            res.status = b'200 OK'
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   224
            res.headers[b'Content-Type'] = b'text/plain'
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   225
            res.setbodybytes(meta['message'] + b'\n')
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   226
            return
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   227
        else:
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   228
            raise error.ProgrammingError(
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   229
                'unhandled action from frame processor: %s' % action)
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   230
37056
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   231
    action, meta = reactor.oninputeof()
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   232
    if action == 'sendframes':
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   233
        # We assume we haven't started sending the response yet. If we're
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   234
        # wrong, the response type will raise an exception.
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   235
        res.status = b'200 OK'
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   236
        res.headers[b'Content-Type'] = FRAMINGTYPE
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   237
        res.setbodygen(meta['framegen'])
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   238
    elif action == 'noop':
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   239
        pass
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   240
    else:
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   241
        raise error.ProgrammingError('unhandled action from frame processor: %s'
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   242
                                     % action)
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   243
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   244
def _httpv2runcommand(ui, repo, req, res, authedperm, reqcommand, reactor,
37289
5fadc63ac99f wireproto: explicit API to create outgoing streams
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37288
diff changeset
   245
                      outstream, command, issubsequent):
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   246
    """Dispatch a wire protocol command made from HTTPv2 requests.
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   247
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   248
    The authenticated permission (``authedperm``) along with the original
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   249
    command from the URL (``reqcommand``) are passed in.
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   250
    """
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   251
    # We already validated that the session has permissions to perform the
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   252
    # actions in ``authedperm``. In the unified frame protocol, the canonical
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   253
    # command to run is expressed in a frame. However, the URL also requested
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   254
    # to run a specific command. We need to be careful that the command we
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   255
    # run doesn't have permissions requirements greater than what was granted
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   256
    # by ``authedperm``.
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   257
    #
37059
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   258
    # Our rule for this is we only allow one command per HTTP request and
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   259
    # that command must match the command in the URL. However, we make
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   260
    # an exception for the ``multirequest`` URL. This URL is allowed to
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   261
    # execute multiple commands. We double check permissions of each command
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   262
    # as it is invoked to ensure there is no privilege escalation.
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   263
    # TODO consider allowing multiple commands to regular command URLs
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   264
    # iff each command is the same.
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   265
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   266
    proto = httpv2protocolhandler(req, ui, args=command['args'])
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   267
37059
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   268
    if reqcommand == b'multirequest':
37784
ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37783
diff changeset
   269
        if not COMMANDS.commandavailable(command['command'], proto):
37059
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   270
            # TODO proper error mechanism
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   271
            res.status = b'200 OK'
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   272
            res.headers[b'Content-Type'] = b'text/plain'
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   273
            res.setbodybytes(_('wire protocol command not available: %s') %
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   274
                             command['command'])
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   275
            return True
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   276
37129
aaabd709df72 wireproto: review fixups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37120
diff changeset
   277
        # TODO don't use assert here, since it may be elided by -O.
37059
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   278
        assert authedperm in (b'ro', b'rw')
37784
ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37783
diff changeset
   279
        wirecommand = COMMANDS[command['command']]
37059
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   280
        assert wirecommand.permission in ('push', 'pull')
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   281
37059
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   282
        if authedperm == b'ro' and wirecommand.permission != 'pull':
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   283
            # TODO proper error mechanism
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   284
            res.status = b'403 Forbidden'
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   285
            res.headers[b'Content-Type'] = b'text/plain'
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   286
            res.setbodybytes(_('insufficient permissions to execute '
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   287
                               'command: %s') % command['command'])
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   288
            return True
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   289
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   290
        # TODO should we also call checkperm() here? Maybe not if we're going
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   291
        # to overhaul that API. The granted scope from the URL check should
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   292
        # be good enough.
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   293
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   294
    else:
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   295
        # Don't allow multiple commands outside of ``multirequest`` URL.
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   296
        if issubsequent:
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   297
            # TODO proper error mechanism
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   298
            res.status = b'200 OK'
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   299
            res.headers[b'Content-Type'] = b'text/plain'
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   300
            res.setbodybytes(_('multiple commands cannot be issued to this '
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   301
                               'URL'))
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   302
            return True
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   303
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   304
        if reqcommand != command['command']:
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   305
            # TODO define proper error mechanism
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   306
            res.status = b'200 OK'
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   307
            res.headers[b'Content-Type'] = b'text/plain'
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   308
            res.setbodybytes(_('command in frame must match command in URL'))
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   309
            return True
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   310
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   311
    res.status = b'200 OK'
37055
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
   312
    res.headers[b'Content-Type'] = FRAMINGTYPE
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   313
39559
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   314
    try:
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   315
        objs = dispatch(repo, proto, command['command'])
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   316
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   317
        action, meta = reactor.oncommandresponsereadyobjects(
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   318
            outstream, command['requestid'], objs)
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   319
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   320
    except error.WireprotoCommandError as e:
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   321
        action, meta = reactor.oncommanderror(
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   322
            outstream, command['requestid'], e.message, e.messageargs)
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   323
39559
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   324
    except Exception as e:
37726
0c184ca594bb wireprotov2: change behavior of error frame
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37725
diff changeset
   325
        action, meta = reactor.onservererror(
39559
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   326
            outstream, command['requestid'],
39837
69b4a5b89dc5 py3: cast exception to bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39828
diff changeset
   327
            _('exception when invoking command: %s') %
69b4a5b89dc5 py3: cast exception to bytes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39828
diff changeset
   328
            stringutil.forcebytestr(e))
37055
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
   329
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
   330
    if action == 'sendframes':
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
   331
        res.setbodygen(meta['framegen'])
37059
bbea991635d0 wireproto: service multiple command requests per HTTP request
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37057
diff changeset
   332
        return True
37056
861e9d37e56e wireproto: buffer output frames when in half duplex mode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37055
diff changeset
   333
    elif action == 'noop':
37129
aaabd709df72 wireproto: review fixups
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37120
diff changeset
   334
        return False
37055
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
   335
    else:
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
   336
        raise error.ProgrammingError('unhandled event from reactor: %s' %
61393f888dfe wireproto: define and implement responses in framing protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37054
diff changeset
   337
                                     action)
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   338
37782
99accae4cc59 wireproto: reimplement dispatch() for version 2 server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37781
diff changeset
   339
def getdispatchrepo(repo, proto, command):
99accae4cc59 wireproto: reimplement dispatch() for version 2 server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37781
diff changeset
   340
    return repo.filtered('served')
99accae4cc59 wireproto: reimplement dispatch() for version 2 server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37781
diff changeset
   341
99accae4cc59 wireproto: reimplement dispatch() for version 2 server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37781
diff changeset
   342
def dispatch(repo, proto, command):
40021
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   343
    """Run a wire protocol command.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   344
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   345
    Returns an iterable of objects that will be sent to the client.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   346
    """
37782
99accae4cc59 wireproto: reimplement dispatch() for version 2 server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37781
diff changeset
   347
    repo = getdispatchrepo(repo, proto, command)
99accae4cc59 wireproto: reimplement dispatch() for version 2 server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37781
diff changeset
   348
40021
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   349
    entry = COMMANDS[command]
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   350
    func = entry.func
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   351
    spec = entry.args
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   352
37782
99accae4cc59 wireproto: reimplement dispatch() for version 2 server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37781
diff changeset
   353
    args = proto.getargs(spec)
99accae4cc59 wireproto: reimplement dispatch() for version 2 server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37781
diff changeset
   354
40021
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   355
    # There is some duplicate boilerplate code here for calling the command and
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   356
    # emitting objects. It is either that or a lot of indented code that looks
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   357
    # like a pyramid (since there are a lot of code paths that result in not
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   358
    # using the cacher).
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   359
    callcommand = lambda: func(repo, proto, **pycompat.strkwargs(args))
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   360
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   361
    # Request is not cacheable. Don't bother instantiating a cacher.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   362
    if not entry.cachekeyfn:
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   363
        for o in callcommand():
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   364
            yield o
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   365
        return
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   366
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   367
    cacher = makeresponsecacher(repo, proto, command, args,
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   368
                                cborutil.streamencode)
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   369
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   370
    # But we have no cacher. Do default handling.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   371
    if not cacher:
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   372
        for o in callcommand():
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   373
            yield o
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   374
        return
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   375
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   376
    with cacher:
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   377
        cachekey = entry.cachekeyfn(repo, proto, cacher, **args)
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   378
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   379
        # No cache key or the cacher doesn't like it. Do default handling.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   380
        if cachekey is None or not cacher.setcachekey(cachekey):
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   381
            for o in callcommand():
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   382
                yield o
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   383
            return
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   384
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   385
        # Serve it from the cache, if possible.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   386
        cached = cacher.lookup()
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   387
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   388
        if cached:
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   389
            for o in cached['objs']:
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   390
                yield o
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   391
            return
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   392
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   393
        # Else call the command and feed its output into the cacher, allowing
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   394
        # the cacher to buffer/mutate objects as it desires.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   395
        for o in callcommand():
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   396
            for o in cacher.onobject(o):
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   397
                yield o
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   398
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   399
        for o in cacher.onfinished():
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   400
            yield o
37782
99accae4cc59 wireproto: reimplement dispatch() for version 2 server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37781
diff changeset
   401
37810
856f381ad74b interfaceutil: module to stub out zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37784
diff changeset
   402
@interfaceutil.implementer(wireprototypes.baseprotocolhandler)
37296
78103e4138b1 wireproto: port protocol handler to zope.interface
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37295
diff changeset
   403
class httpv2protocolhandler(object):
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   404
    def __init__(self, req, ui, args=None):
37047
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   405
        self._req = req
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   406
        self._ui = ui
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   407
        self._args = args
37047
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   408
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   409
    @property
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   410
    def name(self):
37644
77c9ee77687c wireproto: rename HTTPV2 so it less like HTTP/2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37557
diff changeset
   411
        return HTTP_WIREPROTO_V2
37047
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   412
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   413
    def getargs(self, args):
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   414
        # First look for args that were passed but aren't registered on this
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   415
        # command.
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   416
        extra = set(self._args) - set(args)
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   417
        if extra:
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   418
            raise error.WireprotoCommandError(
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   419
                'unsupported argument to command: %s' %
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   420
                ', '.join(sorted(extra)))
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   421
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   422
        # And look for required arguments that are missing.
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   423
        missing = {a for a in args if args[a]['required']} - set(self._args)
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   424
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   425
        if missing:
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   426
            raise error.WireprotoCommandError(
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   427
                'missing required arguments: %s' % ', '.join(sorted(missing)))
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   428
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   429
        # Now derive the arguments to pass to the command, taking into
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   430
        # account the arguments specified by the client.
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   431
        data = {}
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   432
        for k, meta in sorted(args.items()):
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   433
            # This argument wasn't passed by the client.
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   434
            if k not in self._args:
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   435
                data[k] = meta['default']()
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   436
                continue
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   437
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   438
            v = self._args[k]
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   439
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   440
            # Sets may be expressed as lists. Silently normalize.
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   441
            if meta['type'] == 'set' and isinstance(v, list):
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   442
                v = set(v)
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   443
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   444
            # TODO consider more/stronger type validation.
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   445
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   446
            data[k] = v
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   447
37485
0b7475ea38cf wireproto: port heads command to wire protocol v2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37414
diff changeset
   448
        return data
37047
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   449
37393
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37296
diff changeset
   450
    def getprotocaps(self):
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37296
diff changeset
   451
        # Protocol capabilities are currently not implemented for HTTP V2.
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37296
diff changeset
   452
        return set()
afcfdf53e4b5 wireproto: provide accessors for client capabilities
Joerg Sonnenberger <joerg@bec.de>
parents: 37296
diff changeset
   453
37414
2d965bfeb8f6 wireproto: allow direct stream processing for unbundle
Joerg Sonnenberger <joerg@bec.de>
parents: 37393
diff changeset
   454
    def getpayload(self):
37047
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   455
        raise NotImplementedError
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   456
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   457
    @contextlib.contextmanager
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   458
    def mayberedirectstdio(self):
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   459
        raise NotImplementedError
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   460
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   461
    def client(self):
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   462
        raise NotImplementedError
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   463
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   464
    def addcapabilities(self, repo, caps):
37054
e7a012b60d6e wireproto: implement basic command dispatching for HTTPv2
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37053
diff changeset
   465
        return caps
37047
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   466
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   467
    def checkperm(self, perm):
fddcb51b5084 wireproto: define permissions-based routing of HTTPv2 wire protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37046
diff changeset
   468
        raise NotImplementedError
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   469
37557
734515aca84d wireproto: define and implement HTTP handshake to upgrade protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
   470
def httpv2apidescriptor(req, repo):
734515aca84d wireproto: define and implement HTTP handshake to upgrade protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
   471
    proto = httpv2protocolhandler(req, repo.ui)
734515aca84d wireproto: define and implement HTTP handshake to upgrade protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
   472
734515aca84d wireproto: define and implement HTTP handshake to upgrade protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
   473
    return _capabilitiesv2(repo, proto)
734515aca84d wireproto: define and implement HTTP handshake to upgrade protocol
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37546
diff changeset
   474
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   475
def _capabilitiesv2(repo, proto):
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   476
    """Obtain the set of capabilities for version 2 transports.
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   477
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   478
    These capabilities are distinct from the capabilities for version 1
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   479
    transports.
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   480
    """
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   481
    compression = []
37783
9d818539abfa wireproto: move supportedcompengines out of wireproto
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37782
diff changeset
   482
    for engine in wireprototypes.supportedcompengines(repo.ui, util.SERVERROLE):
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   483
        compression.append({
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   484
            b'name': engine.wireprotosupport().name,
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   485
        })
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   486
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   487
    caps = {
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   488
        'commands': {},
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   489
        'compression': compression,
37653
b2fa1591fb44 wireproto: add media type to version 2 capabilities response
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37644
diff changeset
   490
        'framingmediatypes': [FRAMINGTYPE],
39811
ae20f52437e9 wireprotov2: advertise recognized path filter prefixes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39810
diff changeset
   491
        'pathfilterprefixes': set(narrowspec.VALID_PREFIXES),
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   492
    }
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   493
37784
ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37783
diff changeset
   494
    for command, entry in COMMANDS.items():
39812
8e7e822e85ec wireprotov2: expose rich arguments metadata
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39811
diff changeset
   495
        args = {}
8e7e822e85ec wireprotov2: expose rich arguments metadata
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39811
diff changeset
   496
8e7e822e85ec wireprotov2: expose rich arguments metadata
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39811
diff changeset
   497
        for arg, meta in entry.args.items():
8e7e822e85ec wireprotov2: expose rich arguments metadata
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39811
diff changeset
   498
            args[arg] = {
8e7e822e85ec wireprotov2: expose rich arguments metadata
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39811
diff changeset
   499
                # TODO should this be a normalized type using CBOR's
8e7e822e85ec wireprotov2: expose rich arguments metadata
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39811
diff changeset
   500
                # terminology?
8e7e822e85ec wireprotov2: expose rich arguments metadata
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39811
diff changeset
   501
                b'type': meta['type'],
8e7e822e85ec wireprotov2: expose rich arguments metadata
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39811
diff changeset
   502
                b'required': meta['required'],
8e7e822e85ec wireprotov2: expose rich arguments metadata
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39811
diff changeset
   503
            }
8e7e822e85ec wireprotov2: expose rich arguments metadata
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39811
diff changeset
   504
8e7e822e85ec wireprotov2: expose rich arguments metadata
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39811
diff changeset
   505
            if not meta['required']:
8e7e822e85ec wireprotov2: expose rich arguments metadata
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39811
diff changeset
   506
                args[arg][b'default'] = meta['default']()
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   507
39813
c30faea8d02d wireprotov2: advertise set of valid values for requestable fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39812
diff changeset
   508
            if meta['validvalues']:
c30faea8d02d wireprotov2: advertise set of valid values for requestable fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39812
diff changeset
   509
                args[arg][b'validvalues'] = meta['validvalues']
c30faea8d02d wireprotov2: advertise set of valid values for requestable fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39812
diff changeset
   510
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   511
        caps['commands'][command] = {
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   512
            'args': args,
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   513
            'permissions': [entry.permission],
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   514
        }
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   515
37657
23c4ddda7bbe wireproto: expose repository formats via capabilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37653
diff changeset
   516
    if streamclone.allowservergeneration(repo):
23c4ddda7bbe wireproto: expose repository formats via capabilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37653
diff changeset
   517
        caps['rawrepoformats'] = sorted(repo.requirements &
23c4ddda7bbe wireproto: expose repository formats via capabilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37653
diff changeset
   518
                                        repo.supportedformats)
23c4ddda7bbe wireproto: expose repository formats via capabilities
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37653
diff changeset
   519
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   520
    return proto.addcapabilities(repo, caps)
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   521
40021
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   522
def wireprotocommand(name, args=None, permission='push', cachekeyfn=None):
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   523
    """Decorator to declare a wire protocol command.
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   524
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   525
    ``name`` is the name of the wire protocol command being provided.
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   526
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   527
    ``args`` is a dict defining arguments accepted by the command. Keys are
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   528
    the argument name. Values are dicts with the following keys:
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   529
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   530
       ``type``
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   531
          The argument data type. Must be one of the following string
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   532
          literals: ``bytes``, ``int``, ``list``, ``dict``, ``set``,
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   533
          or ``bool``.
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   534
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   535
       ``default``
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   536
          A callable returning the default value for this argument. If not
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   537
          specified, ``None`` will be the default value.
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   538
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   539
       ``example``
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   540
          An example value for this argument.
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   541
39813
c30faea8d02d wireprotov2: advertise set of valid values for requestable fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39812
diff changeset
   542
       ``validvalues``
c30faea8d02d wireprotov2: advertise set of valid values for requestable fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39812
diff changeset
   543
          Set of recognized values for this argument.
c30faea8d02d wireprotov2: advertise set of valid values for requestable fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39812
diff changeset
   544
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   545
    ``permission`` defines the permission type needed to run this command.
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   546
    Can be ``push`` or ``pull``. These roughly map to read-write and read-only,
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   547
    respectively. Default is to assume command requires ``push`` permissions
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   548
    because otherwise commands not declaring their permissions could modify
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   549
    a repository that is supposed to be read-only.
39559
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   550
40021
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   551
    ``cachekeyfn`` defines an optional callable that can derive the
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   552
    cache key for this request.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   553
39559
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   554
    Wire protocol commands are generators of objects to be serialized and
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   555
    sent to the client.
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   556
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   557
    If a command raises an uncaught exception, this will be translated into
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   558
    a command error.
40021
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   559
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   560
    All commands can opt in to being cacheable by defining a function
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   561
    (``cachekeyfn``) that is called to derive a cache key. This function
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   562
    receives the same arguments as the command itself plus a ``cacher``
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   563
    argument containing the active cacher for the request and returns a bytes
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   564
    containing the key in a cache the response to this command may be cached
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   565
    under.
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   566
    """
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   567
    transports = {k for k, v in wireprototypes.TRANSPORTS.items()
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   568
                  if v['version'] == 2}
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   569
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   570
    if permission not in ('push', 'pull'):
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   571
        raise error.ProgrammingError('invalid wire protocol permission; '
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   572
                                     'got %s; expected "push" or "pull"' %
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   573
                                     permission)
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   574
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   575
    if args is None:
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   576
        args = {}
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   577
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   578
    if not isinstance(args, dict):
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   579
        raise error.ProgrammingError('arguments for version 2 commands '
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   580
                                     'must be declared as dicts')
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   581
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   582
    for arg, meta in args.items():
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   583
        if arg == '*':
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   584
            raise error.ProgrammingError('* argument name not allowed on '
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   585
                                         'version 2 commands')
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   586
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   587
        if not isinstance(meta, dict):
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   588
            raise error.ProgrammingError('arguments for version 2 commands '
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   589
                                         'must declare metadata as a dict')
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   590
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   591
        if 'type' not in meta:
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   592
            raise error.ProgrammingError('%s argument for command %s does not '
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   593
                                         'declare type field' % (arg, name))
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   594
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   595
        if meta['type'] not in ('bytes', 'int', 'list', 'dict', 'set', 'bool'):
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   596
            raise error.ProgrammingError('%s argument for command %s has '
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   597
                                         'illegal type: %s' % (arg, name,
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   598
                                                               meta['type']))
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   599
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   600
        if 'example' not in meta:
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   601
            raise error.ProgrammingError('%s argument for command %s does not '
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   602
                                         'declare example field' % (arg, name))
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   603
39995
582676acaf6d wireprotov2: derive "required" from presence of default value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39864
diff changeset
   604
        meta['required'] = 'default' not in meta
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   605
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   606
        meta.setdefault('default', lambda: None)
39813
c30faea8d02d wireprotov2: advertise set of valid values for requestable fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39812
diff changeset
   607
        meta.setdefault('validvalues', None)
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   608
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   609
    def register(func):
37784
ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37783
diff changeset
   610
        if name in COMMANDS:
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   611
            raise error.ProgrammingError('%s command already registered '
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   612
                                         'for version 2' % name)
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   613
37784
ee0d5e9d77b2 wireproto: move version 2 commands dict to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37783
diff changeset
   614
        COMMANDS[name] = wireprototypes.commandentry(
40021
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   615
            func, args=args, transports=transports, permission=permission,
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   616
            cachekeyfn=cachekeyfn)
37780
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   617
8acd3a9ac4fd wireproto: make version 2 @wireprotocommand an independent function
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37728
diff changeset
   618
        return func
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   619
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   620
    return register
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   621
40021
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   622
def makecommandcachekeyfn(command, localversion=None, allargs=False):
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   623
    """Construct a cache key derivation function with common features.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   624
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   625
    By default, the cache key is a hash of:
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   626
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   627
    * The command name.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   628
    * A global cache version number.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   629
    * A local cache version number (passed via ``localversion``).
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   630
    * All the arguments passed to the command.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   631
    * The media type used.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   632
    * Wire protocol version string.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   633
    * The repository path.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   634
    """
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   635
    if not allargs:
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   636
        raise error.ProgrammingError('only allargs=True is currently supported')
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   637
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   638
    if localversion is None:
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   639
        raise error.ProgrammingError('must set localversion argument value')
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   640
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   641
    def cachekeyfn(repo, proto, cacher, **args):
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   642
        spec = COMMANDS[command]
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   643
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   644
        # Commands that mutate the repo can not be cached.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   645
        if spec.permission == 'push':
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   646
            return None
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   647
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   648
        # TODO config option to disable caching.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   649
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   650
        # Our key derivation strategy is to construct a data structure
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   651
        # holding everything that could influence cacheability and to hash
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   652
        # the CBOR representation of that. Using CBOR seems like it might
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   653
        # be overkill. However, simpler hashing mechanisms are prone to
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   654
        # duplicate input issues. e.g. if you just concatenate two values,
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   655
        # "foo"+"bar" is identical to "fo"+"obar". Using CBOR provides
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   656
        # "padding" between values and prevents these problems.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   657
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   658
        # Seed the hash with various data.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   659
        state = {
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   660
            # To invalidate all cache keys.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   661
            b'globalversion': GLOBAL_CACHE_VERSION,
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   662
            # More granular cache key invalidation.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   663
            b'localversion': localversion,
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   664
            # Cache keys are segmented by command.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   665
            b'command': pycompat.sysbytes(command),
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   666
            # Throw in the media type and API version strings so changes
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   667
            # to exchange semantics invalid cache.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   668
            b'mediatype': FRAMINGTYPE,
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   669
            b'version': HTTP_WIREPROTO_V2,
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   670
            # So same requests for different repos don't share cache keys.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   671
            b'repo': repo.root,
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   672
        }
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   673
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   674
        # The arguments passed to us will have already been normalized.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   675
        # Default values will be set, etc. This is important because it
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   676
        # means that it doesn't matter if clients send an explicit argument
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   677
        # or rely on the default value: it will all normalize to the same
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   678
        # set of arguments on the server and therefore the same cache key.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   679
        #
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   680
        # Arguments by their very nature must support being encoded to CBOR.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   681
        # And the CBOR encoder is deterministic. So we hash the arguments
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   682
        # by feeding the CBOR of their representation into the hasher.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   683
        if allargs:
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   684
            state[b'args'] = pycompat.byteskwargs(args)
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   685
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   686
        cacher.adjustcachekeystate(state)
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   687
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   688
        hasher = hashlib.sha1()
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   689
        for chunk in cborutil.streamencode(state):
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   690
            hasher.update(chunk)
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   691
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   692
        return pycompat.sysbytes(hasher.hexdigest())
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   693
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   694
    return cachekeyfn
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   695
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   696
def makeresponsecacher(repo, proto, command, args, objencoderfn):
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   697
    """Construct a cacher for a cacheable command.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   698
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   699
    Returns an ``iwireprotocolcommandcacher`` instance.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   700
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   701
    Extensions can monkeypatch this function to provide custom caching
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   702
    backends.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   703
    """
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   704
    return None
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   705
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   706
@wireprotocommand('branchmap', permission='pull')
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   707
def branchmapv2(repo, proto):
39559
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   708
    yield {encoding.fromlocal(k): v
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   709
           for k, v in repo.branchmap().iteritems()}
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   710
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   711
@wireprotocommand('capabilities', permission='pull')
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   712
def capabilitiesv2(repo, proto):
39559
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   713
    yield _capabilitiesv2(repo, proto)
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   714
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   715
@wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   716
    'changesetdata',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   717
    args={
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   718
        'noderange': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   719
            'type': 'list',
39995
582676acaf6d wireprotov2: derive "required" from presence of default value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39864
diff changeset
   720
            'default': lambda: None,
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   721
            'example': [[b'0123456...'], [b'abcdef...']],
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   722
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   723
        'nodes': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   724
            'type': 'list',
39995
582676acaf6d wireprotov2: derive "required" from presence of default value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39864
diff changeset
   725
            'default': lambda: None,
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   726
            'example': [b'0123456...'],
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   727
        },
39815
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   728
        'nodesdepth': {
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   729
            'type': 'int',
39995
582676acaf6d wireprotov2: derive "required" from presence of default value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39864
diff changeset
   730
            'default': lambda: None,
39815
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   731
            'example': 10,
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   732
        },
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   733
        'fields': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   734
            'type': 'set',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   735
            'default': set,
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   736
            'example': {b'parents', b'revision'},
39813
c30faea8d02d wireprotov2: advertise set of valid values for requestable fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39812
diff changeset
   737
            'validvalues': {b'bookmarks', b'parents', b'phase', b'revision'},
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   738
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   739
    },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   740
    permission='pull')
39815
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   741
def changesetdata(repo, proto, noderange, nodes, nodesdepth, fields):
39636
399ddd3227a4 wireprotov2: add TODOs around extending changesetdata fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39634
diff changeset
   742
    # TODO look for unknown fields and abort when they can't be serviced.
39813
c30faea8d02d wireprotov2: advertise set of valid values for requestable fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39812
diff changeset
   743
    # This could probably be validated by dispatcher using validvalues.
39636
399ddd3227a4 wireprotov2: add TODOs around extending changesetdata fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39634
diff changeset
   744
39630
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   745
    if noderange is None and nodes is None:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   746
        raise error.WireprotoCommandError(
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   747
            'noderange or nodes must be defined')
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   748
39815
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   749
    if nodesdepth is not None and nodes is None:
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   750
        raise error.WireprotoCommandError(
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   751
            'nodesdepth requires the nodes argument')
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   752
39630
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   753
    if noderange is not None:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   754
        if len(noderange) != 2:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   755
            raise error.WireprotoCommandError(
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   756
                'noderange must consist of 2 elements')
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   757
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   758
        if not noderange[1]:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   759
            raise error.WireprotoCommandError(
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   760
                'heads in noderange request cannot be empty')
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   761
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   762
    cl = repo.changelog
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   763
    hasnode = cl.hasnode
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   764
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   765
    seen = set()
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   766
    outgoing = []
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   767
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   768
    if nodes is not None:
39815
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   769
        outgoing = [n for n in nodes if hasnode(n)]
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   770
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   771
        if nodesdepth:
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   772
            outgoing = [cl.node(r) for r in
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   773
                        repo.revs(b'ancestors(%ln, %d)', outgoing,
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   774
                                  nodesdepth - 1)]
d3d333ab167a wireprotov2: teach changesetdata to fetch ancestors until depth
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39814
diff changeset
   775
39630
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   776
        seen |= set(outgoing)
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   777
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   778
    if noderange is not None:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   779
        if noderange[0]:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   780
            common = [n for n in noderange[0] if hasnode(n)]
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   781
        else:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   782
            common = [nullid]
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   783
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   784
        for n in discovery.outgoing(repo, common, noderange[1]).missing:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   785
            if n not in seen:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   786
                outgoing.append(n)
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   787
            # Don't need to add to seen here because this is the final
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   788
            # source of nodes and there should be no duplicates in this
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   789
            # list.
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   790
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   791
    seen.clear()
39632
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   792
    publishing = repo.publishing()
39630
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   793
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   794
    if outgoing:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   795
        repo.hook('preoutgoing', throw=True, source='serve')
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   796
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   797
    yield {
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   798
        b'totalitems': len(outgoing),
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   799
    }
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   800
39632
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   801
    # The phases of nodes already transferred to the client may have changed
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   802
    # since the client last requested data. We send phase-only records
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   803
    # for these revisions, if requested.
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   804
    if b'phase' in fields and noderange is not None:
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   805
        # TODO skip nodes whose phase will be reflected by a node in the
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   806
        # outgoing set. This is purely an optimization to reduce data
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   807
        # size.
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   808
        for node in noderange[0]:
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   809
            yield {
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   810
                b'node': node,
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   811
                b'phase': b'public' if publishing else repo[node].phasestr()
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   812
            }
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   813
39634
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   814
    nodebookmarks = {}
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   815
    for mark, node in repo._bookmarks.items():
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   816
        nodebookmarks.setdefault(node, set()).add(mark)
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   817
39630
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   818
    # It is already topologically sorted by revision number.
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   819
    for node in outgoing:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   820
        d = {
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   821
            b'node': node,
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   822
        }
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   823
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   824
        if b'parents' in fields:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   825
            d[b'parents'] = cl.parents(node)
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   826
39632
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   827
        if b'phase' in fields:
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   828
            if publishing:
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   829
                d[b'phase'] = b'public'
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   830
            else:
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   831
                ctx = repo[node]
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   832
                d[b'phase'] = ctx.phasestr()
c1aacb0d76ff wireprotov2: add phases to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39630
diff changeset
   833
39634
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   834
        if b'bookmarks' in fields and node in nodebookmarks:
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   835
            d[b'bookmarks'] = sorted(nodebookmarks[node])
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   836
            del nodebookmarks[node]
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   837
39814
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   838
        followingmeta = []
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   839
        followingdata = []
39630
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   840
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   841
        if b'revision' in fields:
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   842
            revisiondata = cl.revision(node, raw=True)
39814
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   843
            followingmeta.append((b'revision', len(revisiondata)))
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   844
            followingdata.append(revisiondata)
39630
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   845
39636
399ddd3227a4 wireprotov2: add TODOs around extending changesetdata fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39634
diff changeset
   846
        # TODO make it possible for extensions to wrap a function or register
399ddd3227a4 wireprotov2: add TODOs around extending changesetdata fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39634
diff changeset
   847
        # a handler to service custom fields.
399ddd3227a4 wireprotov2: add TODOs around extending changesetdata fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39634
diff changeset
   848
39814
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   849
        if followingmeta:
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   850
            d[b'fieldsfollowing'] = followingmeta
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   851
39630
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   852
        yield d
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   853
39814
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   854
        for extra in followingdata:
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   855
            yield extra
39630
9c2c77c73f23 wireprotov2: define and implement "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39559
diff changeset
   856
39634
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   857
    # If requested, send bookmarks from nodes that didn't have revision
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   858
    # data sent so receiver is aware of any bookmark updates.
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   859
    if b'bookmarks' in fields:
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   860
        for node, marks in sorted(nodebookmarks.iteritems()):
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   861
            yield {
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   862
                b'node': node,
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   863
                b'bookmarks': sorted(marks),
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   864
            }
9dffa99f9158 wireprotov2: add bookmarks to "changesetdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39632
diff changeset
   865
39639
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   866
class FileAccessError(Exception):
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   867
    """Represents an error accessing a specific file."""
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   868
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   869
    def __init__(self, path, msg, args):
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   870
        self.path = path
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   871
        self.msg = msg
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   872
        self.args = args
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   873
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   874
def getfilestore(repo, proto, path):
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   875
    """Obtain a file storage object for use with wire protocol.
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   876
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   877
    Exists as a standalone function so extensions can monkeypatch to add
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   878
    access control.
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   879
    """
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   880
    # This seems to work even if the file doesn't exist. So catch
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   881
    # "empty" files and return an error.
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   882
    fl = repo.file(path)
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   883
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   884
    if not len(fl):
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   885
        raise FileAccessError(path, 'unknown file: %s', (path,))
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   886
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   887
    return fl
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   888
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   889
@wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   890
    'filedata',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   891
    args={
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   892
        'haveparents': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   893
            'type': 'bool',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   894
            'default': lambda: False,
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   895
            'example': True,
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   896
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   897
        'nodes': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   898
            'type': 'list',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   899
            'example': [b'0123456...'],
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   900
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   901
        'fields': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   902
            'type': 'set',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   903
            'default': set,
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   904
            'example': {b'parents', b'revision'},
39813
c30faea8d02d wireprotov2: advertise set of valid values for requestable fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39812
diff changeset
   905
            'validvalues': {b'parents', b'revision'},
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   906
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   907
        'path': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   908
            'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   909
            'example': b'foo.txt',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   910
        }
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   911
    },
40021
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   912
    permission='pull',
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   913
    # TODO censoring a file revision won't invalidate the cache.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   914
    # Figure out a way to take censoring into account when deriving
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   915
    # the cache key.
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
   916
    cachekeyfn=makecommandcachekeyfn('filedata', 1, allargs=True))
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   917
def filedata(repo, proto, haveparents, nodes, fields, path):
39639
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   918
    try:
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   919
        # Extensions may wish to access the protocol handler.
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   920
        store = getfilestore(repo, proto, path)
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   921
    except FileAccessError as e:
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   922
        raise error.WireprotoCommandError(e.msg, e.args)
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   923
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   924
    # Validate requested nodes.
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   925
    for node in nodes:
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   926
        try:
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   927
            store.rev(node)
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   928
        except error.LookupError:
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   929
            raise error.WireprotoCommandError('unknown file node: %s',
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   930
                                              (hex(node),))
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   931
39864
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
   932
    revisions = store.emitrevisions(nodes,
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
   933
                                    revisiondata=b'revision' in fields,
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
   934
                                    assumehaveparentrevisions=haveparents)
39639
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   935
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   936
    yield {
39864
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
   937
        b'totalitems': len(nodes),
39639
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   938
    }
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   939
39864
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
   940
    for revision in revisions:
39639
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   941
        d = {
39864
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
   942
            b'node': revision.node,
39639
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   943
        }
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   944
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   945
        if b'parents' in fields:
39864
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
   946
            d[b'parents'] = [revision.p1node, revision.p2node]
39639
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   947
39814
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   948
        followingmeta = []
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   949
        followingdata = []
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   950
39639
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   951
        if b'revision' in fields:
39864
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
   952
            if revision.revision is not None:
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
   953
                followingmeta.append((b'revision', len(revision.revision)))
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
   954
                followingdata.append(revision.revision)
39639
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   955
            else:
39864
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
   956
                d[b'deltabasenode'] = revision.basenode
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
   957
                followingmeta.append((b'delta', len(revision.delta)))
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
   958
                followingdata.append(revision.delta)
39814
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   959
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   960
        if followingmeta:
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   961
            d[b'fieldsfollowing'] = followingmeta
39639
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   962
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   963
        yield d
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   964
39814
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   965
        for extra in followingdata:
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
   966
            yield extra
39639
0e03e6a44dee wireprotov2: define and implement "filedata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39637
diff changeset
   967
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   968
@wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   969
    'heads',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   970
    args={
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   971
        'publiconly': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   972
            'type': 'bool',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   973
            'default': lambda: False,
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   974
            'example': False,
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   975
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   976
    },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   977
    permission='pull')
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   978
def headsv2(repo, proto, publiconly):
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   979
    if publiconly:
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   980
        repo = repo.filtered('immutable')
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   981
39559
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   982
    yield repo.heads()
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   983
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   984
@wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   985
    'known',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   986
    args={
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   987
        'nodes': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   988
            'type': 'list',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   989
            'default': list,
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   990
            'example': [b'deadbeef'],
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   991
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   992
    },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   993
    permission='pull')
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   994
def knownv2(repo, proto, nodes):
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   995
    result = b''.join(b'1' if n else b'0' for n in repo.known(nodes))
39559
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
   996
    yield result
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
   997
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   998
@wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
   999
    'listkeys',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1000
    args={
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1001
        'namespace': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1002
            'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1003
            'example': b'ns',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1004
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1005
    },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1006
    permission='pull')
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1007
def listkeysv2(repo, proto, namespace):
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
  1008
    keys = repo.listkeys(encoding.tolocal(namespace))
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
  1009
    keys = {encoding.fromlocal(k): encoding.fromlocal(v)
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
  1010
            for k, v in keys.iteritems()}
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
  1011
39559
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
  1012
    yield keys
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
  1013
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1014
@wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1015
    'lookup',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1016
    args={
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1017
        'key': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1018
            'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1019
            'example': b'foo',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1020
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1021
    },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1022
    permission='pull')
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
  1023
def lookupv2(repo, proto, key):
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
  1024
    key = encoding.tolocal(key)
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
  1025
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
  1026
    # TODO handle exception.
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
  1027
    node = repo.lookup(key)
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
  1028
39559
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
  1029
    yield node
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
  1030
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1031
@wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1032
    'manifestdata',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1033
    args={
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1034
        'nodes': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1035
            'type': 'list',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1036
            'example': [b'0123456...'],
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1037
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1038
        'haveparents': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1039
            'type': 'bool',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1040
            'default': lambda: False,
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1041
            'example': True,
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1042
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1043
        'fields': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1044
            'type': 'set',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1045
            'default': set,
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1046
            'example': {b'parents', b'revision'},
39813
c30faea8d02d wireprotov2: advertise set of valid values for requestable fields
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39812
diff changeset
  1047
            'validvalues': {b'parents', b'revision'},
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1048
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1049
        'tree': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1050
            'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1051
            'example': b'',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1052
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1053
    },
40021
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
  1054
    permission='pull',
c537144fdbef wireprotov2: support response caching
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39995
diff changeset
  1055
    cachekeyfn=makecommandcachekeyfn('manifestdata', 1, allargs=True))
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1056
def manifestdata(repo, proto, haveparents, nodes, fields, tree):
39637
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1057
    store = repo.manifestlog.getstorage(tree)
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1058
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1059
    # Validate the node is known and abort on unknown revisions.
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1060
    for node in nodes:
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1061
        try:
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1062
            store.rev(node)
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1063
        except error.LookupError:
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1064
            raise error.WireprotoCommandError(
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1065
                'unknown node: %s', (node,))
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1066
39864
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
  1067
    revisions = store.emitrevisions(nodes,
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
  1068
                                    revisiondata=b'revision' in fields,
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
  1069
                                    assumehaveparentrevisions=haveparents)
39637
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1070
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1071
    yield {
39864
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
  1072
        b'totalitems': len(nodes),
39637
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1073
    }
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1074
39864
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
  1075
    for revision in revisions:
39637
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1076
        d = {
39864
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
  1077
            b'node': revision.node,
39637
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1078
        }
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1079
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1080
        if b'parents' in fields:
39864
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
  1081
            d[b'parents'] = [revision.p1node, revision.p2node]
39637
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1082
39814
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
  1083
        followingmeta = []
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
  1084
        followingdata = []
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
  1085
39637
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1086
        if b'revision' in fields:
39864
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
  1087
            if revision.revision is not None:
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
  1088
                followingmeta.append((b'revision', len(revision.revision)))
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
  1089
                followingdata.append(revision.revision)
39637
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1090
            else:
39864
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
  1091
                d[b'deltabasenode'] = revision.basenode
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
  1092
                followingmeta.append((b'delta', len(revision.delta)))
7b752bf08435 wireprotov2server: port to emitrevisions()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39837
diff changeset
  1093
                followingdata.append(revision.delta)
39814
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
  1094
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
  1095
        if followingmeta:
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
  1096
            d[b'fieldsfollowing'] = followingmeta
39637
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1097
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1098
        yield d
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1099
39814
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
  1100
        for extra in followingdata:
d059cb669632 wireprotov2: allow multiple fields to follow revision maps
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39813
diff changeset
  1101
            yield extra
39637
c7a7c7e844e5 wireprotov2: define and implement "manifestdata" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39636
diff changeset
  1102
39810
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1103
@wireprotocommand(
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1104
    'pushkey',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1105
    args={
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1106
        'namespace': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1107
            'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1108
            'example': b'ns',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1109
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1110
        'key': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1111
            'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1112
            'example': b'key',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1113
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1114
        'old': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1115
            'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1116
            'example': b'old',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1117
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1118
        'new': {
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1119
            'type': 'bytes',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1120
            'example': 'new',
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1121
        },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1122
    },
0b61d21f05cc wireprotov2: declare command arguments richly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39641
diff changeset
  1123
    permission='push')
37546
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
  1124
def pushkeyv2(repo, proto, namespace, key, old, new):
3a2367e6c6f2 wireproto: move version 2 command handlers to wireprotov2server
Gregory Szorc <gregory.szorc@gmail.com>
parents: 37545
diff changeset
  1125
    # TODO handle ui output redirection
39559
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
  1126
    yield repo.pushkey(encoding.tolocal(namespace),
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
  1127
                       encoding.tolocal(key),
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
  1128
                       encoding.tolocal(old),
07b58266bce3 wireprotov2: implement commands as a generator of objects
Gregory Szorc <gregory.szorc@gmail.com>
parents: 39448
diff changeset
  1129
                       encoding.tolocal(new))