mercurial/hgweb/protocol.py
author Henrik Stuart <henrik.stuart@edlund.dk>
Sat, 23 May 2009 17:02:49 +0200
changeset 8562 e3495c399006
parent 8225 46293a0c7e9f
child 8845 296767acbb55
permissions -rw-r--r--
named branches: server branchmap wire protocol support (issue736) The repository command, 'branchmap', returns a dictionary, branchname -> [branchheads], and will be implemented for localrepo, httprepo and sshrepo. The following wire format is used for returning data: branchname1 branch1head2 branch1head2 ... branchname2 ... ... Branch names are URL encoded to escape white space, and branch heads are sent as hex encoded node ids. All branches and all their heads are sent. The background and motivation for this command is the desire for a richer named branch semantics when pushing changesets. The details are explained in the original proposal which is included below. 1. BACKGROUND The algorithm currently implemented in Mercurial only considers the graph theoretical heads when determining whether new heads are created, rather than using the branch heads as a count (the algorithm considers a branch head effectively closed when it is merged into another branch or a new named branch is started from that point onward). Our particular problem with the algorithm is that we'd like to see the following case working without forcing a push: Upsteam has: (0:dev) ---- (1:dev) \ `--- (2:stable) Someone merges stable into dev: (0:dev) ---- (1:dev) ------(3:dev) \ / `--- (2:stable) --------´ This can be pushed without --force (as it should). Now someone else does some coding on stable (a bug fix, say): (0:dev) ---- (1:dev) ------(3:dev) \ / `--- (2:stable) ---------´---------(4:stable) This time we need --force to push. We allow this to be pushed without using --force by getting all the remote branch heads (by extending the wire protocol with a new function). We would, furthermore, also prefer if it is impossible to push a new branch without --force (or a later --newbranch option so --force isn't shoe-horned into too many disparate functions, if need be), except of course in the case where the remote repository is empty. This is what our patches accomplish. 2. ALTERNATIVES We have, of course, considered some alternatives to reconstructing enough information to decide whether we are creating new remote branch heads, before we added the new wire protocol command. 2.1. LOOKUP ON REMOTE The main alternative is to use the information from remote.heads() and remote.lookup() to try to reconstruct enough graph information to decide whether we are creating new heads. This is not adequate as illustrated below. Remember that each lookup is typically a request-response pair over SSH or HTTP(S). If we have a simple repository at the remote end like this: (0:dev) ---- (1:dev) ---- (3:stable) \ `--- (2:dev) then remote.heads() will yield [2, 3]. Assume we have nodes [0, 1, 2] locally and want to create a new node, 4:dev, as a descendant from (1:dev), which should be OK as 1:dev is a branch head. If we do remote.lookup('dev') we will get [2]. Thus, we can get information about whether a branch exists on the remote server or not, but this does not solve our problem of figuring out whether we are creating new heads or not. Pushing 4:dev ought to be OK, since after the push, we still only have two heads on branch a. Using remote.lookup() and remote.heads() is thus not adequate to consistently decide whether we are creating new remote heads (e.g. in this situation the latter would never return 1:dev). 2.2. USING INCOMING TO RECONSTRUCT THE GRAPH An alternative would be to use information equivalent to hg incoming to get the full remote graph in addition to the local graph. To do this, we would have to get a changegroup(subset) bundle representing the remote end (which may be a substantial amount of data), getting the branch heads from an instantiated bundlerepository, deleting the bundle, and finally, we can compute the prepush logic. While this is backwards compatible, it will cause a possibly substantial slowdown of the push command as it first needs to pull in all changes. 3. FURTHER ARGUMENTS IN FAVOUR OF THE BRANCHMAP WIRE-PROTOCOL EXTENSION Currently, the commands incoming and pull, work based on the tip of a given branch if used with "-r branchname", making it hard to get all revisions of a certain branch only (if it has multiple heads). This can be solved by requesting the remote's branchheads and letting the revisions to be used with the command be these heads. This can be done by extending the commands with a new option, e.g.: hg pull -b branchname which will be turned into the equivalent of: hg pull -r branchhead1 -r branchhead2 -r branchhead3 We have a simple follow-up patch that can do this ready as well (although not submitted yet as it is pending the acceptance of the branch patch). 4. WRAP-UP We generally find that the branchmap wire protocol extension can provide better named branch support to Mercurial. Currently, some things, like the initial push scenario in this mail, are fairly counter-intuitive, and the more often you have to force push, the more it is likely you will get a lot of spurious and unnecessary merge nodes. Also, restricting incoming and pull to all changes on a branch rather than changes on the tip-most head would be a sensible extension to making named branches a first class citizen in Mercurial. Currently, named branches sometimes feel like a late-coming unwanted step-child. We have run it in a production environment for a while, with fewer multiple heads occurring in our repositories and fewer confused users as a result. Also, it fixes the long-standing issue 736. Co-contributor: Sune Foldager <cryo@cyanite.org>
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
     1
#
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
     2
# Copyright 21 May 2005 - (c) 2005 Jake Edge <jake@edge2.net>
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
     3
# Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
     4
#
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8109
diff changeset
     5
# This software may be used and distributed according to the terms of the
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8109
diff changeset
     6
# GNU General Public License version 2, incorporated herein by reference.
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
     7
8562
e3495c399006 named branches: server branchmap wire protocol support (issue736)
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8225
diff changeset
     8
import cStringIO, zlib, tempfile, errno, os, sys, urllib
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
     9
from mercurial import util, streamclone
6211
f89fd07fc51d Expand import * to allow Pyflakes to find problems
Joel Rosdahl <joel@rosdahl.net>
parents: 6155
diff changeset
    10
from mercurial.node import bin, hex
6154
ef1c5a3b653d improve changegroup.readbundle(), use it in hgweb
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6152
diff changeset
    11
from mercurial import changegroup as changegroupmod
7281
f96c20e9b56a fix missing import, spotted by pychecker
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 7180
diff changeset
    12
from common import ErrorResponse, HTTP_OK, HTTP_NOT_FOUND, HTTP_SERVER_ERROR
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    13
5963
5be210afe1b8 hgweb: explicitly check if requested command exists
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5915
diff changeset
    14
# __all__ is populated with the allowed commands. Be sure to add to it if
5be210afe1b8 hgweb: explicitly check if requested command exists
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5915
diff changeset
    15
# you're adding a new command, or the new command won't work.
5be210afe1b8 hgweb: explicitly check if requested command exists
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5915
diff changeset
    16
5be210afe1b8 hgweb: explicitly check if requested command exists
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5915
diff changeset
    17
__all__ = [
5be210afe1b8 hgweb: explicitly check if requested command exists
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5915
diff changeset
    18
   'lookup', 'heads', 'branches', 'between', 'changegroup',
5be210afe1b8 hgweb: explicitly check if requested command exists
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5915
diff changeset
    19
   'changegroupsubset', 'capabilities', 'unbundle', 'stream_out',
8562
e3495c399006 named branches: server branchmap wire protocol support (issue736)
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8225
diff changeset
    20
   'branchmap',
5963
5be210afe1b8 hgweb: explicitly check if requested command exists
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5915
diff changeset
    21
]
5be210afe1b8 hgweb: explicitly check if requested command exists
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5915
diff changeset
    22
5993
948a41e77902 hgweb: explicit response status
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5963
diff changeset
    23
HGTYPE = 'application/mercurial-0.1'
948a41e77902 hgweb: explicit response status
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5963
diff changeset
    24
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
    25
def lookup(repo, req):
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    26
    try:
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
    27
        r = hex(repo.lookup(req.form['key'][0]))
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    28
        success = 1
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    29
    except Exception,inst:
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    30
        r = str(inst)
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    31
        success = 0
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    32
    resp = "%s %s\n" % (success, r)
5993
948a41e77902 hgweb: explicit response status
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5963
diff changeset
    33
    req.respond(HTTP_OK, HGTYPE, length=len(resp))
6784
18c429ea3a0e hgweb: all protocol functions have become generators
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6782
diff changeset
    34
    yield resp
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    35
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
    36
def heads(repo, req):
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
    37
    resp = " ".join(map(hex, repo.heads())) + "\n"
5993
948a41e77902 hgweb: explicit response status
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5963
diff changeset
    38
    req.respond(HTTP_OK, HGTYPE, length=len(resp))
6784
18c429ea3a0e hgweb: all protocol functions have become generators
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6782
diff changeset
    39
    yield resp
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    40
8562
e3495c399006 named branches: server branchmap wire protocol support (issue736)
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8225
diff changeset
    41
def branchmap(repo, req):
e3495c399006 named branches: server branchmap wire protocol support (issue736)
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8225
diff changeset
    42
    branches = repo.branchmap()
e3495c399006 named branches: server branchmap wire protocol support (issue736)
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8225
diff changeset
    43
    heads = []
e3495c399006 named branches: server branchmap wire protocol support (issue736)
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8225
diff changeset
    44
    for branch, nodes in branches.iteritems():
e3495c399006 named branches: server branchmap wire protocol support (issue736)
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8225
diff changeset
    45
        branchname = urllib.quote(branch)
e3495c399006 named branches: server branchmap wire protocol support (issue736)
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8225
diff changeset
    46
        branchnodes = [hex(node) for node in nodes]
e3495c399006 named branches: server branchmap wire protocol support (issue736)
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8225
diff changeset
    47
        heads.append('%s %s' % (branchname, ' '.join(branchnodes)))
e3495c399006 named branches: server branchmap wire protocol support (issue736)
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8225
diff changeset
    48
    resp = '\n'.join(heads)
e3495c399006 named branches: server branchmap wire protocol support (issue736)
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8225
diff changeset
    49
    req.respond(HTTP_OK, HGTYPE, length=len(resp))
e3495c399006 named branches: server branchmap wire protocol support (issue736)
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8225
diff changeset
    50
    yield resp
e3495c399006 named branches: server branchmap wire protocol support (issue736)
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8225
diff changeset
    51
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
    52
def branches(repo, req):
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    53
    nodes = []
5915
d0576d065993 Prefer i in d over d.has_key(i)
Christian Ebert <blacktrash@gmx.net>
parents: 5598
diff changeset
    54
    if 'nodes' in req.form:
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    55
        nodes = map(bin, req.form['nodes'][0].split(" "))
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    56
    resp = cStringIO.StringIO()
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
    57
    for b in repo.branches(nodes):
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    58
        resp.write(" ".join(map(hex, b)) + "\n")
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    59
    resp = resp.getvalue()
5993
948a41e77902 hgweb: explicit response status
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5963
diff changeset
    60
    req.respond(HTTP_OK, HGTYPE, length=len(resp))
6784
18c429ea3a0e hgweb: all protocol functions have become generators
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6782
diff changeset
    61
    yield resp
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    62
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
    63
def between(repo, req):
5915
d0576d065993 Prefer i in d over d.has_key(i)
Christian Ebert <blacktrash@gmx.net>
parents: 5598
diff changeset
    64
    if 'pairs' in req.form:
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    65
        pairs = [map(bin, p.split("-"))
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    66
                 for p in req.form['pairs'][0].split(" ")]
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    67
    resp = cStringIO.StringIO()
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
    68
    for b in repo.between(pairs):
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    69
        resp.write(" ".join(map(hex, b)) + "\n")
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    70
    resp = resp.getvalue()
5993
948a41e77902 hgweb: explicit response status
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5963
diff changeset
    71
    req.respond(HTTP_OK, HGTYPE, length=len(resp))
6784
18c429ea3a0e hgweb: all protocol functions have become generators
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6782
diff changeset
    72
    yield resp
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    73
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
    74
def changegroup(repo, req):
5993
948a41e77902 hgweb: explicit response status
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5963
diff changeset
    75
    req.respond(HTTP_OK, HGTYPE)
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    76
    nodes = []
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    77
5915
d0576d065993 Prefer i in d over d.has_key(i)
Christian Ebert <blacktrash@gmx.net>
parents: 5598
diff changeset
    78
    if 'roots' in req.form:
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    79
        nodes = map(bin, req.form['roots'][0].split(" "))
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    80
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    81
    z = zlib.compressobj()
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
    82
    f = repo.changegroup(nodes, 'serve')
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    83
    while 1:
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    84
        chunk = f.read(4096)
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    85
        if not chunk:
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    86
            break
6784
18c429ea3a0e hgweb: all protocol functions have become generators
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6782
diff changeset
    87
        yield z.compress(chunk)
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    88
6784
18c429ea3a0e hgweb: all protocol functions have become generators
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6782
diff changeset
    89
    yield z.flush()
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    90
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
    91
def changegroupsubset(repo, req):
5993
948a41e77902 hgweb: explicit response status
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5963
diff changeset
    92
    req.respond(HTTP_OK, HGTYPE)
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    93
    bases = []
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    94
    heads = []
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    95
5915
d0576d065993 Prefer i in d over d.has_key(i)
Christian Ebert <blacktrash@gmx.net>
parents: 5598
diff changeset
    96
    if 'bases' in req.form:
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    97
        bases = [bin(x) for x in req.form['bases'][0].split(' ')]
5915
d0576d065993 Prefer i in d over d.has_key(i)
Christian Ebert <blacktrash@gmx.net>
parents: 5598
diff changeset
    98
    if 'heads' in req.form:
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
    99
        heads = [bin(x) for x in req.form['heads'][0].split(' ')]
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   100
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   101
    z = zlib.compressobj()
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
   102
    f = repo.changegroupsubset(bases, heads, 'serve')
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   103
    while 1:
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   104
        chunk = f.read(4096)
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   105
        if not chunk:
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   106
            break
6784
18c429ea3a0e hgweb: all protocol functions have become generators
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6782
diff changeset
   107
        yield z.compress(chunk)
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   108
6784
18c429ea3a0e hgweb: all protocol functions have become generators
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6782
diff changeset
   109
    yield z.flush()
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   110
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
   111
def capabilities(repo, req):
8562
e3495c399006 named branches: server branchmap wire protocol support (issue736)
Henrik Stuart <henrik.stuart@edlund.dk>
parents: 8225
diff changeset
   112
    caps = ['lookup', 'changegroupsubset', 'branchmap']
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
   113
    if repo.ui.configbool('server', 'uncompressed', untrusted=True):
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
   114
        caps.append('stream=%d' % repo.changelog.version)
6780
4c1d67e0fa8c hgweb: move capabilities calculation back into hgweb.protocol
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6779
diff changeset
   115
    if changegroupmod.bundlepriority:
4c1d67e0fa8c hgweb: move capabilities calculation back into hgweb.protocol
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6779
diff changeset
   116
        caps.append('unbundle=%s' % ','.join(changegroupmod.bundlepriority))
4c1d67e0fa8c hgweb: move capabilities calculation back into hgweb.protocol
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6779
diff changeset
   117
    rsp = ' '.join(caps)
4c1d67e0fa8c hgweb: move capabilities calculation back into hgweb.protocol
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6779
diff changeset
   118
    req.respond(HTTP_OK, HGTYPE, length=len(rsp))
6784
18c429ea3a0e hgweb: all protocol functions have become generators
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6782
diff changeset
   119
    yield rsp
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   120
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
   121
def unbundle(repo, req):
6335
e29557d687c9 hgweb: only accept POST requests for unbundle
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6265
diff changeset
   122
6779
d3147b4e3e8a hgweb: centralize permission checks for protocol commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6630
diff changeset
   123
    proto = req.env.get('wsgi.url_scheme') or 'http'
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   124
    their_heads = req.form['heads'][0].split(' ')
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   125
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   126
    def check_heads():
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
   127
        heads = map(hex, repo.heads())
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   128
        return their_heads == [hex('force')] or their_heads == heads
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   129
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   130
    # fail early if possible
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   131
    if not check_heads():
7180
a42d27bc809d hgweb: be sure to drain request data even in early error conditions
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6926
diff changeset
   132
        req.drain()
6926
57b954d8d003 hgweb: raise ErrorResponses to communicate protocol errors
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6925
diff changeset
   133
        raise ErrorResponse(HTTP_OK, 'unsynced changes')
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   134
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   135
    # do not lock repo until all changegroup data is
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   136
    # streamed. save to temporary file.
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   137
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   138
    fd, tempname = tempfile.mkstemp(prefix='hg-unbundle-')
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   139
    fp = os.fdopen(fd, 'wb+')
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   140
    try:
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   141
        length = int(req.env['CONTENT_LENGTH'])
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   142
        for s in util.filechunkiter(req, limit=length):
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   143
            fp.write(s)
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   144
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   145
        try:
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
   146
            lock = repo.lock()
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   147
            try:
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   148
                if not check_heads():
6926
57b954d8d003 hgweb: raise ErrorResponses to communicate protocol errors
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6925
diff changeset
   149
                    raise ErrorResponse(HTTP_OK, 'unsynced changes')
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   150
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   151
                fp.seek(0)
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   152
                header = fp.read(6)
6154
ef1c5a3b653d improve changegroup.readbundle(), use it in hgweb
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6152
diff changeset
   153
                if header.startswith('HG') and not header.startswith('HG10'):
ef1c5a3b653d improve changegroup.readbundle(), use it in hgweb
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6152
diff changeset
   154
                    raise ValueError('unknown bundle version')
ef1c5a3b653d improve changegroup.readbundle(), use it in hgweb
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6152
diff changeset
   155
                elif header not in changegroupmod.bundletypes:
ef1c5a3b653d improve changegroup.readbundle(), use it in hgweb
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6152
diff changeset
   156
                    raise ValueError('unknown bundle compression type')
ef1c5a3b653d improve changegroup.readbundle(), use it in hgweb
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6152
diff changeset
   157
                gen = changegroupmod.unbundle(header, fp)
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   158
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   159
                # send addchangegroup output to client
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   160
6265
be76e54570f0 Issue937: error messages from hooks not sent over HTTP.
Jesse Glick <jesse.glick@sun.com>
parents: 6212
diff changeset
   161
                oldio = sys.stdout, sys.stderr
be76e54570f0 Issue937: error messages from hooks not sent over HTTP.
Jesse Glick <jesse.glick@sun.com>
parents: 6212
diff changeset
   162
                sys.stderr = sys.stdout = cStringIO.StringIO()
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   163
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   164
                try:
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   165
                    url = 'remote:%s:%s' % (proto,
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   166
                                            req.env.get('REMOTE_HOST', ''))
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   167
                    try:
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
   168
                        ret = repo.addchangegroup(gen, 'serve', url)
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   169
                    except util.Abort, inst:
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   170
                        sys.stdout.write("abort: %s\n" % inst)
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   171
                        ret = 0
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   172
                finally:
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   173
                    val = sys.stdout.getvalue()
6265
be76e54570f0 Issue937: error messages from hooks not sent over HTTP.
Jesse Glick <jesse.glick@sun.com>
parents: 6212
diff changeset
   174
                    sys.stdout, sys.stderr = oldio
6926
57b954d8d003 hgweb: raise ErrorResponses to communicate protocol errors
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6925
diff changeset
   175
                req.respond(HTTP_OK, HGTYPE)
6788
88a1bcc5c6a7 hgweb: use a single-element tuple to return from protocol.unbundle()
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6784
diff changeset
   176
                return '%d\n%s' % (ret, val),
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   177
            finally:
8109
496ae1ea4698 switch lock releasing in the core from gc to explicit
Ronny Pfannschmidt <Ronny.Pfannschmidt@gmx.de>
parents: 7281
diff changeset
   178
                lock.release()
6154
ef1c5a3b653d improve changegroup.readbundle(), use it in hgweb
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6152
diff changeset
   179
        except ValueError, inst:
6926
57b954d8d003 hgweb: raise ErrorResponses to communicate protocol errors
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6925
diff changeset
   180
            raise ErrorResponse(HTTP_OK, inst)
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   181
        except (OSError, IOError), inst:
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   182
            filename = getattr(inst, 'filename', '')
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   183
            # Don't send our filesystem layout to the client
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
   184
            if filename.startswith(repo.root):
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
   185
                filename = filename[len(repo.root)+1:]
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   186
            else:
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   187
                filename = ''
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   188
            error = getattr(inst, 'strerror', 'Unknown error')
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   189
            if inst.errno == errno.ENOENT:
5993
948a41e77902 hgweb: explicit response status
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5963
diff changeset
   190
                code = HTTP_NOT_FOUND
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   191
            else:
5993
948a41e77902 hgweb: explicit response status
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5963
diff changeset
   192
                code = HTTP_SERVER_ERROR
6926
57b954d8d003 hgweb: raise ErrorResponses to communicate protocol errors
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6925
diff changeset
   193
            raise ErrorResponse(code, '%s: %s' % (error, filename))
5598
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   194
    finally:
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   195
        fp.close()
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   196
        os.unlink(tempname)
d534ba1c4eb4 separate the wire protocol commands from the user interface commands
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents:
diff changeset
   197
6781
b4b7261164d5 hgweb: protocol functions take repo instead of web
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6780
diff changeset
   198
def stream_out(repo, req):
5993
948a41e77902 hgweb: explicit response status
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 5963
diff changeset
   199
    req.respond(HTTP_OK, HGTYPE)
6925
87abfefafe02 make streamclone.stream_out() a generator
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6795
diff changeset
   200
    try:
87abfefafe02 make streamclone.stream_out() a generator
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6795
diff changeset
   201
        for chunk in streamclone.stream_out(repo, untrusted=True):
87abfefafe02 make streamclone.stream_out() a generator
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6795
diff changeset
   202
            yield chunk
87abfefafe02 make streamclone.stream_out() a generator
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6795
diff changeset
   203
    except streamclone.StreamException, inst:
87abfefafe02 make streamclone.stream_out() a generator
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 6795
diff changeset
   204
        yield str(inst)