mercurial/httpconnection.py
author Siddharth Agarwal <sid0@fb.com>
Fri, 20 Nov 2015 16:43:25 -0800
changeset 27079 a88a10a933b2
parent 26347 e9a35411bbbc
child 27521 b1adf32b0605
permissions -rw-r--r--
mergestate: add a method to compute actions to perform on dirstate We're going to use this to extend the action lists in merge.applyupdates. The somewhat funky return value is to make passing this dict directly into recordactions easier. We're going to exploit that in an upcoming patch.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     1
# httpconnection.py - urllib2 handler for new http support
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     2
#
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     3
# Copyright 2005, 2006, 2007, 2008 Matt Mackall <mpm@selenic.com>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     4
# Copyright 2006, 2007 Alexis S. L. Carvalho <alexis@cecm.usp.br>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     5
# Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     6
# Copyright 2011 Google, Inc.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     7
#
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     8
# This software may be used and distributed according to the terms of the
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
     9
# GNU General Public License version 2 or any later version.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    10
import logging
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    11
import socket
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    12
import urllib
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    13
import urllib2
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    14
import os
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    15
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    16
from mercurial import httpclient
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    17
from mercurial import sslutil
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    18
from mercurial import util
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    19
from mercurial.i18n import _
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    20
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    21
# moved here from url.py to avoid a cycle
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    22
class httpsendfile(object):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    23
    """This is a wrapper around the objects returned by python's "open".
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    24
15152
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
    25
    Its purpose is to send file-like objects via HTTP.
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
    26
    It do however not define a __len__ attribute because the length
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
    27
    might be more than Py_ssize_t can handle.
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    28
    """
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    29
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    30
    def __init__(self, ui, *args, **kwargs):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    31
        # We can't just "self._data = open(*args, **kwargs)" here because there
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    32
        # is an "open" function defined in this module that shadows the global
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    33
        # one
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    34
        self.ui = ui
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    35
        self._data = open(*args, **kwargs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    36
        self.seek = self._data.seek
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    37
        self.close = self._data.close
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    38
        self.write = self._data.write
15152
94b200a11cf7 http: handle push of bundles > 2 GB again (issue3017)
Mads Kiilerich <mads@kiilerich.com>
parents: 15025
diff changeset
    39
        self.length = os.fstat(self._data.fileno()).st_size
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    40
        self._pos = 0
15791
a814f8fcc65a Use explicit integer division
Martin Geisler <mg@aragost.com>
parents: 15288
diff changeset
    41
        self._total = self.length // 1024 * 2
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    42
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    43
    def read(self, *args, **kwargs):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    44
        try:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    45
            ret = self._data.read(*args, **kwargs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    46
        except EOFError:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    47
            self.ui.progress(_('sending'), None)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    48
        self._pos += len(ret)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    49
        # We pass double the max for total because we currently have
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    50
        # to send the bundle twice in the case of a server that
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    51
        # requires authentication. Since we can't know until we try
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    52
        # once whether authentication will be required, just lie to
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    53
        # the user and maybe the push succeeds suddenly at 50%.
15791
a814f8fcc65a Use explicit integer division
Martin Geisler <mg@aragost.com>
parents: 15288
diff changeset
    54
        self.ui.progress(_('sending'), self._pos // 1024,
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    55
                         unit=_('kb'), total=self._total)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    56
        return ret
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    57
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    58
# moved here from url.py to avoid a cycle
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
    59
def readauthforuri(ui, uri, user):
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    60
    # Read configuration
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    61
    config = dict()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    62
    for key, val in ui.configitems('auth'):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    63
        if '.' not in key:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    64
            ui.warn(_("ignoring invalid [auth] key '%s'\n") % key)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    65
            continue
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    66
        group, setting = key.rsplit('.', 1)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    67
        gdict = config.setdefault(group, dict())
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    68
        if setting in ('username', 'cert', 'key'):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    69
            val = util.expandpath(val)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    70
        gdict[setting] = val
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    71
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    72
    # Find the best match
25206
18a032704f0a httpconnection: drop Python 2.4 specify hack
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 19809
diff changeset
    73
    scheme, hostpath = uri.split('://', 1)
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    74
    bestuser = None
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    75
    bestlen = 0
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    76
    bestauth = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    77
    for group, auth in config.iteritems():
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    78
        if user and user != auth.get('username', user):
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    79
            # If a username was set in the URI, the entry username
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    80
            # must either match it or be unset
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    81
            continue
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    82
        prefix = auth.get('prefix')
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    83
        if not prefix:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    84
            continue
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    85
        p = prefix.split('://', 1)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    86
        if len(p) > 1:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    87
            schemes, prefix = [p[0]], p[1]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    88
        else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    89
            schemes = (auth.get('schemes') or 'https').split()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    90
        if (prefix == '*' or hostpath.startswith(prefix)) and \
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    91
            (len(prefix) > bestlen or (len(prefix) == bestlen and \
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    92
                not bestuser and 'username' in auth)) \
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    93
             and scheme in schemes:
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    94
            bestlen = len(prefix)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    95
            bestauth = group, auth
15005
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    96
            bestuser = auth.get('username')
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    97
            if user and not bestuser:
4a43e23b8c55 hgweb: do not ignore [auth] if url has a username (issue2822)
Patrick Mezard <pmezard@gmail.com>
parents: 14430
diff changeset
    98
                auth['username'] = user
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
    99
    return bestauth
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   100
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   101
# Mercurial (at least until we can remove the old codepath) requires
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   102
# that the http response object be sufficiently file-like, so we
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   103
# provide a close() method here.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   104
class HTTPResponse(httpclient.HTTPResponse):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   105
    def close(self):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   106
        pass
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   107
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   108
class HTTPConnection(httpclient.HTTPConnection):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   109
    response_class = HTTPResponse
26347
e9a35411bbbc httpconnection: remove a mutable default argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25837
diff changeset
   110
    def request(self, method, uri, body=None, headers=None):
e9a35411bbbc httpconnection: remove a mutable default argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25837
diff changeset
   111
        if headers is None:
e9a35411bbbc httpconnection: remove a mutable default argument
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25837
diff changeset
   112
            headers = {}
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   113
        if isinstance(body, httpsendfile):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   114
            body.seek(0)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   115
        httpclient.HTTPConnection.request(self, method, uri, body=body,
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   116
                                          headers=headers)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   117
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   118
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   119
_configuredlogging = False
14375
436e5379d7ba httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents: 14346
diff changeset
   120
LOGFMT = '%(levelname)s:%(name)s:%(lineno)d:%(message)s'
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   121
# Subclass BOTH of these because otherwise urllib2 "helpfully"
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   122
# reinserts them since it notices we don't include any subclasses of
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   123
# them.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   124
class http2handler(urllib2.HTTPHandler, urllib2.HTTPSHandler):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   125
    def __init__(self, ui, pwmgr):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   126
        global _configuredlogging
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   127
        urllib2.AbstractHTTPHandler.__init__(self)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   128
        self.ui = ui
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   129
        self.pwmgr = pwmgr
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   130
        self._connections = {}
25837
d343806dcf68 http2: mark experimental and developer options
Matt Mackall <mpm@selenic.com>
parents: 25660
diff changeset
   131
        # developer config: ui.http2debuglevel
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   132
        loglevel = ui.config('ui', 'http2debuglevel', default=None)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   133
        if loglevel and not _configuredlogging:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   134
            _configuredlogging = True
14294
84256ba2fbf7 httpconnection: fix debug logging option for httpclient
Augie Fackler <durin42@gmail.com>
parents: 14244
diff changeset
   135
            logger = logging.getLogger('mercurial.httpclient')
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   136
            logger.setLevel(getattr(logging, loglevel.upper()))
14375
436e5379d7ba httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents: 14346
diff changeset
   137
            handler = logging.StreamHandler()
436e5379d7ba httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents: 14346
diff changeset
   138
            handler.setFormatter(logging.Formatter(LOGFMT))
436e5379d7ba httpconnection: improved logging formatting
Augie Fackler <durin42@gmail.com>
parents: 14346
diff changeset
   139
            logger.addHandler(handler)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   140
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   141
    def close_all(self):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   142
        """Close and remove all connection objects being kept for reuse."""
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   143
        for openconns in self._connections.values():
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   144
            for conn in openconns:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   145
                conn.close()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   146
        self._connections = {}
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   147
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   148
    # shamelessly borrowed from urllib2.AbstractHTTPHandler
14346
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
   149
    def do_open(self, http_class, req, use_ssl):
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   150
        """Return an addinfourl object for the request, using http_class.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   151
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   152
        http_class must implement the HTTPConnection API from httplib.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   153
        The addinfourl return value is a file-like object.  It also
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   154
        has methods and attributes including:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   155
            - info(): return a mimetools.Message object for the headers
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   156
            - geturl(): return the original request URL
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   157
            - code: HTTP status code
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   158
        """
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   159
        # If using a proxy, the host returned by get_host() is
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   160
        # actually the proxy. On Python 2.6.1, the real destination
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   161
        # hostname is encoded in the URI in the urllib2 request
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   162
        # object. On Python 2.6.5, it's stored in the _tunnel_host
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   163
        # attribute which has no accessor.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   164
        tunhost = getattr(req, '_tunnel_host', None)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   165
        host = req.get_host()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   166
        if tunhost:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   167
            proxyhost = host
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   168
            host = tunhost
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   169
        elif req.has_proxy():
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   170
            proxyhost = req.get_host()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   171
            host = req.get_selector().split('://', 1)[1].split('/', 1)[0]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   172
        else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   173
            proxyhost = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   174
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   175
        if proxyhost:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   176
            if ':' in proxyhost:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   177
                # Note: this means we'll explode if we try and use an
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   178
                # IPv6 http proxy. This isn't a regression, so we
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   179
                # won't worry about it for now.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   180
                proxyhost, proxyport = proxyhost.rsplit(':', 1)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   181
            else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   182
                proxyport = 3128 # squid default
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   183
            proxy = (proxyhost, proxyport)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   184
        else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   185
            proxy = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   186
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   187
        if not host:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   188
            raise urllib2.URLError('no host given')
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   189
14346
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
   190
        connkey = use_ssl, host, proxy
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
   191
        allconns = self._connections.get(connkey, [])
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   192
        conns = [c for c in allconns if not c.busy()]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   193
        if conns:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   194
            h = conns[0]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   195
        else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   196
            if allconns:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   197
                self.ui.debug('all connections for %s busy, making a new '
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   198
                              'one\n' % host)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   199
            timeout = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   200
            if req.timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   201
                timeout = req.timeout
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   202
            h = http_class(host, timeout=timeout, proxy_hostport=proxy)
14346
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
   203
            self._connections.setdefault(connkey, []).append(h)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   204
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   205
        headers = dict(req.headers)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   206
        headers.update(req.unredirected_hdrs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   207
        headers = dict(
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   208
            (name.title(), val) for name, val in headers.items())
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   209
        try:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   210
            path = req.get_selector()
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   211
            if '://' in path:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   212
                path = path.split('://', 1)[1].split('/', 1)[1]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   213
            if path[0] != '/':
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   214
                path = '/' + path
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   215
            h.request(req.get_method(), path, req.data, headers)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   216
            r = h.getresponse()
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25429
diff changeset
   217
        except socket.error as err: # XXX what error?
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   218
            raise urllib2.URLError(err)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   219
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   220
        # Pick apart the HTTPResponse object to get the addinfourl
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   221
        # object initialized properly.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   222
        r.recv = r.read
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   223
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   224
        resp = urllib.addinfourl(r, r.headers, req.get_full_url())
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   225
        resp.code = r.status
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   226
        resp.msg = r.reason
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   227
        return resp
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   228
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   229
    # httplib always uses the given host/port as the socket connect
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   230
    # target, and then allows full URIs in the request path, which it
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   231
    # then observes and treats as a signal to do proxying instead.
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   232
    def http_open(self, req):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   233
        if req.get_full_url().startswith('https'):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   234
            return self.https_open(req)
17787
21503aa02d4f http2: make it possible to connect w/o ssl on port 443
Augie Fackler <raf@durin42.com>
parents: 17428
diff changeset
   235
        def makehttpcon(*args, **kwargs):
17836
98347af64593 httpclient: fix calling convention violation
Matt Mackall <mpm@selenic.com>
parents: 17787
diff changeset
   236
            k2 = dict(kwargs)
98347af64593 httpclient: fix calling convention violation
Matt Mackall <mpm@selenic.com>
parents: 17787
diff changeset
   237
            k2['use_ssl'] = False
98347af64593 httpclient: fix calling convention violation
Matt Mackall <mpm@selenic.com>
parents: 17787
diff changeset
   238
            return HTTPConnection(*args, **k2)
17787
21503aa02d4f http2: make it possible to connect w/o ssl on port 443
Augie Fackler <raf@durin42.com>
parents: 17428
diff changeset
   239
        return self.do_open(makehttpcon, req, False)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   240
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   241
    def https_open(self, req):
15025
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
   242
        # req.get_full_url() does not contain credentials and we may
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
   243
        # need them to match the certificates.
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
   244
        url = req.get_full_url()
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
   245
        user, password = self.pwmgr.find_stored_password(url)
0593e8f81c71 http: pass user to readauthforuri() (fix 4a43e23b8c55)
Patrick Mezard <pmezard@gmail.com>
parents: 15005
diff changeset
   246
        res = readauthforuri(self.ui, url, user)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   247
        if res:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   248
            group, auth = res
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   249
            self.auth = auth
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   250
            self.ui.debug("using auth.%s.* for authentication\n" % group)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   251
        else:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   252
            self.auth = None
14346
bf85c2639700 httpconnection: correctly handle redirects from http to https
Augie Fackler <durin42@gmail.com>
parents: 14294
diff changeset
   253
        return self.do_open(self._makesslconnection, req, True)
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   254
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   255
    def _makesslconnection(self, host, port=443, *args, **kwargs):
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   256
        keyfile = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   257
        certfile = None
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   258
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   259
        if args: # key_file
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   260
            keyfile = args.pop(0)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   261
        if args: # cert_file
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   262
            certfile = args.pop(0)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   263
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   264
        # if the user has specified different key/cert files in
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   265
        # hgrc, we prefer these
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   266
        if self.auth and 'key' in self.auth and 'cert' in self.auth:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   267
            keyfile = self.auth['key']
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   268
            certfile = self.auth['cert']
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   269
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   270
        # let host port take precedence
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   271
        if ':' in host and '[' not in host or ']:' in host:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   272
            host, port = host.rsplit(':', 1)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   273
            port = int(port)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   274
            if '[' in host:
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   275
                host = host[1:-1]
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   276
19809
50d721553198 httpconnection: properly inject ssl_wrap_socket into httpclient (issue4038)
Augie Fackler <raf@durin42.com>
parents: 17836
diff changeset
   277
        kwargs['keyfile'] = keyfile
50d721553198 httpconnection: properly inject ssl_wrap_socket into httpclient (issue4038)
Augie Fackler <raf@durin42.com>
parents: 17836
diff changeset
   278
        kwargs['certfile'] = certfile
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   279
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   280
        kwargs.update(sslutil.sslkwargs(self.ui, host))
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   281
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   282
        con = HTTPConnection(host, port, use_ssl=True,
25429
9d1c61715939 ssl: rename ssl_wrap_socket() to conform to our naming convention
Yuya Nishihara <yuya@tcha.org>
parents: 25206
diff changeset
   283
                             ssl_wrap_socket=sslutil.wrapsocket,
14244
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   284
                             ssl_validator=sslutil.validator(self.ui, host),
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   285
                             **kwargs)
e7525a555a64 url: use new http support if requested by the user
Augie Fackler <durin42@gmail.com>
parents:
diff changeset
   286
        return con