hgext/convert/common.py
author Manuel Jacob <me@manueljacob.de>
Mon, 11 Jul 2022 01:51:20 +0200
branchstable
changeset 49378 094a5fa3cf52
parent 49306 2e726c934fcd
child 50752 faccec1edc2c
permissions -rw-r--r--
procutil: make stream detection in make_line_buffered more correct and strict In make_line_buffered(), we don’t want to wrap the stream if we know that lines get flushed to the underlying raw stream already. Previously, the heuristic was too optimistic. It assumed that any stream which is not an instance of io.BufferedIOBase doesn’t need wrapping. However, there are buffered streams that aren’t instances of io.BufferedIOBase, like Mercurial’s own winstdout. The new logic is different in two ways: First, only for the check, if unwraps any combination of WriteAllWrapper and winstdout. Second, it skips wrapping the stream only if it is an instance of io.RawIOBase (or already wrapped). If it is an instance of io.BufferedIOBase, it gets wrapped. In any other case, the function raises an exception. This ensures that, if an unknown stream is passed or we add another wrapper in the future, we don’t wrap the stream if it’s already line buffered or not wrap the stream if it’s not line buffered. In fact, this was already helpful during development of this change. Without it, I possibly would have forgot that WriteAllWrapper needs to be ignored for the check, leading to unnecessary wrapping if stdout is unbuffered. The alternative would have been to always wrap unknown streams. However, I don’t think that anyone would benefit from being less strict. We can expect streams from the standard library to be subclassing either io.RawIOBase or io.BufferedIOBase, so running Mercurial in the standard way should not regress by this change. Py2exe might replace sys.stdout and sys.stderr, but that currently breaks Mercurial anyway and also these streams don’t claim to be interactive, so this function is not called for them.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
8250
1b60efdb8bc5 convert: add copyright and license headers to back-ends
Martin Geisler <mg@lazybytes.net>
parents: 8155
diff changeset
     1
# common.py - common code for the convert extension
1b60efdb8bc5 convert: add copyright and license headers to back-ends
Martin Geisler <mg@lazybytes.net>
parents: 8155
diff changeset
     2
#
46819
d4ba4d51f85f contributor: change mentions of mpm to olivia
Raphaël Gomès <rgomes@octobus.net>
parents: 45942
diff changeset
     3
#  Copyright 2005-2009 Olivia Mackall <olivia@selenic.com> and others
8250
1b60efdb8bc5 convert: add copyright and license headers to back-ends
Martin Geisler <mg@lazybytes.net>
parents: 8155
diff changeset
     4
#
1b60efdb8bc5 convert: add copyright and license headers to back-ends
Martin Geisler <mg@lazybytes.net>
parents: 8155
diff changeset
     5
# This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9542
diff changeset
     6
# GNU General Public License version 2 or any later version.
8250
1b60efdb8bc5 convert: add copyright and license headers to back-ends
Martin Geisler <mg@lazybytes.net>
parents: 8155
diff changeset
     7
28410
48b04018c897 convert: common use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
     8
import base64
48b04018c897 convert: common use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
     9
import datetime
48b04018c897 convert: common use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    10
import os
48870
df56e6bd37f6 py3: use pickle directly
Gregory Szorc <gregory.szorc@gmail.com>
parents: 46819
diff changeset
    11
import pickle
28410
48b04018c897 convert: common use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    12
import re
36557
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    13
import shlex
28410
48b04018c897 convert: common use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    14
import subprocess
48b04018c897 convert: common use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    15
29205
a0939666b836 py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents: 28900
diff changeset
    16
from mercurial.i18n import _
43085
eef9a2d67051 py3: manually import pycompat.open into files that need it
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43077
diff changeset
    17
from mercurial.pycompat import open
28410
48b04018c897 convert: common use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    18
from mercurial import (
34022
d5b2beca16c0 python3: wrap all uses of <exception>.strerror with strtolocal
Augie Fackler <raf@durin42.com>
parents: 30659
diff changeset
    19
    encoding,
28410
48b04018c897 convert: common use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    20
    error,
48b04018c897 convert: common use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    21
    phases,
36329
93943eef696f py3: use pycompat.byteskwargs in hgext/convert/
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36148
diff changeset
    22
    pycompat,
28410
48b04018c897 convert: common use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    23
    util,
48b04018c897 convert: common use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    24
)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
    25
from mercurial.utils import procutil
5127
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5121
diff changeset
    26
15606
2ad4e9b44d8b convert: simplify getargmax() with propertycache
Patrick Mezard <pmezard@gmail.com>
parents: 14271
diff changeset
    27
propertycache = util.propertycache
2ad4e9b44d8b convert: simplify getargmax() with propertycache
Patrick Mezard <pmezard@gmail.com>
parents: 14271
diff changeset
    28
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
    29
36557
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    30
def _encodeornone(d):
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    31
    if d is None:
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    32
        return
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    33
    return d.encode('latin1')
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    34
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
    35
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48941
diff changeset
    36
class _shlexpy3proxy:
36557
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    37
    def __init__(self, l):
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    38
        self._l = l
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    39
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    40
    def __iter__(self):
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    41
        return (_encodeornone(v) for v in self._l)
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    42
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    43
    def get_token(self):
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    44
        return _encodeornone(self._l.get_token())
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    45
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    46
    @property
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    47
    def infile(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    48
        return self._l.infile or b'<unknown>'
36557
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    49
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    50
    @property
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    51
    def lineno(self):
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    52
        return self._l.lineno
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    53
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
    54
36557
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    55
def shlexer(data=None, filepath=None, wordchars=None, whitespace=None):
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    56
    if data is None:
49288
ef5f5f1cbd90 py3: constant-fold some `pycompat.ispy3`
Manuel Jacob <me@manueljacob.de>
parents: 48946
diff changeset
    57
        data = open(filepath, b'r', encoding='latin1')
36557
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    58
    else:
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    59
        if filepath is not None:
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    60
            raise error.ProgrammingError(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    61
                b'shlexer only accepts data or filepath, not both'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
    62
            )
49288
ef5f5f1cbd90 py3: constant-fold some `pycompat.ispy3`
Manuel Jacob <me@manueljacob.de>
parents: 48946
diff changeset
    63
        data = data.decode('latin1')
36557
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    64
    l = shlex.shlex(data, infile=filepath, posix=True)
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    65
    if whitespace is not None:
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    66
        l.whitespace_split = True
49288
ef5f5f1cbd90 py3: constant-fold some `pycompat.ispy3`
Manuel Jacob <me@manueljacob.de>
parents: 48946
diff changeset
    67
        l.whitespace += whitespace.decode('latin1')
36557
d60430dc7853 convert: add some utility code for working with shlex on Python 3
Augie Fackler <augie@google.com>
parents: 36457
diff changeset
    68
    if wordchars is not None:
49288
ef5f5f1cbd90 py3: constant-fold some `pycompat.ispy3`
Manuel Jacob <me@manueljacob.de>
parents: 48946
diff changeset
    69
        l.wordchars += wordchars.decode('latin1')
ef5f5f1cbd90 py3: constant-fold some `pycompat.ispy3`
Manuel Jacob <me@manueljacob.de>
parents: 48946
diff changeset
    70
    return _shlexpy3proxy(l)
44967
de7bdb0e2a95 py3: suppress DeprecationWarning about deprecated base64 module aliases
Manuel Jacob <me@manueljacob.de>
parents: 44964
diff changeset
    71
de7bdb0e2a95 py3: suppress DeprecationWarning about deprecated base64 module aliases
Manuel Jacob <me@manueljacob.de>
parents: 44964
diff changeset
    72
5127
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5121
diff changeset
    73
def encodeargs(args):
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5121
diff changeset
    74
    def encodearg(s):
49288
ef5f5f1cbd90 py3: constant-fold some `pycompat.ispy3`
Manuel Jacob <me@manueljacob.de>
parents: 48946
diff changeset
    75
        lines = base64.encodebytes(s)
44964
d96d8bbbedb1 py3: fix bytes iteration
Manuel Jacob <me@manueljacob.de>
parents: 43506
diff changeset
    76
        lines = [l.splitlines()[0] for l in pycompat.iterbytestr(lines)]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    77
        return b''.join(lines)
5143
d4fa6bafc43a Remove trailing spaces, fix indentation
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5127
diff changeset
    78
5127
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5121
diff changeset
    79
    s = pickle.dumps(args)
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5121
diff changeset
    80
    return encodearg(s)
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5121
diff changeset
    81
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
    82
5127
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5121
diff changeset
    83
def decodeargs(s):
49288
ef5f5f1cbd90 py3: constant-fold some `pycompat.ispy3`
Manuel Jacob <me@manueljacob.de>
parents: 48946
diff changeset
    84
    s = base64.decodebytes(s)
5127
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5121
diff changeset
    85
    return pickle.loads(s)
4513
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
    86
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
    87
10282
08a0f04b56bd many, many trivial check-code fixups
Matt Mackall <mpm@selenic.com>
parents: 10263
diff changeset
    88
class MissingTool(Exception):
08a0f04b56bd many, many trivial check-code fixups
Matt Mackall <mpm@selenic.com>
parents: 10263
diff changeset
    89
    pass
6332
950e72fc7cf3 convert: allow missing tools not to stop source type detection
Patrick Mezard <pmezard@gmail.com>
parents: 6049
diff changeset
    90
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
    91
6332
950e72fc7cf3 convert: allow missing tools not to stop source type detection
Patrick Mezard <pmezard@gmail.com>
parents: 6049
diff changeset
    92
def checktool(exe, name=None, abort=True):
5497
f0a3918abd42 convert: fail if an external required tool is not found
Patrick Mezard <pmezard@gmail.com>
parents: 5441
diff changeset
    93
    name = name or exe
37120
a8a902d7176e procutil: bulk-replace function calls to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36607
diff changeset
    94
    if not procutil.findexe(exe):
24306
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 22411
diff changeset
    95
        if abort:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26035
diff changeset
    96
            exc = error.Abort
24306
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 22411
diff changeset
    97
        else:
6ddc86eedc3b style: kill ersatz if-else ternary operators
Jordi Gutiérrez Hermoso <jordigh@octave.org>
parents: 22411
diff changeset
    98
            exc = MissingTool
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    99
        raise exc(_(b'cannot find required "%s" tool') % name)
5497
f0a3918abd42 convert: fail if an external required tool is not found
Patrick Mezard <pmezard@gmail.com>
parents: 5441
diff changeset
   100
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   101
10282
08a0f04b56bd many, many trivial check-code fixups
Matt Mackall <mpm@selenic.com>
parents: 10263
diff changeset
   102
class NoRepo(Exception):
08a0f04b56bd many, many trivial check-code fixups
Matt Mackall <mpm@selenic.com>
parents: 10263
diff changeset
   103
    pass
3938
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3917
diff changeset
   104
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   105
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   106
SKIPREV = b'SKIP'
5374
e710874247d1 convert: allow the converter_source to say "skip this revision"
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5373
diff changeset
   107
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   108
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48941
diff changeset
   109
class commit:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   110
    def __init__(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   111
        self,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   112
        author,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   113
        date,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   114
        desc,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   115
        parents,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   116
        branch=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   117
        rev=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   118
        extra=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   119
        sortkey=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   120
        saverev=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   121
        phase=phases.draft,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   122
        optparents=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   123
        ctx=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   124
    ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   125
        self.author = author or b'unknown'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   126
        self.date = date or b'0 0'
5024
7963438881f5 convert: empty log messages are OK as of 7f5c3fb0a37d
Bryan O'Sullivan <bos@serpentine.com>
parents: 5012
diff changeset
   127
        self.desc = desc
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   128
        self.parents = parents  # will be converted and used as parents
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   129
        self.optparents = optparents or []  # will be used if already converted
5012
be25decfdb13 convert: make commit constructor clearer and less magical
Bryan O'Sullivan <bos@serpentine.com>
parents: 5011
diff changeset
   130
        self.branch = branch
be25decfdb13 convert: make commit constructor clearer and less magical
Bryan O'Sullivan <bos@serpentine.com>
parents: 5011
diff changeset
   131
        self.rev = rev
30659
1404146157d9 convert: don't use {} as default argument value
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30400
diff changeset
   132
        self.extra = extra or {}
8690
c5b4f662109f convert: add --sourcesort option for source specific sort
Patrick Mezard <pmezard@gmail.com>
parents: 8656
diff changeset
   133
        self.sortkey = sortkey
25570
7cc1d33f0ba6 convert: always track the hg source revision in the internal commit object
Matt Harbison <matt_harbison@yahoo.com>
parents: 24395
diff changeset
   134
        self.saverev = saverev
25571
1abfe639a70c convert: apply the appropriate phases to the destination (issue4165)
Matt Harbison <matt_harbison@yahoo.com>
parents: 25570
diff changeset
   135
        self.phase = phase
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   136
        self.ctx = ctx  # for hg to hg conversions
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   137
3954
9af4b853ed4d convert-repo: add CVS branch support
Matt Mackall <mpm@selenic.com>
parents: 3953
diff changeset
   138
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48941
diff changeset
   139
class converter_source:
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   140
    """Conversion source interface"""
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   141
35176
671aba341d90 convert: save an indicator of the repo type for sources and sinks
Matt Harbison <matt_harbison@yahoo.com>
parents: 34369
diff changeset
   142
    def __init__(self, ui, repotype, path=None, revs=None):
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   143
        """Initialize conversion source (or raise NoRepo("message")
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   144
        exception if path is not a valid repository)"""
4810
c2d529f288a1 convert: move some code into common init function
Brendan Cully <brendan@kublai.com>
parents: 4807
diff changeset
   145
        self.ui = ui
c2d529f288a1 convert: move some code into common init function
Brendan Cully <brendan@kublai.com>
parents: 4807
diff changeset
   146
        self.path = path
25748
baea47cafe75 convert: add support for specifying multiple revs
Durham Goode <durham@fb.com>
parents: 25660
diff changeset
   147
        self.revs = revs
35176
671aba341d90 convert: save an indicator of the repo type for sources and sinks
Matt Harbison <matt_harbison@yahoo.com>
parents: 34369
diff changeset
   148
        self.repotype = repotype
4810
c2d529f288a1 convert: move some code into common init function
Brendan Cully <brendan@kublai.com>
parents: 4807
diff changeset
   149
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   150
        self.encoding = b'utf-8'
4812
a5209b0487e0 convert: export revmap to source.
Brendan Cully <brendan@kublai.com>
parents: 4810
diff changeset
   151
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   152
    def checkhexformat(self, revstr, mapname=b'splicemap'):
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44967
diff changeset
   153
        """fails if revstr is not a 40 byte hex. mercurial and git both uses
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44967
diff changeset
   154
        such format for their revision numbering
19120
58e782f076e7 splicemap: improve error handling when source is hg (issue2084)
Ben Goswami <bengoswami@fb.com>
parents: 19119
diff changeset
   155
        """
37582
39ced3ef586d py3: add b'' prefix to make the regex bytes
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37463
diff changeset
   156
        if not re.match(br'[0-9a-fA-F]{40,40}$', revstr):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   157
            raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43105
diff changeset
   158
                _(b'%s entry %s is not a valid revision identifier')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   159
                % (mapname, revstr)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   160
            )
19120
58e782f076e7 splicemap: improve error handling when source is hg (issue2084)
Ben Goswami <bengoswami@fb.com>
parents: 19119
diff changeset
   161
5356
f0931c0240b4 convert: add before/after hooks for converter sources
Bryan O'Sullivan <bos@serpentine.com>
parents: 5287
diff changeset
   162
    def before(self):
f0931c0240b4 convert: add before/after hooks for converter sources
Bryan O'Sullivan <bos@serpentine.com>
parents: 5287
diff changeset
   163
        pass
f0931c0240b4 convert: add before/after hooks for converter sources
Bryan O'Sullivan <bos@serpentine.com>
parents: 5287
diff changeset
   164
f0931c0240b4 convert: add before/after hooks for converter sources
Bryan O'Sullivan <bos@serpentine.com>
parents: 5287
diff changeset
   165
    def after(self):
f0931c0240b4 convert: add before/after hooks for converter sources
Bryan O'Sullivan <bos@serpentine.com>
parents: 5287
diff changeset
   166
        pass
f0931c0240b4 convert: add before/after hooks for converter sources
Bryan O'Sullivan <bos@serpentine.com>
parents: 5287
diff changeset
   167
26035
86598f4fe1cf convert: add function to test if file is from source
Durham Goode <durham@fb.com>
parents: 25748
diff changeset
   168
    def targetfilebelongstosource(self, targetfilename):
86598f4fe1cf convert: add function to test if file is from source
Durham Goode <durham@fb.com>
parents: 25748
diff changeset
   169
        """Returns true if the given targetfile belongs to the source repo. This
86598f4fe1cf convert: add function to test if file is from source
Durham Goode <durham@fb.com>
parents: 25748
diff changeset
   170
        is useful when only a subdirectory of the target belongs to the source
86598f4fe1cf convert: add function to test if file is from source
Durham Goode <durham@fb.com>
parents: 25748
diff changeset
   171
        repo."""
86598f4fe1cf convert: add function to test if file is from source
Durham Goode <durham@fb.com>
parents: 25748
diff changeset
   172
        # For normal full repo converts, this is always True.
86598f4fe1cf convert: add function to test if file is from source
Durham Goode <durham@fb.com>
parents: 25748
diff changeset
   173
        return True
86598f4fe1cf convert: add function to test if file is from source
Durham Goode <durham@fb.com>
parents: 25748
diff changeset
   174
5510
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   175
    def setrevmap(self, revmap):
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   176
        """set the map of already-converted revisions"""
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   177
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   178
    def getheads(self):
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   179
        """Return a list of this repository's heads"""
16687
e34106fa0dc3 cleanup: "raise SomeException()" -> "raise SomeException"
Brodie Rao <brodie@sf.io>
parents: 16265
diff changeset
   180
        raise NotImplementedError
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   181
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   182
    def getfile(self, name, rev):
11134
33010ff1fd6f convert: merge sources getmode() into getfile()
Patrick Mezard <pmezard@gmail.com>
parents: 10282
diff changeset
   183
        """Return a pair (data, mode) where data is the file content
33010ff1fd6f convert: merge sources getmode() into getfile()
Patrick Mezard <pmezard@gmail.com>
parents: 10282
diff changeset
   184
        as a string and mode one of '', 'x' or 'l'. rev is the
22296
650b5b6e75ed convert: use None value for missing files instead of overloading IOError
Mads Kiilerich <madski@unity3d.com>
parents: 21635
diff changeset
   185
        identifier returned by a previous call to getchanges().
650b5b6e75ed convert: use None value for missing files instead of overloading IOError
Mads Kiilerich <madski@unity3d.com>
parents: 21635
diff changeset
   186
        Data is None if file is missing/deleted in rev.
7055
0f4564b4cc2c convert: improve convert_source documentation
Patrick Mezard <pmezard@gmail.com>
parents: 6956
diff changeset
   187
        """
16687
e34106fa0dc3 cleanup: "raise SomeException()" -> "raise SomeException"
Brodie Rao <brodie@sf.io>
parents: 16265
diff changeset
   188
        raise NotImplementedError
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   189
22300
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22296
diff changeset
   190
    def getchanges(self, version, full):
24395
216fa1ba9993 convert: optimize convert of files that are unmodified from p2 in merges
Mads Kiilerich <madski@unity3d.com>
parents: 24306
diff changeset
   191
        """Returns a tuple of (files, copies, cleanp2).
7055
0f4564b4cc2c convert: improve convert_source documentation
Patrick Mezard <pmezard@gmail.com>
parents: 6956
diff changeset
   192
0f4564b4cc2c convert: improve convert_source documentation
Patrick Mezard <pmezard@gmail.com>
parents: 6956
diff changeset
   193
        files is a sorted list of (filename, id) tuples for all files
8444
057e96fe2955 convert: improve docstrings, comments.
Greg Ward <greg-hg@gerg.ca>
parents: 8250
diff changeset
   194
        changed between version and its first parent returned by
22300
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22296
diff changeset
   195
        getcommit(). If full, all files in that revision is returned.
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22296
diff changeset
   196
        id is the source revision id of the file.
4516
96d8a56d4ef9 Removed trailing whitespace and tabs from python files
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4515
diff changeset
   197
5121
ef338e34a906 convert: look up copies in getchanges instead of getcommit
Brendan Cully <brendan@kublai.com>
parents: 5112
diff changeset
   198
        copies is a dictionary of dest: source
24395
216fa1ba9993 convert: optimize convert of files that are unmodified from p2 in merges
Mads Kiilerich <madski@unity3d.com>
parents: 24306
diff changeset
   199
216fa1ba9993 convert: optimize convert of files that are unmodified from p2 in merges
Mads Kiilerich <madski@unity3d.com>
parents: 24306
diff changeset
   200
        cleanp2 is the set of files filenames that are clean against p2.
216fa1ba9993 convert: optimize convert of files that are unmodified from p2 in merges
Mads Kiilerich <madski@unity3d.com>
parents: 24306
diff changeset
   201
        (Files that are clean against p1 are already not in files (unless
216fa1ba9993 convert: optimize convert of files that are unmodified from p2 in merges
Mads Kiilerich <madski@unity3d.com>
parents: 24306
diff changeset
   202
        full). This makes it possible to handle p2 clean files similarly.)
5121
ef338e34a906 convert: look up copies in getchanges instead of getcommit
Brendan Cully <brendan@kublai.com>
parents: 5112
diff changeset
   203
        """
16687
e34106fa0dc3 cleanup: "raise SomeException()" -> "raise SomeException"
Brodie Rao <brodie@sf.io>
parents: 16265
diff changeset
   204
        raise NotImplementedError
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   205
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   206
    def getcommit(self, version):
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   207
        """Return the commit object for version"""
16687
e34106fa0dc3 cleanup: "raise SomeException()" -> "raise SomeException"
Brodie Rao <brodie@sf.io>
parents: 16265
diff changeset
   208
        raise NotImplementedError
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   209
22411
c497e39d81a3 convert: add support for deterministic progress bar on scanning phase
Augie Fackler <raf@durin42.com>
parents: 22300
diff changeset
   210
    def numcommits(self):
c497e39d81a3 convert: add support for deterministic progress bar on scanning phase
Augie Fackler <raf@durin42.com>
parents: 22300
diff changeset
   211
        """Return the number of commits in this source.
c497e39d81a3 convert: add support for deterministic progress bar on scanning phase
Augie Fackler <raf@durin42.com>
parents: 22300
diff changeset
   212
c497e39d81a3 convert: add support for deterministic progress bar on scanning phase
Augie Fackler <raf@durin42.com>
parents: 22300
diff changeset
   213
        If unknown, return None.
c497e39d81a3 convert: add support for deterministic progress bar on scanning phase
Augie Fackler <raf@durin42.com>
parents: 22300
diff changeset
   214
        """
c497e39d81a3 convert: add support for deterministic progress bar on scanning phase
Augie Fackler <raf@durin42.com>
parents: 22300
diff changeset
   215
        return None
c497e39d81a3 convert: add support for deterministic progress bar on scanning phase
Augie Fackler <raf@durin42.com>
parents: 22300
diff changeset
   216
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   217
    def gettags(self):
8887
0332f8b44e54 convert/svn: test svn tags encoding
Patrick Mezard <pmezard@gmail.com>
parents: 8693
diff changeset
   218
        """Return the tags as a dictionary of name: revision
0332f8b44e54 convert/svn: test svn tags encoding
Patrick Mezard <pmezard@gmail.com>
parents: 8693
diff changeset
   219
0332f8b44e54 convert/svn: test svn tags encoding
Patrick Mezard <pmezard@gmail.com>
parents: 8693
diff changeset
   220
        Tag names must be UTF-8 strings.
0332f8b44e54 convert/svn: test svn tags encoding
Patrick Mezard <pmezard@gmail.com>
parents: 8693
diff changeset
   221
        """
16687
e34106fa0dc3 cleanup: "raise SomeException()" -> "raise SomeException"
Brodie Rao <brodie@sf.io>
parents: 16265
diff changeset
   222
        raise NotImplementedError
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   223
4759
20ec5cc02f18 convert: ove recode method into converter_source
Brendan Cully <brendan@kublai.com>
parents: 4590
diff changeset
   224
    def recode(self, s, encoding=None):
20ec5cc02f18 convert: ove recode method into converter_source
Brendan Cully <brendan@kublai.com>
parents: 4590
diff changeset
   225
        if not encoding:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   226
            encoding = self.encoding or b'utf-8'
4957
cdd33a048289 removed trailing whitespace
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4873
diff changeset
   227
48934
06de08b36c82 py3: use str instead of pycompat.unicode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48913
diff changeset
   228
        if isinstance(s, str):
5287
c6f932d3e0f6 Don't decode unicode strings.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5173
diff changeset
   229
            return s.encode("utf-8")
4759
20ec5cc02f18 convert: ove recode method into converter_source
Brendan Cully <brendan@kublai.com>
parents: 4590
diff changeset
   230
        try:
37622
bfdd20d22a86 py3: make sure decode() first argument is str
Pulkit Goyal <7895pulkit@gmail.com>
parents: 37582
diff changeset
   231
            return s.decode(pycompat.sysstr(encoding)).encode("utf-8")
16688
cfb6682961b8 cleanup: replace naked excepts with more specific ones
Brodie Rao <brodie@sf.io>
parents: 16687
diff changeset
   232
        except UnicodeError:
4759
20ec5cc02f18 convert: ove recode method into converter_source
Brendan Cully <brendan@kublai.com>
parents: 4590
diff changeset
   233
            try:
20ec5cc02f18 convert: ove recode method into converter_source
Brendan Cully <brendan@kublai.com>
parents: 4590
diff changeset
   234
                return s.decode("latin-1").encode("utf-8")
16688
cfb6682961b8 cleanup: replace naked excepts with more specific ones
Brodie Rao <brodie@sf.io>
parents: 16687
diff changeset
   235
            except UnicodeError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   236
                return s.decode(pycompat.sysstr(encoding), "replace").encode(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   237
                    "utf-8"
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   238
                )
4759
20ec5cc02f18 convert: ove recode method into converter_source
Brendan Cully <brendan@kublai.com>
parents: 4590
diff changeset
   239
5377
756a43a30e34 convert: readd --filemap
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
   240
    def getchangedfiles(self, rev, i):
756a43a30e34 convert: readd --filemap
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
   241
        """Return the files changed by rev compared to parent[i].
5760
0145f9afb0e7 Removed tabs and trailing whitespace in python files
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5556
diff changeset
   242
5377
756a43a30e34 convert: readd --filemap
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
   243
        i is an index selecting one of the parents of rev.  The return
756a43a30e34 convert: readd --filemap
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
   244
        value should be the list of files that are different in rev and
756a43a30e34 convert: readd --filemap
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
   245
        this parent.
756a43a30e34 convert: readd --filemap
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
   246
756a43a30e34 convert: readd --filemap
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
   247
        If rev has no parents, i is None.
5760
0145f9afb0e7 Removed tabs and trailing whitespace in python files
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5556
diff changeset
   248
5377
756a43a30e34 convert: readd --filemap
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
   249
        This function is only needed to support --filemap
756a43a30e34 convert: readd --filemap
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
   250
        """
16687
e34106fa0dc3 cleanup: "raise SomeException()" -> "raise SomeException"
Brodie Rao <brodie@sf.io>
parents: 16265
diff changeset
   251
        raise NotImplementedError
5377
756a43a30e34 convert: readd --filemap
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
   252
5554
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5529
diff changeset
   253
    def converted(self, rev, sinkrev):
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5529
diff changeset
   254
        '''Notify the source that a revision has been converted.'''
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5529
diff changeset
   255
8691
a0a541d6fed6 convert: fail fast if source does not support --sourcesort
Patrick Mezard <pmezard@gmail.com>
parents: 8690
diff changeset
   256
    def hasnativeorder(self):
a0a541d6fed6 convert: fail fast if source does not support --sourcesort
Patrick Mezard <pmezard@gmail.com>
parents: 8690
diff changeset
   257
        """Return true if this source has a meaningful, native revision
a0a541d6fed6 convert: fail fast if source does not support --sourcesort
Patrick Mezard <pmezard@gmail.com>
parents: 8690
diff changeset
   258
        order. For instance, Mercurial revisions are store sequentially
a0a541d6fed6 convert: fail fast if source does not support --sourcesort
Patrick Mezard <pmezard@gmail.com>
parents: 8690
diff changeset
   259
        while there is no such global ordering with Darcs.
a0a541d6fed6 convert: fail fast if source does not support --sourcesort
Patrick Mezard <pmezard@gmail.com>
parents: 8690
diff changeset
   260
        """
a0a541d6fed6 convert: fail fast if source does not support --sourcesort
Patrick Mezard <pmezard@gmail.com>
parents: 8690
diff changeset
   261
        return False
a0a541d6fed6 convert: fail fast if source does not support --sourcesort
Patrick Mezard <pmezard@gmail.com>
parents: 8690
diff changeset
   262
18819
05acdf8e1f23 convert: add closesort algorithm to mercurial sources
Constantine Linnick <theaspect@gmail.com>
parents: 17974
diff changeset
   263
    def hasnativeclose(self):
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44967
diff changeset
   264
        """Return true if this source has ability to close branch."""
18819
05acdf8e1f23 convert: add closesort algorithm to mercurial sources
Constantine Linnick <theaspect@gmail.com>
parents: 17974
diff changeset
   265
        return False
05acdf8e1f23 convert: add closesort algorithm to mercurial sources
Constantine Linnick <theaspect@gmail.com>
parents: 17974
diff changeset
   266
8693
68e0a55eee6e convert: rewrite tags when converting from hg to hg
Patrick Mezard <pmezard@gmail.com>
parents: 8691
diff changeset
   267
    def lookuprev(self, rev):
68e0a55eee6e convert: rewrite tags when converting from hg to hg
Patrick Mezard <pmezard@gmail.com>
parents: 8691
diff changeset
   268
        """If rev is a meaningful revision reference in source, return
68e0a55eee6e convert: rewrite tags when converting from hg to hg
Patrick Mezard <pmezard@gmail.com>
parents: 8691
diff changeset
   269
        the referenced identifier in the same format used by getcommit().
68e0a55eee6e convert: rewrite tags when converting from hg to hg
Patrick Mezard <pmezard@gmail.com>
parents: 8691
diff changeset
   270
        return None otherwise.
68e0a55eee6e convert: rewrite tags when converting from hg to hg
Patrick Mezard <pmezard@gmail.com>
parents: 8691
diff changeset
   271
        """
68e0a55eee6e convert: rewrite tags when converting from hg to hg
Patrick Mezard <pmezard@gmail.com>
parents: 8691
diff changeset
   272
        return None
5554
2147a734dcf9 convert: tell the source repository when a rev has been converted
Bryan O'Sullivan <bos@serpentine.com>
parents: 5529
diff changeset
   273
13744
7e525d2f9a75 convert: add bookmark support to common sink/source implementation
Edouard Gomez <ed.gomez@free.fr>
parents: 11134
diff changeset
   274
    def getbookmarks(self):
7e525d2f9a75 convert: add bookmark support to common sink/source implementation
Edouard Gomez <ed.gomez@free.fr>
parents: 11134
diff changeset
   275
        """Return the bookmarks as a dictionary of name: revision
7e525d2f9a75 convert: add bookmark support to common sink/source implementation
Edouard Gomez <ed.gomez@free.fr>
parents: 11134
diff changeset
   276
7e525d2f9a75 convert: add bookmark support to common sink/source implementation
Edouard Gomez <ed.gomez@free.fr>
parents: 11134
diff changeset
   277
        Bookmark names are to be UTF-8 strings.
7e525d2f9a75 convert: add bookmark support to common sink/source implementation
Edouard Gomez <ed.gomez@free.fr>
parents: 11134
diff changeset
   278
        """
7e525d2f9a75 convert: add bookmark support to common sink/source implementation
Edouard Gomez <ed.gomez@free.fr>
parents: 11134
diff changeset
   279
        return {}
7e525d2f9a75 convert: add bookmark support to common sink/source implementation
Edouard Gomez <ed.gomez@free.fr>
parents: 11134
diff changeset
   280
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   281
    def checkrevformat(self, revstr, mapname=b'splicemap'):
19120
58e782f076e7 splicemap: improve error handling when source is hg (issue2084)
Ben Goswami <bengoswami@fb.com>
parents: 19119
diff changeset
   282
        """revstr is a string that describes a revision in the given
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44967
diff changeset
   283
        source control system.  Return true if revstr has correct
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 44967
diff changeset
   284
        format.
19120
58e782f076e7 splicemap: improve error handling when source is hg (issue2084)
Ben Goswami <bengoswami@fb.com>
parents: 19119
diff changeset
   285
        """
58e782f076e7 splicemap: improve error handling when source is hg (issue2084)
Ben Goswami <bengoswami@fb.com>
parents: 19119
diff changeset
   286
        return True
58e782f076e7 splicemap: improve error handling when source is hg (issue2084)
Ben Goswami <bengoswami@fb.com>
parents: 19119
diff changeset
   287
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   288
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48941
diff changeset
   289
class converter_sink:
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   290
    """Conversion sink (target) interface"""
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   291
35176
671aba341d90 convert: save an indicator of the repo type for sources and sinks
Matt Harbison <matt_harbison@yahoo.com>
parents: 34369
diff changeset
   292
    def __init__(self, ui, repotype, path):
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   293
        """Initialize conversion sink (or raise NoRepo("message")
5441
71e7c86adcb7 convert: refactor sink initialisation, to remove hardcoding of hg
Bryan O'Sullivan <bos@serpentine.com>
parents: 5440
diff changeset
   294
        exception if path is not a valid repository)
71e7c86adcb7 convert: refactor sink initialisation, to remove hardcoding of hg
Bryan O'Sullivan <bos@serpentine.com>
parents: 5440
diff changeset
   295
71e7c86adcb7 convert: refactor sink initialisation, to remove hardcoding of hg
Bryan O'Sullivan <bos@serpentine.com>
parents: 5440
diff changeset
   296
        created is a list of paths to remove if a fatal error occurs
71e7c86adcb7 convert: refactor sink initialisation, to remove hardcoding of hg
Bryan O'Sullivan <bos@serpentine.com>
parents: 5440
diff changeset
   297
        later"""
71e7c86adcb7 convert: refactor sink initialisation, to remove hardcoding of hg
Bryan O'Sullivan <bos@serpentine.com>
parents: 5440
diff changeset
   298
        self.ui = ui
5440
b4ae8535f834 convert: add default constructor for converter_sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5439
diff changeset
   299
        self.path = path
5441
71e7c86adcb7 convert: refactor sink initialisation, to remove hardcoding of hg
Bryan O'Sullivan <bos@serpentine.com>
parents: 5440
diff changeset
   300
        self.created = []
35176
671aba341d90 convert: save an indicator of the repo type for sources and sinks
Matt Harbison <matt_harbison@yahoo.com>
parents: 34369
diff changeset
   301
        self.repotype = repotype
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   302
5011
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4957
diff changeset
   303
    def revmapfile(self):
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   304
        """Path to a file that will contain lines
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   305
        source_rev_id sink_rev_id
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   306
        mapping equivalent revision identifiers for each system."""
16687
e34106fa0dc3 cleanup: "raise SomeException()" -> "raise SomeException"
Brodie Rao <brodie@sf.io>
parents: 16265
diff changeset
   307
        raise NotImplementedError
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   308
4589
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4536
diff changeset
   309
    def authorfile(self):
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4536
diff changeset
   310
        """Path to a file that will contain lines
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4536
diff changeset
   311
        srcauthor=dstauthor
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4536
diff changeset
   312
        mapping equivalent authors identifiers for each system."""
4590
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
   313
        return None
4589
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4536
diff changeset
   314
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   315
    def putcommit(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   316
        self, files, copies, parents, commit, source, revmap, full, cleanp2
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   317
    ):
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   318
        """Create a revision with all changed files listed in 'files'
8693
68e0a55eee6e convert: rewrite tags when converting from hg to hg
Patrick Mezard <pmezard@gmail.com>
parents: 8691
diff changeset
   319
        and having listed parents. 'commit' is a commit object
68e0a55eee6e convert: rewrite tags when converting from hg to hg
Patrick Mezard <pmezard@gmail.com>
parents: 8691
diff changeset
   320
        containing at a minimum the author, date, and message for this
68e0a55eee6e convert: rewrite tags when converting from hg to hg
Patrick Mezard <pmezard@gmail.com>
parents: 8691
diff changeset
   321
        changeset.  'files' is a list of (path, version) tuples,
68e0a55eee6e convert: rewrite tags when converting from hg to hg
Patrick Mezard <pmezard@gmail.com>
parents: 8691
diff changeset
   322
        'copies' is a dictionary mapping destinations to sources,
68e0a55eee6e convert: rewrite tags when converting from hg to hg
Patrick Mezard <pmezard@gmail.com>
parents: 8691
diff changeset
   323
        'source' is the source repository, and 'revmap' is a mapfile
11134
33010ff1fd6f convert: merge sources getmode() into getfile()
Patrick Mezard <pmezard@gmail.com>
parents: 10282
diff changeset
   324
        of source revisions to converted revisions. Only getfile() and
22300
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22296
diff changeset
   325
        lookuprev() should be called on 'source'. 'full' means that 'files'
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22296
diff changeset
   326
        is complete and all other files should be removed.
24395
216fa1ba9993 convert: optimize convert of files that are unmodified from p2 in merges
Mads Kiilerich <madski@unity3d.com>
parents: 24306
diff changeset
   327
        'cleanp2' is a set of the filenames that are unchanged from p2
216fa1ba9993 convert: optimize convert of files that are unmodified from p2 in merges
Mads Kiilerich <madski@unity3d.com>
parents: 24306
diff changeset
   328
        (only in the common merge case where there two parents).
6716
c9b8d2565b92 convert: reintegrate file retrieval code in sinks
Patrick Mezard <pmezard@gmail.com>
parents: 6332
diff changeset
   329
c9b8d2565b92 convert: reintegrate file retrieval code in sinks
Patrick Mezard <pmezard@gmail.com>
parents: 6332
diff changeset
   330
        Note that the sink repository is not told to update itself to
c9b8d2565b92 convert: reintegrate file retrieval code in sinks
Patrick Mezard <pmezard@gmail.com>
parents: 6332
diff changeset
   331
        a particular revision (or even what that revision would be)
c9b8d2565b92 convert: reintegrate file retrieval code in sinks
Patrick Mezard <pmezard@gmail.com>
parents: 6332
diff changeset
   332
        before it receives the file data.
c9b8d2565b92 convert: reintegrate file retrieval code in sinks
Patrick Mezard <pmezard@gmail.com>
parents: 6332
diff changeset
   333
        """
16687
e34106fa0dc3 cleanup: "raise SomeException()" -> "raise SomeException"
Brodie Rao <brodie@sf.io>
parents: 16265
diff changeset
   334
        raise NotImplementedError
4447
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   335
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   336
    def puttags(self, tags):
1b75e0eff532 document conversion interface
Daniel Holth <dholth@fastmail.fm>
parents: 4114
diff changeset
   337
        """Put tags into sink.
8887
0332f8b44e54 convert/svn: test svn tags encoding
Patrick Mezard <pmezard@gmail.com>
parents: 8693
diff changeset
   338
0332f8b44e54 convert/svn: test svn tags encoding
Patrick Mezard <pmezard@gmail.com>
parents: 8693
diff changeset
   339
        tags: {tagname: sink_rev_id, ...} where tagname is an UTF-8 string.
9431
d1b135f2f415 convert: fix history topology when using hg.tagsbranch
Patrick Mezard <pmezard@gmail.com>
parents: 8970
diff changeset
   340
        Return a pair (tag_revision, tag_parent_revision), or (None, None)
d1b135f2f415 convert: fix history topology when using hg.tagsbranch
Patrick Mezard <pmezard@gmail.com>
parents: 8970
diff changeset
   341
        if nothing was changed.
8887
0332f8b44e54 convert/svn: test svn tags encoding
Patrick Mezard <pmezard@gmail.com>
parents: 8693
diff changeset
   342
        """
16687
e34106fa0dc3 cleanup: "raise SomeException()" -> "raise SomeException"
Brodie Rao <brodie@sf.io>
parents: 16265
diff changeset
   343
        raise NotImplementedError
5127
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5121
diff changeset
   344
5934
e495f3f35b2d convert: hg.clonebranches must pull missing parents (issue941)
Patrick Mezard <pmezard@gmail.com>
parents: 5441
diff changeset
   345
    def setbranch(self, branch, pbranches):
6716
c9b8d2565b92 convert: reintegrate file retrieval code in sinks
Patrick Mezard <pmezard@gmail.com>
parents: 6332
diff changeset
   346
        """Set the current branch name. Called before the first putcommit
5173
6b4c332f241b convert: hg: optionally create branches as clones
Brendan Cully <brendan@kublai.com>
parents: 5143
diff changeset
   347
        on the branch.
6b4c332f241b convert: hg: optionally create branches as clones
Brendan Cully <brendan@kublai.com>
parents: 5143
diff changeset
   348
        branch: branch name for subsequent commits
5934
e495f3f35b2d convert: hg.clonebranches must pull missing parents (issue941)
Patrick Mezard <pmezard@gmail.com>
parents: 5441
diff changeset
   349
        pbranches: (converted parent revision, parent branch) tuples"""
5378
8a2915f57dfc convert: add a mode where mercurial_sink skips empty revisions.
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5377
diff changeset
   350
8a2915f57dfc convert: add a mode where mercurial_sink skips empty revisions.
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5377
diff changeset
   351
    def setfilemapmode(self, active):
8a2915f57dfc convert: add a mode where mercurial_sink skips empty revisions.
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5377
diff changeset
   352
        """Tell the destination that we're using a filemap
8a2915f57dfc convert: add a mode where mercurial_sink skips empty revisions.
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5377
diff changeset
   353
8a2915f57dfc convert: add a mode where mercurial_sink skips empty revisions.
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5377
diff changeset
   354
        Some converter_sources (svn in particular) can claim that a file
8a2915f57dfc convert: add a mode where mercurial_sink skips empty revisions.
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5377
diff changeset
   355
        was changed in a revision, even if there was no change.  This method
8a2915f57dfc convert: add a mode where mercurial_sink skips empty revisions.
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5377
diff changeset
   356
        tells the destination that we're using a filemap and that it should
8a2915f57dfc convert: add a mode where mercurial_sink skips empty revisions.
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5377
diff changeset
   357
        filter empty revisions.
8a2915f57dfc convert: add a mode where mercurial_sink skips empty revisions.
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5377
diff changeset
   358
        """
5510
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   359
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   360
    def before(self):
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   361
        pass
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   362
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   363
    def after(self):
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   364
        pass
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   365
13744
7e525d2f9a75 convert: add bookmark support to common sink/source implementation
Edouard Gomez <ed.gomez@free.fr>
parents: 11134
diff changeset
   366
    def putbookmarks(self, bookmarks):
7e525d2f9a75 convert: add bookmark support to common sink/source implementation
Edouard Gomez <ed.gomez@free.fr>
parents: 11134
diff changeset
   367
        """Put bookmarks into sink.
7e525d2f9a75 convert: add bookmark support to common sink/source implementation
Edouard Gomez <ed.gomez@free.fr>
parents: 11134
diff changeset
   368
7e525d2f9a75 convert: add bookmark support to common sink/source implementation
Edouard Gomez <ed.gomez@free.fr>
parents: 11134
diff changeset
   369
        bookmarks: {bookmarkname: sink_rev_id, ...}
7e525d2f9a75 convert: add bookmark support to common sink/source implementation
Edouard Gomez <ed.gomez@free.fr>
parents: 11134
diff changeset
   370
        where bookmarkname is an UTF-8 string.
7e525d2f9a75 convert: add bookmark support to common sink/source implementation
Edouard Gomez <ed.gomez@free.fr>
parents: 11134
diff changeset
   371
        """
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   372
21635
5f2cc464e502 convert: introduce hascommitfrommap sink method
Mads Kiilerich <madski@unity3d.com>
parents: 21634
diff changeset
   373
    def hascommitfrommap(self, rev):
5f2cc464e502 convert: introduce hascommitfrommap sink method
Mads Kiilerich <madski@unity3d.com>
parents: 21634
diff changeset
   374
        """Return False if a rev mentioned in a filemap is known to not be
5f2cc464e502 convert: introduce hascommitfrommap sink method
Mads Kiilerich <madski@unity3d.com>
parents: 21634
diff changeset
   375
        present."""
5f2cc464e502 convert: introduce hascommitfrommap sink method
Mads Kiilerich <madski@unity3d.com>
parents: 21634
diff changeset
   376
        raise NotImplementedError
5f2cc464e502 convert: introduce hascommitfrommap sink method
Mads Kiilerich <madski@unity3d.com>
parents: 21634
diff changeset
   377
21634
23b24d6a70c8 convert: rename sink hascommit to hascommitforsplicemap
Mads Kiilerich <madski@unity3d.com>
parents: 21076
diff changeset
   378
    def hascommitforsplicemap(self, rev):
23b24d6a70c8 convert: rename sink hascommit to hascommitforsplicemap
Mads Kiilerich <madski@unity3d.com>
parents: 21076
diff changeset
   379
        """This method is for the special needs for splicemap handling and not
23b24d6a70c8 convert: rename sink hascommit to hascommitforsplicemap
Mads Kiilerich <madski@unity3d.com>
parents: 21076
diff changeset
   380
        for general use. Returns True if the sink contains rev, aborts on some
23b24d6a70c8 convert: rename sink hascommit to hascommitforsplicemap
Mads Kiilerich <madski@unity3d.com>
parents: 21076
diff changeset
   381
        special cases."""
16687
e34106fa0dc3 cleanup: "raise SomeException()" -> "raise SomeException"
Brodie Rao <brodie@sf.io>
parents: 16265
diff changeset
   382
        raise NotImplementedError
16106
d75aa756149b convert: use splicemap entries when sorting revisions (issue1748)
Patrick Mezard <patrick@mezard.eu>
parents: 16105
diff changeset
   383
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   384
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48941
diff changeset
   385
class commandline:
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   386
    def __init__(self, ui, command):
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   387
        self.ui = ui
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   388
        self.command = command
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   389
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   390
    def prerun(self):
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   391
        pass
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   392
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   393
    def postrun(self):
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   394
        pass
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   395
17413
97f1f22c2dba convert: use subprocess for all commandline calls
Patrick Mezard <patrick@mezard.eu>
parents: 17412
diff changeset
   396
    def _cmdline(self, cmd, *args, **kwargs):
36329
93943eef696f py3: use pycompat.byteskwargs in hgext/convert/
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36148
diff changeset
   397
        kwargs = pycompat.byteskwargs(kwargs)
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   398
        cmdline = [self.command, cmd] + list(args)
48913
f254fc73d956 global: bulk replace simple pycompat.iteritems(x) with x.items()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
   399
        for k, v in kwargs.items():
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   400
            if len(k) == 1:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   401
                cmdline.append(b'-' + k)
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   402
            else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   403
                cmdline.append(b'--' + k.replace(b'_', b'-'))
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   404
            try:
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   405
                if len(k) == 1:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   406
                    cmdline.append(b'' + v)
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   407
                else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   408
                    cmdline[-1] += b'=' + v
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   409
            except TypeError:
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   410
                pass
37120
a8a902d7176e procutil: bulk-replace function calls to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36607
diff changeset
   411
        cmdline = [procutil.shellquote(arg) for arg in cmdline]
7611
dafcc96c1285 convert: display child command output if --debug (fix 878466138b57)
Patrick Mezard <pmezard@gmail.com>
parents: 7610
diff changeset
   412
        if not self.ui.debugflag:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   413
            cmdline += [b'2>', pycompat.bytestr(os.devnull)]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   414
        cmdline = b' '.join(cmdline)
5832
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   415
        return cmdline
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   416
5832
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   417
    def _run(self, cmd, *args, **kwargs):
17413
97f1f22c2dba convert: use subprocess for all commandline calls
Patrick Mezard <patrick@mezard.eu>
parents: 17412
diff changeset
   418
        def popen(cmdline):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   419
            p = subprocess.Popen(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   420
                procutil.tonativestr(cmdline),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   421
                shell=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   422
                bufsize=-1,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   423
                close_fds=procutil.closefds,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   424
                stdout=subprocess.PIPE,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   425
            )
17413
97f1f22c2dba convert: use subprocess for all commandline calls
Patrick Mezard <patrick@mezard.eu>
parents: 17412
diff changeset
   426
            return p
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   427
17413
97f1f22c2dba convert: use subprocess for all commandline calls
Patrick Mezard <patrick@mezard.eu>
parents: 17412
diff changeset
   428
        return self._dorun(popen, cmd, *args, **kwargs)
13759
49b818fd26d8 convert: add support to common commandline to access stdin of the process
Daniel Atallah <daniel.atallah@gmail.com>
parents: 13744
diff changeset
   429
49b818fd26d8 convert: add support to common commandline to access stdin of the process
Daniel Atallah <daniel.atallah@gmail.com>
parents: 13744
diff changeset
   430
    def _run2(self, cmd, *args, **kwargs):
37120
a8a902d7176e procutil: bulk-replace function calls to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36607
diff changeset
   431
        return self._dorun(procutil.popen2, cmd, *args, **kwargs)
13759
49b818fd26d8 convert: add support to common commandline to access stdin of the process
Daniel Atallah <daniel.atallah@gmail.com>
parents: 13744
diff changeset
   432
28662
80cac1de6aea convert: rewrite gitpipe to use common.commandline (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26587
diff changeset
   433
    def _run3(self, cmd, *args, **kwargs):
37120
a8a902d7176e procutil: bulk-replace function calls to point to new module
Yuya Nishihara <yuya@tcha.org>
parents: 36607
diff changeset
   434
        return self._dorun(procutil.popen3, cmd, *args, **kwargs)
28662
80cac1de6aea convert: rewrite gitpipe to use common.commandline (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26587
diff changeset
   435
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   436
    def _dorun(self, openfunc, cmd, *args, **kwargs):
17413
97f1f22c2dba convert: use subprocess for all commandline calls
Patrick Mezard <patrick@mezard.eu>
parents: 17412
diff changeset
   437
        cmdline = self._cmdline(cmd, *args, **kwargs)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   438
        self.ui.debug(b'running: %s\n' % (cmdline,))
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   439
        self.prerun()
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   440
        try:
13759
49b818fd26d8 convert: add support to common commandline to access stdin of the process
Daniel Atallah <daniel.atallah@gmail.com>
parents: 13744
diff changeset
   441
            return openfunc(cmdline)
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   442
        finally:
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   443
            self.postrun()
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   444
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   445
    def run(self, cmd, *args, **kwargs):
17413
97f1f22c2dba convert: use subprocess for all commandline calls
Patrick Mezard <patrick@mezard.eu>
parents: 17412
diff changeset
   446
        p = self._run(cmd, *args, **kwargs)
97f1f22c2dba convert: use subprocess for all commandline calls
Patrick Mezard <patrick@mezard.eu>
parents: 17412
diff changeset
   447
        output = p.communicate()[0]
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   448
        self.ui.debug(output)
17413
97f1f22c2dba convert: use subprocess for all commandline calls
Patrick Mezard <patrick@mezard.eu>
parents: 17412
diff changeset
   449
        return output, p.returncode
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   450
6035
df659eb23360 convert: added GNU Arch source converter
Aleix Conchillo Flaque <aleix@member.fsf.org>
parents: 6001
diff changeset
   451
    def runlines(self, cmd, *args, **kwargs):
17413
97f1f22c2dba convert: use subprocess for all commandline calls
Patrick Mezard <patrick@mezard.eu>
parents: 17412
diff changeset
   452
        p = self._run(cmd, *args, **kwargs)
97f1f22c2dba convert: use subprocess for all commandline calls
Patrick Mezard <patrick@mezard.eu>
parents: 17412
diff changeset
   453
        output = p.stdout.readlines()
97f1f22c2dba convert: use subprocess for all commandline calls
Patrick Mezard <patrick@mezard.eu>
parents: 17412
diff changeset
   454
        p.wait()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   455
        self.ui.debug(b''.join(output))
17413
97f1f22c2dba convert: use subprocess for all commandline calls
Patrick Mezard <patrick@mezard.eu>
parents: 17412
diff changeset
   456
        return output, p.returncode
6035
df659eb23360 convert: added GNU Arch source converter
Aleix Conchillo Flaque <aleix@member.fsf.org>
parents: 6001
diff changeset
   457
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   458
    def checkexit(self, status, output=b''):
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   459
        if status:
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   460
            if output:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   461
                self.ui.warn(_(b'%s error:\n') % self.command)
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   462
                self.ui.warn(output)
37463
bbd240f81ac5 procutil: make explainexit() simply return a message (API)
Yuya Nishihara <yuya@tcha.org>
parents: 37120
diff changeset
   463
            msg = procutil.explainexit(status)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   464
            raise error.Abort(b'%s %s' % (self.command, msg))
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   465
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   466
    def run0(self, cmd, *args, **kwargs):
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   467
        output, status = self.run(cmd, *args, **kwargs)
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   468
        self.checkexit(status, output)
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   469
        return output
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   470
6035
df659eb23360 convert: added GNU Arch source converter
Aleix Conchillo Flaque <aleix@member.fsf.org>
parents: 6001
diff changeset
   471
    def runlines0(self, cmd, *args, **kwargs):
df659eb23360 convert: added GNU Arch source converter
Aleix Conchillo Flaque <aleix@member.fsf.org>
parents: 6001
diff changeset
   472
        output, status = self.runlines(cmd, *args, **kwargs)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   473
        self.checkexit(status, b''.join(output))
6035
df659eb23360 convert: added GNU Arch source converter
Aleix Conchillo Flaque <aleix@member.fsf.org>
parents: 6001
diff changeset
   474
        return output
df659eb23360 convert: added GNU Arch source converter
Aleix Conchillo Flaque <aleix@member.fsf.org>
parents: 6001
diff changeset
   475
15606
2ad4e9b44d8b convert: simplify getargmax() with propertycache
Patrick Mezard <pmezard@gmail.com>
parents: 14271
diff changeset
   476
    @propertycache
2ad4e9b44d8b convert: simplify getargmax() with propertycache
Patrick Mezard <pmezard@gmail.com>
parents: 14271
diff changeset
   477
    def argmax(self):
5832
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   478
        # POSIX requires at least 4096 bytes for ARG_MAX
15606
2ad4e9b44d8b convert: simplify getargmax() with propertycache
Patrick Mezard <pmezard@gmail.com>
parents: 14271
diff changeset
   479
        argmax = 4096
5832
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   480
        try:
43503
313e3a279828 cleanup: remove pointless r-prefixes on double-quoted strings
Augie Fackler <augie@google.com>
parents: 43117
diff changeset
   481
            argmax = os.sysconf("SC_ARG_MAX")
16688
cfb6682961b8 cleanup: replace naked excepts with more specific ones
Brodie Rao <brodie@sf.io>
parents: 16687
diff changeset
   482
        except (AttributeError, ValueError):
5832
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   483
            pass
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   484
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   485
        # Windows shells impose their own limits on command line length,
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   486
        # down to 2047 bytes for cmd.exe under Windows NT/2k and 2500 bytes
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   487
        # for older 4nt.exe. See http://support.microsoft.com/kb/830473 for
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   488
        # details about cmd.exe limitations.
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   489
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   490
        # Since ARG_MAX is for command line _and_ environment, lower our limit
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   491
        # (and make happy Windows shells while doing this).
15791
a814f8fcc65a Use explicit integer division
Martin Geisler <mg@aragost.com>
parents: 15611
diff changeset
   492
        return argmax // 2 - 1
5832
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   493
17412
e169b11fa1e0 convert: make commandline.limit_arglist private, simplify
Patrick Mezard <patrick@mezard.eu>
parents: 17391
diff changeset
   494
    def _limit_arglist(self, arglist, cmd, *args, **kwargs):
e169b11fa1e0 convert: make commandline.limit_arglist private, simplify
Patrick Mezard <patrick@mezard.eu>
parents: 17391
diff changeset
   495
        cmdlen = len(self._cmdline(cmd, *args, **kwargs))
15606
2ad4e9b44d8b convert: simplify getargmax() with propertycache
Patrick Mezard <pmezard@gmail.com>
parents: 14271
diff changeset
   496
        limit = self.argmax - cmdlen
36331
e218830f6f0a convert: don't use bytes as a variable name
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36329
diff changeset
   497
        numbytes = 0
5832
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   498
        fl = []
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   499
        for fn in arglist:
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   500
            b = len(fn) + 3
36331
e218830f6f0a convert: don't use bytes as a variable name
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36329
diff changeset
   501
            if numbytes + b < limit or len(fl) == 0:
5832
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   502
                fl.append(fn)
36331
e218830f6f0a convert: don't use bytes as a variable name
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36329
diff changeset
   503
                numbytes += b
5832
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   504
            else:
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   505
                yield fl
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   506
                fl = [fn]
36331
e218830f6f0a convert: don't use bytes as a variable name
Pulkit Goyal <7895pulkit@gmail.com>
parents: 36329
diff changeset
   507
                numbytes = b
5832
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   508
        if fl:
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   509
            yield fl
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   510
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   511
    def xargs(self, arglist, cmd, *args, **kwargs):
17412
e169b11fa1e0 convert: make commandline.limit_arglist private, simplify
Patrick Mezard <patrick@mezard.eu>
parents: 17391
diff changeset
   512
        for l in self._limit_arglist(arglist, cmd, *args, **kwargs):
5832
2192ed187319 convert: add commandline.xargs(), use it in svn_sink class
Maxim Dounin <mdounin@mdounin.ru>
parents: 5760
diff changeset
   513
            self.run0(cmd, *(list(args) + l), **kwargs)
5510
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   514
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   515
5510
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   516
class mapfile(dict):
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   517
    def __init__(self, ui, path):
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   518
        super(mapfile, self).__init__()
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   519
        self.ui = ui
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   520
        self.path = path
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   521
        self.fp = None
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   522
        self.order = []
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   523
        self._read()
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   524
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   525
    def _read(self):
7774
3b8f2750efcf Handle when the slicemap option is an empty string
Stefan Rusek <stefan@rusek.org>
parents: 7611
diff changeset
   526
        if not self.path:
5996
3f9ce63da18c convert: allow synthetic history to be spliced in.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5959
diff changeset
   527
            return
5510
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   528
        try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   529
            fp = open(self.path, b'rb')
49306
2e726c934fcd py3: catch FileNotFoundError instead of checking errno == ENOENT
Manuel Jacob <me@manueljacob.de>
parents: 49288
diff changeset
   530
        except FileNotFoundError:
5510
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   531
            return
48941
fd5b8e696b75 py3: stop using util.iterfile()
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48934
diff changeset
   532
        for i, line in enumerate(fp):
16190
9479c28a22bf convert: ignore blank lines in mapfiles (issue3286)
Patrick Mezard <patrick@mezard.eu>
parents: 16118
diff changeset
   533
            line = line.splitlines()[0].rstrip()
9479c28a22bf convert: ignore blank lines in mapfiles (issue3286)
Patrick Mezard <patrick@mezard.eu>
parents: 16118
diff changeset
   534
            if not line:
9479c28a22bf convert: ignore blank lines in mapfiles (issue3286)
Patrick Mezard <patrick@mezard.eu>
parents: 16118
diff changeset
   535
                # Ignore blank lines
9479c28a22bf convert: ignore blank lines in mapfiles (issue3286)
Patrick Mezard <patrick@mezard.eu>
parents: 16118
diff changeset
   536
                continue
8047
04c62275cbc7 convert: better mapfile parsing errors (issue1581/1)
Patrick Mezard <pmezard@gmail.com>
parents: 7774
diff changeset
   537
            try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   538
                key, value = line.rsplit(b' ', 1)
8047
04c62275cbc7 convert: better mapfile parsing errors (issue1581/1)
Patrick Mezard <pmezard@gmail.com>
parents: 7774
diff changeset
   539
            except ValueError:
26587
56b2bcea2529 error: get Abort from 'error' instead of 'util'
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 26035
diff changeset
   540
                raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   541
                    _(b'syntax error in %s(%d): key/value pair expected')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   542
                    % (self.path, i + 1)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   543
                )
5510
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   544
            if key not in self:
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   545
                self.order.append(key)
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   546
            super(mapfile, self).__setitem__(key, value)
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   547
        fp.close()
5760
0145f9afb0e7 Removed tabs and trailing whitespace in python files
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5556
diff changeset
   548
5510
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   549
    def __setitem__(self, key, value):
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   550
        if self.fp is None:
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   551
            try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   552
                self.fp = open(self.path, b'ab')
25660
328739ea70c3 global: mass rewrite to use modern exception syntax
Gregory Szorc <gregory.szorc@gmail.com>
parents: 25571
diff changeset
   553
            except IOError as err:
34022
d5b2beca16c0 python3: wrap all uses of <exception>.strerror with strtolocal
Augie Fackler <raf@durin42.com>
parents: 30659
diff changeset
   554
                raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   555
                    _(b'could not open map file %r: %s')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   556
                    % (self.path, encoding.strtolocal(err.strerror))
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   557
                )
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   558
        self.fp.write(util.tonativeeol(b'%s %s\n' % (key, value)))
5510
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   559
        self.fp.flush()
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   560
        super(mapfile, self).__setitem__(key, value)
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   561
11d7908a3ea8 convert: abstract map files into a class
Bryan O'Sullivan <bos@serpentine.com>
parents: 5497
diff changeset
   562
    def close(self):
5512
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   563
        if self.fp:
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   564
            self.fp.close()
8cd26ccc68f8 convert: abstract darcs's commandline handling
Bryan O'Sullivan <bos@serpentine.com>
parents: 5510
diff changeset
   565
            self.fp = None
16105
ebaa0aa749e2 convert: turn splicemap into a simple dictionary
Patrick Mezard <patrick@mezard.eu>
parents: 15791
diff changeset
   566
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   567
17974
337d728e644f convert: add config option to use the local time zone
Julian Cowley <julian@lava.net>
parents: 17413
diff changeset
   568
def makedatetimestamp(t):
36607
c6061cadb400 util: extract all date-related utils in utils/dateutil module
Boris Feld <boris.feld@octobus.net>
parents: 36557
diff changeset
   569
    """Like dateutil.makedate() but for time t instead of current time"""
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   570
    delta = datetime.datetime.utcfromtimestamp(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   571
        t
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42620
diff changeset
   572
    ) - datetime.datetime.fromtimestamp(t)
17974
337d728e644f convert: add config option to use the local time zone
Julian Cowley <julian@lava.net>
parents: 17413
diff changeset
   573
    tz = delta.days * 86400 + delta.seconds
337d728e644f convert: add config option to use the local time zone
Julian Cowley <julian@lava.net>
parents: 17413
diff changeset
   574
    return t, tz