hgext/convert/git.py
author Manuel Jacob <me@manueljacob.de>
Mon, 11 Jul 2022 01:51:20 +0200
branchstable
changeset 49378 094a5fa3cf52
parent 48946 642e31cb55f0
permissions -rw-r--r--
procutil: make stream detection in make_line_buffered more correct and strict In make_line_buffered(), we don’t want to wrap the stream if we know that lines get flushed to the underlying raw stream already. Previously, the heuristic was too optimistic. It assumed that any stream which is not an instance of io.BufferedIOBase doesn’t need wrapping. However, there are buffered streams that aren’t instances of io.BufferedIOBase, like Mercurial’s own winstdout. The new logic is different in two ways: First, only for the check, if unwraps any combination of WriteAllWrapper and winstdout. Second, it skips wrapping the stream only if it is an instance of io.RawIOBase (or already wrapped). If it is an instance of io.BufferedIOBase, it gets wrapped. In any other case, the function raises an exception. This ensures that, if an unknown stream is passed or we add another wrapper in the future, we don’t wrap the stream if it’s already line buffered or not wrap the stream if it’s not line buffered. In fact, this was already helpful during development of this change. Without it, I possibly would have forgot that WriteAllWrapper needs to be ignored for the check, leading to unnecessary wrapping if stdout is unbuffered. The alternative would have been to always wrap unknown streams. However, I don’t think that anyone would benefit from being less strict. We can expect streams from the standard library to be subclassing either io.RawIOBase or io.BufferedIOBase, so running Mercurial in the standard way should not regress by this change. Py2exe might replace sys.stdout and sys.stderr, but that currently breaks Mercurial anyway and also these streams don’t claim to be interactive, so this function is not called for them.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
8250
1b60efdb8bc5 convert: add copyright and license headers to back-ends
Martin Geisler <mg@lazybytes.net>
parents: 7875
diff changeset
     1
# git.py - git support for the convert extension
1b60efdb8bc5 convert: add copyright and license headers to back-ends
Martin Geisler <mg@lazybytes.net>
parents: 7875
diff changeset
     2
#
46819
d4ba4d51f85f contributor: change mentions of mpm to olivia
Raphaël Gomès <rgomes@octobus.net>
parents: 46425
diff changeset
     3
#  Copyright 2005-2009 Olivia Mackall <olivia@selenic.com> and others
8250
1b60efdb8bc5 convert: add copyright and license headers to back-ends
Martin Geisler <mg@lazybytes.net>
parents: 7875
diff changeset
     4
#
1b60efdb8bc5 convert: add copyright and license headers to back-ends
Martin Geisler <mg@lazybytes.net>
parents: 7875
diff changeset
     5
# This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 8456
diff changeset
     6
# GNU General Public License version 2 or any later version.
3821
158fce02dc40 Teach convert-repo to deal with mixed charsets in git
Matt Mackall <mpm@selenic.com>
parents: 2657
diff changeset
     7
4536
cc9b79216a76 Split convert extension into common and repository type modules
Brendan Cully <brendan@kublai.com>
parents: 4532
diff changeset
     8
import os
29205
a0939666b836 py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents: 29051
diff changeset
     9
a0939666b836 py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents: 29051
diff changeset
    10
from mercurial.i18n import _
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46819
diff changeset
    11
from mercurial.node import sha1nodeconstants
28365
cd599bc179fb convert: git use absolute_import
timeless <timeless@mozdev.org>
parents: 26779
diff changeset
    12
from mercurial import (
cd599bc179fb convert: git use absolute_import
timeless <timeless@mozdev.org>
parents: 26779
diff changeset
    13
    config,
cd599bc179fb convert: git use absolute_import
timeless <timeless@mozdev.org>
parents: 26779
diff changeset
    14
    error,
41478
8e0dd36f7a97 git: a little pycompat.bytestring() love to make this code work in py3
Augie Fackler <augie@google.com>
parents: 37581
diff changeset
    15
    pycompat,
47633
770fd64a63e0 windows: use abspath in convert.git
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47128
diff changeset
    16
    util,
28365
cd599bc179fb convert: git use absolute_import
timeless <timeless@mozdev.org>
parents: 26779
diff changeset
    17
)
3938
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3917
diff changeset
    18
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    19
from . import common
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    20
3954
9af4b853ed4d convert-repo: add CVS branch support
Matt Mackall <mpm@selenic.com>
parents: 3953
diff changeset
    21
48946
642e31cb55f0 py3: use class X: instead of class X(object):
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
    22
class submodule:
17929
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
    23
    def __init__(self, path, node, url):
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
    24
        self.path = path
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
    25
        self.node = node
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
    26
        self.url = url
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
    27
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
    28
    def hgsub(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    29
        return b"%s = [git]%s" % (self.path, self.url)
17929
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
    30
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
    31
    def hgsubstate(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    32
        return b"%s %s" % (self.node, self.path)
17929
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
    33
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    34
30660
1f21a6835604 convert: add config option to copy extra keys from Git commits
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30646
diff changeset
    35
# Keys in extra fields that should not be copied if the user requests.
32291
bd872f64a8ba cleanup: use set literals
Martin von Zweigbergk <martinvonz@google.com>
parents: 30815
diff changeset
    36
bannedextrakeys = {
30660
1f21a6835604 convert: add config option to copy extra keys from Git commits
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30646
diff changeset
    37
    # Git commit object built-ins.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    38
    b'tree',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    39
    b'parent',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    40
    b'author',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    41
    b'committer',
30660
1f21a6835604 convert: add config option to copy extra keys from Git commits
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30646
diff changeset
    42
    # Mercurial built-ins.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    43
    b'branch',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    44
    b'close',
32291
bd872f64a8ba cleanup: use set literals
Martin von Zweigbergk <martinvonz@google.com>
parents: 30815
diff changeset
    45
}
30660
1f21a6835604 convert: add config option to copy extra keys from Git commits
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30646
diff changeset
    46
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    47
28670
ff0d3b6b287f merge with stable
Matt Mackall <mpm@selenic.com>
parents: 28365 28662
diff changeset
    48
class convert_git(common.converter_source, common.commandline):
5217
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5216
diff changeset
    49
    # Windows does not support GIT_DIR= construct while other systems
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5216
diff changeset
    50
    # cannot remove environment variable. Just assume none have
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5216
diff changeset
    51
    # both issues.
21630
a204fd9b5ba9 convert: drastically speed up git conversions
David Schleimer <dschleimer@fb.com>
parents: 20373
diff changeset
    52
28659
197eed39e3d5 convert: add new, non-clowny interface for shelling out to git (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26779
diff changeset
    53
    def _gitcmd(self, cmd, *args, **kwargs):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    54
        return cmd(b'--git-dir=%s' % self.path, *args, **kwargs)
28659
197eed39e3d5 convert: add new, non-clowny interface for shelling out to git (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26779
diff changeset
    55
197eed39e3d5 convert: add new, non-clowny interface for shelling out to git (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26779
diff changeset
    56
    def gitrun0(self, *args, **kwargs):
197eed39e3d5 convert: add new, non-clowny interface for shelling out to git (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26779
diff changeset
    57
        return self._gitcmd(self.run0, *args, **kwargs)
197eed39e3d5 convert: add new, non-clowny interface for shelling out to git (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26779
diff changeset
    58
197eed39e3d5 convert: add new, non-clowny interface for shelling out to git (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26779
diff changeset
    59
    def gitrun(self, *args, **kwargs):
197eed39e3d5 convert: add new, non-clowny interface for shelling out to git (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26779
diff changeset
    60
        return self._gitcmd(self.run, *args, **kwargs)
197eed39e3d5 convert: add new, non-clowny interface for shelling out to git (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26779
diff changeset
    61
197eed39e3d5 convert: add new, non-clowny interface for shelling out to git (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26779
diff changeset
    62
    def gitrunlines0(self, *args, **kwargs):
197eed39e3d5 convert: add new, non-clowny interface for shelling out to git (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26779
diff changeset
    63
        return self._gitcmd(self.runlines0, *args, **kwargs)
197eed39e3d5 convert: add new, non-clowny interface for shelling out to git (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26779
diff changeset
    64
197eed39e3d5 convert: add new, non-clowny interface for shelling out to git (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26779
diff changeset
    65
    def gitrunlines(self, *args, **kwargs):
197eed39e3d5 convert: add new, non-clowny interface for shelling out to git (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26779
diff changeset
    66
        return self._gitcmd(self.runlines, *args, **kwargs)
197eed39e3d5 convert: add new, non-clowny interface for shelling out to git (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 26779
diff changeset
    67
28662
80cac1de6aea convert: rewrite gitpipe to use common.commandline (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28661
diff changeset
    68
    def gitpipe(self, *args, **kwargs):
80cac1de6aea convert: rewrite gitpipe to use common.commandline (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28661
diff changeset
    69
        return self._gitcmd(self._run3, *args, **kwargs)
80cac1de6aea convert: rewrite gitpipe to use common.commandline (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28661
diff changeset
    70
35176
671aba341d90 convert: save an indicator of the repo type for sources and sinks
Matt Harbison <matt_harbison@yahoo.com>
parents: 34164
diff changeset
    71
    def __init__(self, ui, repotype, path, revs=None):
671aba341d90 convert: save an indicator of the repo type for sources and sinks
Matt Harbison <matt_harbison@yahoo.com>
parents: 34164
diff changeset
    72
        super(convert_git, self).__init__(ui, repotype, path, revs=revs)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    73
        common.commandline.__init__(self, ui, b'git')
25748
baea47cafe75 convert: add support for specifying multiple revs
Durham Goode <durham@fb.com>
parents: 25699
diff changeset
    74
29051
a56296f55a5e convert: pass absolute paths to git (SEC)
Blake Burkhart <bburky@bburky.com>
parents: 28817
diff changeset
    75
        # Pass an absolute path to git to prevent from ever being interpreted
a56296f55a5e convert: pass absolute paths to git (SEC)
Blake Burkhart <bburky@bburky.com>
parents: 28817
diff changeset
    76
        # as a URL
47633
770fd64a63e0 windows: use abspath in convert.git
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 47128
diff changeset
    77
        path = util.abspath(path)
29051
a56296f55a5e convert: pass absolute paths to git (SEC)
Blake Burkhart <bburky@bburky.com>
parents: 28817
diff changeset
    78
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    79
        if os.path.isdir(path + b"/.git"):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    80
            path += b"/.git"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    81
        if not os.path.exists(path + b"/objects"):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    82
            raise common.NoRepo(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    83
                _(b"%s does not look like a Git repository") % path
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
    84
            )
5497
f0a3918abd42 convert: fail if an external required tool is not found
Patrick Mezard <pmezard@gmail.com>
parents: 5404
diff changeset
    85
22512
6b6da715cb96 convert: change default for git rename detection to 50%
Siddharth Agarwal <sid0@fb.com>
parents: 22511
diff changeset
    86
        # The default value (50) is based on the default for 'git diff'.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    87
        similarity = ui.configint(b'convert', b'git.similarity')
22470
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
    88
        if similarity < 0 or similarity > 100:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    89
            raise error.Abort(_(b'similarity must be between 0 and 100'))
22470
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
    90
        if similarity > 0:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    91
            self.simopt = [b'-C%d%%' % similarity]
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    92
            findcopiesharder = ui.configbool(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    93
                b'convert', b'git.findcopiesharder'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    94
            )
22471
cc5f94db672b convert: add support to find git copies from all files in the working copy
Siddharth Agarwal <sid0@fb.com>
parents: 22470
diff changeset
    95
            if findcopiesharder:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    96
                self.simopt.append(b'--find-copies-harder')
30646
ea3540e66fd8 convert: config option for git rename limit
Gregory Szorc <gregory.szorc@gmail.com>
parents: 29205
diff changeset
    97
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    98
            renamelimit = ui.configint(b'convert', b'git.renamelimit')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    99
            self.simopt.append(b'-l%d' % renamelimit)
22470
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
   100
        else:
28660
cdda7b96afff convert: rewrite calls to Git to use the new shelling mechanism (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28659
diff changeset
   101
            self.simopt = []
22470
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
   102
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   103
        common.checktool(b'git', b'git')
5497
f0a3918abd42 convert: fail if an external required tool is not found
Patrick Mezard <pmezard@gmail.com>
parents: 5404
diff changeset
   104
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
   105
        self.path = path
17929
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   106
        self.submodules = []
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
   107
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   108
        self.catfilepipe = self.gitpipe(b'cat-file', b'--batch')
21630
a204fd9b5ba9 convert: drastically speed up git conversions
David Schleimer <dschleimer@fb.com>
parents: 20373
diff changeset
   109
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   110
        self.copyextrakeys = self.ui.configlist(b'convert', b'git.extrakeys')
30660
1f21a6835604 convert: add config option to copy extra keys from Git commits
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30646
diff changeset
   111
        banned = set(self.copyextrakeys) & bannedextrakeys
1f21a6835604 convert: add config option to copy extra keys from Git commits
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30646
diff changeset
   112
        if banned:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   113
            raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   114
                _(b'copying of extra key is forbidden: %s')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   115
                % _(b', ').join(sorted(banned))
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   116
            )
30660
1f21a6835604 convert: add config option to copy extra keys from Git commits
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30646
diff changeset
   117
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   118
        committeractions = self.ui.configlist(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   119
            b'convert', b'git.committeractions'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   120
        )
30813
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   121
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   122
        messagedifferent = None
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   123
        messagealways = None
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   124
        for a in committeractions:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   125
            if a.startswith((b'messagedifferent', b'messagealways')):
30813
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   126
                k = a
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   127
                v = None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   128
                if b'=' in a:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   129
                    k, v = a.split(b'=', 1)
30813
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   130
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   131
                if k == b'messagedifferent':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   132
                    messagedifferent = v or b'committer:'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   133
                elif k == b'messagealways':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   134
                    messagealways = v or b'committer:'
30813
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   135
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   136
        if messagedifferent and messagealways:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   137
            raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   138
                _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   139
                    b'committeractions cannot define both '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   140
                    b'messagedifferent and messagealways'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   141
                )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   142
            )
30813
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   143
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   144
        dropcommitter = b'dropcommitter' in committeractions
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   145
        replaceauthor = b'replaceauthor' in committeractions
30813
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   146
30815
c5bf2e8ec18c convert: remove "replacecommitter" action
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30813
diff changeset
   147
        if dropcommitter and replaceauthor:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   148
            raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   149
                _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   150
                    b'committeractions cannot define both '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   151
                    b'dropcommitter and replaceauthor'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   152
                )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   153
            )
30813
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   154
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   155
        if dropcommitter and messagealways:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   156
            raise error.Abort(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   157
                _(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   158
                    b'committeractions cannot define both '
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   159
                    b'dropcommitter and messagealways'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   160
                )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   161
            )
30813
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   162
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   163
        if not messagedifferent and not messagealways:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   164
            messagedifferent = b'committer:'
30813
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   165
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   166
        self.committeractions = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   167
            b'dropcommitter': dropcommitter,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   168
            b'replaceauthor': replaceauthor,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   169
            b'messagedifferent': messagedifferent,
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   170
            b'messagealways': messagealways,
30813
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   171
        }
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   172
21630
a204fd9b5ba9 convert: drastically speed up git conversions
David Schleimer <dschleimer@fb.com>
parents: 20373
diff changeset
   173
    def after(self):
a204fd9b5ba9 convert: drastically speed up git conversions
David Schleimer <dschleimer@fb.com>
parents: 20373
diff changeset
   174
        for f in self.catfilepipe:
a204fd9b5ba9 convert: drastically speed up git conversions
David Schleimer <dschleimer@fb.com>
parents: 20373
diff changeset
   175
            f.close()
a204fd9b5ba9 convert: drastically speed up git conversions
David Schleimer <dschleimer@fb.com>
parents: 20373
diff changeset
   176
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
   177
    def getheads(self):
25748
baea47cafe75 convert: add support for specifying multiple revs
Durham Goode <durham@fb.com>
parents: 25699
diff changeset
   178
        if not self.revs:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   179
            output, status = self.gitrun(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   180
                b'rev-parse', b'--branches', b'--remotes'
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   181
            )
28660
cdda7b96afff convert: rewrite calls to Git to use the new shelling mechanism (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28659
diff changeset
   182
            heads = output.splitlines()
cdda7b96afff convert: rewrite calls to Git to use the new shelling mechanism (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28659
diff changeset
   183
            if status:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   184
                raise error.Abort(_(b'cannot retrieve git heads'))
4768
f52bfe566583 convert: import all branches from git repositories
Brendan Cully <brendan@kublai.com>
parents: 4767
diff changeset
   185
        else:
25749
f2748cc43b2a convert: support multiple specifed revs in git source
Durham Goode <durham@fb.com>
parents: 25748
diff changeset
   186
            heads = []
f2748cc43b2a convert: support multiple specifed revs in git source
Durham Goode <durham@fb.com>
parents: 25748
diff changeset
   187
            for rev in self.revs:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   188
                rawhead, ret = self.gitrun(b'rev-parse', b'--verify', rev)
25749
f2748cc43b2a convert: support multiple specifed revs in git source
Durham Goode <durham@fb.com>
parents: 25748
diff changeset
   189
                heads.append(rawhead[:-1])
f2748cc43b2a convert: support multiple specifed revs in git source
Durham Goode <durham@fb.com>
parents: 25748
diff changeset
   190
                if ret:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   191
                    raise error.Abort(_(b'cannot retrieve git head "%s"') % rev)
10986
610f047326b9 convert/git: check status when reading the whole output
Patrick Mezard <pmezard@gmail.com>
parents: 10985
diff changeset
   192
        return heads
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
   193
36332
aefb75730ea3 convert: don't use type as a variable name
Pulkit Goyal <7895pulkit@gmail.com>
parents: 35628
diff changeset
   194
    def catfile(self, rev, ftype):
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46819
diff changeset
   195
        if rev == sha1nodeconstants.nullhex:
16687
e34106fa0dc3 cleanup: "raise SomeException()" -> "raise SomeException"
Brodie Rao <brodie@sf.io>
parents: 16683
diff changeset
   196
            raise IOError
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   197
        self.catfilepipe[0].write(rev + b'\n')
21630
a204fd9b5ba9 convert: drastically speed up git conversions
David Schleimer <dschleimer@fb.com>
parents: 20373
diff changeset
   198
        self.catfilepipe[0].flush()
a204fd9b5ba9 convert: drastically speed up git conversions
David Schleimer <dschleimer@fb.com>
parents: 20373
diff changeset
   199
        info = self.catfilepipe[1].readline().split()
36332
aefb75730ea3 convert: don't use type as a variable name
Pulkit Goyal <7895pulkit@gmail.com>
parents: 35628
diff changeset
   200
        if info[1] != ftype:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   201
            raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   202
                _(b'cannot read %r object at %s')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   203
                % (pycompat.bytestr(ftype), rev)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   204
            )
21630
a204fd9b5ba9 convert: drastically speed up git conversions
David Schleimer <dschleimer@fb.com>
parents: 20373
diff changeset
   205
        size = int(info[2])
a204fd9b5ba9 convert: drastically speed up git conversions
David Schleimer <dschleimer@fb.com>
parents: 20373
diff changeset
   206
        data = self.catfilepipe[1].read(size)
a204fd9b5ba9 convert: drastically speed up git conversions
David Schleimer <dschleimer@fb.com>
parents: 20373
diff changeset
   207
        if len(data) < size:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   208
            raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   209
                _(b'cannot read %r object at %s: unexpected size')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   210
                % (ftype, rev)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   211
            )
21630
a204fd9b5ba9 convert: drastically speed up git conversions
David Schleimer <dschleimer@fb.com>
parents: 20373
diff changeset
   212
        # read the trailing newline
a204fd9b5ba9 convert: drastically speed up git conversions
David Schleimer <dschleimer@fb.com>
parents: 20373
diff changeset
   213
        self.catfilepipe[1].read(1)
10986
610f047326b9 convert/git: check status when reading the whole output
Patrick Mezard <pmezard@gmail.com>
parents: 10985
diff changeset
   214
        return data
692
695dd9a491da convert-repo: deal with packed git and other fixes
mpm@selenic.com
parents: 450
diff changeset
   215
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
   216
    def getfile(self, name, rev):
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46819
diff changeset
   217
        if rev == sha1nodeconstants.nullhex:
22296
650b5b6e75ed convert: use None value for missing files instead of overloading IOError
Mads Kiilerich <madski@unity3d.com>
parents: 21958
diff changeset
   218
            return None, None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   219
        if name == b'.hgsub':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   220
            data = b'\n'.join([m.hgsub() for m in self.submoditer()])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   221
            mode = b''
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   222
        elif name == b'.hgsubstate':
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   223
            data = b'\n'.join([m.hgsubstate() for m in self.submoditer()])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   224
            mode = b''
17929
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   225
        else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   226
            data = self.catfile(rev, b"blob")
17929
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   227
            mode = self.modecache[(name, rev)]
11134
33010ff1fd6f convert: merge sources getmode() into getfile()
Patrick Mezard <pmezard@gmail.com>
parents: 10987
diff changeset
   228
        return data, mode
3956
558f52943cd2 convert-repo: add CVS exec bit support
Matt Mackall <mpm@selenic.com>
parents: 3954
diff changeset
   229
17929
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   230
    def submoditer(self):
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46819
diff changeset
   231
        null = sha1nodeconstants.nullhex
17929
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   232
        for m in sorted(self.submodules, key=lambda p: p.path):
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   233
            if m.node != null:
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   234
                yield m
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   235
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   236
    def parsegitmodules(self, content):
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   237
        """Parse the formatted .gitmodules file, example file format:
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   238
        [submodule "sub"]\n
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   239
        \tpath = sub\n
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   240
        \turl = git://giturl\n
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   241
        """
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   242
        self.submodules = []
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   243
        c = config.config()
25698
307370c2dda2 convert: handle .gitmodules with non-tab whitespaces
Durham Goode <durham@fb.com>
parents: 24395
diff changeset
   244
        # Each item in .gitmodules starts with whitespace that cant be parsed
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   245
        c.parse(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   246
            b'.gitmodules',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   247
            b'\n'.join(line.strip() for line in content.split(b'\n')),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   248
        )
17929
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   249
        for sec in c.sections():
46425
f7621fa14b84 config: use the right API to access git-submodule
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46113
diff changeset
   250
            # turn the config object into a real dict
f7621fa14b84 config: use the right API to access git-submodule
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 46113
diff changeset
   251
            s = dict(c.items(sec))
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   252
            if b'url' in s and b'path' in s:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   253
                self.submodules.append(submodule(s[b'path'], b'', s[b'url']))
17929
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   254
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   255
    def retrievegitmodules(self, version):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   256
        modules, ret = self.gitrun(
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   257
            b'show', b'%s:%s' % (version, b'.gitmodules')
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   258
        )
17929
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   259
        if ret:
25699
5c97a4ecbdd4 convert: improve support for unusual .gitmodules
Durham Goode <durham@fb.com>
parents: 25698
diff changeset
   260
            # This can happen if a file is in the repo that has permissions
5c97a4ecbdd4 convert: improve support for unusual .gitmodules
Durham Goode <durham@fb.com>
parents: 25698
diff changeset
   261
            # 160000, but there is no .gitmodules file.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   262
            self.ui.warn(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43077
diff changeset
   263
                _(b"warning: cannot read submodules config file in %s\n")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   264
                % version
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   265
            )
25699
5c97a4ecbdd4 convert: improve support for unusual .gitmodules
Durham Goode <durham@fb.com>
parents: 25698
diff changeset
   266
            return
5c97a4ecbdd4 convert: improve support for unusual .gitmodules
Durham Goode <durham@fb.com>
parents: 25698
diff changeset
   267
5c97a4ecbdd4 convert: improve support for unusual .gitmodules
Durham Goode <durham@fb.com>
parents: 25698
diff changeset
   268
        try:
5c97a4ecbdd4 convert: improve support for unusual .gitmodules
Durham Goode <durham@fb.com>
parents: 25698
diff changeset
   269
            self.parsegitmodules(modules)
5c97a4ecbdd4 convert: improve support for unusual .gitmodules
Durham Goode <durham@fb.com>
parents: 25698
diff changeset
   270
        except error.ParseError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   271
            self.ui.warn(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   272
                _(b"warning: unable to parse .gitmodules in %s\n") % version
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   273
            )
25699
5c97a4ecbdd4 convert: improve support for unusual .gitmodules
Durham Goode <durham@fb.com>
parents: 25698
diff changeset
   274
            return
5c97a4ecbdd4 convert: improve support for unusual .gitmodules
Durham Goode <durham@fb.com>
parents: 25698
diff changeset
   275
17929
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   276
        for m in self.submodules:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   277
            node, ret = self.gitrun(b'rev-parse', b'%s:%s' % (version, m.path))
17929
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   278
            if ret:
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   279
                continue
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   280
            m.node = node.strip()
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   281
22300
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22296
diff changeset
   282
    def getchanges(self, version, full):
35ab037de989 convert: introduce --full for converting all files
Mads Kiilerich <madski@unity3d.com>
parents: 22296
diff changeset
   283
        if full:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   284
            raise error.Abort(_(b"convert from git does not support --full"))
3956
558f52943cd2 convert-repo: add CVS exec bit support
Matt Mackall <mpm@selenic.com>
parents: 3954
diff changeset
   285
        self.modecache = {}
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   286
        cmd = (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   287
            [b'diff-tree', b'-z', b'--root', b'-m', b'-r']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   288
            + self.simopt
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   289
            + [version]
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   290
        )
28660
cdda7b96afff convert: rewrite calls to Git to use the new shelling mechanism (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28659
diff changeset
   291
        output, status = self.gitrun(*cmd)
cdda7b96afff convert: rewrite calls to Git to use the new shelling mechanism (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28659
diff changeset
   292
        if status:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   293
            raise error.Abort(_(b'cannot read changes in %s') % version)
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
   294
        changes = []
22470
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
   295
        copies = {}
8456
e9e2a2c9b294 convert: use set instead of dict
Benoit Boissinot <benoit.boissinot@ens-lyon.org>
parents: 8271
diff changeset
   296
        seen = set()
7242
d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360)
Patrick Mezard <pmezard@gmail.com>
parents: 7222
diff changeset
   297
        entry = None
22469
15bc0431476b convert: for git, factor out code to add entries to a separate function
Siddharth Agarwal <sid0@fb.com>
parents: 22468
diff changeset
   298
        subexists = [False]
15bc0431476b convert: for git, factor out code to add entries to a separate function
Siddharth Agarwal <sid0@fb.com>
parents: 22468
diff changeset
   299
        subdeleted = [False]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   300
        difftree = output.split(b'\x00')
22467
333d654783ad convert: for git's getchanges, use explicit index for iteration
Siddharth Agarwal <sid0@fb.com>
parents: 22413
diff changeset
   301
        lcount = len(difftree)
333d654783ad convert: for git's getchanges, use explicit index for iteration
Siddharth Agarwal <sid0@fb.com>
parents: 22413
diff changeset
   302
        i = 0
22469
15bc0431476b convert: for git, factor out code to add entries to a separate function
Siddharth Agarwal <sid0@fb.com>
parents: 22468
diff changeset
   303
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   304
        skipsubmodules = self.ui.configbool(b'convert', b'git.skipsubmodules')
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   305
22470
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
   306
        def add(entry, f, isdest):
22469
15bc0431476b convert: for git, factor out code to add entries to a separate function
Siddharth Agarwal <sid0@fb.com>
parents: 22468
diff changeset
   307
            seen.add(f)
15bc0431476b convert: for git, factor out code to add entries to a separate function
Siddharth Agarwal <sid0@fb.com>
parents: 22468
diff changeset
   308
            h = entry[3]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   309
            p = entry[1] == b"100755"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   310
            s = entry[1] == b"120000"
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   311
            renamesource = not isdest and entry[4][0] == b'R'
22469
15bc0431476b convert: for git, factor out code to add entries to a separate function
Siddharth Agarwal <sid0@fb.com>
parents: 22468
diff changeset
   312
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   313
            if f == b'.gitmodules':
26077
e63d05fbae84 convert: add convert.git.skipsubmodules option
Durham Goode <durham@fb.com>
parents: 25998
diff changeset
   314
                if skipsubmodules:
e63d05fbae84 convert: add convert.git.skipsubmodules option
Durham Goode <durham@fb.com>
parents: 25998
diff changeset
   315
                    return
e63d05fbae84 convert: add convert.git.skipsubmodules option
Durham Goode <durham@fb.com>
parents: 25998
diff changeset
   316
22469
15bc0431476b convert: for git, factor out code to add entries to a separate function
Siddharth Agarwal <sid0@fb.com>
parents: 22468
diff changeset
   317
                subexists[0] = True
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   318
                if entry[4] == b'D' or renamesource:
22469
15bc0431476b convert: for git, factor out code to add entries to a separate function
Siddharth Agarwal <sid0@fb.com>
parents: 22468
diff changeset
   319
                    subdeleted[0] = True
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46819
diff changeset
   320
                    changes.append((b'.hgsub', sha1nodeconstants.nullhex))
22469
15bc0431476b convert: for git, factor out code to add entries to a separate function
Siddharth Agarwal <sid0@fb.com>
parents: 22468
diff changeset
   321
                else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   322
                    changes.append((b'.hgsub', b''))
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   323
            elif entry[1] == b'160000' or entry[0] == b':160000':
26077
e63d05fbae84 convert: add convert.git.skipsubmodules option
Durham Goode <durham@fb.com>
parents: 25998
diff changeset
   324
                if not skipsubmodules:
e63d05fbae84 convert: add convert.git.skipsubmodules option
Durham Goode <durham@fb.com>
parents: 25998
diff changeset
   325
                    subexists[0] = True
22469
15bc0431476b convert: for git, factor out code to add entries to a separate function
Siddharth Agarwal <sid0@fb.com>
parents: 22468
diff changeset
   326
            else:
22470
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
   327
                if renamesource:
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46819
diff changeset
   328
                    h = sha1nodeconstants.nullhex
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   329
                self.modecache[(f, h)] = (p and b"x") or (s and b"l") or b""
22469
15bc0431476b convert: for git, factor out code to add entries to a separate function
Siddharth Agarwal <sid0@fb.com>
parents: 22468
diff changeset
   330
                changes.append((f, h))
15bc0431476b convert: for git, factor out code to add entries to a separate function
Siddharth Agarwal <sid0@fb.com>
parents: 22468
diff changeset
   331
22467
333d654783ad convert: for git's getchanges, use explicit index for iteration
Siddharth Agarwal <sid0@fb.com>
parents: 22413
diff changeset
   332
        while i < lcount:
333d654783ad convert: for git's getchanges, use explicit index for iteration
Siddharth Agarwal <sid0@fb.com>
parents: 22413
diff changeset
   333
            l = difftree[i]
333d654783ad convert: for git's getchanges, use explicit index for iteration
Siddharth Agarwal <sid0@fb.com>
parents: 22413
diff changeset
   334
            i += 1
7242
d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360)
Patrick Mezard <pmezard@gmail.com>
parents: 7222
diff changeset
   335
            if not entry:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   336
                if not l.startswith(b':'):
7242
d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360)
Patrick Mezard <pmezard@gmail.com>
parents: 7222
diff changeset
   337
                    continue
41478
8e0dd36f7a97 git: a little pycompat.bytestring() love to make this code work in py3
Augie Fackler <augie@google.com>
parents: 37581
diff changeset
   338
                entry = tuple(pycompat.bytestr(p) for p in l.split())
5335
88e931f74e8b convert_git: avoid returning two entries for the same file in getchanges
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5220
diff changeset
   339
                continue
7242
d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360)
Patrick Mezard <pmezard@gmail.com>
parents: 7222
diff changeset
   340
            f = l
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   341
            if entry[4][0] == b'C':
25997
d4e1e947444b convert: fix git copy file content conversions
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   342
                copysrc = f
d4e1e947444b convert: fix git copy file content conversions
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   343
                copydest = difftree[i]
d4e1e947444b convert: fix git copy file content conversions
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   344
                i += 1
d4e1e947444b convert: fix git copy file content conversions
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   345
                f = copydest
d4e1e947444b convert: fix git copy file content conversions
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   346
                copies[copydest] = copysrc
7242
d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360)
Patrick Mezard <pmezard@gmail.com>
parents: 7222
diff changeset
   347
            if f not in seen:
22470
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
   348
                add(entry, f, False)
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
   349
            # A file can be copied multiple times, or modified and copied
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
   350
            # simultaneously. So f can be repeated even if fdest isn't.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   351
            if entry[4][0] == b'R':
25997
d4e1e947444b convert: fix git copy file content conversions
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   352
                # rename: next line is the destination
22470
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
   353
                fdest = difftree[i]
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
   354
                i += 1
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
   355
                if fdest not in seen:
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
   356
                    add(entry, fdest, True)
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
   357
                    # .gitmodules isn't imported at all, so it being copied to
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
   358
                    # and fro doesn't really make sense
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   359
                    if f != b'.gitmodules' and fdest != b'.gitmodules':
22470
8e0c4df28eec convert: add support to detect git renames and copies
Siddharth Agarwal <sid0@fb.com>
parents: 22469
diff changeset
   360
                        copies[fdest] = f
7242
d1dff8c492dd convert: fix non-ASCII filenames retrieval from git sources (issue 1360)
Patrick Mezard <pmezard@gmail.com>
parents: 7222
diff changeset
   361
            entry = None
17929
0eed66327ad4 convert: add support for converting git submodule (issue3528)
YaNan Xu <robot9@fb.com>
parents: 16689
diff changeset
   362
22469
15bc0431476b convert: for git, factor out code to add entries to a separate function
Siddharth Agarwal <sid0@fb.com>
parents: 22468
diff changeset
   363
        if subexists[0]:
15bc0431476b convert: for git, factor out code to add entries to a separate function
Siddharth Agarwal <sid0@fb.com>
parents: 22468
diff changeset
   364
            if subdeleted[0]:
47012
d55b71393907 node: replace nullid and friends with nodeconstants class
Joerg Sonnenberger <joerg@bec.de>
parents: 46819
diff changeset
   365
                changes.append((b'.hgsubstate', sha1nodeconstants.nullhex))
21868
3420346174b1 convert: detect removal of ".gitmodules" at git source revisions correctly
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20373
diff changeset
   366
            else:
3420346174b1 convert: detect removal of ".gitmodules" at git source revisions correctly
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20373
diff changeset
   367
                self.retrievegitmodules(version)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   368
                changes.append((b'.hgsubstate', b''))
24395
216fa1ba9993 convert: optimize convert of files that are unmodified from p2 in merges
Mads Kiilerich <madski@unity3d.com>
parents: 23206
diff changeset
   369
        return (changes, copies, set())
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
   370
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
   371
    def getcommit(self, version):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   372
        c = self.catfile(version, b"commit")  # read the commit hash
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   373
        end = c.find(b"\n\n")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   374
        message = c[end + 2 :]
4759
20ec5cc02f18 convert: ove recode method into converter_source
Brendan Cully <brendan@kublai.com>
parents: 4721
diff changeset
   375
        message = self.recode(message)
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
   376
        l = c[:end].splitlines()
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
   377
        parents = []
8271
e3d3dad805f9 Add committer tag only when needed in git conversion
Richard Quirk <richard.quirk@gmail.com>
parents: 8250
diff changeset
   378
        author = committer = None
30660
1f21a6835604 convert: add config option to copy extra keys from Git commits
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30646
diff changeset
   379
        extra = {}
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
   380
        for e in l[1:]:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   381
            n, v = e.split(b" ", 1)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   382
            if n == b"author":
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
   383
                p = v.split()
1385
adb3de56635b convert-repo: Fix timezone handling
Matt Mackall <mpm@selenic.com>
parents: 1335
diff changeset
   384
                tm, tz = p[-2:]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   385
                author = b" ".join(p[:-2])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   386
                if author[0] == b"<":
35628
ab11af15a149 style: remove multiple statement on a single line
Boris Feld <boris.feld@octobus.net>
parents: 35176
diff changeset
   387
                    author = author[1:-1]
4759
20ec5cc02f18 convert: ove recode method into converter_source
Brendan Cully <brendan@kublai.com>
parents: 4721
diff changeset
   388
                author = self.recode(author)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   389
            if n == b"committer":
431
dfc44f3f587c convert-repo fixups
mpm@selenic.com
parents: 316
diff changeset
   390
                p = v.split()
1385
adb3de56635b convert-repo: Fix timezone handling
Matt Mackall <mpm@selenic.com>
parents: 1335
diff changeset
   391
                tm, tz = p[-2:]
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   392
                committer = b" ".join(p[:-2])
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   393
                if committer[0] == b"<":
35628
ab11af15a149 style: remove multiple statement on a single line
Boris Feld <boris.feld@octobus.net>
parents: 35176
diff changeset
   394
                    committer = committer[1:-1]
4759
20ec5cc02f18 convert: ove recode method into converter_source
Brendan Cully <brendan@kublai.com>
parents: 4721
diff changeset
   395
                committer = self.recode(committer)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   396
            if n == b"parent":
10282
08a0f04b56bd many, many trivial check-code fixups
Matt Mackall <mpm@selenic.com>
parents: 10263
diff changeset
   397
                parents.append(v)
30660
1f21a6835604 convert: add config option to copy extra keys from Git commits
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30646
diff changeset
   398
            if n in self.copyextrakeys:
1f21a6835604 convert: add config option to copy extra keys from Git commits
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30646
diff changeset
   399
                extra[n] = v
1385
adb3de56635b convert-repo: Fix timezone handling
Matt Mackall <mpm@selenic.com>
parents: 1335
diff changeset
   400
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   401
        if self.committeractions[b'dropcommitter']:
30813
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   402
            committer = None
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   403
        elif self.committeractions[b'replaceauthor']:
30813
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   404
            author = committer
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   405
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   406
        if committer:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   407
            messagealways = self.committeractions[b'messagealways']
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   408
            messagedifferent = self.committeractions[b'messagedifferent']
30813
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   409
            if messagealways:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   410
                message += b'\n%s %s\n' % (messagealways, committer)
30813
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   411
            elif messagedifferent and author != committer:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   412
                message += b'\n%s %s\n' % (messagedifferent, committer)
30813
2cbbd4622ab0 convert: config option to control Git committer actions
Gregory Szorc <gregory.szorc@gmail.com>
parents: 30661
diff changeset
   413
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   414
        tzs, tzh, tzm = tz[-5:-4] + b"1", tz[-4:-2], tz[-2:]
2093
5cc414722587 convert-repo: fix reversed time zone offset
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 1715
diff changeset
   415
        tz = -int(tzs) * (int(tzh) * 3600 + int(tzm))
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   416
        date = tm + b" " + (b"%d" % tz)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   417
        saverev = self.ui.configbool(b'convert', b'git.saverev')
3954
9af4b853ed4d convert-repo: add CVS branch support
Matt Mackall <mpm@selenic.com>
parents: 3953
diff changeset
   418
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   419
        c = common.commit(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   420
            parents=parents,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   421
            date=date,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   422
            author=author,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   423
            desc=message,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   424
            rev=version,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   425
            extra=extra,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   426
            saverev=saverev,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   427
        )
3954
9af4b853ed4d convert-repo: add CVS branch support
Matt Mackall <mpm@selenic.com>
parents: 3953
diff changeset
   428
        return c
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
   429
22413
3cb0318bb2dd convert: enable deterministic conversion progress bar for git
Augie Fackler <raf@durin42.com>
parents: 22300
diff changeset
   430
    def numcommits(self):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   431
        output, ret = self.gitrunlines(b'rev-list', b'--all')
28660
cdda7b96afff convert: rewrite calls to Git to use the new shelling mechanism (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28659
diff changeset
   432
        if ret:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   433
            raise error.Abort(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   434
                _(b'cannot retrieve number of commits in %s') % self.path
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   435
            )
28660
cdda7b96afff convert: rewrite calls to Git to use the new shelling mechanism (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28659
diff changeset
   436
        return len(output)
22413
3cb0318bb2dd convert: enable deterministic conversion progress bar for git
Augie Fackler <raf@durin42.com>
parents: 22300
diff changeset
   437
694
51eb248d3348 Teach convert-repo about tags
mpm@selenic.com
parents: 692
diff changeset
   438
    def gettags(self):
51eb248d3348 Teach convert-repo about tags
mpm@selenic.com
parents: 692
diff changeset
   439
        tags = {}
16259
589aab2ca716 convert: support non annotated tags in git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 14945
diff changeset
   440
        alltags = {}
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   441
        output, status = self.gitrunlines(b'ls-remote', b'--tags', self.path)
28660
cdda7b96afff convert: rewrite calls to Git to use the new shelling mechanism (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28659
diff changeset
   442
cdda7b96afff convert: rewrite calls to Git to use the new shelling mechanism (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28659
diff changeset
   443
        if status:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   444
            raise error.Abort(_(b'cannot read tags from %s') % self.path)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   445
        prefix = b'refs/tags/'
16259
589aab2ca716 convert: support non annotated tags in git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 14945
diff changeset
   446
589aab2ca716 convert: support non annotated tags in git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 14945
diff changeset
   447
        # Build complete list of tags, both annotated and bare ones
28660
cdda7b96afff convert: rewrite calls to Git to use the new shelling mechanism (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28659
diff changeset
   448
        for line in output:
4062
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
   449
            line = line.strip()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   450
            if line.startswith(b"error:") or line.startswith(b"fatal:"):
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   451
                raise error.Abort(_(b'cannot read tags from %s') % self.path)
4062
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
   452
            node, tag = line.split(None, 1)
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
   453
            if not tag.startswith(prefix):
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
   454
                continue
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   455
            alltags[tag[len(prefix) :]] = node
4062
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
   456
16259
589aab2ca716 convert: support non annotated tags in git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 14945
diff changeset
   457
        # Filter out tag objects for annotated tag refs
589aab2ca716 convert: support non annotated tags in git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 14945
diff changeset
   458
        for tag in alltags:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   459
            if tag.endswith(b'^{}'):
16259
589aab2ca716 convert: support non annotated tags in git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 14945
diff changeset
   460
                tags[tag[:-3]] = alltags[tag]
589aab2ca716 convert: support non annotated tags in git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 14945
diff changeset
   461
            else:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   462
                if tag + b'^{}' in alltags:
16259
589aab2ca716 convert: support non annotated tags in git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 14945
diff changeset
   463
                    continue
589aab2ca716 convert: support non annotated tags in git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 14945
diff changeset
   464
                else:
589aab2ca716 convert: support non annotated tags in git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 14945
diff changeset
   465
                    tags[tag] = alltags[tag]
589aab2ca716 convert: support non annotated tags in git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 14945
diff changeset
   466
694
51eb248d3348 Teach convert-repo about tags
mpm@selenic.com
parents: 692
diff changeset
   467
        return tags
5380
a5a7f7fd5554 convert_git: add --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5336
diff changeset
   468
a5a7f7fd5554 convert_git: add --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5336
diff changeset
   469
    def getchangedfiles(self, version, i):
a5a7f7fd5554 convert_git: add --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5336
diff changeset
   470
        changes = []
a5a7f7fd5554 convert_git: add --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5336
diff changeset
   471
        if i is None:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   472
            output, status = self.gitrunlines(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   473
                b'diff-tree', b'--root', b'-m', b'-r', version
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   474
            )
28660
cdda7b96afff convert: rewrite calls to Git to use the new shelling mechanism (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28659
diff changeset
   475
            if status:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   476
                raise error.Abort(_(b'cannot read changes in %s') % version)
28660
cdda7b96afff convert: rewrite calls to Git to use the new shelling mechanism (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28659
diff changeset
   477
            for l in output:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   478
                if b"\t" not in l:
5380
a5a7f7fd5554 convert_git: add --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5336
diff changeset
   479
                    continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   480
                m, f = l[:-1].split(b"\t")
5380
a5a7f7fd5554 convert_git: add --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5336
diff changeset
   481
                changes.append(f)
a5a7f7fd5554 convert_git: add --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5336
diff changeset
   482
        else:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   483
            output, status = self.gitrunlines(
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   484
                b'diff-tree',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   485
                b'--name-only',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   486
                b'--root',
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   487
                b'-r',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   488
                version,
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   489
                b'%s^%d' % (version, i + 1),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   490
                b'--',
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 41759
diff changeset
   491
            )
28816
f4a42bb7c2ec convert: don't ignore errors from git diff-tree
Julien Cristau <julien.cristau@logilab.fr>
parents: 28671
diff changeset
   492
            if status:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   493
                raise error.Abort(_(b'cannot read changes in %s') % version)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   494
            changes = [f.rstrip(b'\n') for f in output]
5380
a5a7f7fd5554 convert_git: add --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5336
diff changeset
   495
a5a7f7fd5554 convert_git: add --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5336
diff changeset
   496
        return changes
13756
6b7077df4aa5 convert: add bookmarks reading support to git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 12144
diff changeset
   497
6b7077df4aa5 convert: add bookmarks reading support to git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 12144
diff changeset
   498
    def getbookmarks(self):
6b7077df4aa5 convert: add bookmarks reading support to git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 12144
diff changeset
   499
        bookmarks = {}
6b7077df4aa5 convert: add bookmarks reading support to git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 12144
diff changeset
   500
25905
80149d0b6842 convert: fix git convert using servers branches
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   501
        # Handle local and remote branches
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   502
        remoteprefix = self.ui.config(b'convert', b'git.remoteprefix')
25905
80149d0b6842 convert: fix git convert using servers branches
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   503
        reftypes = [
80149d0b6842 convert: fix git convert using servers branches
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   504
            # (git prefix, hg prefix)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   505
            (b'refs/remotes/origin/', remoteprefix + b'/'),
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   506
            (b'refs/heads/', b''),
25905
80149d0b6842 convert: fix git convert using servers branches
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   507
        ]
13756
6b7077df4aa5 convert: add bookmarks reading support to git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 12144
diff changeset
   508
32291
bd872f64a8ba cleanup: use set literals
Martin von Zweigbergk <martinvonz@google.com>
parents: 30815
diff changeset
   509
        exclude = {
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   510
            b'refs/remotes/origin/HEAD',
32291
bd872f64a8ba cleanup: use set literals
Martin von Zweigbergk <martinvonz@google.com>
parents: 30815
diff changeset
   511
        }
13756
6b7077df4aa5 convert: add bookmarks reading support to git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 12144
diff changeset
   512
25905
80149d0b6842 convert: fix git convert using servers branches
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   513
        try:
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   514
            output, status = self.gitrunlines(b'show-ref')
28660
cdda7b96afff convert: rewrite calls to Git to use the new shelling mechanism (SEC)
Mateusz Kwapich <mitrandir@fb.com>
parents: 28659
diff changeset
   515
            for line in output:
25905
80149d0b6842 convert: fix git convert using servers branches
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   516
                line = line.strip()
80149d0b6842 convert: fix git convert using servers branches
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   517
                rev, name = line.split(None, 1)
80149d0b6842 convert: fix git convert using servers branches
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   518
                # Process each type of branch
80149d0b6842 convert: fix git convert using servers branches
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   519
                for gitprefix, hgprefix in reftypes:
80149d0b6842 convert: fix git convert using servers branches
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   520
                    if not name.startswith(gitprefix) or name in exclude:
13756
6b7077df4aa5 convert: add bookmarks reading support to git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 12144
diff changeset
   521
                        continue
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   522
                    name = b'%s%s' % (hgprefix, name[len(gitprefix) :])
13756
6b7077df4aa5 convert: add bookmarks reading support to git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 12144
diff changeset
   523
                    bookmarks[name] = rev
25905
80149d0b6842 convert: fix git convert using servers branches
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   524
        except Exception:
80149d0b6842 convert: fix git convert using servers branches
Durham Goode <durham@fb.com>
parents: 25787
diff changeset
   525
            pass
13756
6b7077df4aa5 convert: add bookmarks reading support to git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 12144
diff changeset
   526
6b7077df4aa5 convert: add bookmarks reading support to git backend
Edouard Gomez <ed.gomez@free.fr>
parents: 12144
diff changeset
   527
        return bookmarks
19121
478a04605ce1 splicemap: improve error handling when source is git (issue2084)
Ben Goswami <bengoswami@fb.com>
parents: 18572
diff changeset
   528
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   529
    def checkrevformat(self, revstr, mapname=b'splicemap'):
47062
f38bf44e077f black: make codebase compatible with black v21.4b2 and v20.8b1
Kyle Lippincott <spectral@google.com>
parents: 46819
diff changeset
   530
        """git revision string is a 40 byte hex"""
20373
e8203629371b convert: add mapname parameter to checkrevformat
Sean Farley <sean.michael.farley@gmail.com>
parents: 19121
diff changeset
   531
        self.checkhexformat(revstr, mapname)