hgext/win32mbcs.py
author Joerg Sonnenberger <joerg@bec.de>
Fri, 30 Apr 2021 02:11:58 +0200
changeset 47043 12450fbea288
parent 45942 89a2afe31e82
child 48875 6000f5b25c9b
permissions -rw-r--r--
manifests: push down expected node length into the parser This strictly enforces the node length in the manifest lines according to what the repository expects. One test case moves large hash testing into the non-treemanifest part as treemanifests don't provide an interface for overriding just the node length for now. Differential Revision: https://phab.mercurial-scm.org/D10533
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
     1
# win32mbcs.py -- MBCS filename support for Mercurial
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
     2
#
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
     3
# Copyright (c) 2008 Shun-ichi Goto <shunichi.goto@gmail.com>
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
     4
#
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
     5
# Version: 0.3
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
     6
# Author:  Shun-ichi Goto <shunichi.goto@gmail.com>
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
     7
#
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8001
diff changeset
     8
# This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9569
diff changeset
     9
# GNU General Public License version 2 or any later version.
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
    10
#
8228
eee2319c5895 add blank line after copyright notices and after header
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
    11
8932
f87884329419 extensions: fix up description lines some more
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 8894
diff changeset
    12
'''allow the use of MBCS paths with problematic encodings
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
    13
8001
c0e3aca616de win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents: 7983
diff changeset
    14
Some MBCS encodings are not good for some path operations (i.e.
c0e3aca616de win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents: 7983
diff changeset
    15
splitting path, case conversion, etc.) with its encoded bytes. We call
c0e3aca616de win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents: 7983
diff changeset
    16
such a encoding (i.e. shift_jis and big5) as "problematic encoding".
c0e3aca616de win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents: 7983
diff changeset
    17
This extension can be used to fix the issue with those encodings by
8665
e4ad46f9a004 win32mbcs: capitalize Unicode
Martin Geisler <mg@lazybytes.net>
parents: 8491
diff changeset
    18
wrapping some functions to convert to Unicode string before path
8001
c0e3aca616de win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents: 7983
diff changeset
    19
operation.
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
    20
8668
aea3a23151bd fixed typos found in translatable strings
Martin Geisler <mg@lazybytes.net>
parents: 8667
diff changeset
    21
This extension is useful for:
9216
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    22
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    23
- Japanese Windows users using shift_jis encoding.
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    24
- Chinese Windows users using big5 encoding.
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    25
- All users who use a repository with one of problematic encodings on
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    26
  case-insensitive file system.
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
    27
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
    28
This extension is not needed for:
9216
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    29
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    30
- Any user who use only ASCII chars in path.
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    31
- Any user who do not use any of problematic encodings.
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
    32
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
    33
Note that there are some limitations on using this extension:
9216
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    34
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    35
- You should use single encoding in one repository.
13067
9696954415db win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 10264
diff changeset
    36
- If the repository path ends with 0x5c, .hg/hgrc cannot be read.
13330
551856dea9a6 win32mbcs: Fix typo in documentation
Javi Merino <cibervicho@gmail.com>
parents: 13067
diff changeset
    37
- win32mbcs is not compatible with fixutf8 extension.
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    38
10067
fd6729805f44 win32mbcs: fix typos and reST syntax
Martin Geisler <mg@lazybytes.net>
parents: 10050
diff changeset
    39
By default, win32mbcs uses encoding.encoding decided by Mercurial.
fd6729805f44 win32mbcs: fix typos and reST syntax
Martin Geisler <mg@lazybytes.net>
parents: 10050
diff changeset
    40
You can specify the encoding by config option::
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
    41
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    42
 [win32mbcs]
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    43
 encoding = sjis
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    44
10067
fd6729805f44 win32mbcs: fix typos and reST syntax
Martin Geisler <mg@lazybytes.net>
parents: 10050
diff changeset
    45
It is useful for the users who want to commit with UTF-8 log message.
8894
868670dbc237 extensions: improve the consistency of synopses
Cédric Duval <cedricduval@free.fr>
parents: 8866
diff changeset
    46
'''
28417
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    47
from __future__ import absolute_import
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
    48
28417
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    49
import os
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    50
import sys
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    51
29205
a0939666b836 py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents: 28417
diff changeset
    52
from mercurial.i18n import _
43089
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43087
diff changeset
    53
from mercurial.pycompat import getattr, setattr
28417
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    54
from mercurial import (
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    55
    encoding,
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    56
    error,
30616
6f9fcd29e290 py3: replace os.sep with pycompat.ossep (part 4 of 4)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30476
diff changeset
    57
    pycompat,
34180
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    58
    registrar,
28417
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    59
)
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    60
29841
d5883fd055c6 extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents: 29397
diff changeset
    61
# Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
25186
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 19383
diff changeset
    62
# extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 19383
diff changeset
    63
# be specifying the version(s) of Mercurial they are tested with, or
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 19383
diff changeset
    64
# leave the attribute unspecified.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    65
testedwith = b'ships-with-hg-core'
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
    66
34180
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    67
configtable = {}
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    68
configitem = registrar.configitem(configtable)
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    69
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    70
# Encoding.encoding may be updated by --encoding option.
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    71
# Use a lambda do delay the resolution.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
    72
configitem(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43743
diff changeset
    73
    b'win32mbcs',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43743
diff changeset
    74
    b'encoding',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43743
diff changeset
    75
    default=lambda: encoding.encoding,
34180
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    76
)
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    77
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
    78
_encoding = None  # see extsetup
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
    79
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    80
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
    81
def decode(arg):
43743
66210a20f727 win32mbcs: fix a `str` type conditional for py3
Matt Harbison <matt_harbison@yahoo.com>
parents: 43117
diff changeset
    82
    if isinstance(arg, bytes):
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    83
        uarg = arg.decode(_encoding)
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    84
        if arg == uarg.encode(_encoding):
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
    85
            return uarg
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    86
        raise UnicodeError(b"Not local encoding")
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
    87
    elif isinstance(arg, tuple):
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
    88
        return tuple(map(decode, arg))
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
    89
    elif isinstance(arg, list):
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
    90
        return map(decode, arg)
9131
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
    91
    elif isinstance(arg, dict):
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
    92
        for k, v in arg.items():
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
    93
            arg[k] = decode(v)
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
    94
    return arg
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
    95
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
    96
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
    97
def encode(arg):
38312
79dd61a4554f py3: replace `unicode` with pycompat.unicode
Pulkit Goyal <7895pulkit@gmail.com>
parents: 34645
diff changeset
    98
    if isinstance(arg, pycompat.unicode):
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    99
        return arg.encode(_encoding)
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   100
    elif isinstance(arg, tuple):
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   101
        return tuple(map(encode, arg))
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   102
    elif isinstance(arg, list):
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   103
        return map(encode, arg)
9131
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
   104
    elif isinstance(arg, dict):
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
   105
        for k, v in arg.items():
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
   106
            arg[k] = encode(v)
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   107
    return arg
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   108
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   109
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   110
def appendsep(s):
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   111
    # ensure the path ends with os.sep, appending it if necessary.
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   112
    try:
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   113
        us = decode(s)
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   114
    except UnicodeError:
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   115
        us = s
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   116
    if us and us[-1] not in b':/\\':
30616
6f9fcd29e290 py3: replace os.sep with pycompat.ossep (part 4 of 4)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30476
diff changeset
   117
        s += pycompat.ossep
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   118
    return s
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   119
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   120
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   121
def basewrapper(func, argtype, enc, dec, args, kwds):
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   122
    # check check already converted, then call original
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   123
    for arg in args:
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   124
        if isinstance(arg, argtype):
9131
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
   125
            return func(*args, **kwds)
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
   126
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   127
    try:
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   128
        # convert string arguments, call func, then convert back the
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   129
        # return value.
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   130
        return enc(func(*dec(args), **dec(kwds)))
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   131
    except UnicodeError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   132
        raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43094
diff changeset
   133
            _(b"[win32mbcs] filename conversion failed with %s encoding\n")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   134
            % _encoding
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   135
        )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   136
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   137
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   138
def wrapper(func, args, kwds):
38312
79dd61a4554f py3: replace `unicode` with pycompat.unicode
Pulkit Goyal <7895pulkit@gmail.com>
parents: 34645
diff changeset
   139
    return basewrapper(func, pycompat.unicode, encode, decode, args, kwds)
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   140
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   141
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   142
def reversewrapper(func, args, kwds):
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   143
    return basewrapper(func, str, decode, encode, args, kwds)
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   144
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   145
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   146
def wrapperforlistdir(func, args, kwds):
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   147
    # Ensure 'path' argument ends with os.sep to avoids
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   148
    # misinterpreting last 0x5c of MBCS 2nd byte as path separator.
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   149
    if args:
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   150
        args = list(args)
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   151
        args[0] = appendsep(args[0])
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   152
    if b'path' in kwds:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   153
        kwds[b'path'] = appendsep(kwds[b'path'])
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   154
    return func(*args, **kwds)
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   155
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   156
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   157
def wrapname(name, wrapper):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   158
    module, name = name.rsplit(b'.', 1)
9098
5e4654f5522d win32mbcs: look up modules using sys.modules (issue1729)
Brodie Rao <me+hg@dackz.net>
parents: 8932
diff changeset
   159
    module = sys.modules[module]
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   160
    func = getattr(module, name)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   161
9131
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
   162
    def f(*args, **kwds):
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
   163
        return wrapper(func, args, kwds)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   164
30476
8a9681b963a3 win32mbcs: drop code that was catering to Python 2.3 and earlier
Augie Fackler <augie@google.com>
parents: 29889
diff changeset
   165
    f.__name__ = func.__name__
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   166
    setattr(module, name, f)
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   167
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   168
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   169
# List of functions to be wrapped.
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   170
# NOTE: os.path.dirname() and os.path.basename() are safe because
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   171
#       they use result of os.path.split()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   172
funcs = b'''os.path.join os.path.split os.path.splitext
29889
6f447b9ec263 util: rename checkcase() to fscasesensitive() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 29841
diff changeset
   173
 os.path.normpath os.makedirs mercurial.util.endswithsep
6f447b9ec263 util: rename checkcase() to fscasesensitive() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 29841
diff changeset
   174
 mercurial.util.splitpath mercurial.util.fscasesensitive
14841
6990340c57a8 win32mbcs: wrap two more functions to be wrapped.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 13330
diff changeset
   175
 mercurial.util.fspath mercurial.util.pconvert mercurial.util.normpath
19383
41c06a02814e win32mbcs: wrap util.split()
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17798
diff changeset
   176
 mercurial.util.checkwinfilename mercurial.util.checkosfilename
41c06a02814e win32mbcs: wrap util.split()
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17798
diff changeset
   177
 mercurial.util.split'''
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
   178
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   179
# These functions are required to be called with local encoded string
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   180
# because they expects argument is local encoded string and cause
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   181
# problem with unicode string.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   182
rfuncs = b'''mercurial.encoding.upper mercurial.encoding.lower
32572
377c74ef008d win32mbcs: avoid unintentional failure at colorization
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 32207
diff changeset
   183
 mercurial.util._filenamebytestr'''
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   184
15724
9e6a13c2aeb9 win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15672
diff changeset
   185
# List of Windows specific functions to be wrapped.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   186
winfuncs = b'''os.path.splitunc'''
15724
9e6a13c2aeb9 win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15672
diff changeset
   187
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
   188
# codec and alias names of sjis and big5 to be faked.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   189
problematic_encodings = b'''big5 big5-tw csbig5 big5hkscs big5-hkscs
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   190
 hkscs cp932 932 ms932 mskanji ms-kanji shift_jis csshiftjis shiftjis
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   191
 sjis s_jis shift_jis_2004 shiftjis2004 sjis_2004 sjis2004
8714
505a96cbc923 Add cp950 as problematic encoding which is used in chinese windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 8668
diff changeset
   192
 shift_jisx0213 shiftjisx0213 sjisx0213 s_jisx0213 950 cp950 ms950 '''
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
   193
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   194
13067
9696954415db win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 10264
diff changeset
   195
def extsetup(ui):
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   196
    # TODO: decide use of config section for this extension
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   197
    if (not os.path.supports_unicode_filenames) and (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   198
        pycompat.sysplatform != b'cygwin'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   199
    ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   200
        ui.warn(_(b"[win32mbcs] cannot activate on this platform.\n"))
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   201
        return
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
   202
    # determine encoding for filename
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
   203
    global _encoding
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   204
    _encoding = ui.config(b'win32mbcs', b'encoding')
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   205
    # fake is only for relevant environment.
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
   206
    if _encoding.lower() in problematic_encodings.split():
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   207
        for f in funcs.split():
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   208
            wrapname(f, wrapper)
34645
75979c8d4572 codemod: use pycompat.iswindows
Jun Wu <quark@fb.com>
parents: 34180
diff changeset
   209
        if pycompat.iswindows:
15724
9e6a13c2aeb9 win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15672
diff changeset
   210
            for f in winfuncs.split():
9e6a13c2aeb9 win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15672
diff changeset
   211
                wrapname(f, wrapper)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   212
        wrapname(b"mercurial.util.listdir", wrapperforlistdir)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   213
        wrapname(b"mercurial.windows.listdir", wrapperforlistdir)
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   214
        # wrap functions to be called with local byte string arguments
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   215
        for f in rfuncs.split():
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   216
            wrapname(f, reversewrapper)
13067
9696954415db win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 10264
diff changeset
   217
        # Check sys.args manually instead of using ui.debug() because
9696954415db win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 10264
diff changeset
   218
        # command line options is not yet applied when
9696954415db win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 10264
diff changeset
   219
        # extensions.loadall() is called.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   220
        if b'--debug' in sys.argv:
43094
e8cf9ad52a78 formatting: run black on all file again
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43089
diff changeset
   221
            ui.writenoi18n(
e8cf9ad52a78 formatting: run black on all file again
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43089
diff changeset
   222
                b"[win32mbcs] activated with encoding: %s\n" % _encoding
e8cf9ad52a78 formatting: run black on all file again
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43089
diff changeset
   223
            )