hgext/win32mbcs.py
author Manuel Jacob <me@manueljacob.de>
Thu, 15 Sep 2022 01:48:38 +0200
changeset 49494 c96ed4029fda
parent 48934 06de08b36c82
child 50742 48d9af6bd043
permissions -rw-r--r--
templates: add filter to reverse list The filter supports only lists because for lists, it’s straightforward to implement. Reversing text doesn’t seem very useful and is hard to implement. Reversing the bytes would break multi-bytes encodings. Reversing the code points would break characters consisting of multiple code points. Reversing graphemes is non-trivial without using a library not included in the standard library.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
     1
# win32mbcs.py -- MBCS filename support for Mercurial
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
     2
#
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
     3
# Copyright (c) 2008 Shun-ichi Goto <shunichi.goto@gmail.com>
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
     4
#
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
     5
# Version: 0.3
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
     6
# Author:  Shun-ichi Goto <shunichi.goto@gmail.com>
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
     7
#
8225
46293a0c7e9f updated license to be explicit about GPL version 2
Martin Geisler <mg@lazybytes.net>
parents: 8001
diff changeset
     8
# This software may be used and distributed according to the terms of the
10263
25e572394f5c Update license to GPLv2+
Matt Mackall <mpm@selenic.com>
parents: 9569
diff changeset
     9
# GNU General Public License version 2 or any later version.
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
    10
#
8228
eee2319c5895 add blank line after copyright notices and after header
Martin Geisler <mg@lazybytes.net>
parents: 8225
diff changeset
    11
8932
f87884329419 extensions: fix up description lines some more
Dirkjan Ochtman <dirkjan@ochtman.nl>
parents: 8894
diff changeset
    12
'''allow the use of MBCS paths with problematic encodings
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
    13
8001
c0e3aca616de win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents: 7983
diff changeset
    14
Some MBCS encodings are not good for some path operations (i.e.
c0e3aca616de win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents: 7983
diff changeset
    15
splitting path, case conversion, etc.) with its encoded bytes. We call
c0e3aca616de win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents: 7983
diff changeset
    16
such a encoding (i.e. shift_jis and big5) as "problematic encoding".
c0e3aca616de win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents: 7983
diff changeset
    17
This extension can be used to fix the issue with those encodings by
8665
e4ad46f9a004 win32mbcs: capitalize Unicode
Martin Geisler <mg@lazybytes.net>
parents: 8491
diff changeset
    18
wrapping some functions to convert to Unicode string before path
8001
c0e3aca616de win32mbcs: word-wrap help texts at 70 characters
Martin Geisler <mg@daimi.au.dk>
parents: 7983
diff changeset
    19
operation.
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
    20
8668
aea3a23151bd fixed typos found in translatable strings
Martin Geisler <mg@lazybytes.net>
parents: 8667
diff changeset
    21
This extension is useful for:
9216
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    22
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    23
- Japanese Windows users using shift_jis encoding.
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    24
- Chinese Windows users using big5 encoding.
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    25
- All users who use a repository with one of problematic encodings on
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    26
  case-insensitive file system.
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
    27
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
    28
This extension is not needed for:
9216
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    29
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    30
- Any user who use only ASCII chars in path.
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    31
- Any user who do not use any of problematic encodings.
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
    32
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
    33
Note that there are some limitations on using this extension:
9216
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    34
9b2649b6ce5c win32mbcs: fix formatting of lists with proper reST markup
Martin Geisler <mg@lazybytes.net>
parents: 9154
diff changeset
    35
- You should use single encoding in one repository.
13067
9696954415db win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 10264
diff changeset
    36
- If the repository path ends with 0x5c, .hg/hgrc cannot be read.
13330
551856dea9a6 win32mbcs: Fix typo in documentation
Javi Merino <cibervicho@gmail.com>
parents: 13067
diff changeset
    37
- win32mbcs is not compatible with fixutf8 extension.
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    38
10067
fd6729805f44 win32mbcs: fix typos and reST syntax
Martin Geisler <mg@lazybytes.net>
parents: 10050
diff changeset
    39
By default, win32mbcs uses encoding.encoding decided by Mercurial.
fd6729805f44 win32mbcs: fix typos and reST syntax
Martin Geisler <mg@lazybytes.net>
parents: 10050
diff changeset
    40
You can specify the encoding by config option::
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
    41
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    42
 [win32mbcs]
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    43
 encoding = sjis
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    44
10067
fd6729805f44 win32mbcs: fix typos and reST syntax
Martin Geisler <mg@lazybytes.net>
parents: 10050
diff changeset
    45
It is useful for the users who want to commit with UTF-8 log message.
8894
868670dbc237 extensions: improve the consistency of synopses
Cédric Duval <cedricduval@free.fr>
parents: 8866
diff changeset
    46
'''
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
    47
28417
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    48
import os
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    49
import sys
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    50
29205
a0939666b836 py3: move up symbol imports to enforce import-checker rules
Yuya Nishihara <yuya@tcha.org>
parents: 28417
diff changeset
    51
from mercurial.i18n import _
43089
c59eb1560c44 py3: manually import getattr where it is needed
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43087
diff changeset
    52
from mercurial.pycompat import getattr, setattr
28417
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    53
from mercurial import (
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    54
    encoding,
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    55
    error,
30616
6f9fcd29e290 py3: replace os.sep with pycompat.ossep (part 4 of 4)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30476
diff changeset
    56
    pycompat,
34180
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    57
    registrar,
28417
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    58
)
588874c33b4d win32mbcs: use absolute_import
timeless <timeless@mozdev.org>
parents: 26587
diff changeset
    59
29841
d5883fd055c6 extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents: 29397
diff changeset
    60
# Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
25186
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 19383
diff changeset
    61
# extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 19383
diff changeset
    62
# be specifying the version(s) of Mercurial they are tested with, or
80c5b2666a96 extensions: document that `testedwith = 'internal'` is special
Augie Fackler <augie@google.com>
parents: 19383
diff changeset
    63
# leave the attribute unspecified.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    64
testedwith = b'ships-with-hg-core'
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
    65
34180
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    66
configtable = {}
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    67
configitem = registrar.configitem(configtable)
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    68
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    69
# Encoding.encoding may be updated by --encoding option.
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    70
# Use a lambda do delay the resolution.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
    71
configitem(
45942
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43743
diff changeset
    72
    b'win32mbcs',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43743
diff changeset
    73
    b'encoding',
89a2afe31e82 formating: upgrade to black 20.8b1
Augie Fackler <raf@durin42.com>
parents: 43743
diff changeset
    74
    default=lambda: encoding.encoding,
34180
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    75
)
37513324f620 configitems: register the 'win32mbcs.encoding' config
Boris Feld <boris.feld@octobus.net>
parents: 32652
diff changeset
    76
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
    77
_encoding = None  # see extsetup
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
    78
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    79
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
    80
def decode(arg):
43743
66210a20f727 win32mbcs: fix a `str` type conditional for py3
Matt Harbison <matt_harbison@yahoo.com>
parents: 43117
diff changeset
    81
    if isinstance(arg, bytes):
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    82
        uarg = arg.decode(_encoding)
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    83
        if arg == uarg.encode(_encoding):
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
    84
            return uarg
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    85
        raise UnicodeError(b"Not local encoding")
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
    86
    elif isinstance(arg, tuple):
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
    87
        return tuple(map(decode, arg))
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
    88
    elif isinstance(arg, list):
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
    89
        return map(decode, arg)
9131
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
    90
    elif isinstance(arg, dict):
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
    91
        for k, v in arg.items():
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
    92
            arg[k] = decode(v)
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
    93
    return arg
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
    94
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
    95
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
    96
def encode(arg):
48934
06de08b36c82 py3: use str instead of pycompat.unicode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
    97
    if isinstance(arg, str):
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
    98
        return arg.encode(_encoding)
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
    99
    elif isinstance(arg, tuple):
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   100
        return tuple(map(encode, arg))
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   101
    elif isinstance(arg, list):
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   102
        return map(encode, arg)
9131
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
   103
    elif isinstance(arg, dict):
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
   104
        for k, v in arg.items():
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
   105
            arg[k] = encode(v)
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   106
    return arg
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   107
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   108
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   109
def appendsep(s):
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   110
    # ensure the path ends with os.sep, appending it if necessary.
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   111
    try:
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   112
        us = decode(s)
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   113
    except UnicodeError:
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   114
        us = s
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   115
    if us and us[-1] not in b':/\\':
30616
6f9fcd29e290 py3: replace os.sep with pycompat.ossep (part 4 of 4)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30476
diff changeset
   116
        s += pycompat.ossep
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   117
    return s
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   118
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   119
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   120
def basewrapper(func, argtype, enc, dec, args, kwds):
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   121
    # check check already converted, then call original
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   122
    for arg in args:
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   123
        if isinstance(arg, argtype):
9131
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
   124
            return func(*args, **kwds)
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
   125
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   126
    try:
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   127
        # convert string arguments, call func, then convert back the
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   128
        # return value.
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   129
        return enc(func(*dec(args), **dec(kwds)))
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   130
    except UnicodeError:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   131
        raise error.Abort(
43117
8ff1ecfadcd1 cleanup: join string literals that are already on one line
Martin von Zweigbergk <martinvonz@google.com>
parents: 43094
diff changeset
   132
            _(b"[win32mbcs] filename conversion failed with %s encoding\n")
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   133
            % _encoding
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   134
        )
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   135
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   136
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   137
def wrapper(func, args, kwds):
48934
06de08b36c82 py3: use str instead of pycompat.unicode
Gregory Szorc <gregory.szorc@gmail.com>
parents: 48875
diff changeset
   138
    return basewrapper(func, str, encode, decode, args, kwds)
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   139
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   140
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   141
def reversewrapper(func, args, kwds):
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   142
    return basewrapper(func, str, decode, encode, args, kwds)
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   143
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   144
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   145
def wrapperforlistdir(func, args, kwds):
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   146
    # Ensure 'path' argument ends with os.sep to avoids
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   147
    # misinterpreting last 0x5c of MBCS 2nd byte as path separator.
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   148
    if args:
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   149
        args = list(args)
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   150
        args[0] = appendsep(args[0])
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   151
    if b'path' in kwds:
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   152
        kwds[b'path'] = appendsep(kwds[b'path'])
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   153
    return func(*args, **kwds)
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   154
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   155
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   156
def wrapname(name, wrapper):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   157
    module, name = name.rsplit(b'.', 1)
9098
5e4654f5522d win32mbcs: look up modules using sys.modules (issue1729)
Brodie Rao <me+hg@dackz.net>
parents: 8932
diff changeset
   158
    module = sys.modules[module]
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   159
    func = getattr(module, name)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   160
9131
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
   161
    def f(*args, **kwds):
2bbb8419720d win32mbcs: wrapper supports keyword arguments and dict result.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9100
diff changeset
   162
        return wrapper(func, args, kwds)
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   163
30476
8a9681b963a3 win32mbcs: drop code that was catering to Python 2.3 and earlier
Augie Fackler <augie@google.com>
parents: 29889
diff changeset
   164
    f.__name__ = func.__name__
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   165
    setattr(module, name, f)
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   166
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   167
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   168
# List of functions to be wrapped.
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   169
# NOTE: os.path.dirname() and os.path.basename() are safe because
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   170
#       they use result of os.path.split()
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   171
funcs = b'''os.path.join os.path.split os.path.splitext
29889
6f447b9ec263 util: rename checkcase() to fscasesensitive() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 29841
diff changeset
   172
 os.path.normpath os.makedirs mercurial.util.endswithsep
6f447b9ec263 util: rename checkcase() to fscasesensitive() (API)
Martin von Zweigbergk <martinvonz@google.com>
parents: 29841
diff changeset
   173
 mercurial.util.splitpath mercurial.util.fscasesensitive
14841
6990340c57a8 win32mbcs: wrap two more functions to be wrapped.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 13330
diff changeset
   174
 mercurial.util.fspath mercurial.util.pconvert mercurial.util.normpath
19383
41c06a02814e win32mbcs: wrap util.split()
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17798
diff changeset
   175
 mercurial.util.checkwinfilename mercurial.util.checkosfilename
41c06a02814e win32mbcs: wrap util.split()
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17798
diff changeset
   176
 mercurial.util.split'''
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
   177
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   178
# These functions are required to be called with local encoded string
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   179
# because they expects argument is local encoded string and cause
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   180
# problem with unicode string.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   181
rfuncs = b'''mercurial.encoding.upper mercurial.encoding.lower
32572
377c74ef008d win32mbcs: avoid unintentional failure at colorization
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 32207
diff changeset
   182
 mercurial.util._filenamebytestr'''
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   183
15724
9e6a13c2aeb9 win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15672
diff changeset
   184
# List of Windows specific functions to be wrapped.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   185
winfuncs = b'''os.path.splitunc'''
15724
9e6a13c2aeb9 win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15672
diff changeset
   186
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
   187
# codec and alias names of sjis and big5 to be faked.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   188
problematic_encodings = b'''big5 big5-tw csbig5 big5hkscs big5-hkscs
6887
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   189
 hkscs cp932 932 ms932 mskanji ms-kanji shift_jis csshiftjis shiftjis
304484c7e0ba Update win32mbcs extension
Shun-ichi Goto <shunichi.goto@gmail.com>
parents: 6210
diff changeset
   190
 sjis s_jis shift_jis_2004 shiftjis2004 sjis_2004 sjis2004
8714
505a96cbc923 Add cp950 as problematic encoding which is used in chinese windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 8668
diff changeset
   191
 shift_jisx0213 shiftjisx0213 sjisx0213 s_jisx0213 950 cp950 ms950 '''
5846
02884e56c217 New extension to support problematic MBCS on Windows.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents:
diff changeset
   192
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   193
13067
9696954415db win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 10264
diff changeset
   194
def extsetup(ui):
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   195
    # TODO: decide use of config section for this extension
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   196
    if (not os.path.supports_unicode_filenames) and (
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   197
        pycompat.sysplatform != b'cygwin'
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 38312
diff changeset
   198
    ):
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   199
        ui.warn(_(b"[win32mbcs] cannot activate on this platform.\n"))
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   200
        return
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
   201
    # determine encoding for filename
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
   202
    global _encoding
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   203
    _encoding = ui.config(b'win32mbcs', b'encoding')
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   204
    # fake is only for relevant environment.
10050
dd37f044f1fa win32mbcs: Add configuration to specify path encoding
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9569
diff changeset
   205
    if _encoding.lower() in problematic_encodings.split():
7877
eba7f12b0c51 cleanup: whitespace cleanup
Peter Arrenbrecht <peter.arrenbrecht@gmail.com>
parents: 7598
diff changeset
   206
        for f in funcs.split():
9132
b47d7b440c5c win32mbcs: add special wrapper for osutil.listdir().
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 9131
diff changeset
   207
            wrapname(f, wrapper)
34645
75979c8d4572 codemod: use pycompat.iswindows
Jun Wu <quark@fb.com>
parents: 34180
diff changeset
   208
        if pycompat.iswindows:
15724
9e6a13c2aeb9 win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15672
diff changeset
   209
            for f in winfuncs.split():
9e6a13c2aeb9 win32mbcs: allow win32mbcs extension to be enabled on cygwin platform
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 15672
diff changeset
   210
                wrapname(f, wrapper)
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   211
        wrapname(b"mercurial.util.listdir", wrapperforlistdir)
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   212
        wrapname(b"mercurial.windows.listdir", wrapperforlistdir)
17798
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   213
        # wrap functions to be called with local byte string arguments
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   214
        for f in rfuncs.split():
4091b0322918 win32mbcs: add reversing wrapper for some unicode-incompatible functions.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 17428
diff changeset
   215
            wrapname(f, reversewrapper)
13067
9696954415db win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 10264
diff changeset
   216
        # Check sys.args manually instead of using ui.debug() because
9696954415db win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 10264
diff changeset
   217
        # command line options is not yet applied when
9696954415db win32mbcs: use extsetup() to wrap functions only once.
Shun-ichi GOTO <shunichi.goto@gmail.com>
parents: 10264
diff changeset
   218
        # extensions.loadall() is called.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
   219
        if b'--debug' in sys.argv:
43094
e8cf9ad52a78 formatting: run black on all file again
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43089
diff changeset
   220
            ui.writenoi18n(
e8cf9ad52a78 formatting: run black on all file again
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43089
diff changeset
   221
                b"[win32mbcs] activated with encoding: %s\n" % _encoding
e8cf9ad52a78 formatting: run black on all file again
Pierre-Yves David <pierre-yves.david@octobus.net>
parents: 43089
diff changeset
   222
            )