contrib/hgfixes/fix_leftover_imports.py
author Mads Kiilerich <madski@unity3d.com>
Fri, 06 Feb 2015 02:52:10 +0100
branchstable
changeset 24159 5b4ed033390b
parent 19378 9de689d20230
permissions -rw-r--r--
revisionbranchcache: fall back to slow path if starting readonly (issue4531) Transitioning to Mercurial versions with revision branch cache could be slow as long as all operations were readonly (revset queries) and the cache would be populated but not written back. Instead, fall back to using the consistently slow path when readonly and the cache doesn't exist yet. That avoids the overhead of populating the cache without writing it back. If not readonly, it will still populate all missing entries initially. That avoids repeated writing of the cache file with small updates, and it also makes sure a fully populated cache available for the readonly operations.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
11949
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     1
"Fixer that translates some APIs ignored by the default 2to3 fixers."
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     2
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     3
# FIXME: This fixer has some ugly hacks. Its main design is based on that of
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     4
# fix_imports, from lib2to3. Unfortunately, the fix_imports framework only
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     5
# changes module names "without dots", meaning it won't work for some changes
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     6
# in the email module/package. Thus this fixer was born. I believe that with a
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     7
# bit more thinking, a more generic fixer can be implemented, but I'll leave
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     8
# that as future work.
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     9
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    10
from lib2to3.fixer_util import Name
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    11
from lib2to3.fixes import fix_imports
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    12
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    13
# This maps the old names to the new names. Note that a drawback of the current
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    14
# design is that the dictionary keys MUST have EXACTLY one dot (.) in them,
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    15
# otherwise things will break. (If you don't need a module hierarchy, you're
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    16
# better of just inherit from fix_imports and overriding the MAPPING dict.)
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    17
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    18
MAPPING = {'email.Utils': 'email.utils',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    19
           'email.Errors': 'email.errors',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    20
           'email.Header': 'email.header',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    21
           'email.Parser': 'email.parser',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    22
           'email.Encoders': 'email.encoders',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    23
           'email.MIMEText': 'email.mime.text',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    24
           'email.MIMEBase': 'email.mime.base',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    25
           'email.Generator': 'email.generator',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    26
           'email.MIMEMultipart': 'email.mime.multipart',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    27
}
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    28
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    29
def alternates(members):
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    30
    return "(" + "|".join(map(repr, members)) + ")"
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    31
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    32
def build_pattern(mapping=MAPPING):
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    33
    packages = {}
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    34
    for key in mapping:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    35
        # What we are doing here is the following: with dotted names, we'll
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    36
        # have something like package_name <trailer '.' module>. Then, we are
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    37
        # making a dictionary to copy this structure. For example, if
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    38
        # mapping={'A.B': 'a.b', 'A.C': 'a.c'}, it will generate the dictionary
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    39
        # {'A': ['b', 'c']} to, then, generate something like "A <trailer '.'
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    40
        # ('b' | 'c')".
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    41
        name = key.split('.')
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    42
        prefix = name[0]
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    43
        if prefix in packages:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    44
            packages[prefix].append(name[1:][0])
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    45
        else:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    46
            packages[prefix] = name[1:]
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    47
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    48
    mod_list = ' | '.join(["'%s' '.' ('%s')" %
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    49
        (key, "' | '".join(packages[key])) for key in packages])
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    50
    mod_list = '(' + mod_list + ' )'
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    51
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    52
    yield """name_import=import_name< 'import' module_name=dotted_name< %s > >
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    53
          """ % mod_list
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    54
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    55
    yield """name_import=import_name< 'import'
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    56
            multiple_imports=dotted_as_names< any*
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    57
            module_name=dotted_name< %s >
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    58
            any* >
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    59
            >""" % mod_list
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    60
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    61
    packs = ' | '.join(["'%s' trailer<'.' ('%s')>" % (key,
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    62
               "' | '".join(packages[key])) for key in packages])
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    63
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    64
    yield "power< package=(%s) trailer<'.' any > any* >" % packs
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    65
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    66
class FixLeftoverImports(fix_imports.FixImports):
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    67
    # We want to run this fixer after fix_import has run (this shouldn't matter
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    68
    # for hg, though, as setup3k prefers to run the default fixers first)
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    69
    mapping = MAPPING
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    70
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    71
    def build_pattern(self):
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    72
        return "|".join(build_pattern(self.mapping))
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    73
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    74
    def transform(self, node, results):
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    75
        # Mostly copied from fix_imports.py
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    76
        import_mod = results.get("module_name")
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    77
        if import_mod:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    78
            try:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    79
                mod_name = import_mod.value
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    80
            except AttributeError:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    81
                # XXX: A hack to remove whitespace prefixes and suffixes
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    82
                mod_name = str(import_mod).strip()
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    83
            new_name = self.mapping[mod_name]
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    84
            import_mod.replace(Name(new_name, prefix=import_mod.prefix))
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    85
            if "name_import" in results:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    86
                # If it's not a "from x import x, y" or "import x as y" import,
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    87
                # marked its usage to be replaced.
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    88
                self.replace[mod_name] = new_name
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    89
            if "multiple_imports" in results:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    90
                # This is a nasty hack to fix multiple imports on a line (e.g.,
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    91
                # "import StringIO, urlparse"). The problem is that I can't
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    92
                # figure out an easy way to make a pattern recognize the keys of
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    93
                # MAPPING randomly sprinkled in an import statement.
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    94
                results = self.match(node)
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    95
                if results:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    96
                    self.transform(node, results)
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    97
        else:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    98
            # Replace usage of the module.
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    99
            # Now this is, mostly, a hack
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   100
            bare_name = results["package"][0]
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   101
            bare_name_text = ''.join(map(str, results['package'])).strip()
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   102
            new_name = self.replace.get(bare_name_text)
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   103
            prefix = results['package'][0].prefix
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   104
            if new_name:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   105
                bare_name.replace(Name(new_name, prefix=prefix))
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   106
                results["package"][1].replace(Name(''))
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   107