contrib/hgfixes/fix_leftover_imports.py
author David Soria Parra <davidsp@fb.com>
Wed, 24 Sep 2014 13:16:20 -0700
changeset 22540 9a860ac8c216
parent 19378 9de689d20230
permissions -rw-r--r--
parsers: fix uninitialize variable warning The heads pointer is not initialized correctly if filter is false, causing both clang and gcc to issue a warning. Correctly initialize heads to NULL.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
11949
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     1
"Fixer that translates some APIs ignored by the default 2to3 fixers."
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     2
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     3
# FIXME: This fixer has some ugly hacks. Its main design is based on that of
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     4
# fix_imports, from lib2to3. Unfortunately, the fix_imports framework only
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     5
# changes module names "without dots", meaning it won't work for some changes
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     6
# in the email module/package. Thus this fixer was born. I believe that with a
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     7
# bit more thinking, a more generic fixer can be implemented, but I'll leave
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     8
# that as future work.
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
     9
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    10
from lib2to3.fixer_util import Name
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    11
from lib2to3.fixes import fix_imports
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    12
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    13
# This maps the old names to the new names. Note that a drawback of the current
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    14
# design is that the dictionary keys MUST have EXACTLY one dot (.) in them,
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    15
# otherwise things will break. (If you don't need a module hierarchy, you're
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    16
# better of just inherit from fix_imports and overriding the MAPPING dict.)
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    17
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    18
MAPPING = {'email.Utils': 'email.utils',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    19
           'email.Errors': 'email.errors',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    20
           'email.Header': 'email.header',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    21
           'email.Parser': 'email.parser',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    22
           'email.Encoders': 'email.encoders',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    23
           'email.MIMEText': 'email.mime.text',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    24
           'email.MIMEBase': 'email.mime.base',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    25
           'email.Generator': 'email.generator',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    26
           'email.MIMEMultipart': 'email.mime.multipart',
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    27
}
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    28
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    29
def alternates(members):
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    30
    return "(" + "|".join(map(repr, members)) + ")"
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    31
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    32
def build_pattern(mapping=MAPPING):
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    33
    packages = {}
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    34
    for key in mapping:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    35
        # What we are doing here is the following: with dotted names, we'll
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    36
        # have something like package_name <trailer '.' module>. Then, we are
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    37
        # making a dictionary to copy this structure. For example, if
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    38
        # mapping={'A.B': 'a.b', 'A.C': 'a.c'}, it will generate the dictionary
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    39
        # {'A': ['b', 'c']} to, then, generate something like "A <trailer '.'
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    40
        # ('b' | 'c')".
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    41
        name = key.split('.')
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    42
        prefix = name[0]
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    43
        if prefix in packages:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    44
            packages[prefix].append(name[1:][0])
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    45
        else:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    46
            packages[prefix] = name[1:]
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    47
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    48
    mod_list = ' | '.join(["'%s' '.' ('%s')" %
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    49
        (key, "' | '".join(packages[key])) for key in packages])
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    50
    mod_list = '(' + mod_list + ' )'
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    51
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    52
    yield """name_import=import_name< 'import' module_name=dotted_name< %s > >
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    53
          """ % mod_list
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    54
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    55
    yield """name_import=import_name< 'import'
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    56
            multiple_imports=dotted_as_names< any*
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    57
            module_name=dotted_name< %s >
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    58
            any* >
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    59
            >""" % mod_list
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    60
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    61
    packs = ' | '.join(["'%s' trailer<'.' ('%s')>" % (key,
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    62
               "' | '".join(packages[key])) for key in packages])
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    63
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    64
    yield "power< package=(%s) trailer<'.' any > any* >" % packs
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    65
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    66
class FixLeftoverImports(fix_imports.FixImports):
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    67
    # We want to run this fixer after fix_import has run (this shouldn't matter
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    68
    # for hg, though, as setup3k prefers to run the default fixers first)
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    69
    mapping = MAPPING
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    70
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    71
    def build_pattern(self):
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    72
        return "|".join(build_pattern(self.mapping))
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    73
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    74
    def transform(self, node, results):
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    75
        # Mostly copied from fix_imports.py
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    76
        import_mod = results.get("module_name")
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    77
        if import_mod:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    78
            try:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    79
                mod_name = import_mod.value
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    80
            except AttributeError:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    81
                # XXX: A hack to remove whitespace prefixes and suffixes
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    82
                mod_name = str(import_mod).strip()
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    83
            new_name = self.mapping[mod_name]
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    84
            import_mod.replace(Name(new_name, prefix=import_mod.prefix))
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    85
            if "name_import" in results:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    86
                # If it's not a "from x import x, y" or "import x as y" import,
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    87
                # marked its usage to be replaced.
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    88
                self.replace[mod_name] = new_name
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    89
            if "multiple_imports" in results:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    90
                # This is a nasty hack to fix multiple imports on a line (e.g.,
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    91
                # "import StringIO, urlparse"). The problem is that I can't
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    92
                # figure out an easy way to make a pattern recognize the keys of
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    93
                # MAPPING randomly sprinkled in an import statement.
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    94
                results = self.match(node)
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    95
                if results:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    96
                    self.transform(node, results)
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    97
        else:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    98
            # Replace usage of the module.
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
    99
            # Now this is, mostly, a hack
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   100
            bare_name = results["package"][0]
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   101
            bare_name_text = ''.join(map(str, results['package'])).strip()
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   102
            new_name = self.replace.get(bare_name_text)
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   103
            prefix = results['package'][0].prefix
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   104
            if new_name:
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   105
                bare_name.replace(Name(new_name, prefix=prefix))
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   106
                results["package"][1].replace(Name(''))
626fe5c99231 hgfixes: added a fixer to convert changes in the email package
Renato Cunha <renatoc@gmail.com>
parents:
diff changeset
   107