i18n/check-translation.py
author timeless@mozdev.org
Thu, 17 Sep 2015 21:20:17 -0400
changeset 26276 93395bee98ba
parent 26261 8fb92ff63ccf
child 26277 ad4d6c7aea6a
permissions -rw-r--r--
tests: cleanup check-translation deprecated
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
20152
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
     1
#!/usr/bin/env python
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
     2
#
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
     3
# check-translation.py - check Mercurial specific translation problems
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
     4
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
     5
import polib
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
     6
import re
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
     7
26261
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
     8
scanners = []
20152
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
     9
checkers = []
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    10
26261
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    11
def scanner():
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    12
    def decorator(func):
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    13
        scanners.append(func)
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    14
        return func
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    15
    return decorator
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    16
22203
35c2ea4ca26f cleanup: rename check-translation.py checker function - don't hide global var
Mads Kiilerich <madski@unity3d.com>
parents: 20515
diff changeset
    17
def levelchecker(level, msgidpat):
20152
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    18
    def decorator(func):
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    19
        if msgidpat:
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    20
            match = re.compile(msgidpat).search
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    21
        else:
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    22
            match = lambda msgid: True
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    23
        checkers.append((func, level))
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    24
        func.match = match
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    25
        return func
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    26
    return decorator
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    27
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    28
def match(checker, pe):
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    29
    """Examine whether POEntry "pe" is target of specified checker or not
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    30
    """
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    31
    if not checker.match(pe.msgid):
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    32
        return
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    33
    # examine suppression by translator comment
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    34
    nochecker = 'no-%s-check' % checker.__name__
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    35
    for tc in pe.tcomment.split():
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    36
        if nochecker == tc:
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    37
            return
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    38
    return True
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    39
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    40
####################
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    41
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    42
def fatalchecker(msgidpat=None):
22203
35c2ea4ca26f cleanup: rename check-translation.py checker function - don't hide global var
Mads Kiilerich <madski@unity3d.com>
parents: 20515
diff changeset
    43
    return levelchecker('fatal', msgidpat)
20152
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    44
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    45
@fatalchecker(r'\$\$')
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    46
def promptchoice(pe):
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    47
    """Check translation of the string given to "ui.promptchoice()"
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    48
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    49
    >>> pe = polib.POEntry(
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    50
    ...     msgid ='prompt$$missing &sep$$missing &amp$$followed by &none',
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    51
    ...     msgstr='prompt  missing &sep$$missing  amp$$followed by none&')
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    52
    >>> match(promptchoice, pe)
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    53
    True
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    54
    >>> for e in promptchoice(pe): print e
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    55
    number of choices differs between msgid and msgstr
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    56
    msgstr has invalid choice missing '&'
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    57
    msgstr has invalid '&' followed by none
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    58
    """
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    59
    idchoices = [c.rstrip(' ') for c in pe.msgid.split('$$')[1:]]
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    60
    strchoices = [c.rstrip(' ') for c in pe.msgstr.split('$$')[1:]]
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    61
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    62
    if len(idchoices) != len(strchoices):
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    63
        yield "number of choices differs between msgid and msgstr"
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    64
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    65
    indices = [(c, c.find('&')) for c in strchoices]
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    66
    if [c for c, i in indices if i == -1]:
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    67
        yield "msgstr has invalid choice missing '&'"
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    68
    if [c for c, i in indices if len(c) == i + 1]:
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    69
        yield "msgstr has invalid '&' followed by none"
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
    70
26261
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    71
deprecatedpe = None
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    72
@scanner()
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    73
def deprecatedsetup(pofile):
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    74
    pes = [p for p in pofile if p.msgid == 'DEPRECATED']
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    75
    if len(pes):
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    76
        global deprecatedpe
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    77
        deprecatedpe = pes[0]
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    78
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    79
@fatalchecker('(DEPRECATED)')
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    80
def deprecated(pe):
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    81
    """Check for DEPRECATED
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    82
    >>> ped = polib.POEntry(
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    83
    ...     msgid = 'DEPRECATED',
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    84
    ...     msgstr= 'DETACERPED')
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    85
    >>> deprecatedsetup([ped])
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    86
    >>> pe = polib.POEntry(
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    87
    ...     msgid = 'Something (DEPRECATED)',
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    88
    ...     msgstr= 'something (DETACERPED)')
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    89
    >>> match(deprecated, pe)
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    90
    True
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    91
    >>> pe = polib.POEntry(
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    92
    ...     msgid = 'Something (DEPRECATED)',
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    93
    ...     msgstr= 'something')
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    94
    >>> match(deprecated, pe)
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    95
    True
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    96
    >>> for e in deprecated(pe): print e
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    97
    msgstr inconsistently translated (DEPRECATED)
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
    98
    """
26276
93395bee98ba tests: cleanup check-translation deprecated
timeless@mozdev.org
parents: 26261
diff changeset
    99
    if not ('(DEPRECATED)' in pe.msgstr or
93395bee98ba tests: cleanup check-translation deprecated
timeless@mozdev.org
parents: 26261
diff changeset
   100
            (deprecatedpe and deprecatedpe.msgstr and
93395bee98ba tests: cleanup check-translation deprecated
timeless@mozdev.org
parents: 26261
diff changeset
   101
             deprecatedpe.msgstr in pe.msgstr)):
93395bee98ba tests: cleanup check-translation deprecated
timeless@mozdev.org
parents: 26261
diff changeset
   102
        yield "msgstr inconsistently translated (DEPRECATED)"
26261
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
   103
20152
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   104
####################
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   105
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   106
def warningchecker(msgidpat=None):
22203
35c2ea4ca26f cleanup: rename check-translation.py checker function - don't hide global var
Mads Kiilerich <madski@unity3d.com>
parents: 20515
diff changeset
   107
    return levelchecker('warning', msgidpat)
20152
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   108
20514
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   109
@warningchecker()
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   110
def taildoublecolons(pe):
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   111
    """Check equality of tail '::'-ness between msgid and msgstr
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   112
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   113
    >>> pe = polib.POEntry(
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   114
    ...     msgid ='ends with ::',
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   115
    ...     msgstr='ends with ::')
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   116
    >>> for e in taildoublecolons(pe): print e
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   117
    >>> pe = polib.POEntry(
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   118
    ...     msgid ='ends with ::',
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   119
    ...     msgstr='ends without double-colons')
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   120
    >>> for e in taildoublecolons(pe): print e
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   121
    tail '::'-ness differs between msgid and msgstr
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   122
    >>> pe = polib.POEntry(
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   123
    ...     msgid ='ends without double-colons',
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   124
    ...     msgstr='ends with ::')
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   125
    >>> for e in taildoublecolons(pe): print e
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   126
    tail '::'-ness differs between msgid and msgstr
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   127
    """
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   128
    if pe.msgid.endswith('::') != pe.msgstr.endswith('::'):
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   129
        yield "tail '::'-ness differs between msgid and msgstr"
410c80539c5c i18n: check equality of tail '::'-ness between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20164
diff changeset
   130
20515
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   131
@warningchecker()
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   132
def indentation(pe):
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   133
    """Check equality of initial indentation between msgid and msgstr
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   134
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   135
    This may report unexpected warning, because this doesn't aware
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   136
    the syntax of rst document and the context of msgstr.
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   137
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   138
    >>> pe = polib.POEntry(
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   139
    ...     msgid ='    indented text',
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   140
    ...     msgstr='  narrowed indentation')
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   141
    >>> for e in indentation(pe): print e
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   142
    initial indentation width differs betweeen msgid and msgstr
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   143
    """
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   144
    idindent = len(pe.msgid) - len(pe.msgid.lstrip())
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   145
    strindent = len(pe.msgstr) - len(pe.msgstr.lstrip())
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   146
    if idindent != strindent:
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   147
        yield "initial indentation width differs betweeen msgid and msgstr"
6afbfb9b1af1 i18n: check equality of initial indentation between msgid and msgstr
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 20514
diff changeset
   148
20152
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   149
####################
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   150
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   151
def check(pofile, fatal=True, warning=False):
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   152
    targetlevel = { 'fatal': fatal, 'warning': warning }
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   153
    targetcheckers = [(checker, level)
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   154
                      for checker, level in checkers
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   155
                      if targetlevel[level]]
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   156
    if not targetcheckers:
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   157
        return []
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   158
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   159
    detected = []
26261
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
   160
    for checker in scanners:
8fb92ff63ccf tests: check for inconsistently translated DEPRECATED
timeless@mozdev.org
parents: 22203
diff changeset
   161
        checker(pofile)
20152
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   162
    for pe in pofile.translated_entries():
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   163
        errors = []
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   164
        for checker, level in targetcheckers:
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   165
            if match(checker, pe):
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   166
                errors.extend((level, checker.__name__, error)
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   167
                              for error in checker(pe))
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   168
        if errors:
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   169
            detected.append((pe, errors))
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   170
    return detected
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   171
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   172
########################################
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   173
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   174
if __name__ == "__main__":
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   175
    import sys
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   176
    import optparse
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   177
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   178
    optparser = optparse.OptionParser("""%prog [options] pofile ...
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   179
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   180
This checks Mercurial specific translation problems in specified
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   181
'*.po' files.
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   182
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   183
Each detected problems are shown in the format below::
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   184
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   185
    filename:linenum:type(checker): problem detail .....
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   186
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   187
"type" is "fatal" or "warning". "checker" is the name of the function
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   188
detecting corresponded error.
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   189
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   190
Checking by checker "foo" on the specific msgstr can be suppressed by
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   191
the "translator comment" like below. Multiple "no-xxxx-check" should
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   192
be separated by whitespaces::
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   193
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   194
    # no-foo-check
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   195
    msgid = "....."
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   196
    msgstr = "....."
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   197
""")
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   198
    optparser.add_option("", "--warning",
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   199
                         help="show also warning level problems",
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   200
                         action="store_true")
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   201
    optparser.add_option("", "--doctest",
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   202
                         help="run doctest of this tool, instead of check",
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   203
                         action="store_true")
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   204
    (options, args) = optparser.parse_args()
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   205
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   206
    if options.doctest:
20164
1ddf4409229f tests: fix missing import in check-translations
Matt Mackall <mpm@selenic.com>
parents: 20158
diff changeset
   207
        import os
20158
209e04a06467 tests: fix Mac doctest escape code garbage for check-translations
Matt Mackall <mpm@selenic.com>
parents: 20152
diff changeset
   208
        if 'TERM' in os.environ:
209e04a06467 tests: fix Mac doctest escape code garbage for check-translations
Matt Mackall <mpm@selenic.com>
parents: 20152
diff changeset
   209
            del os.environ['TERM']
20152
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   210
        import doctest
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   211
        failures, tests = doctest.testmod()
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   212
        sys.exit(failures and 1 or 0)
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   213
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   214
    # replace polib._POFileParser to show linenum of problematic msgstr
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   215
    class ExtPOFileParser(polib._POFileParser):
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   216
        def process(self, symbol, linenum):
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   217
            super(ExtPOFileParser, self).process(symbol, linenum)
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   218
            if symbol == 'MS': # msgstr
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   219
                self.current_entry.linenum = linenum
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   220
    polib._POFileParser = ExtPOFileParser
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   221
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   222
    detected = []
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   223
    warning = options.warning
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   224
    for f in args:
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   225
        detected.extend((f, pe, errors)
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   226
                        for pe, errors in check(polib.pofile(f),
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   227
                                                warning=warning))
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   228
    if detected:
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   229
        for f, pe, errors in detected:
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   230
            for level, checker, error in errors:
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   231
                sys.stderr.write('%s:%d:%s(%s): %s\n'
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   232
                                 % (f, pe.linenum, level, checker, error))
84939b728749 i18n: add the tool to check Mercurial specific translation problems in *.po
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents:
diff changeset
   233
        sys.exit(1)