mercurial/minirst.py
author Martin Geisler <mg@lazybytes.net>
Sat, 07 Nov 2009 03:14:56 +0100
changeset 9739 75cff8f12910
parent 9738 f52c4f7a4732
child 9741 245689e7f869
permissions -rw-r--r--
minirst: support enumerated lists
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     1
# minirst.py - minimal reStructuredText parser
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     2
#
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     3
# Copyright 2009 Matt Mackall <mpm@selenic.com> and others
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     4
#
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     6
# GNU General Public License version 2, incorporated herein by reference.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     7
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     8
"""simplified reStructuredText parser.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
     9
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    10
This parser knows just enough about reStructuredText to parse the
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    11
Mercurial docstrings.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    12
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    13
It cheats in a major way: nested blocks are not really nested. They
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    14
are just indented blocks that look like they are nested. This relies
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    15
on the user to keep the right indentation for the blocks.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    16
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    17
It only supports a small subset of reStructuredText:
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    18
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    19
- paragraphs
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    20
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    21
- definition lists (must use '  ' to indent definitions)
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    22
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    23
- lists (items must start with '-')
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    24
9293
e48a48b754d3 minirst: parse field lists
Martin Geisler <mg@lazybytes.net>
parents: 9292
diff changeset
    25
- field lists (colons cannot be escaped)
e48a48b754d3 minirst: parse field lists
Martin Geisler <mg@lazybytes.net>
parents: 9292
diff changeset
    26
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    27
- literal blocks
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    28
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    29
- option lists (supports only long options without arguments)
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    30
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    31
- inline markup is not recognized at all.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    32
"""
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    33
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    34
import re, sys, textwrap
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    35
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    36
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    37
def findblocks(text):
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    38
    """Find continuous blocks of lines in text.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    39
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    40
    Returns a list of dictionaries representing the blocks. Each block
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    41
    has an 'indent' field and a 'lines' field.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    42
    """
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    43
    blocks = [[]]
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    44
    lines = text.splitlines()
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    45
    for line in lines:
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    46
        if line.strip():
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    47
            blocks[-1].append(line)
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    48
        elif blocks[-1]:
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    49
            blocks.append([])
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    50
    if not blocks[-1]:
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    51
        del blocks[-1]
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    52
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    53
    for i, block in enumerate(blocks):
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    54
        indent = min((len(l) - len(l.lstrip())) for l in block)
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    55
        blocks[i] = dict(indent=indent, lines=[l[indent:] for l in block])
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    56
    return blocks
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    57
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    58
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    59
def findliteralblocks(blocks):
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    60
    """Finds literal blocks and adds a 'type' field to the blocks.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    61
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    62
    Literal blocks are given the type 'literal', all other blocks are
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    63
    given type the 'paragraph'.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    64
    """
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    65
    i = 0
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    66
    while i < len(blocks):
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    67
        # Searching for a block that looks like this:
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    68
        #
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    69
        # +------------------------------+
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    70
        # | paragraph                    |
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    71
        # | (ends with "::")             |
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    72
        # +------------------------------+
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    73
        #    +---------------------------+
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    74
        #    | indented literal block    |
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    75
        #    +---------------------------+
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    76
        blocks[i]['type'] = 'paragraph'
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    77
        if blocks[i]['lines'][-1].endswith('::') and i+1 < len(blocks):
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    78
            indent = blocks[i]['indent']
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    79
            adjustment = blocks[i+1]['indent'] - indent
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    80
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    81
            if blocks[i]['lines'] == ['::']:
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    82
                # Expanded form: remove block
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    83
                del blocks[i]
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    84
                i -= 1
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    85
            elif blocks[i]['lines'][-1].endswith(' ::'):
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    86
                # Partially minimized form: remove space and both
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    87
                # colons.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    88
                blocks[i]['lines'][-1] = blocks[i]['lines'][-1][:-3]
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    89
            else:
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    90
                # Fully minimized form: remove just one colon.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    91
                blocks[i]['lines'][-1] = blocks[i]['lines'][-1][:-1]
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    92
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    93
            # List items are formatted with a hanging indent. We must
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    94
            # correct for this here while we still have the original
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    95
            # information on the indentation of the subsequent literal
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
    96
            # blocks available.
9738
f52c4f7a4732 minirst: prepare for general types of bullet lists
Martin Geisler <mg@lazybytes.net>
parents: 9737
diff changeset
    97
            m = _bulletre.match(blocks[i]['lines'][0])
f52c4f7a4732 minirst: prepare for general types of bullet lists
Martin Geisler <mg@lazybytes.net>
parents: 9737
diff changeset
    98
            if m:
f52c4f7a4732 minirst: prepare for general types of bullet lists
Martin Geisler <mg@lazybytes.net>
parents: 9737
diff changeset
    99
                indent += m.end()
f52c4f7a4732 minirst: prepare for general types of bullet lists
Martin Geisler <mg@lazybytes.net>
parents: 9737
diff changeset
   100
                adjustment -= m.end()
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   101
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   102
            # Mark the following indented blocks.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   103
            while i+1 < len(blocks) and blocks[i+1]['indent'] > indent:
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   104
                blocks[i+1]['type'] = 'literal'
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   105
                blocks[i+1]['indent'] -= adjustment
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   106
                i += 1
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   107
        i += 1
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   108
    return blocks
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   109
9739
75cff8f12910 minirst: support enumerated lists
Martin Geisler <mg@lazybytes.net>
parents: 9738
diff changeset
   110
_bulletre = re.compile(r'(-|[0-9A-Za-z]+\.|\(?[0-9A-Za-z]+\)) ')
9737
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   111
_optionre = re.compile(r'^(--[a-z-]+)((?:[ =][a-zA-Z][\w-]*)?  +)(.*)$')
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   112
_fieldre = re.compile(r':(?![: ])([^:]*)(?<! ):( +)(.*)')
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   113
_definitionre = re.compile(r'[^ ]')
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   114
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   115
def splitparagraphs(blocks):
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   116
    """Split paragraphs into lists."""
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   117
    # Tuples with (list type, item regexp, single line items?). Order
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   118
    # matters: definition lists has the least specific regexp and must
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   119
    # come last.
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   120
    listtypes = [('bullet', _bulletre, True),
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   121
                 ('option', _optionre, True),
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   122
                 ('field', _fieldre, True),
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   123
                 ('definition', _definitionre, False)]
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   124
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   125
    def match(lines, i, itemre, singleline):
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   126
        """Does itemre match an item at line i?
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   127
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   128
        A list item can be followed by an idented line or another list
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   129
        item (but only if singleline is True).
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   130
        """
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   131
        line1 = lines[i]
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   132
        line2 = i+1 < len(lines) and lines[i+1] or ''
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   133
        if not itemre.match(line1):
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   134
            return False
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   135
        if singleline:
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   136
            return line2 == '' or line2[0] == ' ' or itemre.match(line2)
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   137
        else:
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   138
            return line2.startswith(' ')
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   139
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   140
    i = 0
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   141
    while i < len(blocks):
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   142
        if blocks[i]['type'] == 'paragraph':
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   143
            lines = blocks[i]['lines']
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   144
            for type, itemre, singleline in listtypes:
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   145
                if match(lines, 0, itemre, singleline):
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   146
                    items = []
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   147
                    for j, line in enumerate(lines):
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   148
                        if match(lines, j, itemre, singleline):
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   149
                            items.append(dict(type=type, lines=[],
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   150
                                              indent=blocks[i]['indent']))
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   151
                        items[-1]['lines'].append(line)
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   152
                    blocks[i:i+1] = items
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   153
                    break
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   154
        i += 1
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   155
    return blocks
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   156
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   157
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   158
def findsections(blocks):
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   159
    """Finds sections.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   160
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   161
    The blocks must have a 'type' field, i.e., they should have been
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   162
    run through findliteralblocks first.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   163
    """
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   164
    for block in blocks:
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   165
        # Searching for a block that looks like this:
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   166
        #
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   167
        # +------------------------------+
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   168
        # | Section title                |
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   169
        # | -------------                |
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   170
        # +------------------------------+
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   171
        if (block['type'] == 'paragraph' and
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   172
            len(block['lines']) == 2 and
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   173
            block['lines'][1] == '-' * len(block['lines'][0])):
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   174
            block['type'] = 'section'
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   175
    return blocks
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   176
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   177
9623
32727ce029de minirst: convert ``foo`` into "foo" upon display
Martin Geisler <mg@lazybytes.net>
parents: 9540
diff changeset
   178
def inlineliterals(blocks):
32727ce029de minirst: convert ``foo`` into "foo" upon display
Martin Geisler <mg@lazybytes.net>
parents: 9540
diff changeset
   179
    for b in blocks:
32727ce029de minirst: convert ``foo`` into "foo" upon display
Martin Geisler <mg@lazybytes.net>
parents: 9540
diff changeset
   180
        if b['type'] == 'paragraph':
32727ce029de minirst: convert ``foo`` into "foo" upon display
Martin Geisler <mg@lazybytes.net>
parents: 9540
diff changeset
   181
            b['lines'] = [l.replace('``', '"') for l in b['lines']]
32727ce029de minirst: convert ``foo`` into "foo" upon display
Martin Geisler <mg@lazybytes.net>
parents: 9540
diff changeset
   182
    return blocks
32727ce029de minirst: convert ``foo`` into "foo" upon display
Martin Geisler <mg@lazybytes.net>
parents: 9540
diff changeset
   183
32727ce029de minirst: convert ``foo`` into "foo" upon display
Martin Geisler <mg@lazybytes.net>
parents: 9540
diff changeset
   184
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   185
def addmargins(blocks):
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   186
    """Adds empty blocks for vertical spacing.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   187
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   188
    This groups bullets, options, and definitions together with no vertical
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   189
    space between them, and adds an empty block between all other blocks.
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   190
    """
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   191
    i = 1
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   192
    while i < len(blocks):
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   193
        if (blocks[i]['type'] == blocks[i-1]['type'] and
9293
e48a48b754d3 minirst: parse field lists
Martin Geisler <mg@lazybytes.net>
parents: 9292
diff changeset
   194
            blocks[i]['type'] in ('bullet', 'option', 'field', 'definition')):
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   195
            i += 1
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   196
        else:
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   197
            blocks.insert(i, dict(lines=[''], indent=0, type='margin'))
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   198
            i += 2
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   199
    return blocks
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   200
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   201
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   202
def formatblock(block, width):
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   203
    """Format a block according to width."""
9417
4c3fb45123e5 util, minirst: do not crash with COLUMNS=0
Martin Geisler <mg@lazybytes.net>
parents: 9293
diff changeset
   204
    if width <= 0:
4c3fb45123e5 util, minirst: do not crash with COLUMNS=0
Martin Geisler <mg@lazybytes.net>
parents: 9293
diff changeset
   205
        width = 78
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   206
    indent = ' ' * block['indent']
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   207
    if block['type'] == 'margin':
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   208
        return ''
9735
97d0d910fa5d minirst: remove unnecessary "elif:" statements
Martin Geisler <mg@lazybytes.net>
parents: 9623
diff changeset
   209
    if block['type'] == 'literal':
9291
cd5b6a11b607 minirst: indent literal blocks with two spaces
Martin Geisler <mg@lazybytes.net>
parents: 9156
diff changeset
   210
        indent += '  '
cd5b6a11b607 minirst: indent literal blocks with two spaces
Martin Geisler <mg@lazybytes.net>
parents: 9156
diff changeset
   211
        return indent + ('\n' + indent).join(block['lines'])
9735
97d0d910fa5d minirst: remove unnecessary "elif:" statements
Martin Geisler <mg@lazybytes.net>
parents: 9623
diff changeset
   212
    if block['type'] == 'section':
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   213
        return indent + ('\n' + indent).join(block['lines'])
9735
97d0d910fa5d minirst: remove unnecessary "elif:" statements
Martin Geisler <mg@lazybytes.net>
parents: 9623
diff changeset
   214
    if block['type'] == 'definition':
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   215
        term = indent + block['lines'][0]
9737
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   216
        hang = len(block['lines'][-1]) - len(block['lines'][-1].lstrip())
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   217
        defindent = indent + hang * ' '
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   218
        text = ' '.join(map(str.strip, block['lines'][1:]))
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   219
        return "%s\n%s" % (term, textwrap.fill(text, width=width,
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   220
                                               initial_indent=defindent,
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   221
                                               subsequent_indent=defindent))
9735
97d0d910fa5d minirst: remove unnecessary "elif:" statements
Martin Geisler <mg@lazybytes.net>
parents: 9623
diff changeset
   222
    initindent = subindent = indent
97d0d910fa5d minirst: remove unnecessary "elif:" statements
Martin Geisler <mg@lazybytes.net>
parents: 9623
diff changeset
   223
    if block['type'] == 'bullet':
9738
f52c4f7a4732 minirst: prepare for general types of bullet lists
Martin Geisler <mg@lazybytes.net>
parents: 9737
diff changeset
   224
        m = _bulletre.match(block['lines'][0])
f52c4f7a4732 minirst: prepare for general types of bullet lists
Martin Geisler <mg@lazybytes.net>
parents: 9737
diff changeset
   225
        if m:
f52c4f7a4732 minirst: prepare for general types of bullet lists
Martin Geisler <mg@lazybytes.net>
parents: 9737
diff changeset
   226
            subindent = indent + m.end() * ' '
9737
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   227
    elif block['type'] == 'field':
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   228
        m = _fieldre.match(block['lines'][0])
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   229
        if m:
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   230
            key, spaces, rest = m.groups()
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   231
            # Turn ":foo: bar" into "foo   bar".
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   232
            block['lines'][0] = '%s  %s%s' % (key, spaces, rest)
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   233
            subindent = indent + (2 + len(key) + len(spaces)) * ' '
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   234
    elif block['type'] == 'option':
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   235
        m = _optionre.match(block['lines'][0])
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   236
        if m:
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   237
            option, arg, rest = m.groups()
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   238
            subindent = indent + (len(option) + len(arg)) * ' '
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   239
9737
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   240
    text = ' '.join(map(str.strip, block['lines']))
9735
97d0d910fa5d minirst: remove unnecessary "elif:" statements
Martin Geisler <mg@lazybytes.net>
parents: 9623
diff changeset
   241
    return textwrap.fill(text, width=width,
97d0d910fa5d minirst: remove unnecessary "elif:" statements
Martin Geisler <mg@lazybytes.net>
parents: 9623
diff changeset
   242
                         initial_indent=initindent,
97d0d910fa5d minirst: remove unnecessary "elif:" statements
Martin Geisler <mg@lazybytes.net>
parents: 9623
diff changeset
   243
                         subsequent_indent=subindent)
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   244
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   245
9540
cad36e496640 help: un-indent help topics
Martin Geisler <mg@lazybytes.net>
parents: 9417
diff changeset
   246
def format(text, width, indent=0):
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   247
    """Parse and format the text according to width."""
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   248
    blocks = findblocks(text)
9540
cad36e496640 help: un-indent help topics
Martin Geisler <mg@lazybytes.net>
parents: 9417
diff changeset
   249
    for b in blocks:
cad36e496640 help: un-indent help topics
Martin Geisler <mg@lazybytes.net>
parents: 9417
diff changeset
   250
        b['indent'] += indent
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   251
    blocks = findliteralblocks(blocks)
9623
32727ce029de minirst: convert ``foo`` into "foo" upon display
Martin Geisler <mg@lazybytes.net>
parents: 9540
diff changeset
   252
    blocks = inlineliterals(blocks)
9737
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   253
    blocks = splitparagraphs(blocks)
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   254
    blocks = findsections(blocks)
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   255
    blocks = addmargins(blocks)
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   256
    return '\n'.join(formatblock(b, width) for b in blocks)
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   257
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   258
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   259
if __name__ == "__main__":
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   260
    from pprint import pprint
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   261
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   262
    def debug(func, blocks):
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   263
        blocks = func(blocks)
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   264
        print "*** after %s:" % func.__name__
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   265
        pprint(blocks)
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   266
        print
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   267
        return blocks
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   268
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   269
    text = open(sys.argv[1]).read()
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   270
    blocks = debug(findblocks, text)
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   271
    blocks = debug(findliteralblocks, blocks)
9737
5f101af4a921 minirst: combine list parsing in one function
Martin Geisler <mg@lazybytes.net>
parents: 9735
diff changeset
   272
    blocks = debug(splitparagraphs, blocks)
9156
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   273
    blocks = debug(findsections, blocks)
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   274
    blocks = debug(addmargins, blocks)
c9c7e8cdac9c minimal reStructuredText parser
Martin Geisler <mg@lazybytes.net>
parents:
diff changeset
   275
    print '\n'.join(formatblock(b, 30) for b in blocks)