mercurial/minifileset.py
author Boris Feld <boris.feld@octobus.net>
Wed, 17 Jan 2018 17:07:55 +0100
changeset 35726 45b678bf3a78
parent 35691 735f47b41521
child 35740 06a757b9e334
permissions -rw-r--r--
atomicupdate: add an experimental option to use atomictemp when updating In some cases Mercurial truncating files when updating causes problems. It can happens when processes are currently reading the file or with big file or on NFS mounts. We add an experimental option to use the atomictemp option of vfs.__call__ in order to avoid the problem. The localrepository.wwrite seems to assume the files are created without the `x` flag; with atomictempfile, the new file might inherit the `x` flag from the destination. We force remove it afterward. This code could be refactored and the flag processing could be moved inside vfs. This patch should be tested with `--extra-config-opt experimental.update.atomic-file=True` as we disabled the option by default. Differential Revision: https://phab.mercurial-scm.org/D1882
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
     1
# minifileset.py - a simple language to select files
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
     2
#
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
     3
# Copyright 2017 Facebook, Inc.
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
     4
#
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
     7
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
     8
from __future__ import absolute_import
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
     9
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    10
from .i18n import _
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    11
from . import (
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    12
    error,
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    13
    fileset,
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    14
)
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    15
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    16
def _compile(tree):
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    17
    if not tree:
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    18
        raise error.ParseError(_("missing argument"))
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    19
    op = tree[0]
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    20
    if op == 'symbol':
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    21
        name = fileset.getstring(tree, _('invalid file pattern'))
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    22
        if name.startswith('**'): # file extension test, ex. "**.tar.gz"
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    23
            ext = name[2:]
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    24
            for c in ext:
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    25
                if c in '*{}[]?/\\':
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    26
                    raise error.ParseError(_('reserved character: %s') % c)
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    27
            return lambda n, s: n.endswith(ext)
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    28
        raise error.ParseError(_('invalid symbol: %s') % name)
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    29
    elif op == 'string':
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    30
        # TODO: teach fileset about 'path:', so that this can be a symbol and
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    31
        # not require quoting.
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    32
        name = fileset.getstring(tree, _('invalid path literal'))
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    33
        if name.startswith('path:'): # directory or full path test
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    34
            p = name[5:] # prefix
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    35
            pl = len(p)
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    36
            f = lambda n, s: n.startswith(p) and (len(n) == pl or n[pl] == '/')
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    37
            return f
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    38
        raise error.ParseError(_("invalid string"),
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    39
                               hint=_('paths must be prefixed with "path:"'))
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    40
    elif op == 'or':
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    41
        func1 = _compile(tree[1])
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    42
        func2 = _compile(tree[2])
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    43
        return lambda n, s: func1(n, s) or func2(n, s)
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    44
    elif op == 'and':
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    45
        func1 = _compile(tree[1])
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    46
        func2 = _compile(tree[2])
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    47
        return lambda n, s: func1(n, s) and func2(n, s)
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    48
    elif op == 'not':
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    49
        return lambda n, s: not _compile(tree[1])(n, s)
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    50
    elif op == 'group':
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    51
        return _compile(tree[1])
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    52
    elif op == 'func':
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    53
        symbols = {
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    54
            'all': lambda n, s: True,
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    55
            'none': lambda n, s: False,
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    56
            'size': lambda n, s: fileset.sizematcher(tree[2])(s),
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    57
        }
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    58
35691
735f47b41521 fileset: make it robust for bad function calls
Yuya Nishihara <yuya@tcha.org>
parents: 35616
diff changeset
    59
        name = fileset.getsymbol(tree[1])
735f47b41521 fileset: make it robust for bad function calls
Yuya Nishihara <yuya@tcha.org>
parents: 35616
diff changeset
    60
        if name in symbols:
35616
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    61
            return symbols[name]
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    62
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    63
        raise error.UnknownIdentifier(name, symbols.keys())
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    64
    elif op == 'minus':     # equivalent to 'x and not y'
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    65
        func1 = _compile(tree[1])
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    66
        func2 = _compile(tree[2])
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    67
        return lambda n, s: func1(n, s) and not func2(n, s)
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    68
    elif op == 'negate':
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    69
        raise error.ParseError(_("can't use negate operator in this context"))
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    70
    elif op == 'list':
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    71
        raise error.ParseError(_("can't use a list in this context"),
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    72
                               hint=_('see hg help "filesets.x or y"'))
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    73
    raise error.ProgrammingError('illegal tree: %r' % (tree,))
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    74
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    75
def compile(text):
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    76
    """generate a function (path, size) -> bool from filter specification.
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    77
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    78
    "text" could contain the operators defined by the fileset language for
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    79
    common logic operations, and parenthesis for grouping.  The supported path
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    80
    tests are '**.extname' for file extension test, and '"path:dir/subdir"'
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    81
    for prefix test.  The ``size()`` predicate is borrowed from filesets to test
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    82
    file size.  The predicates ``all()`` and ``none()`` are also supported.
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    83
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    84
    '(**.php & size(">10MB")) | **.zip | ("path:bin" & !"path:bin/README")' for
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    85
    example, will catch all php files whose size is greater than 10 MB, all
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    86
    files whose name ends with ".zip", and all files under "bin" in the repo
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    87
    root except for "bin/README".
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    88
    """
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    89
    tree = fileset.parse(text)
706aa203b396 fileset: add a lightweight file filtering language
Matt Harbison <matt_harbison@yahoo.com>
parents:
diff changeset
    90
    return _compile(tree)