hgext/logtoprocess.py
author Simon Sapin <simon.sapin@octobus.net>
Fri, 21 Jan 2022 17:54:03 +0100
changeset 48745 94e36b230990
parent 44417 7c0b8652fd8c
child 48875 6000f5b25c9b
permissions -rw-r--r--
status: prefer relative paths in Rust code … when the repository root is under the current directory, so the kernel needs to traverse fewer directory in every call to `read_dir` or `symlink_metadata`. Better yet would be to use libc functions like `openat` and `fstatat` to remove such repeated traversals entirely, but the standard library does not provide APIs based on those. Maybe with a crate like https://crates.io/crates/openat instead? Benchmarks of `rhg status` show that this patch is neutral in some configurations, and makes the command up to ~20% faster in others. Below is semi-arbitrary subset of results. The four numeric columns are: time (in seconds) with this changeset’s parent, time with this changeset, time difference (negative is better), time ratio (less than 1 is better). ``` mercurial-dirstate-v1 | default-plain-clean.no-iu.pbr | 0.0061 -> 0.0059: -0.0002 (0.97) mercurial-dirstate-v2 | default-plain-clean.no-iu.pbr | 0.0029 -> 0.0028: -0.0001 (0.97) mozilla-dirstate-v1 | default-plain-clean.no-iu.pbr | 0.2110 -> 0.2102: -0.0007 (1.00) mozilla-dirstate-v2 | default-copies-clean.ignored.pbr | 0.0489 -> 0.0401: -0.0088 (0.82) mozilla-dirstate-v2 | default-copies-clean.no-iu.pbr | 0.0479 -> 0.0393: -0.0085 (0.82) mozilla-dirstate-v2 | default-copies-large.all.pbr | 0.1262 -> 0.1210: -0.0051 (0.96) mozilla-dirstate-v2 | default-copies-small.ignored-unknown.pbr | 0.1262 -> 0.1200: -0.0062 (0.95) mozilla-dirstate-v2 | default-copies-small.ignored.pbr | 0.0536 -> 0.0417: -0.0119 (0.78) mozilla-dirstate-v2 | default-copies-small.no-iu.pbr | 0.0482 -> 0.0393: -0.0089 (0.81) mozilla-dirstate-v2 | default-plain-clean.ignored.pbr | 0.0518 -> 0.0402: -0.0116 (0.78) mozilla-dirstate-v2 | default-plain-clean.no-iu.pbr | 0.0481 -> 0.0392: -0.0088 (0.82) mozilla-dirstate-v2 | default-plain-large.all.pbr | 0.1271 -> 0.1218: -0.0052 (0.96) mozilla-dirstate-v2 | default-plain-small.ignored-unknown.pbr | 0.1225 -> 0.1202: -0.0022 (0.98) mozilla-dirstate-v2 | default-plain-small.ignored.pbr | 0.0510 -> 0.0418: -0.0092 (0.82) mozilla-dirstate-v2 | default-plain-small.no-iu.pbr | 0.0480 -> 0.0394: -0.0086 (0.82) netbeans-dirstate-v1 | default-plain-clean.no-iu.pbr | 0.1442 -> 0.1422: -0.0020 (0.99) netbeans-dirstate-v2 | default-plain-clean.no-iu.pbr | 0.0325 -> 0.0282: -0.0043 (0.87) ``` Differential Revision: https://phab.mercurial-scm.org/D12175
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     1
# logtoprocess.py - send ui.log() data to a subprocess
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     2
#
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     3
# Copyright 2016 Facebook, Inc.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     4
#
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
31601
8e7feaad2d8d logtoprocess: use lowercase for docstring title
Jun Wu <quark@fb.com>
parents: 30638
diff changeset
     7
"""send ui.log() data to a subprocess (EXPERIMENTAL)
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     8
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
     9
This extension lets you specify a shell command per ui.log() event,
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    10
sending all remaining arguments to as environment variables to that command.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    11
40622
d2c997b8001f logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 40620
diff changeset
    12
Positional arguments construct a log message, which is passed in the `MSG1`
d2c997b8001f logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 40620
diff changeset
    13
environment variables. Each keyword argument is set as a `OPT_UPPERCASE_KEY`
d2c997b8001f logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 40620
diff changeset
    14
variable (so the key is uppercased, and prefixed with `OPT_`). The original
d2c997b8001f logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 40620
diff changeset
    15
event name is passed in the `EVENT` environment variable, and the process ID
d2c997b8001f logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 40620
diff changeset
    16
of mercurial is given in `HGPID`.
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    17
40622
d2c997b8001f logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 40620
diff changeset
    18
So given a call `ui.log('foo', 'bar %s\n', 'baz', spam='eggs'), a script
d2c997b8001f logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 40620
diff changeset
    19
configured for the `foo` event can expect an environment with `MSG1=bar baz`,
d2c997b8001f logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 40620
diff changeset
    20
and `OPT_SPAM=eggs`.
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    21
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    22
Scripts are configured in the `[logtoprocess]` section, each key an event name.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    23
For example::
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    24
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    25
  [logtoprocess]
40622
d2c997b8001f logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)
Yuya Nishihara <yuya@tcha.org>
parents: 40620
diff changeset
    26
  commandexception = echo "$MSG1" > /var/log/mercurial_exceptions.log
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    27
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    28
would log the warning message and traceback of any failed command dispatch.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    29
30332
318a24b52eeb spelling: fixes of non-dictionary words
Mads Kiilerich <madski@unity3d.com>
parents: 29841
diff changeset
    30
Scripts are run asynchronously as detached daemon processes; mercurial will
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    31
not ensure that they exit cleanly.
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    32
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    33
"""
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    34
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    35
from __future__ import absolute_import
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    36
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    37
import os
30638
1c5cbf28f007 py3: replace os.environ with encoding.environ (part 5 of 5)
Pulkit Goyal <7895pulkit@gmail.com>
parents: 30332
diff changeset
    38
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40761
diff changeset
    39
from mercurial.utils import procutil
39826
c31ce080eb75 py3: convert arguments, cwd and env to native strings when spawning subprocess
Matt Harbison <matt_harbison@yahoo.com>
parents: 35063
diff changeset
    40
29841
d5883fd055c6 extensions: change magic "shipped with hg" string
Augie Fackler <augie@google.com>
parents: 29463
diff changeset
    41
# Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    42
# extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    43
# be specifying the version(s) of Mercurial they are tested with, or
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    44
# leave the attribute unspecified.
43077
687b865b95ad formatting: byteify all mercurial/ and hgext/ string literals
Augie Fackler <augie@google.com>
parents: 43076
diff changeset
    45
testedwith = b'ships-with-hg-core'
28901
a368da441b32 logtoprocess: new experimental extension
Martijn Pieters <mjpieters@fb.com>
parents:
diff changeset
    46
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40761
diff changeset
    47
40677
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    48
class processlogger(object):
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    49
    """Map log events to external commands
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    50
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    51
    Arguments are passed on as environment variables.
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    52
    """
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    53
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    54
    def __init__(self, ui):
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    55
        self._scripts = dict(ui.configitems(b'logtoprocess'))
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    56
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    57
    def tracked(self, event):
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    58
        return bool(self._scripts.get(event))
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    59
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    60
    def log(self, ui, event, msg, opts):
40730
55b053af7196 ui: manage logger instances and event filtering by core ui
Yuya Nishihara <yuya@tcha.org>
parents: 40677
diff changeset
    61
        script = self._scripts[event]
44417
7c0b8652fd8c logtoprocess: avoid traceback when running long commands
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 43077
diff changeset
    62
        maxmsg = 100000
7c0b8652fd8c logtoprocess: avoid traceback when running long commands
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 43077
diff changeset
    63
        if len(msg) > maxmsg:
7c0b8652fd8c logtoprocess: avoid traceback when running long commands
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 43077
diff changeset
    64
            # Each env var has a 128KiB limit on linux. msg can be long, in
7c0b8652fd8c logtoprocess: avoid traceback when running long commands
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 43077
diff changeset
    65
            # particular for command event, where it's the full command line.
7c0b8652fd8c logtoprocess: avoid traceback when running long commands
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 43077
diff changeset
    66
            # Prefer truncating the message than raising "Argument list too
7c0b8652fd8c logtoprocess: avoid traceback when running long commands
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 43077
diff changeset
    67
            # long" error.
7c0b8652fd8c logtoprocess: avoid traceback when running long commands
Valentin Gatien-Baron <vgatien-baron@janestreet.com>
parents: 43077
diff changeset
    68
            msg = msg[:maxmsg] + b' (truncated)'
40677
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    69
        env = {
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    70
            b'EVENT': event,
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    71
            b'HGPID': os.getpid(),
40760
ffd574c144d2 ui: pass in formatted message to logger.log()
Yuya Nishihara <yuya@tcha.org>
parents: 40730
diff changeset
    72
            b'MSG1': msg,
40677
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    73
        }
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    74
        # keyword arguments get prefixed with OPT_ and uppercased
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40761
diff changeset
    75
        env.update(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40761
diff changeset
    76
            (b'OPT_%s' % key.upper(), value) for key, value in opts.items()
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40761
diff changeset
    77
        )
40677
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    78
        fullenv = procutil.shellenviron(env)
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    79
        procutil.runbgcommand(script, fullenv, shell=True)
2b859742ea15 logtoprocess: extract logger class from ui wrapper
Yuya Nishihara <yuya@tcha.org>
parents: 40622
diff changeset
    80
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 40761
diff changeset
    81
40730
55b053af7196 ui: manage logger instances and event filtering by core ui
Yuya Nishihara <yuya@tcha.org>
parents: 40677
diff changeset
    82
def uipopulate(ui):
55b053af7196 ui: manage logger instances and event filtering by core ui
Yuya Nishihara <yuya@tcha.org>
parents: 40677
diff changeset
    83
    ui.setlogger(b'logtoprocess', processlogger(ui))