tests/killdaemons.py
author Martin von Zweigbergk <martinvonz@google.com>
Mon, 23 Jan 2017 10:48:55 -0800
changeset 30866 5249b6470de9
parent 29811 4ddfb730789d
child 32677 f840b2621cce
permissions -rwxr-xr-x
verify: replace _validpath() by matcher The verifier calls out to _validpath() to check if it should verify that path and the narrowhg extension overrides _validpath() to tell the verifier to skip that path. In treemanifest repos, the verifier calls the same method to check if it should visit a directory. However, the decision to visit a directory is different from the condition that it's a matching path, and narrowhg was working around it by returning True from its _validpath() override if *either* was true. Similar to how one can do "hg files -I foo/bar/ -X foo/" (making the include pointless), narrowhg can be configured to track the same paths. In that case match("foo/bar/baz") would be false, but match.visitdir("foo/bar/baz") turns out to be true, causing verify to fail. This may seem like a bug in visitdir(), but it's explicitly documented to be undefined for subdirectories of excluded directories. When using treemanifests, the walk would not descend into foo/, so verification would pass. However, when using flat manifests, there is no recursive directory walk and the file path "foo/bar/baz" would be passed to _validpath() without "foo/" (actually without the slash) being passed first. As explained above, _validpath() would return true for the file path and "hg verify" would fail. Replacing the _validpath() method by a matcher seems like the obvious fix. Narrowhg can then pass in its own matcher and not have to conflate the two matching functions (for dirs and files). I think it also makes the code clearer.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
7344
58fd3c718ca4 tests: add killdaemons helper script
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     1
#!/usr/bin/env python
58fd3c718ca4 tests: add killdaemons helper script
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     2
28942
05cb9c6f310e py3: use absolute_import in killdaemons.py
Robert Stanca <robert.stanca7@gmail.com>
parents: 25473
diff changeset
     3
from __future__ import absolute_import
05cb9c6f310e py3: use absolute_import in killdaemons.py
Robert Stanca <robert.stanca7@gmail.com>
parents: 25473
diff changeset
     4
import errno
05cb9c6f310e py3: use absolute_import in killdaemons.py
Robert Stanca <robert.stanca7@gmail.com>
parents: 25473
diff changeset
     5
import os
05cb9c6f310e py3: use absolute_import in killdaemons.py
Robert Stanca <robert.stanca7@gmail.com>
parents: 25473
diff changeset
     6
import signal
05cb9c6f310e py3: use absolute_import in killdaemons.py
Robert Stanca <robert.stanca7@gmail.com>
parents: 25473
diff changeset
     7
import sys
05cb9c6f310e py3: use absolute_import in killdaemons.py
Robert Stanca <robert.stanca7@gmail.com>
parents: 25473
diff changeset
     8
import time
7344
58fd3c718ca4 tests: add killdaemons helper script
Matt Mackall <mpm@selenic.com>
parents:
diff changeset
     9
17465
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    10
if os.name =='nt':
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    11
    import ctypes
20493
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    12
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    13
    def _check(ret, expectederr=None):
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    14
        if ret == 0:
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    15
            winerrno = ctypes.GetLastError()
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    16
            if winerrno == expectederr:
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    17
                return True
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    18
            raise ctypes.WinError(winerrno)
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    19
17465
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    20
    def kill(pid, logfn, tryhard=True):
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    21
        logfn('# Killing daemon process %d' % pid)
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    22
        PROCESS_TERMINATE = 1
20496
acbd19b9fbe1 tests: killdaemons.py for windows distinguishes access violation and terminated
Simon Heimberg <simohe@besonet.ch>
parents: 20495
diff changeset
    23
        PROCESS_QUERY_INFORMATION = 0x400
20698
1147563faf62 killdaemons: drop superfluous L suffix from constant
Augie Fackler <raf@durin42.com>
parents: 20496
diff changeset
    24
        SYNCHRONIZE = 0x00100000
20494
ea3adeac5248 tests: killdaemons.py for windows waits for killed process to terminate
Simon Heimberg <simohe@besonet.ch>
parents: 20493
diff changeset
    25
        WAIT_OBJECT_0 = 0
ea3adeac5248 tests: killdaemons.py for windows waits for killed process to terminate
Simon Heimberg <simohe@besonet.ch>
parents: 20493
diff changeset
    26
        WAIT_TIMEOUT = 258
17465
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    27
        handle = ctypes.windll.kernel32.OpenProcess(
20496
acbd19b9fbe1 tests: killdaemons.py for windows distinguishes access violation and terminated
Simon Heimberg <simohe@besonet.ch>
parents: 20495
diff changeset
    28
                PROCESS_TERMINATE|SYNCHRONIZE|PROCESS_QUERY_INFORMATION,
acbd19b9fbe1 tests: killdaemons.py for windows distinguishes access violation and terminated
Simon Heimberg <simohe@besonet.ch>
parents: 20495
diff changeset
    29
                False, pid)
20493
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    30
        if handle == 0:
20495
0cf1c8c452d3 tests: killdaemons.py for checks reason when getting no process handle
Simon Heimberg <simohe@besonet.ch>
parents: 20494
diff changeset
    31
            _check(0, 87) # err 87 when process not found
20493
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    32
            return # process not found, already finished
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    33
        try:
20496
acbd19b9fbe1 tests: killdaemons.py for windows distinguishes access violation and terminated
Simon Heimberg <simohe@besonet.ch>
parents: 20495
diff changeset
    34
            r = ctypes.windll.kernel32.WaitForSingleObject(handle, 100)
acbd19b9fbe1 tests: killdaemons.py for windows distinguishes access violation and terminated
Simon Heimberg <simohe@besonet.ch>
parents: 20495
diff changeset
    35
            if r == WAIT_OBJECT_0:
acbd19b9fbe1 tests: killdaemons.py for windows distinguishes access violation and terminated
Simon Heimberg <simohe@besonet.ch>
parents: 20495
diff changeset
    36
                pass # terminated, but process handle still available
acbd19b9fbe1 tests: killdaemons.py for windows distinguishes access violation and terminated
Simon Heimberg <simohe@besonet.ch>
parents: 20495
diff changeset
    37
            elif r == WAIT_TIMEOUT:
acbd19b9fbe1 tests: killdaemons.py for windows distinguishes access violation and terminated
Simon Heimberg <simohe@besonet.ch>
parents: 20495
diff changeset
    38
                _check(ctypes.windll.kernel32.TerminateProcess(handle, -1))
acbd19b9fbe1 tests: killdaemons.py for windows distinguishes access violation and terminated
Simon Heimberg <simohe@besonet.ch>
parents: 20495
diff changeset
    39
            else:
acbd19b9fbe1 tests: killdaemons.py for windows distinguishes access violation and terminated
Simon Heimberg <simohe@besonet.ch>
parents: 20495
diff changeset
    40
                _check(r)
20494
ea3adeac5248 tests: killdaemons.py for windows waits for killed process to terminate
Simon Heimberg <simohe@besonet.ch>
parents: 20493
diff changeset
    41
ea3adeac5248 tests: killdaemons.py for windows waits for killed process to terminate
Simon Heimberg <simohe@besonet.ch>
parents: 20493
diff changeset
    42
            # TODO?: forcefully kill when timeout
ea3adeac5248 tests: killdaemons.py for windows waits for killed process to terminate
Simon Heimberg <simohe@besonet.ch>
parents: 20493
diff changeset
    43
            #        and ?shorter waiting time? when tryhard==True
ea3adeac5248 tests: killdaemons.py for windows waits for killed process to terminate
Simon Heimberg <simohe@besonet.ch>
parents: 20493
diff changeset
    44
            r = ctypes.windll.kernel32.WaitForSingleObject(handle, 100)
ea3adeac5248 tests: killdaemons.py for windows waits for killed process to terminate
Simon Heimberg <simohe@besonet.ch>
parents: 20493
diff changeset
    45
                                                       # timeout = 100 ms
ea3adeac5248 tests: killdaemons.py for windows waits for killed process to terminate
Simon Heimberg <simohe@besonet.ch>
parents: 20493
diff changeset
    46
            if r == WAIT_OBJECT_0:
ea3adeac5248 tests: killdaemons.py for windows waits for killed process to terminate
Simon Heimberg <simohe@besonet.ch>
parents: 20493
diff changeset
    47
                pass # process is terminated
ea3adeac5248 tests: killdaemons.py for windows waits for killed process to terminate
Simon Heimberg <simohe@besonet.ch>
parents: 20493
diff changeset
    48
            elif r == WAIT_TIMEOUT:
ea3adeac5248 tests: killdaemons.py for windows waits for killed process to terminate
Simon Heimberg <simohe@besonet.ch>
parents: 20493
diff changeset
    49
                logfn('# Daemon process %d is stuck')
ea3adeac5248 tests: killdaemons.py for windows waits for killed process to terminate
Simon Heimberg <simohe@besonet.ch>
parents: 20493
diff changeset
    50
            else:
21194
476069509e72 killdaemons: correct typo of _check() function caught by pyflakes
Yuya Nishihara <yuya@tcha.org>
parents: 20698
diff changeset
    51
                _check(r) # any error
20493
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    52
        except: #re-raises
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    53
            ctypes.windll.kernel32.CloseHandle(handle) # no _check, keep error
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    54
            raise
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    55
        _check(ctypes.windll.kernel32.CloseHandle(handle))
b5f43dbf64ca tests: kill for windows in killdaemons.py checks return values
Simon Heimberg <simohe@besonet.ch>
parents: 17466
diff changeset
    56
17465
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    57
else:
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    58
    def kill(pid, logfn, tryhard=True):
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    59
        try:
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    60
            os.kill(pid, 0)
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    61
            logfn('# Killing daemon process %d' % pid)
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    62
            os.kill(pid, signal.SIGTERM)
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    63
            if tryhard:
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    64
                for i in range(10):
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    65
                    time.sleep(0.05)
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    66
                    os.kill(pid, 0)
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    67
            else:
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    68
                time.sleep(0.1)
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    69
                os.kill(pid, 0)
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    70
            logfn('# Daemon process %d is stuck - really killing it' % pid)
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    71
            os.kill(pid, signal.SIGKILL)
25031
0adc22a0b6b3 python3: update killdaemons and run-tests print and exception syntax
Augie Fackler <augie@google.com>
parents: 21194
diff changeset
    72
        except OSError as err:
17465
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    73
            if err.errno != errno.ESRCH:
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    74
                raise
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    75
17464
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    76
def killdaemons(pidfile, tryhard=True, remove=False, logfn=None):
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    77
    if not logfn:
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    78
        logfn = lambda s: s
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    79
    # Kill off any leftover daemon processes
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    80
    try:
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    81
        fp = open(pidfile)
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    82
        for line in fp:
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    83
            try:
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    84
                pid = int(line)
29811
4ddfb730789d killdaemons: prevent killing all user processes by bad pid file
Yuya Nishihara <yuya@tcha.org>
parents: 28942
diff changeset
    85
                if pid <= 0:
4ddfb730789d killdaemons: prevent killing all user processes by bad pid file
Yuya Nishihara <yuya@tcha.org>
parents: 28942
diff changeset
    86
                    raise ValueError
17464
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    87
            except ValueError:
29811
4ddfb730789d killdaemons: prevent killing all user processes by bad pid file
Yuya Nishihara <yuya@tcha.org>
parents: 28942
diff changeset
    88
                logfn('# Not killing daemon process %s - invalid pid'
4ddfb730789d killdaemons: prevent killing all user processes by bad pid file
Yuya Nishihara <yuya@tcha.org>
parents: 28942
diff changeset
    89
                      % line.rstrip())
17464
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    90
                continue
17465
2d4a096e213c killdaemons: add windows implementation
Patrick Mezard <patrick@mezard.eu>
parents: 17464
diff changeset
    91
            kill(pid, logfn, tryhard)
17464
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    92
        fp.close()
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    93
        if remove:
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    94
            os.unlink(pidfile)
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    95
    except IOError:
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    96
        pass
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    97
eddfb9a550d0 run-tests: do not duplicate killdaemons() code
Patrick Mezard <patrick@mezard.eu>
parents: 10905
diff changeset
    98
if __name__ == '__main__':
25473
123c99034cb6 tests: make killdaemons.py use DAEMON_PIDS by default
Matt Mackall <mpm@selenic.com>
parents: 25031
diff changeset
    99
    if len(sys.argv) > 1:
123c99034cb6 tests: make killdaemons.py use DAEMON_PIDS by default
Matt Mackall <mpm@selenic.com>
parents: 25031
diff changeset
   100
        path, = sys.argv[1:]
123c99034cb6 tests: make killdaemons.py use DAEMON_PIDS by default
Matt Mackall <mpm@selenic.com>
parents: 25031
diff changeset
   101
    else:
123c99034cb6 tests: make killdaemons.py use DAEMON_PIDS by default
Matt Mackall <mpm@selenic.com>
parents: 25031
diff changeset
   102
        path = os.environ["DAEMON_PIDS"]
123c99034cb6 tests: make killdaemons.py use DAEMON_PIDS by default
Matt Mackall <mpm@selenic.com>
parents: 25031
diff changeset
   103
17466
d5a3bda6e170 killdaemons: take file argument explicitely
Patrick Mezard <patrick@mezard.eu>
parents: 17465
diff changeset
   104
    killdaemons(path)