tests/test-stdio.py
author Manuel Jacob <me@manueljacob.de>
Fri, 10 Jul 2020 12:27:58 +0200
changeset 45095 8e04607023e5
parent 45078 a59aab6078eb
child 45096 e9e452eafbfb
permissions -rwxr-xr-x
procutil: ensure that procutil.std{out,err}.write() writes all bytes Python 3 offers different kind of streams and it’s not guaranteed for all of them that calling write() writes all bytes. When Python is started in unbuffered mode, sys.std{out,err}.buffer are instances of io.FileIO, whose write() can write less bytes for platform-specific reasons (e.g. Linux has a 0x7ffff000 bytes maximum and could write less if interrupted by a signal; when writing to Windows consoles, it’s limited to 32767 bytes to avoid the "not enough space" error). This can lead to silent loss of data, both when using sys.std{out,err}.buffer (which may in fact not be a buffered stream) and when using the text streams sys.std{out,err} (I’ve created a CPython bug report for that: https://bugs.python.org/issue41221). Python may fix the problem at some point. For now, we implement our own wrapper for procutil.std{out,err} that calls the raw stream’s write() method until all bytes have been written. We don’t use sys.std{out,err} for larger writes, so I think it’s not worth the effort to patch them.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
     1
#!/usr/bin/env python
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
     2
"""
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
     3
Tests the buffering behavior of stdio streams in `mercurial.utils.procutil`.
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
     4
"""
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
     5
from __future__ import absolute_import
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
     6
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
     7
import contextlib
45068
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
     8
import errno
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
     9
import os
45095
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    10
import signal
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    11
import subprocess
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    12
import sys
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    13
import unittest
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    14
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    15
from mercurial import pycompat
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    16
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    17
45078
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
    18
TEST_BUFFERING_CHILD_SCRIPT = r'''
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    19
import os
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    20
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    21
from mercurial import dispatch
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    22
from mercurial.utils import procutil
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    23
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    24
dispatch.initstdio()
45044
359884685eab tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45043
diff changeset
    25
procutil.{stream}.write(b'aaa')
359884685eab tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45043
diff changeset
    26
os.write(procutil.{stream}.fileno(), b'[written aaa]')
359884685eab tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45043
diff changeset
    27
procutil.{stream}.write(b'bbb\n')
359884685eab tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45043
diff changeset
    28
os.write(procutil.{stream}.fileno(), b'[written bbb\\n]')
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    29
'''
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    30
UNBUFFERED = b'aaa[written aaa]bbb\n[written bbb\\n]'
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    31
LINE_BUFFERED = b'[written aaa]aaabbb\n[written bbb\\n]'
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    32
FULLY_BUFFERED = b'[written aaa][written bbb\\n]aaabbb\n'
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    33
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    34
45095
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    35
TEST_LARGE_WRITE_CHILD_SCRIPT = r'''
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    36
import signal
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    37
import sys
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    38
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    39
from mercurial import dispatch
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    40
from mercurial.utils import procutil
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    41
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    42
signal.signal(signal.SIGINT, lambda *x: None)
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    43
dispatch.initstdio()
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    44
procutil.{stream}.write(b'x' * 1048576)
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    45
'''
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    46
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    47
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    48
@contextlib.contextmanager
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    49
def _closing(fds):
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    50
    try:
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    51
        yield
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    52
    finally:
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    53
        for fd in fds:
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    54
            try:
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    55
                os.close(fd)
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    56
            except EnvironmentError:
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    57
                pass
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    58
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    59
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    60
@contextlib.contextmanager
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    61
def _pipes():
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    62
    rwpair = os.pipe()
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    63
    with _closing(rwpair):
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    64
        yield rwpair
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    65
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    66
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    67
@contextlib.contextmanager
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    68
def _ptys():
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    69
    if pycompat.iswindows:
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    70
        raise unittest.SkipTest("PTYs are not supported on Windows")
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    71
    import pty
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    72
    import tty
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    73
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    74
    rwpair = pty.openpty()
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    75
    with _closing(rwpair):
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    76
        tty.setraw(rwpair[0])
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    77
        yield rwpair
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    78
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
    79
45095
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    80
def _readall(fd, buffer_size, initial_buf=None):
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
    81
    buf = initial_buf or []
45068
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
    82
    while True:
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
    83
        try:
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
    84
            s = os.read(fd, buffer_size)
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
    85
        except OSError as e:
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
    86
            if e.errno == errno.EIO:
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
    87
                # If the child-facing PTY got closed, reading from the
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
    88
                # parent-facing PTY raises EIO.
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
    89
                break
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
    90
            raise
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
    91
        if not s:
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
    92
            break
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
    93
        buf.append(s)
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
    94
    return b''.join(buf)
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
    95
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
    96
45044
359884685eab tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45043
diff changeset
    97
class TestStdio(unittest.TestCase):
45078
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
    98
    def _test(
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
    99
        self,
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   100
        child_script,
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   101
        stream,
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   102
        rwpair_generator,
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   103
        check_output,
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   104
        python_args=[],
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   105
    ):
45044
359884685eab tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45043
diff changeset
   106
        assert stream in ('stdout', 'stderr')
359884685eab tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45043
diff changeset
   107
        with rwpair_generator() as (stream_receiver, child_stream), open(
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   108
            os.devnull, 'rb'
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   109
        ) as child_stdin:
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   110
            proc = subprocess.Popen(
45078
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   111
                [sys.executable] + python_args + ['-c', child_script],
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   112
                stdin=child_stdin,
45044
359884685eab tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45043
diff changeset
   113
                stdout=child_stream if stream == 'stdout' else None,
359884685eab tests: generalize common test case code in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45043
diff changeset
   114
                stderr=child_stream if stream == 'stderr' else None,
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   115
            )
45068
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
   116
            try:
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
   117
                os.close(child_stream)
45095
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   118
                check_output(stream_receiver, proc)
45069
9172fd511999 tests: terminate subprocess in test-stdio.py in case of exception
Manuel Jacob <me@manueljacob.de>
parents: 45068
diff changeset
   119
            except:  # re-raises
9172fd511999 tests: terminate subprocess in test-stdio.py in case of exception
Manuel Jacob <me@manueljacob.de>
parents: 45068
diff changeset
   120
                proc.terminate()
9172fd511999 tests: terminate subprocess in test-stdio.py in case of exception
Manuel Jacob <me@manueljacob.de>
parents: 45068
diff changeset
   121
                raise
45068
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
   122
            finally:
8cd18aba5e6c tests: proof test-stdio.py against buffer fill-up
Manuel Jacob <me@manueljacob.de>
parents: 45045
diff changeset
   123
                retcode = proc.wait()
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   124
            self.assertEqual(retcode, 0)
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   125
45078
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   126
    def _test_buffering(
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   127
        self, stream, rwpair_generator, expected_output, python_args=[]
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   128
    ):
45095
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   129
        def check_output(stream_receiver, proc):
45078
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   130
            self.assertEqual(_readall(stream_receiver, 1024), expected_output)
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   131
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   132
        self._test(
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   133
            TEST_BUFFERING_CHILD_SCRIPT.format(stream=stream),
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   134
            stream,
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   135
            rwpair_generator,
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   136
            check_output,
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   137
            python_args,
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   138
        )
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   139
45070
bc05c13e246f tests: make names in test-stdio.py more distinctive
Manuel Jacob <me@manueljacob.de>
parents: 45069
diff changeset
   140
    def test_buffering_stdout_pipes(self):
45078
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   141
        self._test_buffering('stdout', _pipes, FULLY_BUFFERED)
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   142
45070
bc05c13e246f tests: make names in test-stdio.py more distinctive
Manuel Jacob <me@manueljacob.de>
parents: 45069
diff changeset
   143
    def test_buffering_stdout_ptys(self):
45078
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   144
        self._test_buffering('stdout', _ptys, LINE_BUFFERED)
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   145
45070
bc05c13e246f tests: make names in test-stdio.py more distinctive
Manuel Jacob <me@manueljacob.de>
parents: 45069
diff changeset
   146
    def test_buffering_stdout_pipes_unbuffered(self):
45078
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   147
        self._test_buffering('stdout', _pipes, UNBUFFERED, python_args=['-u'])
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   148
45070
bc05c13e246f tests: make names in test-stdio.py more distinctive
Manuel Jacob <me@manueljacob.de>
parents: 45069
diff changeset
   149
    def test_buffering_stdout_ptys_unbuffered(self):
45078
a59aab6078eb tests: make subprocess handling reusable for different tests in test-stdio.py
Manuel Jacob <me@manueljacob.de>
parents: 45077
diff changeset
   150
        self._test_buffering('stdout', _ptys, UNBUFFERED, python_args=['-u'])
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   151
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   152
    if not pycompat.ispy3 and not pycompat.iswindows:
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   153
        # On Python 2 on non-Windows, we manually open stdout in line-buffered
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   154
        # mode if connected to a TTY. We should check if Python was configured
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   155
        # to use unbuffered stdout, but it's hard to do that.
45070
bc05c13e246f tests: make names in test-stdio.py more distinctive
Manuel Jacob <me@manueljacob.de>
parents: 45069
diff changeset
   156
        test_buffering_stdout_ptys_unbuffered = unittest.expectedFailure(
bc05c13e246f tests: make names in test-stdio.py more distinctive
Manuel Jacob <me@manueljacob.de>
parents: 45069
diff changeset
   157
            test_buffering_stdout_ptys_unbuffered
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   158
        )
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   159
45095
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   160
    def _test_large_write(self, stream, rwpair_generator, python_args=[]):
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   161
        if not pycompat.ispy3 and pycompat.isdarwin:
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   162
            # Python 2 doesn't always retry on EINTR, but the libc might retry.
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   163
            # So far, it was observed only on macOS that EINTR is raised at the
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   164
            # Python level. As Python 2 support will be dropped soon-ish, we
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   165
            # won't attempt to fix it.
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   166
            raise unittest.SkipTest("raises EINTR on macOS")
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   167
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   168
        def check_output(stream_receiver, proc):
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   169
            if not pycompat.iswindows:
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   170
                # On Unix, we can provoke a partial write() by interrupting it
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   171
                # by a signal handler as soon as a bit of data was written.
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   172
                # We test that write() is called until all data is written.
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   173
                buf = [os.read(stream_receiver, 1)]
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   174
                proc.send_signal(signal.SIGINT)
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   175
            else:
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   176
                # On Windows, there doesn't seem to be a way to cause partial
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   177
                # writes.
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   178
                buf = []
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   179
            self.assertEqual(
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   180
                _readall(stream_receiver, 131072, buf), b'x' * 1048576
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   181
            )
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   182
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   183
        self._test(
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   184
            TEST_LARGE_WRITE_CHILD_SCRIPT.format(stream=stream),
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   185
            stream,
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   186
            rwpair_generator,
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   187
            check_output,
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   188
            python_args,
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   189
        )
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   190
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   191
    def test_large_write_stdout_pipes(self):
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   192
        self._test_large_write('stdout', _pipes)
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   193
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   194
    def test_large_write_stdout_ptys(self):
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   195
        self._test_large_write('stdout', _ptys)
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   196
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   197
    def test_large_write_stdout_pipes_unbuffered(self):
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   198
        self._test_large_write('stdout', _pipes, python_args=['-u'])
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   199
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   200
    def test_large_write_stdout_ptys_unbuffered(self):
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   201
        self._test_large_write('stdout', _ptys, python_args=['-u'])
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   202
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   203
    def test_large_write_stderr_pipes(self):
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   204
        self._test_large_write('stderr', _pipes)
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   205
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   206
    def test_large_write_stderr_ptys(self):
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   207
        self._test_large_write('stderr', _ptys)
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   208
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   209
    def test_large_write_stderr_pipes_unbuffered(self):
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   210
        self._test_large_write('stderr', _pipes, python_args=['-u'])
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   211
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   212
    def test_large_write_stderr_ptys_unbuffered(self):
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   213
        self._test_large_write('stderr', _ptys, python_args=['-u'])
8e04607023e5 procutil: ensure that procutil.std{out,err}.write() writes all bytes
Manuel Jacob <me@manueljacob.de>
parents: 45078
diff changeset
   214
45038
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   215
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   216
if __name__ == '__main__':
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   217
    import silenttestrunner
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   218
c7d109c400a4 tests: add tests for buffering behavior of mercurial.utils.procutil.stdout
Manuel Jacob <me@manueljacob.de>
parents:
diff changeset
   219
    silenttestrunner.main(__name__)