tests/test-run-tests.t
author Pierre-Yves David <pierre-yves.david@fb.com>
Thu, 07 May 2015 23:40:29 -0700
changeset 25098 bf84ab53c2fd
parent 25097 a4fce7905721
child 25388 6025cac3d02f
permissions -rw-r--r--
run-tests: include 'start' and 'end' in --time output This is useful information to understand what is taking time in tests. Both are included because I can see myself sorting this output using shell script. Having both data makes it much easier than extracting 'start'+'real'.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     1
This file tests the behavior of run-tests.py itself.
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     2
24960
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     3
Avoid interference from actual test env:
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     4
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     5
  $ unset HGTEST_JOBS
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     6
  $ unset HGTEST_TIMEOUT
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     7
  $ unset HGTEST_PORT
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     8
  $ unset HGTEST_SHELL
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     9
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    10
Smoke test
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    11
============
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    12
23935
d64dd1252386 run-tests.py: inherit --pure option from outer run-tests.py execution
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 23728
diff changeset
    13
  $ $TESTDIR/run-tests.py $HGTEST_RUN_TESTS_PURE
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    14
  
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    15
  # Ran 0 tests, 0 skipped, 0 warned, 0 failed.
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    16
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    17
a succesful test
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    18
=======================
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    19
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    20
  $ cat > test-success.t << EOF
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    21
  >   $ echo babar
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    22
  >   babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    23
  >   $ echo xyzzy
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    24
  >   xyzzy
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    25
  > EOF
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    26
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    27
  $ $TESTDIR/run-tests.py --with-hg=`which hg`
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    28
  .
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    29
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    30
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    31
failing test
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    32
==================
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    33
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    34
  $ cat > test-failure.t << EOF
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    35
  >   $ echo babar
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    36
  >   rataxes
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    37
  > This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    38
  > this test is still more bytes than success.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    39
  > EOF
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    40
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    41
  >>> fh = open('test-failure-unicode.t', 'wb')
25054
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
    42
  >>> fh.write(u'  $ echo babar\u03b1\n'.encode('utf-8')) and None
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
    43
  >>> fh.write(u'  l\u03b5\u03b5t\n'.encode('utf-8')) and None
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    44
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    45
  $ $TESTDIR/run-tests.py --with-hg=`which hg`
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    46
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
    47
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
    48
  +++ $TESTTMP/test-failure.t.err
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    49
  @@ -1,4 +1,4 @@
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    50
     $ echo babar
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    51
  -  rataxes
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    52
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    53
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    54
   this test is still more bytes than success.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    55
  
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    56
  ERROR: test-failure.t output changed
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    57
  !.
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    58
  --- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    59
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    60
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    61
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    62
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    63
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    64
  
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    65
  ERROR: test-failure-unicode.t output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    66
  !
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    67
  Failed test-failure.t: output changed
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    68
  Failed test-failure-unicode.t: output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    69
  # Ran 3 tests, 0 skipped, 0 warned, 2 failed.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    70
  python hash seed: * (glob)
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    71
  [1]
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    72
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    73
test --xunit support
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    74
  $ $TESTDIR/run-tests.py --with-hg=`which hg` --xunit=xunit.xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    75
  
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    76
  --- $TESTTMP/test-failure.t
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    77
  +++ $TESTTMP/test-failure.t.err
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    78
  @@ -1,4 +1,4 @@
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    79
     $ echo babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    80
  -  rataxes
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    81
  +  babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    82
   This is a noop statement so that
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    83
   this test is still more bytes than success.
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    84
  
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    85
  ERROR: test-failure.t output changed
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    86
  !.
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    87
  --- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    88
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    89
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    90
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    91
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    92
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    93
  
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    94
  ERROR: test-failure-unicode.t output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    95
  !
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    96
  Failed test-failure.t: output changed
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    97
  Failed test-failure-unicode.t: output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
    98
  # Ran 3 tests, 0 skipped, 0 warned, 2 failed.
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
    99
  python hash seed: * (glob)
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   100
  [1]
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   101
  $ cat xunit.xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   102
  <?xml version="1.0" encoding="utf-8"?>
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   103
  <testsuite errors="0" failures="2" name="run-tests" skipped="0" tests="3">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   104
    <testcase name="test-success.t" time="*"/> (glob)
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   105
    <testcase name="test-failure-unicode.t" time="*"> (glob)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   106
  <![CDATA[--- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   107
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   108
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   109
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   110
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   111
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   112
  ]]>  </testcase>
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   113
    <testcase name="test-failure.t" time="*"> (glob)
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   114
  <![CDATA[--- $TESTTMP/test-failure.t
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   115
  +++ $TESTTMP/test-failure.t.err
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   116
  @@ -1,4 +1,4 @@
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   117
     $ echo babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   118
  -  rataxes
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   119
  +  babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   120
   This is a noop statement so that
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   121
   this test is still more bytes than success.
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   122
  ]]>  </testcase>
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   123
  </testsuite>
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   124
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   125
  $ rm test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   126
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   127
test for --retest
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   128
====================
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   129
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   130
  $ $TESTDIR/run-tests.py --with-hg=`which hg` --retest
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   131
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   132
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   133
  +++ $TESTTMP/test-failure.t.err
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   134
  @@ -1,4 +1,4 @@
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   135
     $ echo babar
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   136
  -  rataxes
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   137
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   138
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   139
   this test is still more bytes than success.
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   140
  
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   141
  ERROR: test-failure.t output changed
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   142
  !
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   143
  Failed test-failure.t: output changed
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   144
  # Ran 2 tests, 1 skipped, 0 warned, 1 failed.
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   145
  python hash seed: * (glob)
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   146
  [1]
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   147
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   148
Selecting Tests To Run
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   149
======================
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   150
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   151
successful
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   152
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   153
  $ $TESTDIR/run-tests.py --with-hg=`which hg` test-success.t
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   154
  .
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   155
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   156
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   157
success w/ keyword
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   158
  $ $TESTDIR/run-tests.py --with-hg=`which hg` -k xyzzy
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   159
  .
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   160
  # Ran 2 tests, 1 skipped, 0 warned, 0 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   161
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   162
failed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   163
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   164
  $ $TESTDIR/run-tests.py --with-hg=`which hg` test-failure.t
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   165
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   166
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   167
  +++ $TESTTMP/test-failure.t.err
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   168
  @@ -1,4 +1,4 @@
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   169
     $ echo babar
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   170
  -  rataxes
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   171
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   172
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   173
   this test is still more bytes than success.
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   174
  
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   175
  ERROR: test-failure.t output changed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   176
  !
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   177
  Failed test-failure.t: output changed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   178
  # Ran 1 tests, 0 skipped, 0 warned, 1 failed.
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   179
  python hash seed: * (glob)
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   180
  [1]
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   181
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   182
failure w/ keyword
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   183
  $ $TESTDIR/run-tests.py --with-hg=`which hg` -k rataxes
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   184
  
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   185
  --- $TESTTMP/test-failure.t
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   186
  +++ $TESTTMP/test-failure.t.err
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   187
  @@ -1,4 +1,4 @@
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   188
     $ echo babar
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   189
  -  rataxes
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   190
  +  babar
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   191
   This is a noop statement so that
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   192
   this test is still more bytes than success.
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   193
  
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   194
  ERROR: test-failure.t output changed
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   195
  !
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   196
  Failed test-failure.t: output changed
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   197
  # Ran 2 tests, 1 skipped, 0 warned, 1 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   198
  python hash seed: * (glob)
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   199
  [1]
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   200
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   201
Verify that when a process fails to start we show a useful message
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   202
==================================================================
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   203
NOTE: there is currently a bug where this shows "2 failed" even though
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   204
it's actually the same test being reported for failure twice.
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   205
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   206
  $ cat > test-serve-fail.t <<EOF
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   207
  >   $ echo 'abort: child process failed to start blah'
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   208
  > EOF
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   209
  $ $TESTDIR/run-tests.py --with-hg=`which hg` test-serve-fail.t
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   210
  
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   211
  ERROR: test-serve-fail.t output changed
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   212
  !
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   213
  ERROR: test-serve-fail.t output changed
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   214
  !
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   215
  Failed test-serve-fail.t: server failed to start (HGPORT=*) (glob)
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   216
  Failed test-serve-fail.t: output changed
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   217
  # Ran 1 tests, 0 skipped, 0 warned, 2 failed.
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   218
  python hash seed: * (glob)
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   219
  [1]
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   220
  $ rm test-serve-fail.t
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   221
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   222
Running In Debug Mode
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   223
======================
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   224
22134
90c8b36d1a2e test-run-tests: fix stdout/stderr io ordering
Matt Mackall <mpm@selenic.com>
parents: 22121
diff changeset
   225
  $ $TESTDIR/run-tests.py --with-hg=`which hg` --debug 2>&1 | grep -v pwd
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   226
  + echo *SALT* 0 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   227
  *SALT* 0 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   228
  + echo babar
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   229
  babar
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   230
  + echo *SALT* 4 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   231
  *SALT* 4 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   232
  .+ echo *SALT* 0 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   233
  *SALT* 0 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   234
  + echo babar
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   235
  babar
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   236
  + echo *SALT* 2 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   237
  *SALT* 2 0 (glob)
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   238
  + echo xyzzy
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   239
  xyzzy
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   240
  + echo *SALT* 4 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   241
  *SALT* 4 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   242
  .
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   243
  # Ran 2 tests, 0 skipped, 0 warned, 0 failed.
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   244
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   245
Parallel runs
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   246
==============
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   247
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   248
(duplicate the failing test to get predictable output)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   249
  $ cp test-failure.t test-failure-copy.t
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   250
23107
5459b30aa498 tests: silence output race in test-run-tests.t
Matt Mackall <mpm@selenic.com>
parents: 22840
diff changeset
   251
  $ $TESTDIR/run-tests.py --with-hg=`which hg` --jobs 2 test-failure*.t -n
5459b30aa498 tests: silence output race in test-run-tests.t
Matt Mackall <mpm@selenic.com>
parents: 22840
diff changeset
   252
  !!
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   253
  Failed test-failure*.t: output changed (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   254
  Failed test-failure*.t: output changed (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   255
  # Ran 2 tests, 0 skipped, 0 warned, 2 failed.
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   256
  python hash seed: * (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   257
  [1]
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   258
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   259
failures in parallel with --first should only print one failure
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   260
  >>> f = open('test-nothing.t', 'w')
25054
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   261
  >>> f.write('foo\n' * 1024) and None
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   262
  >>> f.write('  $ sleep 1') and None
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   263
  $ $TESTDIR/run-tests.py --with-hg=`which hg` --jobs 2 --first
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   264
  
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   265
  --- $TESTTMP/test-failure*.t (glob)
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   266
  +++ $TESTTMP/test-failure*.t.err (glob)
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   267
  @@ -1,4 +1,4 @@
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   268
     $ echo babar
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   269
  -  rataxes
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   270
  +  babar
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   271
   This is a noop statement so that
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   272
   this test is still more bytes than success.
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   273
  
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   274
  Failed test-failure*.t: output changed (glob)
24507
a0668a587c04 run-tests: wait for test threads after first error
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
   275
  Failed test-nothing.t: output changed
a0668a587c04 run-tests: wait for test threads after first error
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
   276
  # Ran 2 tests, 0 skipped, 0 warned, 2 failed.
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   277
  python hash seed: * (glob)
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   278
  [1]
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   279
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   280
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   281
(delete the duplicated test file)
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   282
  $ rm test-failure-copy.t test-nothing.t
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   283
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   284
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   285
Interactive run
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   286
===============
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   287
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   288
(backup the failing test)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   289
  $ cp test-failure.t backup
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   290
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   291
Refuse the fix
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   292
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   293
  $ echo 'n' | $TESTDIR/run-tests.py --with-hg=`which hg` -i
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   294
  
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   295
  --- $TESTTMP/test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   296
  +++ $TESTTMP/test-failure.t.err
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   297
  @@ -1,4 +1,4 @@
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   298
     $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   299
  -  rataxes
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   300
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   301
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   302
   this test is still more bytes than success.
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21756
diff changeset
   303
  Accept this change? [n] 
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   304
  ERROR: test-failure.t output changed
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21756
diff changeset
   305
  !.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   306
  Failed test-failure.t: output changed
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   307
  # Ran 2 tests, 0 skipped, 0 warned, 1 failed.
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   308
  python hash seed: * (glob)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   309
  [1]
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   310
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   311
  $ cat test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   312
    $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   313
    rataxes
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   314
  This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   315
  this test is still more bytes than success.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   316
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   317
Interactive with custom view
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   318
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   319
  $ echo 'n' | $TESTDIR/run-tests.py --with-hg=`which hg` -i --view echo
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   320
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err (glob)
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   321
  Accept this change? [n]* (glob)
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   322
  ERROR: test-failure.t output changed
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   323
  !.
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   324
  Failed test-failure.t: output changed
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   325
  # Ran 2 tests, 0 skipped, 0 warned, 1 failed.
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   326
  python hash seed: * (glob)
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   327
  [1]
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   328
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   329
View the fix
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   330
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   331
  $ echo 'y' | $TESTDIR/run-tests.py --with-hg=`which hg` --view echo
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   332
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err (glob)
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   333
  
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   334
  ERROR: test-failure.t output changed
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   335
  !.
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   336
  Failed test-failure.t: output changed
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   337
  # Ran 2 tests, 0 skipped, 0 warned, 1 failed.
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   338
  python hash seed: * (glob)
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   339
  [1]
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   340
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   341
Accept the fix
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   342
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   343
  $ echo "  $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   344
  $ echo "  saved backup bundle to \$TESTTMP/foo.hg" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   345
  $ echo "  $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   346
  $ echo "  saved backup bundle to \$TESTTMP/foo.hg (glob)" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   347
  $ echo "  $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   348
  $ echo "  saved backup bundle to \$TESTTMP/*.hg (glob)" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   349
  $ echo 'y' | $TESTDIR/run-tests.py --with-hg=`which hg` -i 2>&1 | \
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   350
  >   sed -e 's,(glob)$,&<,g'
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   351
  
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   352
  --- $TESTTMP/test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   353
  +++ $TESTTMP/test-failure.t.err
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   354
  @@ -1,9 +1,9 @@
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   355
     $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   356
  -  rataxes
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   357
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   358
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   359
   this test is still more bytes than success.
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   360
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   361
  -  saved backup bundle to $TESTTMP/foo.hg
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   362
  +  saved backup bundle to $TESTTMP/foo.hg (glob)<
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   363
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   364
     saved backup bundle to $TESTTMP/foo.hg (glob)<
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   365
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   366
  Accept this change? [n] ..
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   367
  # Ran 2 tests, 0 skipped, 0 warned, 0 failed.
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   368
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   369
  $ sed -e 's,(glob)$,&<,g' test-failure.t
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   370
    $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   371
    babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   372
  This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   373
  this test is still more bytes than success.
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   374
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   375
    saved backup bundle to $TESTTMP/foo.hg (glob)<
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   376
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   377
    saved backup bundle to $TESTTMP/foo.hg (glob)<
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   378
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   379
    saved backup bundle to $TESTTMP/*.hg (glob)<
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   380
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   381
(reinstall)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   382
  $ mv backup test-failure.t
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   383
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   384
No Diff
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   385
===============
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   386
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   387
  $ $TESTDIR/run-tests.py --with-hg=`which hg` --nodiff
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   388
  !.
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   389
  Failed test-failure.t: output changed
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   390
  # Ran 2 tests, 0 skipped, 0 warned, 1 failed.
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   391
  python hash seed: * (glob)
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   392
  [1]
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   393
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   394
test for --time
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   395
==================
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   396
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   397
  $ $TESTDIR/run-tests.py --with-hg=`which hg` test-success.t --time
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   398
  .
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   399
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   400
  # Producing time report
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   401
  start   end     cuser   csys    real      Test
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   402
  \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   test-success.t (re)
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   403
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   404
test for --time with --job enabled
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   405
====================================
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   406
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   407
  $ $TESTDIR/run-tests.py --with-hg=`which hg` test-success.t --time --jobs 2
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   408
  .
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   409
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   410
  # Producing time report
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   411
  start   end     cuser   csys    real      Test
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   412
  \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   test-success.t (re)
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   413
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   414
Skips
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   415
================
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   416
  $ cat > test-skip.t <<EOF
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   417
  >   $ echo xyzzy
22045
769198c6a62d run-tests: add #require to abort full test
Matt Mackall <mpm@selenic.com>
parents: 22044
diff changeset
   418
  > #require false
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   419
  > EOF
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   420
  $ $TESTDIR/run-tests.py --with-hg=`which hg` --nodiff
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   421
  !.s
22292
102f0e926668 run-tests: report skipped tests as "skipped" - they might still be "relevant"
Mads Kiilerich <madski@unity3d.com>
parents: 22173
diff changeset
   422
  Skipped test-skip.t: skipped
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   423
  Failed test-failure.t: output changed
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   424
  # Ran 2 tests, 1 skipped, 0 warned, 1 failed.
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   425
  python hash seed: * (glob)
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   426
  [1]
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   427
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   428
  $ $TESTDIR/run-tests.py --with-hg=`which hg` --keyword xyzzy
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   429
  .s
22292
102f0e926668 run-tests: report skipped tests as "skipped" - they might still be "relevant"
Mads Kiilerich <madski@unity3d.com>
parents: 22173
diff changeset
   430
  Skipped test-skip.t: skipped
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   431
  # Ran 2 tests, 2 skipped, 0 warned, 0 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   432
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   433
Skips with xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   434
  $ $TESTDIR/run-tests.py --with-hg=`which hg` --keyword xyzzy \
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   435
  >  --xunit=xunit.xml
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   436
  .s
22292
102f0e926668 run-tests: report skipped tests as "skipped" - they might still be "relevant"
Mads Kiilerich <madski@unity3d.com>
parents: 22173
diff changeset
   437
  Skipped test-skip.t: skipped
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   438
  # Ran 2 tests, 2 skipped, 0 warned, 0 failed.
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   439
  $ cat xunit.xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   440
  <?xml version="1.0" encoding="utf-8"?>
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   441
  <testsuite errors="0" failures="0" name="run-tests" skipped="2" tests="2">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   442
    <testcase name="test-success.t" time="*"/> (glob)
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   443
  </testsuite>
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   444
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   445
Missing skips or blacklisted skips don't count as executed:
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   446
  $ echo test-failure.t > blacklist
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   447
  $ $TESTDIR/run-tests.py --with-hg=`which hg` --blacklist=blacklist \
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   448
  >   test-failure.t test-bogus.t
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   449
  ss
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   450
  Skipped test-bogus.t: Doesn't exist
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   451
  Skipped test-failure.t: blacklisted
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   452
  # Ran 0 tests, 2 skipped, 0 warned, 0 failed.
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   453
22579
36a940d82f88 test: protect the run-tests.py --json test behind an hghave rule
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22486
diff changeset
   454
#if json
36a940d82f88 test: protect the run-tests.py --json test behind an hghave rule
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22486
diff changeset
   455
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   456
test for --json
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   457
==================
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   458
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   459
  $ $TESTDIR/run-tests.py --with-hg=`which hg` --json
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   460
  
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   461
  --- $TESTTMP/test-failure.t
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   462
  +++ $TESTTMP/test-failure.t.err
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   463
  @@ -1,4 +1,4 @@
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   464
     $ echo babar
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   465
  -  rataxes
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   466
  +  babar
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   467
   This is a noop statement so that
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   468
   this test is still more bytes than success.
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   469
  
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   470
  ERROR: test-failure.t output changed
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   471
  !.s
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   472
  Skipped test-skip.t: skipped
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   473
  Failed test-failure.t: output changed
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   474
  # Ran 2 tests, 1 skipped, 0 warned, 1 failed.
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   475
  python hash seed: * (glob)
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   476
  [1]
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   477
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   478
  $ cat report.json
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   479
  testreport ={
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   480
      "test-failure.t": [\{] (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   481
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   482
          "cuser": "\s*[\d\.]{4,5}", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   483
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   484
          "result": "failure", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   485
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   486
          "time": "\s*[\d\.]{4,5}" (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   487
      }, ? (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   488
      "test-skip.t": {
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   489
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   490
          "cuser": "\s*[\d\.]{4,5}", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   491
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   492
          "result": "skip", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   493
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   494
          "time": "\s*[\d\.]{4,5}" (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   495
      }, ? (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   496
      "test-success.t": [\{] (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   497
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   498
          "cuser": "\s*[\d\.]{4,5}", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   499
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   500
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   501
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   502
          "time": "\s*[\d\.]{4,5}" (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   503
      }
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   504
  } (no-eol)
22579
36a940d82f88 test: protect the run-tests.py --json test behind an hghave rule
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22486
diff changeset
   505
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   506
Test that failed test accepted through interactive are properly reported:
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   507
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   508
  $ cp test-failure.t backup
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   509
  $ echo y | $TESTDIR/run-tests.py --with-hg=`which hg` --json -i
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   510
  
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   511
  --- $TESTTMP/test-failure.t
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   512
  +++ $TESTTMP/test-failure.t.err
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   513
  @@ -1,4 +1,4 @@
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   514
     $ echo babar
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   515
  -  rataxes
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   516
  +  babar
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   517
   This is a noop statement so that
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   518
   this test is still more bytes than success.
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   519
  Accept this change? [n] ..s
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   520
  Skipped test-skip.t: skipped
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   521
  # Ran 2 tests, 1 skipped, 0 warned, 0 failed.
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   522
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   523
  $ cat report.json
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   524
  testreport ={
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   525
      "test-failure.t": [\{] (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   526
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   527
          "cuser": "\s*[\d\.]{4,5}", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   528
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   529
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   530
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   531
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   532
      }, ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   533
      "test-skip.t": {
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   534
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   535
          "cuser": "\s*[\d\.]{4,5}", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   536
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   537
          "result": "skip", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   538
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   539
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   540
      }, ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   541
      "test-success.t": [\{] (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   542
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   543
          "cuser": "\s*[\d\.]{4,5}", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   544
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   545
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   546
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   547
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   548
      }
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   549
  } (no-eol)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   550
  $ mv backup test-failure.t
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   551
22579
36a940d82f88 test: protect the run-tests.py --json test behind an hghave rule
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22486
diff changeset
   552
#endif
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   553
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   554
backslash on end of line with glob matching is handled properly
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   555
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   556
  $ cat > test-glob-backslash.t << EOF
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   557
  >   $ echo 'foo bar \\'
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   558
  >   foo * \ (glob)
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   559
  > EOF
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   560
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   561
  $ $TESTDIR/run-tests.py --with-hg=`which hg` test-glob-backslash.t
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   562
  .
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   563
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   564
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   565
  $ rm -f test-glob-backslash.t
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   566