tests/test-run-tests.t
author Kyle Lippincott <spectral@google.com>
Fri, 26 May 2017 14:52:54 -0700
changeset 32474 c2b7fb580794
parent 32316 7340465bd788
child 32622 931bb962e0eb
permissions -rw-r--r--
tests: hint how to run slow tests when rejecting
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     1
This file tests the behavior of run-tests.py itself.
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     2
24960
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     3
Avoid interference from actual test env:
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     4
29220
054bcdad6042 tests: refactor run-tests helpers
timeless <timeless@mozdev.org>
parents: 29199
diff changeset
     5
  $ . "$TESTDIR/helper-runtests.sh"
24960
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     6
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
     7
Smoke test with install
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     8
============
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     9
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    10
  $ run-tests.py $HGTEST_RUN_TESTS_PURE -l
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    11
  
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    12
  # Ran 0 tests, 0 skipped, 0 warned, 0 failed.
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    13
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    14
Define a helper to avoid the install step
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    15
=============
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    16
  $ rt()
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    17
  > {
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    18
  >     run-tests.py --with-hg=`which hg` "$@"
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    19
  > }
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    20
28037
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    21
error paths
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    22
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    23
#if symlink
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    24
  $ ln -s `which true` hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    25
  $ run-tests.py --with-hg=./hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    26
  warning: --with-hg should specify an hg script
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    27
  
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    28
  # Ran 0 tests, 0 skipped, 0 warned, 0 failed.
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    29
  $ rm hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    30
#endif
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    31
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    32
#if execbit
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    33
  $ touch hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    34
  $ run-tests.py --with-hg=./hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    35
  Usage: run-tests.py [options] [tests]
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    36
  
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    37
  run-tests.py: error: --with-hg must specify an executable hg script
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    38
  [2]
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    39
  $ rm hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    40
#endif
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    41
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    42
Features for testing optional lines
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    43
===================================
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    44
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    45
  $ cat > hghaveaddon.py <<EOF
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    46
  > import hghave
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    47
  > @hghave.check("custom", "custom hghave feature")
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    48
  > def has_custom():
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    49
  >     return True
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    50
  > @hghave.check("missing", "missing hghave feature")
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    51
  > def has_missing():
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    52
  >     return False
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    53
  > EOF
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    54
28812
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    55
an empty test
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    56
=======================
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    57
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    58
  $ touch test-empty.t
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    59
  $ rt
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    60
  .
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    61
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    62
  $ rm test-empty.t
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    63
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    64
a succesful test
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    65
=======================
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    66
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    67
  $ cat > test-success.t << EOF
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    68
  >   $ echo babar
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    69
  >   babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    70
  >   $ echo xyzzy
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
    71
  >   dont_print (?)
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
    72
  >   nothing[42]line (re) (?)
28701
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28619
diff changeset
    73
  >   never*happens (glob) (?)
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
    74
  >   more_nothing (?)
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    75
  >   xyzzy
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25098
diff changeset
    76
  >   nor this (?)
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    77
  >   $ printf 'abc\ndef\nxyz\n'
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    78
  >   123 (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    79
  >   abc
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    80
  >   def (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    81
  >   456 (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    82
  >   xyz
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    83
  >   $ printf 'zyx\nwvu\ntsr\n'
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    84
  >   abc (?)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    85
  >   zyx (custom !)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    86
  >   wvu
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    87
  >   no_print (no-custom !)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    88
  >   tsr (no-missing !)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    89
  >   missing (missing !)
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    90
  > EOF
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    91
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    92
  $ rt
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    93
  .
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    94
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    95
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    96
failing test
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    97
==================
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    98
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    99
test churn with globs
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   100
  $ cat > test-failure.t <<EOF
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   101
  >   $ echo "bar-baz"; echo "bar-bad"
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   102
  >   bar*bad (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   103
  >   bar*baz (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   104
  > EOF
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   105
  $ rt test-failure.t
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   106
  
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   107
  --- $TESTTMP/test-failure.t
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   108
  +++ $TESTTMP/test-failure.t.err
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   109
  @@ -1,3 +1,3 @@
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   110
     $ echo "bar-baz"; echo "bar-bad"
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   111
  +  bar*baz (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   112
     bar*bad (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   113
  -  bar*baz (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   114
  
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   115
  ERROR: test-failure.t output changed
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   116
  !
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   117
  Failed test-failure.t: output changed
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   118
  # Ran 1 tests, 0 skipped, 0 warned, 1 failed.
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   119
  python hash seed: * (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   120
  [1]
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   121
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   122
basic failing test
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   123
  $ cat > test-failure.t << EOF
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   124
  >   $ echo babar
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   125
  >   rataxes
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   126
  > This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   127
  > this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   128
  > pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   129
  > pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   130
  > pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   131
  > pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   132
  > pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   133
  > pad pad pad pad............................................................
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   134
  > EOF
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   135
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   136
  >>> fh = open('test-failure-unicode.t', 'wb')
25054
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   137
  >>> fh.write(u'  $ echo babar\u03b1\n'.encode('utf-8')) and None
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   138
  >>> fh.write(u'  l\u03b5\u03b5t\n'.encode('utf-8')) and None
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   139
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   140
  $ rt
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   141
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   142
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   143
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   144
  @@ -1,5 +1,5 @@
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   145
     $ echo babar
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   146
  -  rataxes
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   147
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   148
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   149
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   150
   pad pad pad pad............................................................
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   151
  
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   152
  ERROR: test-failure.t output changed
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   153
  !.
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   154
  --- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   155
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   156
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   157
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   158
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   159
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   160
  
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   161
  ERROR: test-failure-unicode.t output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   162
  !
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   163
  Failed test-failure.t: output changed
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   164
  Failed test-failure-unicode.t: output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   165
  # Ran 3 tests, 0 skipped, 0 warned, 2 failed.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   166
  python hash seed: * (glob)
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   167
  [1]
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   168
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   169
test --xunit support
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   170
  $ rt --xunit=xunit.xml
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   171
  
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   172
  --- $TESTTMP/test-failure.t
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   173
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   174
  @@ -1,5 +1,5 @@
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   175
     $ echo babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   176
  -  rataxes
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   177
  +  babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   178
   This is a noop statement so that
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   179
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   180
   pad pad pad pad............................................................
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   181
  
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   182
  ERROR: test-failure.t output changed
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   183
  !.
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   184
  --- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   185
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   186
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   187
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   188
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   189
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   190
  
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   191
  ERROR: test-failure-unicode.t output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   192
  !
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   193
  Failed test-failure.t: output changed
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   194
  Failed test-failure-unicode.t: output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   195
  # Ran 3 tests, 0 skipped, 0 warned, 2 failed.
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   196
  python hash seed: * (glob)
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   197
  [1]
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   198
  $ cat xunit.xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   199
  <?xml version="1.0" encoding="utf-8"?>
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   200
  <testsuite errors="0" failures="2" name="run-tests" skipped="0" tests="3">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   201
    <testcase name="test-success.t" time="*"/> (glob)
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   202
    <testcase name="test-failure-unicode.t" time="*"> (glob)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   203
  <![CDATA[--- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   204
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   205
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   206
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   207
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   208
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   209
  ]]>  </testcase>
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   210
    <testcase name="test-failure.t" time="*"> (glob)
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   211
  <![CDATA[--- $TESTTMP/test-failure.t
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   212
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   213
  @@ -1,5 +1,5 @@
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   214
     $ echo babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   215
  -  rataxes
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   216
  +  babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   217
   This is a noop statement so that
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   218
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   219
   pad pad pad pad............................................................
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   220
  ]]>  </testcase>
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   221
  </testsuite>
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   222
29280
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   223
  $ cat .testtimes
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   224
  test-failure-unicode.t * (glob)
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   225
  test-failure.t * (glob)
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   226
  test-success.t * (glob)
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   227
  $ rm test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   228
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   229
test for --retest
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   230
====================
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   231
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   232
  $ rt --retest
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   233
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   234
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   235
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   236
  @@ -1,5 +1,5 @@
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   237
     $ echo babar
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   238
  -  rataxes
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   239
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   240
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   241
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   242
   pad pad pad pad............................................................
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   243
  
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   244
  ERROR: test-failure.t output changed
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   245
  !
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   246
  Failed test-failure.t: output changed
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   247
  # Ran 2 tests, 1 skipped, 0 warned, 1 failed.
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   248
  python hash seed: * (glob)
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   249
  [1]
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   250
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   251
Selecting Tests To Run
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   252
======================
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   253
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   254
successful
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   255
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   256
  $ rt test-success.t
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   257
  .
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   258
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   259
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   260
success w/ keyword
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   261
  $ rt -k xyzzy
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   262
  .
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   263
  # Ran 2 tests, 1 skipped, 0 warned, 0 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   264
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   265
failed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   266
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   267
  $ rt test-failure.t
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   268
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   269
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   270
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   271
  @@ -1,5 +1,5 @@
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   272
     $ echo babar
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   273
  -  rataxes
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   274
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   275
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   276
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   277
   pad pad pad pad............................................................
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   278
  
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   279
  ERROR: test-failure.t output changed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   280
  !
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   281
  Failed test-failure.t: output changed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   282
  # Ran 1 tests, 0 skipped, 0 warned, 1 failed.
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   283
  python hash seed: * (glob)
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   284
  [1]
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   285
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   286
failure w/ keyword
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   287
  $ rt -k rataxes
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   288
  
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   289
  --- $TESTTMP/test-failure.t
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   290
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   291
  @@ -1,5 +1,5 @@
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   292
     $ echo babar
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   293
  -  rataxes
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   294
  +  babar
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   295
   This is a noop statement so that
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   296
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   297
   pad pad pad pad............................................................
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   298
  
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   299
  ERROR: test-failure.t output changed
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   300
  !
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   301
  Failed test-failure.t: output changed
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   302
  # Ran 2 tests, 1 skipped, 0 warned, 1 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   303
  python hash seed: * (glob)
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   304
  [1]
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   305
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   306
Verify that when a process fails to start we show a useful message
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   307
==================================================================
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   308
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   309
  $ cat > test-serve-fail.t <<EOF
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   310
  >   $ echo 'abort: child process failed to start blah'
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   311
  > EOF
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   312
  $ rt test-serve-fail.t
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   313
  
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   314
  ERROR: test-serve-fail.t output changed
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   315
  !
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   316
  Failed test-serve-fail.t: server failed to start (HGPORT=*) (glob)
27567
54bc0e884385 run-tests: avoid double counting server fails
timeless <timeless@mozdev.org>
parents: 27564
diff changeset
   317
  # Ran 1 tests, 0 skipped, 0 warned, 1 failed.
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   318
  python hash seed: * (glob)
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   319
  [1]
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   320
  $ rm test-serve-fail.t
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   321
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   322
Verify that we can try other ports
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   323
===================================
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   324
  $ hg init inuse
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   325
  $ hg serve -R inuse -p $HGPORT -d --pid-file=blocks.pid
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   326
  $ cat blocks.pid >> $DAEMON_PIDS
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   327
  $ cat > test-serve-inuse.t <<EOF
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   328
  >   $ hg serve -R `pwd`/inuse -p \$HGPORT -d --pid-file=hg.pid
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   329
  >   $ cat hg.pid >> \$DAEMON_PIDS
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   330
  > EOF
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   331
  $ rt test-serve-inuse.t
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   332
  .
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   333
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   334
  $ rm test-serve-inuse.t
29221
73f58eb14684 test-run-tests: clean up inuse server eagerly
timeless <timeless@mozdev.org>
parents: 29220
diff changeset
   335
  $ killdaemons.py $DAEMON_PIDS
73f58eb14684 test-run-tests: clean up inuse server eagerly
timeless <timeless@mozdev.org>
parents: 29220
diff changeset
   336
  $ rm $DAEMON_PIDS
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   337
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   338
Running In Debug Mode
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   339
======================
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   340
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   341
  $ rt --debug 2>&1 | grep -v pwd
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   342
  + echo *SALT* 0 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   343
  *SALT* 0 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   344
  + echo babar
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   345
  babar
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   346
  + echo *SALT* 10 0 (glob)
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   347
  *SALT* 10 0 (glob)
27509
6ff90d618d72 test-run-tests: glob away a --debug run difference on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 27395
diff changeset
   348
  *+ echo *SALT* 0 0 (glob)
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   349
  *SALT* 0 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   350
  + echo babar
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   351
  babar
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   352
  + echo *SALT* 2 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   353
  *SALT* 2 0 (glob)
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   354
  + echo xyzzy
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   355
  xyzzy
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   356
  + echo *SALT* 9 0 (glob)
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   357
  *SALT* 9 0 (glob)
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   358
  + printf *abc\ndef\nxyz\n* (glob)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   359
  abc
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   360
  def
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   361
  xyz
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   362
  + echo *SALT* 15 0 (glob)
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   363
  *SALT* 15 0 (glob)
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   364
  + printf *zyx\nwvu\ntsr\n* (glob)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   365
  zyx
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   366
  wvu
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   367
  tsr
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   368
  + echo *SALT* 22 0 (glob)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   369
  *SALT* 22 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   370
  .
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   371
  # Ran 2 tests, 0 skipped, 0 warned, 0 failed.
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   372
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   373
Parallel runs
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   374
==============
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   375
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   376
(duplicate the failing test to get predictable output)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   377
  $ cp test-failure.t test-failure-copy.t
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   378
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   379
  $ rt --jobs 2 test-failure*.t -n
23107
5459b30aa498 tests: silence output race in test-run-tests.t
Matt Mackall <mpm@selenic.com>
parents: 22840
diff changeset
   380
  !!
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   381
  Failed test-failure*.t: output changed (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   382
  Failed test-failure*.t: output changed (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   383
  # Ran 2 tests, 0 skipped, 0 warned, 2 failed.
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   384
  python hash seed: * (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   385
  [1]
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   386
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   387
failures in parallel with --first should only print one failure
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   388
  >>> f = open('test-nothing.t', 'w')
25054
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   389
  >>> f.write('foo\n' * 1024) and None
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   390
  >>> f.write('  $ sleep 1') and None
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   391
  $ rt --jobs 2 --first
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   392
  
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   393
  --- $TESTTMP/test-failure*.t (glob)
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   394
  +++ $TESTTMP/test-failure*.t.err (glob)
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   395
  @@ -1,5 +1,5 @@
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   396
     $ echo babar
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   397
  -  rataxes
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   398
  +  babar
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   399
   This is a noop statement so that
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   400
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   401
   pad pad pad pad............................................................
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   402
  
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   403
  Failed test-failure*.t: output changed (glob)
24507
a0668a587c04 run-tests: wait for test threads after first error
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
   404
  Failed test-nothing.t: output changed
a0668a587c04 run-tests: wait for test threads after first error
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
   405
  # Ran 2 tests, 0 skipped, 0 warned, 2 failed.
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   406
  python hash seed: * (glob)
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   407
  [1]
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   408
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   409
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   410
(delete the duplicated test file)
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   411
  $ rm test-failure-copy.t test-nothing.t
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   412
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   413
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   414
Interactive run
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   415
===============
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   416
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   417
(backup the failing test)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   418
  $ cp test-failure.t backup
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   419
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   420
Refuse the fix
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   421
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   422
  $ echo 'n' | rt -i
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   423
  
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   424
  --- $TESTTMP/test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   425
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   426
  @@ -1,5 +1,5 @@
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   427
     $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   428
  -  rataxes
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   429
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   430
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   431
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   432
   pad pad pad pad............................................................
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21756
diff changeset
   433
  Accept this change? [n] 
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   434
  ERROR: test-failure.t output changed
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21756
diff changeset
   435
  !.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   436
  Failed test-failure.t: output changed
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   437
  # Ran 2 tests, 0 skipped, 0 warned, 1 failed.
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   438
  python hash seed: * (glob)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   439
  [1]
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   440
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   441
  $ cat test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   442
    $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   443
    rataxes
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   444
  This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   445
  this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   446
  pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   447
  pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   448
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   449
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   450
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   451
  pad pad pad pad............................................................
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   452
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   453
Interactive with custom view
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   454
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   455
  $ echo 'n' | rt -i --view echo
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   456
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err (glob)
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   457
  Accept this change? [n]* (glob)
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   458
  ERROR: test-failure.t output changed
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   459
  !.
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   460
  Failed test-failure.t: output changed
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   461
  # Ran 2 tests, 0 skipped, 0 warned, 1 failed.
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   462
  python hash seed: * (glob)
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   463
  [1]
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   464
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   465
View the fix
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   466
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   467
  $ echo 'y' | rt --view echo
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   468
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err (glob)
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   469
  
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   470
  ERROR: test-failure.t output changed
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   471
  !.
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   472
  Failed test-failure.t: output changed
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   473
  # Ran 2 tests, 0 skipped, 0 warned, 1 failed.
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   474
  python hash seed: * (glob)
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   475
  [1]
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   476
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   477
Accept the fix
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   478
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   479
  $ echo "  $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   480
  $ echo "  saved backup bundle to \$TESTTMP/foo.hg" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   481
  $ echo "  $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   482
  $ echo "  saved backup bundle to \$TESTTMP/foo.hg (glob)" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   483
  $ echo "  $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   484
  $ echo "  saved backup bundle to \$TESTTMP/*.hg (glob)" >> test-failure.t
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   485
  $ echo 'y' | rt -i 2>&1
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   486
  
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   487
  --- $TESTTMP/test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   488
  +++ $TESTTMP/test-failure.t.err
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   489
  @@ -1,5 +1,5 @@
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   490
     $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   491
  -  rataxes
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   492
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   493
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   494
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   495
   pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   496
  @@ -9,7 +9,7 @@
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   497
   pad pad pad pad............................................................
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   498
   pad pad pad pad............................................................
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   499
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   500
  -  saved backup bundle to $TESTTMP/foo.hg
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   501
  +  saved backup bundle to $TESTTMP/foo.hg* (glob)
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   502
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   503
     saved backup bundle to $TESTTMP/foo.hg* (glob)
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   504
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   505
  Accept this change? [n] ..
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   506
  # Ran 2 tests, 0 skipped, 0 warned, 0 failed.
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   507
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   508
  $ sed -e 's,(glob)$,&<,g' test-failure.t
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   509
    $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   510
    babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   511
  This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   512
  this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   513
  pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   514
  pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   515
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   516
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   517
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   518
  pad pad pad pad............................................................
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   519
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   520
    saved backup bundle to $TESTTMP/foo.hg (glob)<
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   521
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   522
    saved backup bundle to $TESTTMP/foo.hg (glob)<
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   523
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   524
    saved backup bundle to $TESTTMP/*.hg (glob)<
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   525
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   526
(reinstall)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   527
  $ mv backup test-failure.t
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   528
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   529
No Diff
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   530
===============
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   531
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   532
  $ rt --nodiff
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   533
  !.
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   534
  Failed test-failure.t: output changed
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   535
  # Ran 2 tests, 0 skipped, 0 warned, 1 failed.
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   536
  python hash seed: * (glob)
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   537
  [1]
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   538
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   539
test --tmpdir support
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   540
  $ rt --tmpdir=$TESTTMP/keep test-success.t
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   541
  
26432
39577d4520ab test-run-tests: add globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 26422
diff changeset
   542
  Keeping testtmp dir: $TESTTMP/keep/child1/test-success.t (glob)
39577d4520ab test-run-tests: add globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 26422
diff changeset
   543
  Keeping threadtmp dir: $TESTTMP/keep/child1  (glob)
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   544
  .
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   545
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   546
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   547
timeouts
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   548
========
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   549
  $ cat > test-timeout.t <<EOF
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   550
  >   $ sleep 2
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   551
  >   $ echo pass
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   552
  >   pass
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   553
  > EOF
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   554
  > echo '#require slow' > test-slow-timeout.t
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   555
  > cat test-timeout.t >> test-slow-timeout.t
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   556
  $ rt --timeout=1 --slowtimeout=3 test-timeout.t test-slow-timeout.t
27393
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
   557
  st
32474
c2b7fb580794 tests: hint how to run slow tests when rejecting
Kyle Lippincott <spectral@google.com>
parents: 32316
diff changeset
   558
  Skipped test-slow-timeout.t: missing feature: allow slow tests (use --allow-slow-tests)
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   559
  Failed test-timeout.t: timed out
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   560
  # Ran 1 tests, 1 skipped, 0 warned, 1 failed.
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   561
  python hash seed: * (glob)
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   562
  [1]
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   563
  $ rt --timeout=1 --slowtimeout=3 \
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   564
  > test-timeout.t test-slow-timeout.t --allow-slow-tests
27393
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
   565
  .t
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   566
  Failed test-timeout.t: timed out
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   567
  # Ran 2 tests, 0 skipped, 0 warned, 1 failed.
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   568
  python hash seed: * (glob)
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   569
  [1]
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   570
  $ rm test-timeout.t test-slow-timeout.t
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   571
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   572
test for --time
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   573
==================
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   574
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   575
  $ rt test-success.t --time
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   576
  .
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   577
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   578
  # Producing time report
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   579
  start   end     cuser   csys    real      Test
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   580
  \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   test-success.t (re)
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   581
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   582
test for --time with --job enabled
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   583
====================================
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   584
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   585
  $ rt test-success.t --time --jobs 2
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   586
  .
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   587
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   588
  # Producing time report
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   589
  start   end     cuser   csys    real      Test
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   590
  \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   test-success.t (re)
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   591
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   592
Skips
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   593
================
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   594
  $ cat > test-skip.t <<EOF
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   595
  >   $ echo xyzzy
22045
769198c6a62d run-tests: add #require to abort full test
Matt Mackall <mpm@selenic.com>
parents: 22044
diff changeset
   596
  > #require false
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   597
  > EOF
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   598
  $ rt --nodiff
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   599
  !.s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   600
  Skipped test-skip.t: missing feature: nail clipper
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   601
  Failed test-failure.t: output changed
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   602
  # Ran 2 tests, 1 skipped, 0 warned, 1 failed.
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   603
  python hash seed: * (glob)
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   604
  [1]
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   605
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   606
  $ rt --keyword xyzzy
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   607
  .s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   608
  Skipped test-skip.t: missing feature: nail clipper
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   609
  # Ran 2 tests, 2 skipped, 0 warned, 0 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   610
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   611
Skips with xml
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   612
  $ rt --keyword xyzzy \
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   613
  >  --xunit=xunit.xml
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   614
  .s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   615
  Skipped test-skip.t: missing feature: nail clipper
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   616
  # Ran 2 tests, 2 skipped, 0 warned, 0 failed.
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   617
  $ cat xunit.xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   618
  <?xml version="1.0" encoding="utf-8"?>
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   619
  <testsuite errors="0" failures="0" name="run-tests" skipped="2" tests="2">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   620
    <testcase name="test-success.t" time="*"/> (glob)
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   621
  </testsuite>
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   622
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   623
Missing skips or blacklisted skips don't count as executed:
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   624
  $ echo test-failure.t > blacklist
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   625
  $ rt --blacklist=blacklist --json\
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   626
  >   test-failure.t test-bogus.t
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   627
  ss
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   628
  Skipped test-bogus.t: Doesn't exist
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   629
  Skipped test-failure.t: blacklisted
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   630
  # Ran 0 tests, 2 skipped, 0 warned, 0 failed.
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   631
  $ cat report.json
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   632
  testreport ={
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   633
      "test-bogus.t": {
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   634
          "result": "skip"
29199
daff05dcd184 run-tests: handle json.dumps divergence
timeless <timeless@mozdev.org>
parents: 29173
diff changeset
   635
      },
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   636
      "test-failure.t": {
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   637
          "result": "skip"
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   638
      }
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   639
  } (no-eol)
29173
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   640
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   641
Whitelist trumps blacklist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   642
  $ echo test-failure.t > whitelist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   643
  $ rt --blacklist=blacklist --whitelist=whitelist --json\
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   644
  >   test-failure.t test-bogus.t
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   645
  s
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   646
  --- $TESTTMP/test-failure.t
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   647
  +++ $TESTTMP/test-failure.t.err
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   648
  @@ -1,5 +1,5 @@
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   649
     $ echo babar
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   650
  -  rataxes
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   651
  +  babar
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   652
   This is a noop statement so that
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   653
   this test is still more bytes than success.
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   654
   pad pad pad pad............................................................
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   655
  
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   656
  ERROR: test-failure.t output changed
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   657
  !
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   658
  Skipped test-bogus.t: Doesn't exist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   659
  Failed test-failure.t: output changed
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   660
  # Ran 1 tests, 1 skipped, 0 warned, 1 failed.
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   661
  python hash seed: * (glob)
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   662
  [1]
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   663
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   664
test for --json
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   665
==================
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   666
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   667
  $ rt --json
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   668
  
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   669
  --- $TESTTMP/test-failure.t
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   670
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   671
  @@ -1,5 +1,5 @@
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   672
     $ echo babar
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   673
  -  rataxes
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   674
  +  babar
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   675
   This is a noop statement so that
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   676
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   677
   pad pad pad pad............................................................
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   678
  
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   679
  ERROR: test-failure.t output changed
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   680
  !.s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   681
  Skipped test-skip.t: missing feature: nail clipper
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   682
  Failed test-failure.t: output changed
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   683
  # Ran 2 tests, 1 skipped, 0 warned, 1 failed.
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   684
  python hash seed: * (glob)
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   685
  [1]
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   686
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   687
  $ cat report.json
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   688
  testreport ={
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   689
      "test-failure.t": [\{] (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   690
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   691
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   692
          "diff": "---.+\+\+\+.+", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   693
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   694
          "result": "failure", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   695
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   696
          "time": "\s*[\d\.]{4,5}" (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   697
      }, ? (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   698
      "test-skip.t": {
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   699
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   700
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   701
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   702
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   703
          "result": "skip", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   704
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   705
          "time": "\s*[\d\.]{4,5}" (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   706
      }, ? (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   707
      "test-success.t": [\{] (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   708
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   709
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   710
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   711
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   712
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   713
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   714
          "time": "\s*[\d\.]{4,5}" (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   715
      }
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   716
  } (no-eol)
22579
36a940d82f88 test: protect the run-tests.py --json test behind an hghave rule
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22486
diff changeset
   717
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   718
Test that failed test accepted through interactive are properly reported:
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   719
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   720
  $ cp test-failure.t backup
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   721
  $ echo y | rt --json -i
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   722
  
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   723
  --- $TESTTMP/test-failure.t
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   724
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   725
  @@ -1,5 +1,5 @@
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   726
     $ echo babar
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   727
  -  rataxes
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   728
  +  babar
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   729
   This is a noop statement so that
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   730
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   731
   pad pad pad pad............................................................
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   732
  Accept this change? [n] ..s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   733
  Skipped test-skip.t: missing feature: nail clipper
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   734
  # Ran 2 tests, 1 skipped, 0 warned, 0 failed.
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   735
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   736
  $ cat report.json
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   737
  testreport ={
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   738
      "test-failure.t": [\{] (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   739
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   740
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   741
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   742
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   743
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   744
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   745
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   746
      }, ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   747
      "test-skip.t": {
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   748
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   749
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   750
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   751
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   752
          "result": "skip", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   753
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   754
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   755
      }, ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   756
      "test-success.t": [\{] (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   757
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   758
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   759
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   760
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   761
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   762
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   763
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   764
      }
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   765
  } (no-eol)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   766
  $ mv backup test-failure.t
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   767
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   768
backslash on end of line with glob matching is handled properly
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   769
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   770
  $ cat > test-glob-backslash.t << EOF
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   771
  >   $ echo 'foo bar \\'
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   772
  >   foo * \ (glob)
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   773
  > EOF
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   774
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   775
  $ rt test-glob-backslash.t
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   776
  .
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   777
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   778
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   779
  $ rm -f test-glob-backslash.t
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   780
31673
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
   781
Test globbing of local IP addresses
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29280
diff changeset
   782
  $ echo 172.16.18.1
31673
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
   783
  $LOCALIP (glob)
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
   784
  $ echo dead:beef::1
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
   785
  $LOCALIP (glob)
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29280
diff changeset
   786
25728
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   787
Test reusability for third party tools
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   788
======================================
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   789
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   790
  $ mkdir "$TESTTMP"/anothertests
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   791
  $ cd "$TESTTMP"/anothertests
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   792
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   793
test that `run-tests.py` can execute hghave, even if it runs not in
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   794
Mercurial source tree.
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   795
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   796
  $ cat > test-hghave.t <<EOF
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   797
  > #require true
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   798
  >   $ echo foo
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   799
  >   foo
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   800
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
   801
  $ rt test-hghave.t
25728
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   802
  .
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   803
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   804
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   805
test that RUNTESTDIR refers the directory, in which `run-tests.py` now
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   806
running is placed.
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   807
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   808
  $ cat > test-runtestdir.t <<EOF
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   809
  > - $TESTDIR, in which test-run-tests.t is placed
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   810
  > - \$TESTDIR, in which test-runtestdir.t is placed (expanded at runtime)
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   811
  > - \$RUNTESTDIR, in which run-tests.py is placed (expanded at runtime)
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   812
  > 
27057
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
   813
  > #if windows
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
   814
  >   $ test "\$TESTDIR" = "$TESTTMP\anothertests"
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
   815
  > #else
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   816
  >   $ test "\$TESTDIR" = "$TESTTMP"/anothertests
27057
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
   817
  > #endif
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   818
  >   $ test "\$RUNTESTDIR" = "$TESTDIR"
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   819
  >   $ head -n 3 "\$RUNTESTDIR"/../contrib/check-code.py
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   820
  >   #!/usr/bin/env python
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   821
  >   #
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   822
  >   # check-code - a style and portability checker for Mercurial
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   823
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
   824
  $ rt test-runtestdir.t
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   825
  .
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   826
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   827
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   828
#if execbit
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   829
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   830
test that TESTDIR is referred in PATH
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   831
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   832
  $ cat > custom-command.sh <<EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   833
  > #!/bin/sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   834
  > echo "hello world"
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   835
  > EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   836
  $ chmod +x custom-command.sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   837
  $ cat > test-testdir-path.t <<EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   838
  >   $ custom-command.sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   839
  >   hello world
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   840
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
   841
  $ rt test-testdir-path.t
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   842
  .
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   843
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   844
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   845
#endif
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   846
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   847
test support for --allow-slow-tests
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   848
  $ cat > test-very-slow-test.t <<EOF
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   849
  > #require slow
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   850
  >   $ echo pass
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   851
  >   pass
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   852
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
   853
  $ rt test-very-slow-test.t
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   854
  s
32474
c2b7fb580794 tests: hint how to run slow tests when rejecting
Kyle Lippincott <spectral@google.com>
parents: 32316
diff changeset
   855
  Skipped test-very-slow-test.t: missing feature: allow slow tests (use --allow-slow-tests)
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   856
  # Ran 0 tests, 1 skipped, 0 warned, 0 failed.
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   857
  $ rt $HGTEST_RUN_TESTS_PURE --allow-slow-tests test-very-slow-test.t
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   858
  .
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   859
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
28180
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   860
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   861
support for running a test outside the current directory
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   862
  $ mkdir nonlocal
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   863
  $ cat > nonlocal/test-is-not-here.t << EOF
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   864
  >   $ echo pass
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   865
  >   pass
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   866
  > EOF
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   867
  $ rt nonlocal/test-is-not-here.t
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   868
  .
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   869
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   870
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   871
support for bisecting failed tests automatically
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   872
  $ hg init bisect
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   873
  $ cd bisect
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   874
  $ cat >> test-bisect.t <<EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   875
  >   $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   876
  >   pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   877
  > EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   878
  $ hg add test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   879
  $ hg ci -m 'good'
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   880
  $ cat >> test-bisect.t <<EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   881
  >   $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   882
  >   fail
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   883
  > EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   884
  $ hg ci -m 'bad'
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   885
  $ rt --known-good-rev=0 test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   886
  
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   887
  --- $TESTTMP/anothertests/bisect/test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   888
  +++ $TESTTMP/anothertests/bisect/test-bisect.t.err
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   889
  @@ -1,4 +1,4 @@
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   890
     $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   891
     pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   892
     $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   893
  -  fail
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   894
  +  pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   895
  
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   896
  ERROR: test-bisect.t output changed
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   897
  !
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   898
  Failed test-bisect.t: output changed
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   899
  test-bisect.t broken by 72cbf122d116 (bad)
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   900
  # Ran 1 tests, 0 skipped, 0 warned, 1 failed.
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   901
  python hash seed: * (glob)
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   902
  [1]
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   903
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   904
  $ cd ..
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   905
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   906
Test cases in .t files
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   907
======================
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   908
  $ mkdir cases
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   909
  $ cd cases
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   910
  $ cat > test-cases-abc.t <<'EOF'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   911
  > #testcases A B C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   912
  >   $ V=B
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   913
  > #if A
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   914
  >   $ V=A
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   915
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   916
  > #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   917
  >   $ V=C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   918
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   919
  >   $ echo $V | sed 's/A/C/'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   920
  >   C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   921
  > #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   922
  >   $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   923
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   924
  > #if A
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   925
  >   $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   926
  >   [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   927
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   928
  > #if no-C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   929
  >   $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   930
  >   [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   931
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   932
  >   $ [ $V = D ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   933
  >   [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   934
  > EOF
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   935
  $ rt
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   936
  .
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   937
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   938
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t.B.err
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   939
  @@ -7,7 +7,7 @@
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   940
     $ V=C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   941
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   942
     $ echo $V | sed 's/A/C/'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   943
  -  C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   944
  +  B
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   945
   #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   946
     $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   947
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   948
  
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   949
  ERROR: test-cases-abc.t (case B) output changed
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   950
  !.
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   951
  Failed test-cases-abc.t (case B): output changed
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   952
  # Ran 3 tests, 0 skipped, 0 warned, 1 failed.
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   953
  python hash seed: * (glob)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   954
  [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   955
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   956
--restart works
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   957
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   958
  $ rt --restart
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   959
  
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   960
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   961
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t.B.err
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   962
  @@ -7,7 +7,7 @@
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   963
     $ V=C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   964
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   965
     $ echo $V | sed 's/A/C/'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   966
  -  C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   967
  +  B
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   968
   #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   969
     $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   970
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   971
  
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   972
  ERROR: test-cases-abc.t (case B) output changed
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   973
  !.
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   974
  Failed test-cases-abc.t (case B): output changed
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   975
  # Ran 2 tests, 0 skipped, 0 warned, 1 failed.
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   976
  python hash seed: * (glob)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
   977
  [1]