tests/test-run-tests.t
author Gregory Szorc <gregory.szorc@gmail.com>
Wed, 02 May 2018 19:16:01 -0700
branchstable
changeset 37829 2ead51dcde71
parent 37740 1b71a397d6b2
child 37846 89793289c891
permissions -rw-r--r--
paper: don't register click handlers with inline javascript (issue5812) The use of inline href="javascript:" undermines CSP policies that don't allow inline javascript. This commit changes the registering of the diffstat and line wrapping toggle handlers to the the global DOMContentLoaded handler, thus eliminating all inline javascript from the paper template. Differential Revision: https://phab.mercurial-scm.org/D3437
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     1
This file tests the behavior of run-tests.py itself.
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     2
24960
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     3
Avoid interference from actual test env:
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     4
29220
054bcdad6042 tests: refactor run-tests helpers
timeless <timeless@mozdev.org>
parents: 29199
diff changeset
     5
  $ . "$TESTDIR/helper-runtests.sh"
24960
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     6
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
     7
Smoke test with install
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     8
============
37740
1b71a397d6b2 tests: make sure test-run-tests.t actually runs run-tests.py under Python 3
Augie Fackler <augie@google.com>
parents: 37434
diff changeset
     9
  $ $PYTHON $TESTDIR/run-tests.py $HGTEST_RUN_TESTS_PURE -l
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    10
  
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
    11
  # Ran 0 tests, 0 skipped, 0 failed.
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    12
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    13
Define a helper to avoid the install step
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    14
=============
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    15
  $ rt()
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    16
  > {
37740
1b71a397d6b2 tests: make sure test-run-tests.t actually runs run-tests.py under Python 3
Augie Fackler <augie@google.com>
parents: 37434
diff changeset
    17
  >     $PYTHON $TESTDIR/run-tests.py --with-hg=`which hg` "$@"
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    18
  > }
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    19
28037
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    20
error paths
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    21
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    22
#if symlink
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    23
  $ ln -s `which true` hg
37740
1b71a397d6b2 tests: make sure test-run-tests.t actually runs run-tests.py under Python 3
Augie Fackler <augie@google.com>
parents: 37434
diff changeset
    24
  $ $PYTHON $TESTDIR/run-tests.py --with-hg=./hg
28037
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    25
  warning: --with-hg should specify an hg script
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    26
  
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
    27
  # Ran 0 tests, 0 skipped, 0 failed.
28037
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    28
  $ rm hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    29
#endif
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    30
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    31
#if execbit
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    32
  $ touch hg
37740
1b71a397d6b2 tests: make sure test-run-tests.t actually runs run-tests.py under Python 3
Augie Fackler <augie@google.com>
parents: 37434
diff changeset
    33
  $ $PYTHON $TESTDIR/run-tests.py --with-hg=./hg
35187
b4b0aed7bfaf run-tests: convert to argparse
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35155
diff changeset
    34
  usage: run-tests.py [options] [tests]
28037
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    35
  run-tests.py: error: --with-hg must specify an executable hg script
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    36
  [2]
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    37
  $ rm hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    38
#endif
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    39
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    40
Features for testing optional lines
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    41
===================================
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    42
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    43
  $ cat > hghaveaddon.py <<EOF
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    44
  > import hghave
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    45
  > @hghave.check("custom", "custom hghave feature")
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    46
  > def has_custom():
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    47
  >     return True
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    48
  > @hghave.check("missing", "missing hghave feature")
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    49
  > def has_missing():
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    50
  >     return False
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    51
  > EOF
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    52
28812
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    53
an empty test
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    54
=======================
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    55
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    56
  $ touch test-empty.t
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    57
  $ rt
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    58
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
    59
  # Ran 1 tests, 0 skipped, 0 failed.
28812
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    60
  $ rm test-empty.t
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    61
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    62
a succesful test
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    63
=======================
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    64
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    65
  $ cat > test-success.t << EOF
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    66
  >   $ echo babar
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    67
  >   babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    68
  >   $ echo xyzzy
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
    69
  >   dont_print (?)
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
    70
  >   nothing[42]line (re) (?)
28701
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28619
diff changeset
    71
  >   never*happens (glob) (?)
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
    72
  >   more_nothing (?)
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    73
  >   xyzzy
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25098
diff changeset
    74
  >   nor this (?)
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    75
  >   $ printf 'abc\ndef\nxyz\n'
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    76
  >   123 (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    77
  >   abc
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    78
  >   def (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    79
  >   456 (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    80
  >   xyz
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    81
  >   $ printf 'zyx\nwvu\ntsr\n'
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    82
  >   abc (?)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    83
  >   zyx (custom !)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    84
  >   wvu
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    85
  >   no_print (no-custom !)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    86
  >   tsr (no-missing !)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    87
  >   missing (missing !)
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    88
  > EOF
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    89
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    90
  $ rt
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    91
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
    92
  # Ran 1 tests, 0 skipped, 0 failed.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    93
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    94
failing test
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    95
==================
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    96
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    97
test churn with globs
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    98
  $ cat > test-failure.t <<EOF
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
    99
  >   $ echo "bar-baz"; echo "bar-bad"; echo foo
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   100
  >   bar*bad (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   101
  >   bar*baz (glob)
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   102
  >   | fo (re)
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   103
  > EOF
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   104
  $ rt test-failure.t
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   105
  
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   106
  --- $TESTTMP/test-failure.t
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   107
  +++ $TESTTMP/test-failure.t.err
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   108
  @@ -1,4 +1,4 @@
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   109
     $ echo "bar-baz"; echo "bar-bad"; echo foo
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   110
  +  bar*baz (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   111
     bar*bad (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   112
  -  bar*baz (glob)
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   113
  -  | fo (re)
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   114
  +  foo
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   115
  
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   116
  ERROR: test-failure.t output changed
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   117
  !
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   118
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   119
  # Ran 1 tests, 0 skipped, 1 failed.
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   120
  python hash seed: * (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   121
  [1]
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   122
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   123
test diff colorisation
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   124
33552
754569f5e999 run-tests: make sure to check if pygments is installed before using it
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33500
diff changeset
   125
#if no-windows pygments
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   126
  $ rt test-failure.t --color always
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   127
  
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   128
  \x1b[38;5;124m--- $TESTTMP/test-failure.t\x1b[39m (esc)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   129
  \x1b[38;5;34m+++ $TESTTMP/test-failure.t.err\x1b[39m (esc)
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   130
  \x1b[38;5;90;01m@@ -1,4 +1,4 @@\x1b[39;00m (esc)
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   131
     $ echo "bar-baz"; echo "bar-bad"; echo foo
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   132
  \x1b[38;5;34m+  bar*baz (glob)\x1b[39m (esc)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   133
     bar*bad (glob)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   134
  \x1b[38;5;124m-  bar*baz (glob)\x1b[39m (esc)
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   135
  \x1b[38;5;124m-  | fo (re)\x1b[39m (esc)
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   136
  \x1b[38;5;34m+  foo\x1b[39m (esc)
34842
8bce3e51b101 run-tests: move newline out of colorized message
Martin von Zweigbergk <martinvonz@google.com>
parents: 34263
diff changeset
   137
  
33814
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33696
diff changeset
   138
  \x1b[38;5;88mERROR: \x1b[39m\x1b[38;5;9mtest-failure.t\x1b[39m\x1b[38;5;88m output changed\x1b[39m (esc)
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   139
  !
33814
81b12f69ef5b run-tests: also color the summary messages (skipped, failed...)
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 33696
diff changeset
   140
  \x1b[38;5;88mFailed \x1b[39m\x1b[38;5;9mtest-failure.t\x1b[39m\x1b[38;5;88m: output changed\x1b[39m (esc)
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   141
  # Ran 1 tests, 0 skipped, 1 failed.
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   142
  python hash seed: * (glob)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   143
  [1]
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   144
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   145
  $ rt test-failure.t 2> tmp.log
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   146
  [1]
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   147
  $ cat tmp.log
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   148
  
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   149
  --- $TESTTMP/test-failure.t
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   150
  +++ $TESTTMP/test-failure.t.err
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   151
  @@ -1,4 +1,4 @@
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   152
     $ echo "bar-baz"; echo "bar-bad"; echo foo
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   153
  +  bar*baz (glob)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   154
     bar*bad (glob)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   155
  -  bar*baz (glob)
35155
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   156
  -  | fo (re)
6d5718e39657 run-tests: make "| foo (re)" not match everything
Martin von Zweigbergk <martinvonz@google.com>
parents: 35097
diff changeset
   157
  +  foo
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   158
  
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   159
  ERROR: test-failure.t output changed
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   160
  !
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   161
  Failed test-failure.t: output changed
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   162
  # Ran 1 tests, 0 skipped, 1 failed.
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   163
  python hash seed: * (glob)
33500
9c6e64911de0 run-tests: disable color on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 33420
diff changeset
   164
#endif
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   165
33695
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   166
  $ cat > test-failure.t << EOF
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   167
  >   $ true
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   168
  >   should go away (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   169
  >   $ true
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   170
  >   should stay (false !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   171
  > 
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   172
  > Should remove first line, not second or third
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   173
  >   $ echo 'testing'
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   174
  >   baz*foo (glob) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   175
  >   foobar*foo (glob) (false !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   176
  >   te*ting (glob) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   177
  > 
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   178
  > Should keep first two lines, remove third and last
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   179
  >   $ echo 'testing'
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   180
  >   test.ng (re) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   181
  >   foo.ar (re) (false !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   182
  >   b.r (re) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   183
  >   missing (?)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   184
  >   awol (true !)
33696
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   185
  > 
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   186
  > The "missing" line should stay, even though awol is dropped
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   187
  >   $ echo 'testing'
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   188
  >   test.ng (re) (true !)
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   189
  >   foo.ar (?)
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   190
  >   awol
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   191
  >   missing (?)
33695
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   192
  > EOF
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   193
  $ rt test-failure.t
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   194
  
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   195
  --- $TESTTMP/test-failure.t
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   196
  +++ $TESTTMP/test-failure.t.err
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   197
  @@ -1,11 +1,9 @@
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   198
     $ true
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   199
  -  should go away (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   200
     $ true
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   201
     should stay (false !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   202
   
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   203
   Should remove first line, not second or third
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   204
     $ echo 'testing'
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   205
  -  baz*foo (glob) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   206
     foobar*foo (glob) (false !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   207
     te*ting (glob) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   208
   
33696
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   209
     foo.ar (re) (false !)
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   210
     missing (?)
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   211
  @@ -13,13 +11,10 @@
33695
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   212
     $ echo 'testing'
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   213
     test.ng (re) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   214
     foo.ar (re) (false !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   215
  -  b.r (re) (true !)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   216
     missing (?)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   217
  -  awol (true !)
33696
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   218
   
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   219
   The "missing" line should stay, even though awol is dropped
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   220
     $ echo 'testing'
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   221
     test.ng (re) (true !)
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   222
     foo.ar (?)
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   223
  -  awol
1fa6023240f4 run-tests: don't drop optional lines after a missing unconditional line
Matt Harbison <matt_harbison@yahoo.com>
parents: 33695
diff changeset
   224
     missing (?)
33695
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   225
  
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   226
  ERROR: test-failure.t output changed
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   227
  !
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   228
  Failed test-failure.t: output changed
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   229
  # Ran 1 tests, 0 skipped, 1 failed.
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   230
  python hash seed: * (glob)
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   231
  [1]
eeed23508383 run-tests: drop required (feature !) style lines when the output is missing
Matt Harbison <matt_harbison@yahoo.com>
parents: 33552
diff changeset
   232
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   233
basic failing test
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   234
  $ cat > test-failure.t << EOF
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   235
  >   $ echo babar
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   236
  >   rataxes
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   237
  > This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   238
  > this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   239
  > pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   240
  > pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   241
  > pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   242
  > pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   243
  > pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   244
  > pad pad pad pad............................................................
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   245
  > EOF
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   246
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   247
  >>> fh = open('test-failure-unicode.t', 'wb')
25054
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   248
  >>> fh.write(u'  $ echo babar\u03b1\n'.encode('utf-8')) and None
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   249
  >>> fh.write(u'  l\u03b5\u03b5t\n'.encode('utf-8')) and None
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   250
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   251
  $ rt
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   252
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   253
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   254
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   255
  @@ -1,5 +1,5 @@
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   256
     $ echo babar
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   257
  -  rataxes
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   258
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   259
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   260
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   261
   pad pad pad pad............................................................
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   262
  
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   263
  ERROR: test-failure.t output changed
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   264
  !.
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   265
  --- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   266
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   267
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   268
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   269
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   270
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   271
  
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   272
  ERROR: test-failure-unicode.t output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   273
  !
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   274
  Failed test-failure.t: output changed
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   275
  Failed test-failure-unicode.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   276
  # Ran 3 tests, 0 skipped, 2 failed.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   277
  python hash seed: * (glob)
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   278
  [1]
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   279
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   280
test --outputdir
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   281
  $ mkdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   282
  $ rt --outputdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   283
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   284
  --- $TESTTMP/test-failure.t
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   285
  +++ $TESTTMP/output/test-failure.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   286
  @@ -1,5 +1,5 @@
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   287
     $ echo babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   288
  -  rataxes
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   289
  +  babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   290
   This is a noop statement so that
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   291
   this test is still more bytes than success.
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   292
   pad pad pad pad............................................................
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   293
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   294
  ERROR: test-failure.t output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   295
  !.
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   296
  --- $TESTTMP/test-failure-unicode.t
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   297
  +++ $TESTTMP/output/test-failure-unicode.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   298
  @@ -1,2 +1,2 @@
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   299
     $ echo babar\xce\xb1 (esc)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   300
  -  l\xce\xb5\xce\xb5t (esc)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   301
  +  babar\xce\xb1 (esc)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   302
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   303
  ERROR: test-failure-unicode.t output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   304
  !
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   305
  Failed test-failure.t: output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   306
  Failed test-failure-unicode.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   307
  # Ran 3 tests, 0 skipped, 2 failed.
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   308
  python hash seed: * (glob)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   309
  [1]
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   310
  $ ls -a output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   311
  .
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   312
  ..
32717
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
   313
  .testtimes
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   314
  test-failure-unicode.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   315
  test-failure.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   316
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   317
test --xunit support
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   318
  $ rt --xunit=xunit.xml
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   319
  
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   320
  --- $TESTTMP/test-failure.t
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   321
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   322
  @@ -1,5 +1,5 @@
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   323
     $ echo babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   324
  -  rataxes
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   325
  +  babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   326
   This is a noop statement so that
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   327
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   328
   pad pad pad pad............................................................
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   329
  
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   330
  ERROR: test-failure.t output changed
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   331
  !.
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   332
  --- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   333
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   334
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   335
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   336
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   337
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   338
  
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   339
  ERROR: test-failure-unicode.t output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   340
  !
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   341
  Failed test-failure.t: output changed
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   342
  Failed test-failure-unicode.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   343
  # Ran 3 tests, 0 skipped, 2 failed.
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   344
  python hash seed: * (glob)
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   345
  [1]
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   346
  $ cat xunit.xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   347
  <?xml version="1.0" encoding="utf-8"?>
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   348
  <testsuite errors="0" failures="2" name="run-tests" skipped="0" tests="3">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   349
    <testcase name="test-success.t" time="*"/> (glob)
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   350
    <testcase name="test-failure-unicode.t" time="*"> (glob)
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   351
      <failure message="output changed" type="output-mismatch">
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   352
  <![CDATA[--- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   353
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   354
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   355
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   356
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   357
  +  babar\xce\xb1 (esc)
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   358
  ]]>    </failure>
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   359
    </testcase>
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   360
    <testcase name="test-failure.t" time="*"> (glob)
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   361
      <failure message="output changed" type="output-mismatch">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   362
  <![CDATA[--- $TESTTMP/test-failure.t
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   363
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   364
  @@ -1,5 +1,5 @@
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   365
     $ echo babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   366
  -  rataxes
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   367
  +  babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   368
   This is a noop statement so that
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   369
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   370
   pad pad pad pad............................................................
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   371
  ]]>    </failure>
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   372
    </testcase>
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   373
  </testsuite>
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   374
29280
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   375
  $ cat .testtimes
35855
69d7fcd91696 testrunner: fix updating of .testtimes file
Martin von Zweigbergk <martinvonz@google.com>
parents: 35536
diff changeset
   376
  test-empty.t * (glob)
29280
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   377
  test-failure-unicode.t * (glob)
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   378
  test-failure.t * (glob)
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   379
  test-success.t * (glob)
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   380
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   381
  $ rt --list-tests
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   382
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   383
  test-failure.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   384
  test-success.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   385
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   386
  $ rt --list-tests --json
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   387
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   388
  test-failure.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   389
  test-success.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   390
  $ cat report.json
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   391
  testreport ={
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   392
      "test-failure-unicode.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   393
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   394
      },
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   395
      "test-failure.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   396
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   397
      },
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   398
      "test-success.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   399
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   400
      }
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   401
  } (no-eol)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   402
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   403
  $ rt --list-tests --xunit=xunit.xml
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   404
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   405
  test-failure.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   406
  test-success.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   407
  $ cat xunit.xml
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   408
  <?xml version="1.0" encoding="utf-8"?>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   409
  <testsuite errors="0" failures="0" name="run-tests" skipped="0" tests="0">
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   410
    <testcase name="test-failure-unicode.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   411
    <testcase name="test-failure.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   412
    <testcase name="test-success.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   413
  </testsuite>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   414
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   415
  $ rt --list-tests test-failure* --json --xunit=xunit.xml --outputdir output
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   416
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   417
  test-failure.t
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   418
  $ cat output/report.json
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   419
  testreport ={
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   420
      "test-failure-unicode.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   421
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   422
      },
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   423
      "test-failure.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   424
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   425
      }
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   426
  } (no-eol)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   427
  $ cat xunit.xml
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   428
  <?xml version="1.0" encoding="utf-8"?>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   429
  <testsuite errors="0" failures="0" name="run-tests" skipped="0" tests="0">
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   430
    <testcase name="test-failure-unicode.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   431
    <testcase name="test-failure.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   432
  </testsuite>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   433
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   434
  $ rm test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   435
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   436
test for --retest
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   437
====================
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   438
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   439
  $ rt --retest
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   440
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   441
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   442
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   443
  @@ -1,5 +1,5 @@
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   444
     $ echo babar
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   445
  -  rataxes
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   446
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   447
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   448
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   449
   pad pad pad pad............................................................
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   450
  
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   451
  ERROR: test-failure.t output changed
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   452
  !
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   453
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   454
  # Ran 2 tests, 1 skipped, 1 failed.
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   455
  python hash seed: * (glob)
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   456
  [1]
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   457
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   458
--retest works with --outputdir
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   459
  $ rm -r output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   460
  $ mkdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   461
  $ mv test-failure.t.err output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   462
  $ rt --retest --outputdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   463
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   464
  --- $TESTTMP/test-failure.t
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   465
  +++ $TESTTMP/output/test-failure.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   466
  @@ -1,5 +1,5 @@
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   467
     $ echo babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   468
  -  rataxes
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   469
  +  babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   470
   This is a noop statement so that
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   471
   this test is still more bytes than success.
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   472
   pad pad pad pad............................................................
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   473
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   474
  ERROR: test-failure.t output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   475
  !
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   476
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   477
  # Ran 2 tests, 1 skipped, 1 failed.
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   478
  python hash seed: * (glob)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   479
  [1]
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   480
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   481
Selecting Tests To Run
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   482
======================
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   483
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   484
successful
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   485
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   486
  $ rt test-success.t
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   487
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   488
  # Ran 1 tests, 0 skipped, 0 failed.
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   489
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   490
success w/ keyword
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   491
  $ rt -k xyzzy
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   492
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   493
  # Ran 2 tests, 1 skipped, 0 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   494
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   495
failed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   496
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   497
  $ rt test-failure.t
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   498
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   499
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   500
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   501
  @@ -1,5 +1,5 @@
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   502
     $ echo babar
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   503
  -  rataxes
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   504
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   505
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   506
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   507
   pad pad pad pad............................................................
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   508
  
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   509
  ERROR: test-failure.t output changed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   510
  !
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   511
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   512
  # Ran 1 tests, 0 skipped, 1 failed.
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   513
  python hash seed: * (glob)
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   514
  [1]
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   515
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   516
failure w/ keyword
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   517
  $ rt -k rataxes
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   518
  
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   519
  --- $TESTTMP/test-failure.t
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   520
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   521
  @@ -1,5 +1,5 @@
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   522
     $ echo babar
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   523
  -  rataxes
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   524
  +  babar
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   525
   This is a noop statement so that
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   526
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   527
   pad pad pad pad............................................................
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   528
  
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   529
  ERROR: test-failure.t output changed
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   530
  !
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   531
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   532
  # Ran 2 tests, 1 skipped, 1 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   533
  python hash seed: * (glob)
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   534
  [1]
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   535
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   536
Verify that when a process fails to start we show a useful message
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   537
==================================================================
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   538
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   539
  $ cat > test-serve-fail.t <<EOF
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   540
  >   $ echo 'abort: child process failed to start blah'
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   541
  > EOF
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   542
  $ rt test-serve-fail.t
36461
51a9f0246931 run-tests: resume raising an exception when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36438
diff changeset
   543
  
36438
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 35855
diff changeset
   544
  --- $TESTTMP/test-serve-fail.t
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 35855
diff changeset
   545
  +++ $TESTTMP/test-serve-fail.t.err
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 35855
diff changeset
   546
  @@ -1* +1,2 @@ (glob)
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 35855
diff changeset
   547
     $ echo 'abort: child process failed to start blah'
93228b2a1fc0 run-tests: don't mask errors when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 35855
diff changeset
   548
  +  abort: child process failed to start blah
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   549
  
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   550
  ERROR: test-serve-fail.t output changed
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   551
  !
36461
51a9f0246931 run-tests: resume raising an exception when a server fails to start
Matt Harbison <matt_harbison@yahoo.com>
parents: 36438
diff changeset
   552
  Failed test-serve-fail.t: server failed to start (HGPORT=*) (glob)
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   553
  # Ran 1 tests, 0 skipped, 1 failed.
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   554
  python hash seed: * (glob)
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   555
  [1]
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   556
  $ rm test-serve-fail.t
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   557
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   558
Verify that we can try other ports
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   559
===================================
37434
3fbd8b862d66 tests: work around potential repo incompatibility
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36677
diff changeset
   560
3fbd8b862d66 tests: work around potential repo incompatibility
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36677
diff changeset
   561
Extensions aren't inherited by the invoked run-tests.py. An extension
3fbd8b862d66 tests: work around potential repo incompatibility
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36677
diff changeset
   562
introducing a repository requirement could cause this to fail. So we force
3fbd8b862d66 tests: work around potential repo incompatibility
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36677
diff changeset
   563
HGRCPATH to get a clean environment.
3fbd8b862d66 tests: work around potential repo incompatibility
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36677
diff changeset
   564
3fbd8b862d66 tests: work around potential repo incompatibility
Gregory Szorc <gregory.szorc@gmail.com>
parents: 36677
diff changeset
   565
  $ HGRCPATH= hg init inuse
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   566
  $ hg serve -R inuse -p $HGPORT -d --pid-file=blocks.pid
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   567
  $ cat blocks.pid >> $DAEMON_PIDS
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   568
  $ cat > test-serve-inuse.t <<EOF
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   569
  >   $ hg serve -R `pwd`/inuse -p \$HGPORT -d --pid-file=hg.pid
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   570
  >   $ cat hg.pid >> \$DAEMON_PIDS
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   571
  > EOF
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   572
  $ rt test-serve-inuse.t
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   573
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   574
  # Ran 1 tests, 0 skipped, 0 failed.
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   575
  $ rm test-serve-inuse.t
29221
73f58eb14684 test-run-tests: clean up inuse server eagerly
timeless <timeless@mozdev.org>
parents: 29220
diff changeset
   576
  $ killdaemons.py $DAEMON_PIDS
73f58eb14684 test-run-tests: clean up inuse server eagerly
timeless <timeless@mozdev.org>
parents: 29220
diff changeset
   577
  $ rm $DAEMON_PIDS
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   578
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   579
Running In Debug Mode
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   580
======================
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   581
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   582
  $ rt --debug 2>&1 | grep -v pwd
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   583
  + echo *SALT* 0 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   584
  *SALT* 0 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   585
  + echo babar
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   586
  babar
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   587
  + echo *SALT* 10 0 (glob)
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   588
  *SALT* 10 0 (glob)
27509
6ff90d618d72 test-run-tests: glob away a --debug run difference on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 27395
diff changeset
   589
  *+ echo *SALT* 0 0 (glob)
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   590
  *SALT* 0 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   591
  + echo babar
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   592
  babar
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   593
  + echo *SALT* 2 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   594
  *SALT* 2 0 (glob)
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   595
  + echo xyzzy
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   596
  xyzzy
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   597
  + echo *SALT* 9 0 (glob)
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   598
  *SALT* 9 0 (glob)
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   599
  + printf *abc\ndef\nxyz\n* (glob)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   600
  abc
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   601
  def
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   602
  xyz
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   603
  + echo *SALT* 15 0 (glob)
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   604
  *SALT* 15 0 (glob)
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   605
  + printf *zyx\nwvu\ntsr\n* (glob)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   606
  zyx
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   607
  wvu
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   608
  tsr
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   609
  + echo *SALT* 22 0 (glob)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   610
  *SALT* 22 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   611
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   612
  # Ran 2 tests, 0 skipped, 0 failed.
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   613
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   614
Parallel runs
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   615
==============
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   616
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   617
(duplicate the failing test to get predictable output)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   618
  $ cp test-failure.t test-failure-copy.t
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   619
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   620
  $ rt --jobs 2 test-failure*.t -n
23107
5459b30aa498 tests: silence output race in test-run-tests.t
Matt Mackall <mpm@selenic.com>
parents: 22840
diff changeset
   621
  !!
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   622
  Failed test-failure*.t: output changed (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   623
  Failed test-failure*.t: output changed (glob)
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   624
  # Ran 2 tests, 0 skipped, 2 failed.
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   625
  python hash seed: * (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   626
  [1]
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   627
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   628
failures in parallel with --first should only print one failure
34894
b63a7d839c33 tests: de-flake test-run-tests.t's "--jobs=2 --first" test
Martin von Zweigbergk <martinvonz@google.com>
parents: 34842
diff changeset
   629
  $ rt --jobs 2 --first test-failure*.t
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   630
  
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   631
  --- $TESTTMP/test-failure*.t (glob)
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   632
  +++ $TESTTMP/test-failure*.t.err (glob)
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   633
  @@ -1,5 +1,5 @@
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   634
     $ echo babar
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   635
  -  rataxes
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   636
  +  babar
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   637
   This is a noop statement so that
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   638
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   639
   pad pad pad pad............................................................
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   640
  
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   641
  Failed test-failure*.t: output changed (glob)
34894
b63a7d839c33 tests: de-flake test-run-tests.t's "--jobs=2 --first" test
Martin von Zweigbergk <martinvonz@google.com>
parents: 34842
diff changeset
   642
  Failed test-failure*.t: output changed (glob)
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   643
  # Ran 2 tests, 0 skipped, 2 failed.
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   644
  python hash seed: * (glob)
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   645
  [1]
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   646
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   647
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   648
(delete the duplicated test file)
34894
b63a7d839c33 tests: de-flake test-run-tests.t's "--jobs=2 --first" test
Martin von Zweigbergk <martinvonz@google.com>
parents: 34842
diff changeset
   649
  $ rm test-failure-copy.t
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   650
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   651
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   652
Interactive run
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   653
===============
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   654
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   655
(backup the failing test)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   656
  $ cp test-failure.t backup
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   657
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   658
Refuse the fix
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   659
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   660
  $ echo 'n' | rt -i
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   661
  
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   662
  --- $TESTTMP/test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   663
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   664
  @@ -1,5 +1,5 @@
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   665
     $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   666
  -  rataxes
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   667
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   668
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   669
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   670
   pad pad pad pad............................................................
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21756
diff changeset
   671
  Accept this change? [n] 
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   672
  ERROR: test-failure.t output changed
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21756
diff changeset
   673
  !.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   674
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   675
  # Ran 2 tests, 0 skipped, 1 failed.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   676
  python hash seed: * (glob)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   677
  [1]
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   678
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   679
  $ cat test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   680
    $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   681
    rataxes
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   682
  This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   683
  this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   684
  pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   685
  pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   686
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   687
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   688
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   689
  pad pad pad pad............................................................
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   690
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   691
Interactive with custom view
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   692
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   693
  $ echo 'n' | rt -i --view echo
35393
4441705b7111 tests: remove (glob) annotations that were only for '\' matches
Matt Harbison <matt_harbison@yahoo.com>
parents: 35391
diff changeset
   694
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   695
  Accept this change? [n]* (glob)
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   696
  ERROR: test-failure.t output changed
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   697
  !.
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   698
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   699
  # Ran 2 tests, 0 skipped, 1 failed.
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   700
  python hash seed: * (glob)
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   701
  [1]
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   702
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   703
View the fix
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   704
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   705
  $ echo 'y' | rt --view echo
35393
4441705b7111 tests: remove (glob) annotations that were only for '\' matches
Matt Harbison <matt_harbison@yahoo.com>
parents: 35391
diff changeset
   706
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   707
  
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   708
  ERROR: test-failure.t output changed
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   709
  !.
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   710
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   711
  # Ran 2 tests, 0 skipped, 1 failed.
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   712
  python hash seed: * (glob)
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   713
  [1]
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   714
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   715
Accept the fix
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   716
35416
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   717
  $ cat >> test-failure.t <<EOF
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   718
  >   $ echo 'saved backup bundle to \$TESTTMP/foo.hg'
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   719
  >   saved backup bundle to \$TESTTMP/foo.hg
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   720
  >   $ echo 'saved backup bundle to \$TESTTMP/foo.hg'
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   721
  >   saved backup bundle to $TESTTMP\\foo.hg
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   722
  >   $ echo 'saved backup bundle to \$TESTTMP/foo.hg'
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   723
  >   saved backup bundle to \$TESTTMP/*.hg (glob)
417fa23017f9 tests: avoid echo with backslash escapes
Martin von Zweigbergk <martinvonz@google.com>
parents: 35393
diff changeset
   724
  > EOF
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   725
  $ echo 'y' | rt -i 2>&1
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   726
  
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   727
  --- $TESTTMP/test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   728
  +++ $TESTTMP/test-failure.t.err
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   729
  @@ -1,5 +1,5 @@
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   730
     $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   731
  -  rataxes
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   732
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   733
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   734
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   735
   pad pad pad pad............................................................
35391
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   736
  @@ -11,6 +11,6 @@
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   737
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   738
     saved backup bundle to $TESTTMP/foo.hg
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   739
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
35391
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   740
  -  saved backup bundle to $TESTTMP\foo.hg
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   741
  +  saved backup bundle to $TESTTMP/foo.hg
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   742
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
35391
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   743
     saved backup bundle to $TESTTMP/*.hg (glob)
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   744
  Accept this change? [n] ..
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   745
  # Ran 2 tests, 0 skipped, 0 failed.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   746
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   747
  $ sed -e 's,(glob)$,&<,g' test-failure.t
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   748
    $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   749
    babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   750
  This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   751
  this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   752
  pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   753
  pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   754
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   755
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   756
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   757
  pad pad pad pad............................................................
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   758
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
35391
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   759
    saved backup bundle to $TESTTMP/foo.hg
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   760
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
35391
0432e54f4dbe run-tests: stop automatically adding a (glob) for bundle backup lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 35387
diff changeset
   761
    saved backup bundle to $TESTTMP/foo.hg
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   762
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   763
    saved backup bundle to $TESTTMP/*.hg (glob)<
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   764
32980
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   765
Race condition - test file was modified when test is running
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   766
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   767
  $ TESTRACEDIR=`pwd`
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   768
  $ export TESTRACEDIR
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   769
  $ cat > test-race.t <<EOF
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   770
  >   $ echo 1
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   771
  >   $ echo "# a new line" >> $TESTRACEDIR/test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   772
  > EOF
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   773
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   774
  $ rt -i test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   775
  
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   776
  --- $TESTTMP/test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   777
  +++ $TESTTMP/test-race.t.err
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   778
  @@ -1,2 +1,3 @@
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   779
     $ echo 1
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   780
  +  1
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   781
     $ echo "# a new line" >> $TESTTMP/test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   782
  Reference output has changed (run again to prompt changes)
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   783
  ERROR: test-race.t output changed
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   784
  !
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   785
  Failed test-race.t: output changed
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   786
  # Ran 1 tests, 0 skipped, 1 failed.
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   787
  python hash seed: * (glob)
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   788
  [1]
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   789
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   790
  $ rm test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   791
32982
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   792
When "#testcases" is used in .t files
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   793
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   794
  $ cat >> test-cases.t <<EOF
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   795
  > #testcases a b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   796
  > #if a
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   797
  >   $ echo 1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   798
  > #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   799
  > #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   800
  >   $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   801
  > #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   802
  > EOF
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   803
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   804
  $ cat <<EOF | rt -i test-cases.t 2>&1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   805
  > y
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   806
  > y
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   807
  > EOF
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   808
  
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   809
  --- $TESTTMP/test-cases.t
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   810
  +++ $TESTTMP/test-cases.t.a.err
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   811
  @@ -1,6 +1,7 @@
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   812
   #testcases a b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   813
   #if a
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   814
     $ echo 1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   815
  +  1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   816
   #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   817
   #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   818
     $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   819
  Accept this change? [n] .
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   820
  --- $TESTTMP/test-cases.t
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   821
  +++ $TESTTMP/test-cases.t.b.err
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   822
  @@ -5,4 +5,5 @@
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   823
   #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   824
   #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   825
     $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   826
  +  2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   827
   #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   828
  Accept this change? [n] .
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   829
  # Ran 2 tests, 0 skipped, 0 failed.
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   830
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   831
  $ cat test-cases.t
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   832
  #testcases a b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   833
  #if a
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   834
    $ echo 1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   835
    1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   836
  #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   837
  #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   838
    $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   839
    2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   840
  #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   841
33934
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   842
  $ cat >> test-cases.t <<'EOF'
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   843
  > #if a
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   844
  >   $ NAME=A
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   845
  > #else
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   846
  >   $ NAME=B
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   847
  > #endif
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   848
  >   $ echo $NAME
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   849
  >   A (a !)
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   850
  >   B (b !)
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   851
  > EOF
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   852
  $ rt test-cases.t
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   853
  ..
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   854
  # Ran 2 tests, 0 skipped, 0 failed.
6cc8f848b4c3 run-tests: make per-line condition support testcase names
Jun Wu <quark@fb.com>
parents: 33814
diff changeset
   855
32982
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   856
  $ rm test-cases.t
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   857
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   858
(reinstall)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   859
  $ mv backup test-failure.t
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   860
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   861
No Diff
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   862
===============
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   863
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   864
  $ rt --nodiff
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   865
  !.
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   866
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   867
  # Ran 2 tests, 0 skipped, 1 failed.
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   868
  python hash seed: * (glob)
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   869
  [1]
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   870
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   871
test --tmpdir support
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   872
  $ rt --tmpdir=$TESTTMP/keep test-success.t
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   873
  
35393
4441705b7111 tests: remove (glob) annotations that were only for '\' matches
Matt Harbison <matt_harbison@yahoo.com>
parents: 35391
diff changeset
   874
  Keeping testtmp dir: $TESTTMP/keep/child1/test-success.t
4441705b7111 tests: remove (glob) annotations that were only for '\' matches
Matt Harbison <matt_harbison@yahoo.com>
parents: 35391
diff changeset
   875
  Keeping threadtmp dir: $TESTTMP/keep/child1 
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   876
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   877
  # Ran 1 tests, 0 skipped, 0 failed.
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   878
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   879
timeouts
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   880
========
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   881
  $ cat > test-timeout.t <<EOF
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   882
  >   $ sleep 2
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   883
  >   $ echo pass
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   884
  >   pass
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   885
  > EOF
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   886
  > echo '#require slow' > test-slow-timeout.t
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   887
  > cat test-timeout.t >> test-slow-timeout.t
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   888
  $ rt --timeout=1 --slowtimeout=3 test-timeout.t test-slow-timeout.t
27393
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
   889
  st
32474
c2b7fb580794 tests: hint how to run slow tests when rejecting
Kyle Lippincott <spectral@google.com>
parents: 32316
diff changeset
   890
  Skipped test-slow-timeout.t: missing feature: allow slow tests (use --allow-slow-tests)
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   891
  Failed test-timeout.t: timed out
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   892
  # Ran 1 tests, 1 skipped, 1 failed.
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   893
  python hash seed: * (glob)
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   894
  [1]
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   895
  $ rt --timeout=1 --slowtimeout=3 \
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   896
  > test-timeout.t test-slow-timeout.t --allow-slow-tests
27393
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
   897
  .t
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   898
  Failed test-timeout.t: timed out
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   899
  # Ran 2 tests, 0 skipped, 1 failed.
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   900
  python hash seed: * (glob)
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   901
  [1]
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   902
  $ rm test-timeout.t test-slow-timeout.t
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   903
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   904
test for --time
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   905
==================
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   906
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   907
  $ rt test-success.t --time
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   908
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   909
  # Ran 1 tests, 0 skipped, 0 failed.
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   910
  # Producing time report
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   911
  start   end     cuser   csys    real      Test
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   912
  \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   test-success.t (re)
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   913
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   914
test for --time with --job enabled
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   915
====================================
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   916
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   917
  $ rt test-success.t --time --jobs 2
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   918
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   919
  # Ran 1 tests, 0 skipped, 0 failed.
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   920
  # Producing time report
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   921
  start   end     cuser   csys    real      Test
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   922
  \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   test-success.t (re)
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   923
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   924
Skips
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   925
================
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   926
  $ cat > test-skip.t <<EOF
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   927
  >   $ echo xyzzy
36677
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
   928
  > #if true
22045
769198c6a62d run-tests: add #require to abort full test
Matt Mackall <mpm@selenic.com>
parents: 22044
diff changeset
   929
  > #require false
36677
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
   930
  > #end
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
   931
  > EOF
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
   932
  $ cat > test-noskip.t <<EOF
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
   933
  > #if false
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
   934
  > #require false
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
   935
  > #endif
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   936
  > EOF
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   937
  $ rt --nodiff
36677
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
   938
  !.s.
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   939
  Skipped test-skip.t: missing feature: nail clipper
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   940
  Failed test-failure.t: output changed
36677
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
   941
  # Ran 3 tests, 1 skipped, 1 failed.
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   942
  python hash seed: * (glob)
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   943
  [1]
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   944
36677
a247a0e82e7d run-tests: allow #require inside #if
Jun Wu <quark@fb.com>
parents: 36461
diff changeset
   945
  $ rm test-noskip.t
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   946
  $ rt --keyword xyzzy
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   947
  .s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   948
  Skipped test-skip.t: missing feature: nail clipper
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   949
  # Ran 2 tests, 2 skipped, 0 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   950
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   951
Skips with xml
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   952
  $ rt --keyword xyzzy \
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   953
  >  --xunit=xunit.xml
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   954
  .s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   955
  Skipped test-skip.t: missing feature: nail clipper
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   956
  # Ran 2 tests, 2 skipped, 0 failed.
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   957
  $ cat xunit.xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   958
  <?xml version="1.0" encoding="utf-8"?>
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   959
  <testsuite errors="0" failures="0" name="run-tests" skipped="2" tests="2">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   960
    <testcase name="test-success.t" time="*"/> (glob)
32715
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
   961
    <testcase name="test-skip.t">
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
   962
      <skipped>
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
   963
  <![CDATA[missing feature: nail clipper]]>    </skipped>
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
   964
    </testcase>
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   965
  </testsuite>
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   966
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   967
Missing skips or blacklisted skips don't count as executed:
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   968
  $ echo test-failure.t > blacklist
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   969
  $ rt --blacklist=blacklist --json\
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   970
  >   test-failure.t test-bogus.t
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   971
  ss
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   972
  Skipped test-bogus.t: Doesn't exist
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   973
  Skipped test-failure.t: blacklisted
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   974
  # Ran 0 tests, 2 skipped, 0 failed.
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   975
  $ cat report.json
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   976
  testreport ={
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   977
      "test-bogus.t": {
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   978
          "result": "skip"
29199
daff05dcd184 run-tests: handle json.dumps divergence
timeless <timeless@mozdev.org>
parents: 29173
diff changeset
   979
      },
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   980
      "test-failure.t": {
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   981
          "result": "skip"
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   982
      }
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   983
  } (no-eol)
29173
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   984
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   985
Whitelist trumps blacklist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   986
  $ echo test-failure.t > whitelist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   987
  $ rt --blacklist=blacklist --whitelist=whitelist --json\
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   988
  >   test-failure.t test-bogus.t
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   989
  s
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   990
  --- $TESTTMP/test-failure.t
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   991
  +++ $TESTTMP/test-failure.t.err
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   992
  @@ -1,5 +1,5 @@
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   993
     $ echo babar
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   994
  -  rataxes
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   995
  +  babar
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   996
   This is a noop statement so that
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   997
   this test is still more bytes than success.
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   998
   pad pad pad pad............................................................
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   999
  
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1000
  ERROR: test-failure.t output changed
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1001
  !
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1002
  Skipped test-bogus.t: Doesn't exist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1003
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1004
  # Ran 1 tests, 1 skipped, 1 failed.
29173
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1005
  python hash seed: * (glob)
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1006
  [1]
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
  1007
34263
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1008
Ensure that --test-list causes only the tests listed in that file to
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1009
be executed.
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1010
  $ echo test-success.t >> onlytest
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1011
  $ rt --test-list=onlytest
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1012
  .
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1013
  # Ran 1 tests, 0 skipped, 0 failed.
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1014
  $ echo test-bogus.t >> anothertest
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1015
  $ rt --test-list=onlytest --test-list=anothertest
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1016
  s.
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1017
  Skipped test-bogus.t: Doesn't exist
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1018
  # Ran 1 tests, 1 skipped, 0 failed.
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1019
  $ rm onlytest anothertest
1533371769b5 tests: add support for listing tests to run in a file
Augie Fackler <augie@google.com>
parents: 34041
diff changeset
  1020
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1021
test for --json
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1022
==================
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1023
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1024
  $ rt --json
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1025
  
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1026
  --- $TESTTMP/test-failure.t
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1027
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
  1028
  @@ -1,5 +1,5 @@
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1029
     $ echo babar
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1030
  -  rataxes
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1031
  +  babar
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1032
   This is a noop statement so that
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1033
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
  1034
   pad pad pad pad............................................................
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1035
  
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1036
  ERROR: test-failure.t output changed
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1037
  !.s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
  1038
  Skipped test-skip.t: missing feature: nail clipper
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1039
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1040
  # Ran 2 tests, 1 skipped, 1 failed.
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1041
  python hash seed: * (glob)
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1042
  [1]
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1043
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1044
  $ cat report.json
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1045
  testreport ={
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1046
      "test-failure.t": [\{] (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1047
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1048
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1049
          "diff": "---.+\+\+\+.+", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1050
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1051
          "result": "failure", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1052
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1053
          "time": "\s*[\d\.]{4,5}" (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1054
      }, ? (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1055
      "test-skip.t": {
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1056
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1057
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1058
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1059
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1060
          "result": "skip", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1061
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1062
          "time": "\s*[\d\.]{4,5}" (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1063
      }, ? (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1064
      "test-success.t": [\{] (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1065
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1066
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1067
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1068
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1069
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1070
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
  1071
          "time": "\s*[\d\.]{4,5}" (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1072
      }
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
  1073
  } (no-eol)
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1074
--json with --outputdir
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1075
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1076
  $ rm report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1077
  $ rm -r output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1078
  $ mkdir output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1079
  $ rt --json --outputdir output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1080
  
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1081
  --- $TESTTMP/test-failure.t
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1082
  +++ $TESTTMP/output/test-failure.t.err
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1083
  @@ -1,5 +1,5 @@
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1084
     $ echo babar
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1085
  -  rataxes
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1086
  +  babar
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1087
   This is a noop statement so that
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1088
   this test is still more bytes than success.
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1089
   pad pad pad pad............................................................
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1090
  
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1091
  ERROR: test-failure.t output changed
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1092
  !.s
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1093
  Skipped test-skip.t: missing feature: nail clipper
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1094
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1095
  # Ran 2 tests, 1 skipped, 1 failed.
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1096
  python hash seed: * (glob)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1097
  [1]
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1098
  $ f report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1099
  report.json: file not found
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1100
  $ cat output/report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1101
  testreport ={
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1102
      "test-failure.t": [\{] (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1103
          "csys": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1104
          "cuser": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1105
          "diff": "---.+\+\+\+.+", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1106
          "end": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1107
          "result": "failure", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1108
          "start": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1109
          "time": "\s*[\d\.]{4,5}" (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1110
      }, ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1111
      "test-skip.t": {
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1112
          "csys": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1113
          "cuser": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1114
          "diff": "", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1115
          "end": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1116
          "result": "skip", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1117
          "start": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1118
          "time": "\s*[\d\.]{4,5}" (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1119
      }, ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1120
      "test-success.t": [\{] (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1121
          "csys": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1122
          "cuser": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1123
          "diff": "", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1124
          "end": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1125
          "result": "success", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1126
          "start": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1127
          "time": "\s*[\d\.]{4,5}" (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1128
      }
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1129
  } (no-eol)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1130
  $ ls -a output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1131
  .
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1132
  ..
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1133
  .testtimes
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1134
  report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1135
  test-failure.t.err
22579
36a940d82f88 test: protect the run-tests.py --json test behind an hghave rule
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22486
diff changeset
  1136
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1137
Test that failed test accepted through interactive are properly reported:
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1138
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1139
  $ cp test-failure.t backup
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1140
  $ echo y | rt --json -i
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1141
  
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1142
  --- $TESTTMP/test-failure.t
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1143
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
  1144
  @@ -1,5 +1,5 @@
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1145
     $ echo babar
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1146
  -  rataxes
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1147
  +  babar
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1148
   This is a noop statement so that
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1149
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
  1150
   pad pad pad pad............................................................
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1151
  Accept this change? [n] ..s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
  1152
  Skipped test-skip.t: missing feature: nail clipper
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1153
  # Ran 2 tests, 1 skipped, 0 failed.
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1154
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1155
  $ cat report.json
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1156
  testreport ={
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1157
      "test-failure.t": [\{] (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1158
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1159
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1160
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1161
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1162
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1163
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1164
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1165
      }, ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1166
      "test-skip.t": {
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1167
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1168
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1169
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1170
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1171
          "result": "skip", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1172
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1173
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1174
      }, ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1175
      "test-success.t": [\{] (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1176
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1177
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1178
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1179
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1180
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1181
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1182
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1183
      }
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1184
  } (no-eol)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1185
  $ mv backup test-failure.t
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1186
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1187
backslash on end of line with glob matching is handled properly
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1188
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1189
  $ cat > test-glob-backslash.t << EOF
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1190
  >   $ echo 'foo bar \\'
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1191
  >   foo * \ (glob)
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1192
  > EOF
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1193
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1194
  $ rt test-glob-backslash.t
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1195
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1196
  # Ran 1 tests, 0 skipped, 0 failed.
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1197
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1198
  $ rm -f test-glob-backslash.t
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1199
31673
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1200
Test globbing of local IP addresses
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29280
diff changeset
  1201
  $ echo 172.16.18.1
31673
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1202
  $LOCALIP (glob)
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1203
  $ echo dead:beef::1
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1204
  $LOCALIP (glob)
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29280
diff changeset
  1205
25728
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1206
Test reusability for third party tools
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1207
======================================
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1208
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1209
  $ mkdir "$TESTTMP"/anothertests
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1210
  $ cd "$TESTTMP"/anothertests
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1211
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1212
test that `run-tests.py` can execute hghave, even if it runs not in
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1213
Mercurial source tree.
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1214
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1215
  $ cat > test-hghave.t <<EOF
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1216
  > #require true
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1217
  >   $ echo foo
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1218
  >   foo
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1219
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1220
  $ rt test-hghave.t
25728
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1221
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1222
  # Ran 1 tests, 0 skipped, 0 failed.
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1223
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1224
test that RUNTESTDIR refers the directory, in which `run-tests.py` now
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1225
running is placed.
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1226
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1227
  $ cat > test-runtestdir.t <<EOF
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1228
  > - $TESTDIR, in which test-run-tests.t is placed
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1229
  > - \$TESTDIR, in which test-runtestdir.t is placed (expanded at runtime)
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1230
  > - \$RUNTESTDIR, in which run-tests.py is placed (expanded at runtime)
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1231
  > 
27057
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1232
  > #if windows
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1233
  >   $ test "\$TESTDIR" = "$TESTTMP\anothertests"
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1234
  > #else
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1235
  >   $ test "\$TESTDIR" = "$TESTTMP"/anothertests
27057
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1236
  > #endif
35387
c0c6df81c9bb tests: add some commentary and diagnostics to test-run-tests.t
Augie Fackler <augie@google.com>
parents: 35241
diff changeset
  1237
  > If this prints a path, that means RUNTESTDIR didn't equal
c0c6df81c9bb tests: add some commentary and diagnostics to test-run-tests.t
Augie Fackler <augie@google.com>
parents: 35241
diff changeset
  1238
  > TESTDIR as it should have.
c0c6df81c9bb tests: add some commentary and diagnostics to test-run-tests.t
Augie Fackler <augie@google.com>
parents: 35241
diff changeset
  1239
  >   $ test "\$RUNTESTDIR" = "$TESTDIR" || echo "\$RUNTESTDIR"
c0c6df81c9bb tests: add some commentary and diagnostics to test-run-tests.t
Augie Fackler <augie@google.com>
parents: 35241
diff changeset
  1240
  > This should print the start of check-code. If this passes but the
c0c6df81c9bb tests: add some commentary and diagnostics to test-run-tests.t
Augie Fackler <augie@google.com>
parents: 35241
diff changeset
  1241
  > previous check failed, that means we found a copy of check-code at whatever
c0c6df81c9bb tests: add some commentary and diagnostics to test-run-tests.t
Augie Fackler <augie@google.com>
parents: 35241
diff changeset
  1242
  > RUNTESTSDIR ended up containing, even though it doesn't match TESTDIR.
32937
7fe1f9785c75 tests: sed away python #! in test-run-tests.t to avoid some upcoming insanity
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
  1243
  >   $ head -n 3 "\$RUNTESTDIR"/../contrib/check-code.py | sed 's@.!.*python@#!USRBINENVPY@'
7fe1f9785c75 tests: sed away python #! in test-run-tests.t to avoid some upcoming insanity
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
  1244
  >   #!USRBINENVPY
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1245
  >   #
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1246
  >   # check-code - a style and portability checker for Mercurial
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1247
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1248
  $ rt test-runtestdir.t
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1249
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1250
  # Ran 1 tests, 0 skipped, 0 failed.
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1251
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1252
#if execbit
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1253
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1254
test that TESTDIR is referred in PATH
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1255
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1256
  $ cat > custom-command.sh <<EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1257
  > #!/bin/sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1258
  > echo "hello world"
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1259
  > EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1260
  $ chmod +x custom-command.sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1261
  $ cat > test-testdir-path.t <<EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1262
  >   $ custom-command.sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1263
  >   hello world
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1264
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1265
  $ rt test-testdir-path.t
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1266
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1267
  # Ran 1 tests, 0 skipped, 0 failed.
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1268
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1269
#endif
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1270
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1271
test support for --allow-slow-tests
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1272
  $ cat > test-very-slow-test.t <<EOF
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1273
  > #require slow
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1274
  >   $ echo pass
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1275
  >   pass
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1276
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1277
  $ rt test-very-slow-test.t
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1278
  s
32474
c2b7fb580794 tests: hint how to run slow tests when rejecting
Kyle Lippincott <spectral@google.com>
parents: 32316
diff changeset
  1279
  Skipped test-very-slow-test.t: missing feature: allow slow tests (use --allow-slow-tests)
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1280
  # Ran 0 tests, 1 skipped, 0 failed.
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1281
  $ rt $HGTEST_RUN_TESTS_PURE --allow-slow-tests test-very-slow-test.t
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1282
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1283
  # Ran 1 tests, 0 skipped, 0 failed.
28180
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1284
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1285
support for running a test outside the current directory
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1286
  $ mkdir nonlocal
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1287
  $ cat > nonlocal/test-is-not-here.t << EOF
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1288
  >   $ echo pass
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1289
  >   pass
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1290
  > EOF
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1291
  $ rt nonlocal/test-is-not-here.t
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1292
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1293
  # Ran 1 tests, 0 skipped, 0 failed.
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1294
34969
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1295
support for automatically discovering test if arg is a folder
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1296
  $ mkdir tmp && cd tmp
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1297
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1298
  $ cat > test-uno.t << EOF
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1299
  >   $ echo line
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1300
  >   line
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1301
  > EOF
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1302
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1303
  $ cp test-uno.t test-dos.t
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1304
  $ cd ..
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1305
  $ cp -R tmp tmpp
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1306
  $ cp tmp/test-uno.t test-solo.t
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1307
35241
2a99c5bfea47 test-run-tests: do not rebuild hg in the test
Jun Wu <quark@fb.com>
parents: 35189
diff changeset
  1308
  $ rt tmp/ test-solo.t tmpp
34969
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1309
  .....
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1310
  # Ran 5 tests, 0 skipped, 0 failed.
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1311
  $ rm -rf tmp tmpp
d600bda4a3e1 run-tests: allow automatic test discovery when providing folder as argument
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34962
diff changeset
  1312
34962
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1313
support for running run-tests.py from another directory
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1314
  $ mkdir tmp && cd tmp
35097
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1315
34962
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1316
  $ cat > useful-file.sh << EOF
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1317
  > important command
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1318
  > EOF
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1319
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1320
  $ cat > test-folder.t << EOF
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1321
  >   $ cat \$TESTDIR/useful-file.sh
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1322
  >   important command
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1323
  > EOF
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1324
35097
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1325
  $ cat > test-folder-fail.t << EOF
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1326
  >   $ cat \$TESTDIR/useful-file.sh
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1327
  >   important commando
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1328
  > EOF
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1329
34962
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1330
  $ cd ..
35241
2a99c5bfea47 test-run-tests: do not rebuild hg in the test
Jun Wu <quark@fb.com>
parents: 35189
diff changeset
  1331
  $ rt tmp/test-*.t
35097
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1332
  
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1333
  --- $TESTTMP/anothertests/tmp/test-folder-fail.t
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1334
  +++ $TESTTMP/anothertests/tmp/test-folder-fail.t.err
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1335
  @@ -1,2 +1,2 @@
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1336
     $ cat $TESTDIR/useful-file.sh
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1337
  -  important commando
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1338
  +  important command
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1339
  
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1340
  ERROR: test-folder-fail.t output changed
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1341
  !.
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1342
  Failed test-folder-fail.t: output changed
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1343
  # Ran 2 tests, 0 skipped, 1 failed.
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1344
  python hash seed: * (glob)
fc0f3ed071fc run-tests: outputdir also has to be changed if $TESTDIR is not $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 35069
diff changeset
  1345
  [1]
34962
a18eef03d879 run-tests: $TESTDIR can be something else than $PWD
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 34894
diff changeset
  1346
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1347
support for bisecting failed tests automatically
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1348
  $ hg init bisect
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1349
  $ cd bisect
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1350
  $ cat >> test-bisect.t <<EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1351
  >   $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1352
  >   pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1353
  > EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1354
  $ hg add test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1355
  $ hg ci -m 'good'
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1356
  $ cat >> test-bisect.t <<EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1357
  >   $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1358
  >   fail
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1359
  > EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1360
  $ hg ci -m 'bad'
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1361
  $ rt --known-good-rev=0 test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1362
  
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1363
  --- $TESTTMP/anothertests/bisect/test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1364
  +++ $TESTTMP/anothertests/bisect/test-bisect.t.err
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1365
  @@ -1,4 +1,4 @@
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1366
     $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1367
     pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1368
     $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1369
  -  fail
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1370
  +  pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1371
  
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1372
  ERROR: test-bisect.t output changed
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1373
  !
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1374
  Failed test-bisect.t: output changed
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1375
  test-bisect.t broken by 72cbf122d116 (bad)
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1376
  # Ran 1 tests, 0 skipped, 1 failed.
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1377
  python hash seed: * (glob)
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1378
  [1]
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1379
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1380
  $ cd ..
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1381
34041
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1382
support bisecting a separate repo
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1383
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1384
  $ hg init bisect-dependent
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1385
  $ cd bisect-dependent
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1386
  $ cat > test-bisect-dependent.t <<EOF
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1387
  >   $ tail -1 \$TESTDIR/../bisect/test-bisect.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1388
  >     pass
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1389
  > EOF
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1390
  $ hg commit -Am dependent test-bisect-dependent.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1391
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1392
  $ rt --known-good-rev=0 test-bisect-dependent.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1393
  
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1394
  --- $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1395
  +++ $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t.err
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1396
  @@ -1,2 +1,2 @@
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1397
     $ tail -1 $TESTDIR/../bisect/test-bisect.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1398
  -    pass
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1399
  +    fail
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1400
  
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1401
  ERROR: test-bisect-dependent.t output changed
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1402
  !
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1403
  Failed test-bisect-dependent.t: output changed
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1404
  Failed to identify failure point for test-bisect-dependent.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1405
  # Ran 1 tests, 0 skipped, 1 failed.
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1406
  python hash seed: * (glob)
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1407
  [1]
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1408
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1409
  $ rt --bisect-repo=../test-bisect test-bisect-dependent.t
35187
b4b0aed7bfaf run-tests: convert to argparse
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35155
diff changeset
  1410
  usage: run-tests.py [options] [tests]
34041
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1411
  run-tests.py: error: --bisect-repo cannot be used without --known-good-rev
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1412
  [2]
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1413
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1414
  $ rt --known-good-rev=0 --bisect-repo=../bisect test-bisect-dependent.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1415
  
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1416
  --- $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1417
  +++ $TESTTMP/anothertests/bisect-dependent/test-bisect-dependent.t.err
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1418
  @@ -1,2 +1,2 @@
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1419
     $ tail -1 $TESTDIR/../bisect/test-bisect.t
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1420
  -    pass
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1421
  +    fail
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1422
  
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1423
  ERROR: test-bisect-dependent.t output changed
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1424
  !
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1425
  Failed test-bisect-dependent.t: output changed
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1426
  test-bisect-dependent.t broken by 72cbf122d116 (bad)
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1427
  # Ran 1 tests, 0 skipped, 1 failed.
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1428
  python hash seed: * (glob)
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1429
  [1]
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1430
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1431
  $ cd ..
40313c63da87 run-tests: allow bisecting a different repo
Jun Wu <quark@fb.com>
parents: 33934
diff changeset
  1432
32622
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1433
Test a broken #if statement doesn't break run-tests threading.
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1434
==============================================================
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1435
  $ mkdir broken
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1436
  $ cd broken
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1437
  $ cat > test-broken.t <<EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1438
  > true
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1439
  > #if notarealhghavefeature
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1440
  >   $ false
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1441
  > #endif
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1442
  > EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1443
  $ for f in 1 2 3 4 ; do
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1444
  > cat > test-works-$f.t <<EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1445
  > This is test case $f
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1446
  >   $ sleep 1
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1447
  > EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1448
  > done
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1449
  $ rt -j 2
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1450
  ....
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1451
  # Ran 5 tests, 0 skipped, 0 failed.
32622
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1452
  skipped: unknown feature: notarealhghavefeature
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1453
  
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1454
  $ cd ..
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1455
  $ rm -rf broken
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1456
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1457
Test cases in .t files
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1458
======================
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1459
  $ mkdir cases
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1460
  $ cd cases
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1461
  $ cat > test-cases-abc.t <<'EOF'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1462
  > #testcases A B C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1463
  >   $ V=B
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1464
  > #if A
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1465
  >   $ V=A
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1466
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1467
  > #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1468
  >   $ V=C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1469
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1470
  >   $ echo $V | sed 's/A/C/'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1471
  >   C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1472
  > #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1473
  >   $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1474
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1475
  > #if A
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1476
  >   $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1477
  >   [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1478
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1479
  > #if no-C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1480
  >   $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1481
  >   [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1482
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1483
  >   $ [ $V = D ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1484
  >   [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1485
  > EOF
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1486
  $ rt
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1487
  .
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1488
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1489
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t.B.err
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1490
  @@ -7,7 +7,7 @@
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1491
     $ V=C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1492
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1493
     $ echo $V | sed 's/A/C/'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1494
  -  C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1495
  +  B
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1496
   #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1497
     $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1498
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1499
  
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1500
  ERROR: test-cases-abc.t (case B) output changed
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1501
  !.
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1502
  Failed test-cases-abc.t (case B): output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1503
  # Ran 3 tests, 0 skipped, 1 failed.
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1504
  python hash seed: * (glob)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1505
  [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1506
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1507
--restart works
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1508
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1509
  $ rt --restart
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1510
  
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1511
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1512
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t.B.err
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1513
  @@ -7,7 +7,7 @@
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1514
     $ V=C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1515
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1516
     $ echo $V | sed 's/A/C/'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1517
  -  C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1518
  +  B
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1519
   #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1520
     $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1521
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1522
  
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1523
  ERROR: test-cases-abc.t (case B) output changed
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1524
  !.
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1525
  Failed test-cases-abc.t (case B): output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1526
  # Ran 2 tests, 0 skipped, 1 failed.
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1527
  python hash seed: * (glob)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1528
  [1]
32720
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1529
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1530
--restart works with outputdir
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1531
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1532
  $ mkdir output
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1533
  $ mv test-cases-abc.t.B.err output
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1534
  $ rt --restart --outputdir output
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1535
  
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1536
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1537
  +++ $TESTTMP/anothertests/cases/output/test-cases-abc.t.B.err
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1538
  @@ -7,7 +7,7 @@
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1539
     $ V=C
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1540
   #endif
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1541
     $ echo $V | sed 's/A/C/'
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1542
  -  C
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1543
  +  B
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1544
   #if C
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1545
     $ [ $V = C ]
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1546
   #endif
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1547
  
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1548
  ERROR: test-cases-abc.t (case B) output changed
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1549
  !.
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1550
  Failed test-cases-abc.t (case B): output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1551
  # Ran 2 tests, 0 skipped, 1 failed.
32720
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1552
  python hash seed: * (glob)
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1553
  [1]
35069
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1554
35536
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1555
Test TESTCASE variable
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1556
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1557
  $ cat > test-cases-ab.t <<'EOF'
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1558
  >   $ dostuff() {
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1559
  >   >   echo "In case $TESTCASE"
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1560
  >   > }
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1561
  > #testcases A B
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1562
  > #if A
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1563
  >   $ dostuff
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1564
  >   In case A
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1565
  > #endif
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1566
  > #if B
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1567
  >   $ dostuff
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1568
  >   In case B
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1569
  > #endif
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1570
  > EOF
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1571
  $ rt test-cases-ab.t
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1572
  ..
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1573
  # Ran 2 tests, 0 skipped, 0 failed.
f04d16bef2c7 tests: make #testcase available as env var in test
Martin von Zweigbergk <martinvonz@google.com>
parents: 35416
diff changeset
  1574
35069
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1575
Test automatic pattern replacement
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1576
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1577
  $ cat << EOF >> common-pattern.py
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1578
  > substitutions = [
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1579
  >     (br'foo-(.*)\\b',
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1580
  >      br'\$XXX=\\1\$'),
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1581
  >     (br'bar\\n',
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1582
  >      br'\$YYY$\\n'),
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1583
  > ]
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1584
  > EOF
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1585
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1586
  $ cat << EOF >> test-substitution.t
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1587
  >   $ echo foo-12
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1588
  >   \$XXX=12$
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1589
  >   $ echo foo-42
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1590
  >   \$XXX=42$
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1591
  >   $ echo bar prior
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1592
  >   bar prior
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1593
  >   $ echo lastbar
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1594
  >   last\$YYY$
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1595
  >   $ echo foo-bar foo-baz
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1596
  > EOF
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1597
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1598
  $ rt test-substitution.t
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1599
  
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1600
  --- $TESTTMP/anothertests/cases/test-substitution.t
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1601
  +++ $TESTTMP/anothertests/cases/test-substitution.t.err
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1602
  @@ -7,3 +7,4 @@
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1603
     $ echo lastbar
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1604
     last$YYY$
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1605
     $ echo foo-bar foo-baz
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1606
  +  $XXX=bar foo-baz$
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1607
  
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1608
  ERROR: test-substitution.t output changed
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1609
  !
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1610
  Failed test-substitution.t: output changed
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1611
  # Ran 1 tests, 0 skipped, 1 failed.
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1612
  python hash seed: * (glob)
4fb489a998c9 run-tests: allow to register any arbitrary pattern for replacement
Boris Feld <boris.feld@octobus.net>
parents: 34969
diff changeset
  1613
  [1]
35189
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1614
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1615
--extra-config-opt works
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1616
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1617
  $ cat << EOF >> test-config-opt.t
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1618
  >   $ hg init test-config-opt
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1619
  >   $ hg -R test-config-opt purge
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1620
  > EOF
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1621
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1622
  $ rt --extra-config-opt extensions.purge= test-config-opt.t
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1623
  .
073843b5e353 run-tests: make --extra-config-opt work with Python 3
Gregory Szorc <gregory.szorc@gmail.com>
parents: 35187
diff changeset
  1624
  # Ran 1 tests, 0 skipped, 0 failed.