tests/test-run-tests.t
author Pulkit Goyal <7895pulkit@gmail.com>
Sat, 15 Jul 2017 02:17:05 +0530
changeset 33552 754569f5e999
parent 33500 9c6e64911de0
child 33695 eeed23508383
permissions -rw-r--r--
run-tests: make sure to check if pygments is installed before using it e80041832e introduced support to color the output of tests but used pygments without checking whether it's installed or not. That breaks test-run-tests.t for machines which don't have pygments installed. This patch conditionalize the color test in test-run-tests.t and also add a check to make sure pygments is installed before using that.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     1
This file tests the behavior of run-tests.py itself.
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     2
24960
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     3
Avoid interference from actual test env:
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     4
29220
054bcdad6042 tests: refactor run-tests helpers
timeless <timeless@mozdev.org>
parents: 29199
diff changeset
     5
  $ . "$TESTDIR/helper-runtests.sh"
24960
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     6
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
     7
Smoke test with install
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     8
============
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     9
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    10
  $ run-tests.py $HGTEST_RUN_TESTS_PURE -l
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    11
  
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
    12
  # Ran 0 tests, 0 skipped, 0 failed.
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    13
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    14
Define a helper to avoid the install step
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    15
=============
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    16
  $ rt()
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    17
  > {
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    18
  >     run-tests.py --with-hg=`which hg` "$@"
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    19
  > }
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    20
28037
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    21
error paths
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    22
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    23
#if symlink
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    24
  $ ln -s `which true` hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    25
  $ run-tests.py --with-hg=./hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    26
  warning: --with-hg should specify an hg script
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    27
  
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
    28
  # Ran 0 tests, 0 skipped, 0 failed.
28037
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    29
  $ rm hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    30
#endif
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    31
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    32
#if execbit
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    33
  $ touch hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    34
  $ run-tests.py --with-hg=./hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    35
  Usage: run-tests.py [options] [tests]
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    36
  
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    37
  run-tests.py: error: --with-hg must specify an executable hg script
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    38
  [2]
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    39
  $ rm hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    40
#endif
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    41
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    42
Features for testing optional lines
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    43
===================================
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    44
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    45
  $ cat > hghaveaddon.py <<EOF
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    46
  > import hghave
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    47
  > @hghave.check("custom", "custom hghave feature")
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    48
  > def has_custom():
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    49
  >     return True
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    50
  > @hghave.check("missing", "missing hghave feature")
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    51
  > def has_missing():
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    52
  >     return False
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    53
  > EOF
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    54
28812
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    55
an empty test
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    56
=======================
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    57
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    58
  $ touch test-empty.t
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    59
  $ rt
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    60
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
    61
  # Ran 1 tests, 0 skipped, 0 failed.
28812
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    62
  $ rm test-empty.t
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    63
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    64
a succesful test
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    65
=======================
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    66
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    67
  $ cat > test-success.t << EOF
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    68
  >   $ echo babar
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    69
  >   babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    70
  >   $ echo xyzzy
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
    71
  >   dont_print (?)
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
    72
  >   nothing[42]line (re) (?)
28701
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28619
diff changeset
    73
  >   never*happens (glob) (?)
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
    74
  >   more_nothing (?)
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    75
  >   xyzzy
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25098
diff changeset
    76
  >   nor this (?)
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    77
  >   $ printf 'abc\ndef\nxyz\n'
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    78
  >   123 (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    79
  >   abc
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    80
  >   def (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    81
  >   456 (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    82
  >   xyz
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    83
  >   $ printf 'zyx\nwvu\ntsr\n'
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    84
  >   abc (?)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    85
  >   zyx (custom !)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    86
  >   wvu
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    87
  >   no_print (no-custom !)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    88
  >   tsr (no-missing !)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
    89
  >   missing (missing !)
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    90
  > EOF
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    91
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    92
  $ rt
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    93
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
    94
  # Ran 1 tests, 0 skipped, 0 failed.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    95
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    96
failing test
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    97
==================
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    98
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    99
test churn with globs
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   100
  $ cat > test-failure.t <<EOF
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   101
  >   $ echo "bar-baz"; echo "bar-bad"
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   102
  >   bar*bad (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   103
  >   bar*baz (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   104
  > EOF
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   105
  $ rt test-failure.t
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   106
  
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   107
  --- $TESTTMP/test-failure.t
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   108
  +++ $TESTTMP/test-failure.t.err
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   109
  @@ -1,3 +1,3 @@
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   110
     $ echo "bar-baz"; echo "bar-bad"
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   111
  +  bar*baz (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   112
     bar*bad (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   113
  -  bar*baz (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   114
  
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   115
  ERROR: test-failure.t output changed
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   116
  !
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   117
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   118
  # Ran 1 tests, 0 skipped, 1 failed.
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   119
  python hash seed: * (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   120
  [1]
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   121
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   122
test diff colorisation
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   123
33552
754569f5e999 run-tests: make sure to check if pygments is installed before using it
Pulkit Goyal <7895pulkit@gmail.com>
parents: 33500
diff changeset
   124
#if no-windows pygments
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   125
  $ rt test-failure.t --color always
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   126
  
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   127
  \x1b[38;5;124m--- $TESTTMP/test-failure.t\x1b[39m (esc)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   128
  \x1b[38;5;34m+++ $TESTTMP/test-failure.t.err\x1b[39m (esc)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   129
  \x1b[38;5;90;01m@@ -1,3 +1,3 @@\x1b[39;00m (esc)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   130
     $ echo "bar-baz"; echo "bar-bad"
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   131
  \x1b[38;5;34m+  bar*baz (glob)\x1b[39m (esc)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   132
     bar*bad (glob)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   133
  \x1b[38;5;124m-  bar*baz (glob)\x1b[39m (esc)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   134
  
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   135
  ERROR: test-failure.t output changed
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   136
  !
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   137
  Failed test-failure.t: output changed
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   138
  # Ran 1 tests, 0 skipped, 1 failed.
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   139
  python hash seed: * (glob)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   140
  [1]
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   141
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   142
  $ rt test-failure.t 2> tmp.log
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   143
  [1]
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   144
  $ cat tmp.log
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   145
  
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   146
  --- $TESTTMP/test-failure.t
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   147
  +++ $TESTTMP/test-failure.t.err
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   148
  @@ -1,3 +1,3 @@
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   149
     $ echo "bar-baz"; echo "bar-bad"
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   150
  +  bar*baz (glob)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   151
     bar*bad (glob)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   152
  -  bar*baz (glob)
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   153
  
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   154
  ERROR: test-failure.t output changed
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   155
  !
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   156
  Failed test-failure.t: output changed
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   157
  # Ran 1 tests, 0 skipped, 1 failed.
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   158
  python hash seed: * (glob)
33500
9c6e64911de0 run-tests: disable color on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 33420
diff changeset
   159
#endif
33420
e80041832eec run-tests: add color to output if pygments is available
Matthieu Laneuville <matthieu.laneuville@octobus.net>
parents: 32982
diff changeset
   160
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
   161
basic failing test
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   162
  $ cat > test-failure.t << EOF
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   163
  >   $ echo babar
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   164
  >   rataxes
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   165
  > This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   166
  > this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   167
  > pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   168
  > pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   169
  > pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   170
  > pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   171
  > pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   172
  > pad pad pad pad............................................................
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   173
  > EOF
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   174
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   175
  >>> fh = open('test-failure-unicode.t', 'wb')
25054
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   176
  >>> fh.write(u'  $ echo babar\u03b1\n'.encode('utf-8')) and None
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   177
  >>> fh.write(u'  l\u03b5\u03b5t\n'.encode('utf-8')) and None
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   178
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   179
  $ rt
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   180
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   181
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   182
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   183
  @@ -1,5 +1,5 @@
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   184
     $ echo babar
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   185
  -  rataxes
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   186
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   187
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   188
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   189
   pad pad pad pad............................................................
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   190
  
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   191
  ERROR: test-failure.t output changed
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   192
  !.
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   193
  --- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   194
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   195
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   196
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   197
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   198
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   199
  
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   200
  ERROR: test-failure-unicode.t output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   201
  !
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   202
  Failed test-failure.t: output changed
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   203
  Failed test-failure-unicode.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   204
  # Ran 3 tests, 0 skipped, 2 failed.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   205
  python hash seed: * (glob)
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   206
  [1]
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   207
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   208
test --outputdir
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   209
  $ mkdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   210
  $ rt --outputdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   211
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   212
  --- $TESTTMP/test-failure.t
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   213
  +++ $TESTTMP/output/test-failure.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   214
  @@ -1,5 +1,5 @@
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   215
     $ echo babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   216
  -  rataxes
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   217
  +  babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   218
   This is a noop statement so that
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   219
   this test is still more bytes than success.
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   220
   pad pad pad pad............................................................
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   221
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   222
  ERROR: test-failure.t output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   223
  !.
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   224
  --- $TESTTMP/test-failure-unicode.t
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   225
  +++ $TESTTMP/output/test-failure-unicode.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   226
  @@ -1,2 +1,2 @@
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   227
     $ echo babar\xce\xb1 (esc)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   228
  -  l\xce\xb5\xce\xb5t (esc)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   229
  +  babar\xce\xb1 (esc)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   230
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   231
  ERROR: test-failure-unicode.t output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   232
  !
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   233
  Failed test-failure.t: output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   234
  Failed test-failure-unicode.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   235
  # Ran 3 tests, 0 skipped, 2 failed.
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   236
  python hash seed: * (glob)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   237
  [1]
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   238
  $ ls -a output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   239
  .
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   240
  ..
32717
e5680cb1414f run-tests: write test times to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32716
diff changeset
   241
  .testtimes
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   242
  test-failure-unicode.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   243
  test-failure.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   244
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   245
test --xunit support
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   246
  $ rt --xunit=xunit.xml
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   247
  
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   248
  --- $TESTTMP/test-failure.t
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   249
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   250
  @@ -1,5 +1,5 @@
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   251
     $ echo babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   252
  -  rataxes
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   253
  +  babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   254
   This is a noop statement so that
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   255
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   256
   pad pad pad pad............................................................
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   257
  
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   258
  ERROR: test-failure.t output changed
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   259
  !.
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   260
  --- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   261
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   262
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   263
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   264
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   265
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   266
  
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   267
  ERROR: test-failure-unicode.t output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   268
  !
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   269
  Failed test-failure.t: output changed
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   270
  Failed test-failure-unicode.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   271
  # Ran 3 tests, 0 skipped, 2 failed.
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   272
  python hash seed: * (glob)
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   273
  [1]
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   274
  $ cat xunit.xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   275
  <?xml version="1.0" encoding="utf-8"?>
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   276
  <testsuite errors="0" failures="2" name="run-tests" skipped="0" tests="3">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   277
    <testcase name="test-success.t" time="*"/> (glob)
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   278
    <testcase name="test-failure-unicode.t" time="*"> (glob)
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   279
      <failure message="output changed" type="output-mismatch">
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   280
  <![CDATA[--- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   281
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   282
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   283
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   284
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   285
  +  babar\xce\xb1 (esc)
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   286
  ]]>    </failure>
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   287
    </testcase>
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   288
    <testcase name="test-failure.t" time="*"> (glob)
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   289
      <failure message="output changed" type="output-mismatch">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   290
  <![CDATA[--- $TESTTMP/test-failure.t
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   291
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   292
  @@ -1,5 +1,5 @@
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   293
     $ echo babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   294
  -  rataxes
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   295
  +  babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   296
   This is a noop statement so that
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   297
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   298
   pad pad pad pad............................................................
32714
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   299
  ]]>    </failure>
ef8d24539612 run-tests: wrap failures in an XUnit 'failure' element
Siddharth Agarwal <sid0@fb.com>
parents: 32704
diff changeset
   300
    </testcase>
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   301
  </testsuite>
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   302
29280
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   303
  $ cat .testtimes
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   304
  test-failure-unicode.t * (glob)
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   305
  test-failure.t * (glob)
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   306
  test-success.t * (glob)
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   307
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   308
  $ rt --list-tests
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   309
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   310
  test-failure.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   311
  test-success.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   312
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   313
  $ rt --list-tests --json
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   314
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   315
  test-failure.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   316
  test-success.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   317
  $ cat report.json
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   318
  testreport ={
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   319
      "test-failure-unicode.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   320
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   321
      },
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   322
      "test-failure.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   323
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   324
      },
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   325
      "test-success.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   326
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   327
      }
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   328
  } (no-eol)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   329
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   330
  $ rt --list-tests --xunit=xunit.xml
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   331
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   332
  test-failure.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   333
  test-success.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   334
  $ cat xunit.xml
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   335
  <?xml version="1.0" encoding="utf-8"?>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   336
  <testsuite errors="0" failures="0" name="run-tests" skipped="0" tests="0">
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   337
    <testcase name="test-failure-unicode.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   338
    <testcase name="test-failure.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   339
    <testcase name="test-success.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   340
  </testsuite>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   341
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   342
  $ rt --list-tests test-failure* --json --xunit=xunit.xml --outputdir output
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   343
  test-failure-unicode.t
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   344
  test-failure.t
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   345
  $ cat output/report.json
32704
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   346
  testreport ={
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   347
      "test-failure-unicode.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   348
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   349
      },
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   350
      "test-failure.t": {
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   351
          "result": "success"
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   352
      }
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   353
  } (no-eol)
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   354
  $ cat xunit.xml
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   355
  <?xml version="1.0" encoding="utf-8"?>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   356
  <testsuite errors="0" failures="0" name="run-tests" skipped="0" tests="0">
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   357
    <testcase name="test-failure-unicode.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   358
    <testcase name="test-failure.t"/>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   359
  </testsuite>
1270b00a385d run-tests: add a way to list tests, with JSON and XUnit support
Siddharth Agarwal <sid0@fb.com>
parents: 32622
diff changeset
   360
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   361
  $ rm test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   362
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   363
test for --retest
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   364
====================
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   365
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   366
  $ rt --retest
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   367
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   368
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   369
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   370
  @@ -1,5 +1,5 @@
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   371
     $ echo babar
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   372
  -  rataxes
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   373
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   374
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   375
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   376
   pad pad pad pad............................................................
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   377
  
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   378
  ERROR: test-failure.t output changed
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   379
  !
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   380
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   381
  # Ran 2 tests, 1 skipped, 1 failed.
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   382
  python hash seed: * (glob)
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   383
  [1]
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   384
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   385
--retest works with --outputdir
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   386
  $ rm -r output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   387
  $ mkdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   388
  $ mv test-failure.t.err output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   389
  $ rt --retest --outputdir output
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   390
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   391
  --- $TESTTMP/test-failure.t
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   392
  +++ $TESTTMP/output/test-failure.t.err
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   393
  @@ -1,5 +1,5 @@
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   394
     $ echo babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   395
  -  rataxes
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   396
  +  babar
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   397
   This is a noop statement so that
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   398
   this test is still more bytes than success.
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   399
   pad pad pad pad............................................................
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   400
  
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   401
  ERROR: test-failure.t output changed
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   402
  !
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   403
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   404
  # Ran 2 tests, 1 skipped, 1 failed.
32716
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   405
  python hash seed: * (glob)
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   406
  [1]
2146f01a2577 run-tests: allow specifying an output dir to write .errs to
Siddharth Agarwal <sid0@fb.com>
parents: 32715
diff changeset
   407
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   408
Selecting Tests To Run
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   409
======================
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   410
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   411
successful
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   412
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   413
  $ rt test-success.t
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   414
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   415
  # Ran 1 tests, 0 skipped, 0 failed.
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   416
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   417
success w/ keyword
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   418
  $ rt -k xyzzy
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   419
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   420
  # Ran 2 tests, 1 skipped, 0 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   421
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   422
failed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   423
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   424
  $ rt test-failure.t
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   425
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   426
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   427
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   428
  @@ -1,5 +1,5 @@
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   429
     $ echo babar
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   430
  -  rataxes
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   431
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   432
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   433
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   434
   pad pad pad pad............................................................
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   435
  
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   436
  ERROR: test-failure.t output changed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   437
  !
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   438
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   439
  # Ran 1 tests, 0 skipped, 1 failed.
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   440
  python hash seed: * (glob)
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   441
  [1]
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   442
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   443
failure w/ keyword
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   444
  $ rt -k rataxes
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   445
  
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   446
  --- $TESTTMP/test-failure.t
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   447
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   448
  @@ -1,5 +1,5 @@
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   449
     $ echo babar
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   450
  -  rataxes
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   451
  +  babar
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   452
   This is a noop statement so that
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   453
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   454
   pad pad pad pad............................................................
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   455
  
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   456
  ERROR: test-failure.t output changed
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   457
  !
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   458
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   459
  # Ran 2 tests, 1 skipped, 1 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   460
  python hash seed: * (glob)
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   461
  [1]
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   462
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   463
Verify that when a process fails to start we show a useful message
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   464
==================================================================
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   465
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   466
  $ cat > test-serve-fail.t <<EOF
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   467
  >   $ echo 'abort: child process failed to start blah'
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   468
  > EOF
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   469
  $ rt test-serve-fail.t
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   470
  
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   471
  ERROR: test-serve-fail.t output changed
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   472
  !
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   473
  Failed test-serve-fail.t: server failed to start (HGPORT=*) (glob)
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   474
  # Ran 1 tests, 0 skipped, 1 failed.
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   475
  python hash seed: * (glob)
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   476
  [1]
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   477
  $ rm test-serve-fail.t
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   478
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   479
Verify that we can try other ports
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   480
===================================
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   481
  $ hg init inuse
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   482
  $ hg serve -R inuse -p $HGPORT -d --pid-file=blocks.pid
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   483
  $ cat blocks.pid >> $DAEMON_PIDS
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   484
  $ cat > test-serve-inuse.t <<EOF
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   485
  >   $ hg serve -R `pwd`/inuse -p \$HGPORT -d --pid-file=hg.pid
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   486
  >   $ cat hg.pid >> \$DAEMON_PIDS
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   487
  > EOF
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   488
  $ rt test-serve-inuse.t
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   489
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   490
  # Ran 1 tests, 0 skipped, 0 failed.
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   491
  $ rm test-serve-inuse.t
29221
73f58eb14684 test-run-tests: clean up inuse server eagerly
timeless <timeless@mozdev.org>
parents: 29220
diff changeset
   492
  $ killdaemons.py $DAEMON_PIDS
73f58eb14684 test-run-tests: clean up inuse server eagerly
timeless <timeless@mozdev.org>
parents: 29220
diff changeset
   493
  $ rm $DAEMON_PIDS
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   494
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   495
Running In Debug Mode
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   496
======================
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   497
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   498
  $ rt --debug 2>&1 | grep -v pwd
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   499
  + echo *SALT* 0 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   500
  *SALT* 0 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   501
  + echo babar
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   502
  babar
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   503
  + echo *SALT* 10 0 (glob)
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   504
  *SALT* 10 0 (glob)
27509
6ff90d618d72 test-run-tests: glob away a --debug run difference on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 27395
diff changeset
   505
  *+ echo *SALT* 0 0 (glob)
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   506
  *SALT* 0 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   507
  + echo babar
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   508
  babar
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   509
  + echo *SALT* 2 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   510
  *SALT* 2 0 (glob)
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   511
  + echo xyzzy
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   512
  xyzzy
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   513
  + echo *SALT* 9 0 (glob)
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   514
  *SALT* 9 0 (glob)
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   515
  + printf *abc\ndef\nxyz\n* (glob)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   516
  abc
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   517
  def
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   518
  xyz
31827
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   519
  + echo *SALT* 15 0 (glob)
220d4bffd23e run-tests: prevent a (glob) declaration from reordering (?) lines
Matt Harbison <matt_harbison@yahoo.com>
parents: 31673
diff changeset
   520
  *SALT* 15 0 (glob)
31829
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   521
  + printf *zyx\nwvu\ntsr\n* (glob)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   522
  zyx
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   523
  wvu
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   524
  tsr
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   525
  + echo *SALT* 22 0 (glob)
4eec2f04a672 run-tests: support per-line conditional output in tests
Matt Harbison <matt_harbison@yahoo.com>
parents: 31828
diff changeset
   526
  *SALT* 22 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   527
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   528
  # Ran 2 tests, 0 skipped, 0 failed.
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   529
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   530
Parallel runs
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   531
==============
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   532
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   533
(duplicate the failing test to get predictable output)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   534
  $ cp test-failure.t test-failure-copy.t
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   535
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   536
  $ rt --jobs 2 test-failure*.t -n
23107
5459b30aa498 tests: silence output race in test-run-tests.t
Matt Mackall <mpm@selenic.com>
parents: 22840
diff changeset
   537
  !!
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   538
  Failed test-failure*.t: output changed (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   539
  Failed test-failure*.t: output changed (glob)
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   540
  # Ran 2 tests, 0 skipped, 2 failed.
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   541
  python hash seed: * (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   542
  [1]
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   543
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   544
failures in parallel with --first should only print one failure
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   545
  >>> f = open('test-nothing.t', 'w')
25054
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   546
  >>> f.write('foo\n' * 1024) and None
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   547
  >>> f.write('  $ sleep 1') and None
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   548
  $ rt --jobs 2 --first
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   549
  
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   550
  --- $TESTTMP/test-failure*.t (glob)
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   551
  +++ $TESTTMP/test-failure*.t.err (glob)
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   552
  @@ -1,5 +1,5 @@
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   553
     $ echo babar
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   554
  -  rataxes
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   555
  +  babar
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   556
   This is a noop statement so that
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   557
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   558
   pad pad pad pad............................................................
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   559
  
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   560
  Failed test-failure*.t: output changed (glob)
24507
a0668a587c04 run-tests: wait for test threads after first error
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
   561
  Failed test-nothing.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   562
  # Ran 2 tests, 0 skipped, 2 failed.
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   563
  python hash seed: * (glob)
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   564
  [1]
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   565
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   566
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   567
(delete the duplicated test file)
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   568
  $ rm test-failure-copy.t test-nothing.t
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   569
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   570
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   571
Interactive run
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   572
===============
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   573
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   574
(backup the failing test)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   575
  $ cp test-failure.t backup
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   576
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   577
Refuse the fix
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   578
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   579
  $ echo 'n' | rt -i
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   580
  
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   581
  --- $TESTTMP/test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   582
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   583
  @@ -1,5 +1,5 @@
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   584
     $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   585
  -  rataxes
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   586
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   587
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   588
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   589
   pad pad pad pad............................................................
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21756
diff changeset
   590
  Accept this change? [n] 
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   591
  ERROR: test-failure.t output changed
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21756
diff changeset
   592
  !.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   593
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   594
  # Ran 2 tests, 0 skipped, 1 failed.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   595
  python hash seed: * (glob)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   596
  [1]
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   597
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   598
  $ cat test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   599
    $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   600
    rataxes
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   601
  This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   602
  this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   603
  pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   604
  pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   605
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   606
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   607
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   608
  pad pad pad pad............................................................
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   609
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   610
Interactive with custom view
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   611
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   612
  $ echo 'n' | rt -i --view echo
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   613
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err (glob)
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   614
  Accept this change? [n]* (glob)
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   615
  ERROR: test-failure.t output changed
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   616
  !.
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   617
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   618
  # Ran 2 tests, 0 skipped, 1 failed.
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   619
  python hash seed: * (glob)
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   620
  [1]
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   621
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   622
View the fix
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   623
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   624
  $ echo 'y' | rt --view echo
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   625
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err (glob)
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   626
  
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   627
  ERROR: test-failure.t output changed
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   628
  !.
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   629
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   630
  # Ran 2 tests, 0 skipped, 1 failed.
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   631
  python hash seed: * (glob)
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   632
  [1]
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   633
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   634
Accept the fix
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   635
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   636
  $ echo "  $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   637
  $ echo "  saved backup bundle to \$TESTTMP/foo.hg" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   638
  $ echo "  $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   639
  $ echo "  saved backup bundle to \$TESTTMP/foo.hg (glob)" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   640
  $ echo "  $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   641
  $ echo "  saved backup bundle to \$TESTTMP/*.hg (glob)" >> test-failure.t
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   642
  $ echo 'y' | rt -i 2>&1
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   643
  
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   644
  --- $TESTTMP/test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   645
  +++ $TESTTMP/test-failure.t.err
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   646
  @@ -1,5 +1,5 @@
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   647
     $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   648
  -  rataxes
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   649
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   650
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   651
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   652
   pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   653
  @@ -9,7 +9,7 @@
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   654
   pad pad pad pad............................................................
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   655
   pad pad pad pad............................................................
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   656
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   657
  -  saved backup bundle to $TESTTMP/foo.hg
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   658
  +  saved backup bundle to $TESTTMP/foo.hg* (glob)
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   659
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   660
     saved backup bundle to $TESTTMP/foo.hg* (glob)
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   661
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   662
  Accept this change? [n] ..
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   663
  # Ran 2 tests, 0 skipped, 0 failed.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   664
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   665
  $ sed -e 's,(glob)$,&<,g' test-failure.t
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   666
    $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   667
    babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   668
  This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   669
  this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   670
  pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   671
  pad pad pad pad............................................................
31828
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   672
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   673
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   674
  pad pad pad pad............................................................
ff60498211f3 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 31827
diff changeset
   675
  pad pad pad pad............................................................
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   676
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   677
    saved backup bundle to $TESTTMP/foo.hg (glob)<
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   678
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   679
    saved backup bundle to $TESTTMP/foo.hg (glob)<
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   680
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   681
    saved backup bundle to $TESTTMP/*.hg (glob)<
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   682
32980
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   683
Race condition - test file was modified when test is running
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   684
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   685
  $ TESTRACEDIR=`pwd`
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   686
  $ export TESTRACEDIR
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   687
  $ cat > test-race.t <<EOF
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   688
  >   $ echo 1
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   689
  >   $ echo "# a new line" >> $TESTRACEDIR/test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   690
  > EOF
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   691
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   692
  $ rt -i test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   693
  
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   694
  --- $TESTTMP/test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   695
  +++ $TESTTMP/test-race.t.err
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   696
  @@ -1,2 +1,3 @@
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   697
     $ echo 1
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   698
  +  1
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   699
     $ echo "# a new line" >> $TESTTMP/test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   700
  Reference output has changed (run again to prompt changes)
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   701
  ERROR: test-race.t output changed
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   702
  !
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   703
  Failed test-race.t: output changed
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   704
  # Ran 1 tests, 0 skipped, 1 failed.
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   705
  python hash seed: * (glob)
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   706
  [1]
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   707
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   708
  $ rm test-race.t
8dc62c97a665 run-tests: do not prompt changes (-i) if a race condition is detected
Jun Wu <quark@fb.com>
parents: 32942
diff changeset
   709
32982
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   710
When "#testcases" is used in .t files
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   711
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   712
  $ cat >> test-cases.t <<EOF
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   713
  > #testcases a b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   714
  > #if a
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   715
  >   $ echo 1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   716
  > #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   717
  > #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   718
  >   $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   719
  > #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   720
  > EOF
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   721
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   722
  $ cat <<EOF | rt -i test-cases.t 2>&1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   723
  > y
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   724
  > y
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   725
  > EOF
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   726
  
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   727
  --- $TESTTMP/test-cases.t
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   728
  +++ $TESTTMP/test-cases.t.a.err
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   729
  @@ -1,6 +1,7 @@
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   730
   #testcases a b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   731
   #if a
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   732
     $ echo 1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   733
  +  1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   734
   #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   735
   #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   736
     $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   737
  Accept this change? [n] .
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   738
  --- $TESTTMP/test-cases.t
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   739
  +++ $TESTTMP/test-cases.t.b.err
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   740
  @@ -5,4 +5,5 @@
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   741
   #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   742
   #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   743
     $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   744
  +  2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   745
   #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   746
  Accept this change? [n] .
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   747
  # Ran 2 tests, 0 skipped, 0 failed.
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   748
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   749
  $ cat test-cases.t
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   750
  #testcases a b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   751
  #if a
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   752
    $ echo 1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   753
    1
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   754
  #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   755
  #if b
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   756
    $ echo 2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   757
    2
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   758
  #endif
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   759
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   760
  $ rm test-cases.t
573baab2a797 run-tests: fix -i when "#testcases" is used in .t test
Jun Wu <quark@fb.com>
parents: 32980
diff changeset
   761
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   762
(reinstall)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   763
  $ mv backup test-failure.t
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   764
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   765
No Diff
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   766
===============
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   767
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   768
  $ rt --nodiff
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   769
  !.
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   770
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   771
  # Ran 2 tests, 0 skipped, 1 failed.
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   772
  python hash seed: * (glob)
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   773
  [1]
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   774
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   775
test --tmpdir support
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   776
  $ rt --tmpdir=$TESTTMP/keep test-success.t
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   777
  
26432
39577d4520ab test-run-tests: add globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 26422
diff changeset
   778
  Keeping testtmp dir: $TESTTMP/keep/child1/test-success.t (glob)
39577d4520ab test-run-tests: add globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 26422
diff changeset
   779
  Keeping threadtmp dir: $TESTTMP/keep/child1  (glob)
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   780
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   781
  # Ran 1 tests, 0 skipped, 0 failed.
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   782
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   783
timeouts
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   784
========
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   785
  $ cat > test-timeout.t <<EOF
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   786
  >   $ sleep 2
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   787
  >   $ echo pass
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   788
  >   pass
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   789
  > EOF
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   790
  > echo '#require slow' > test-slow-timeout.t
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   791
  > cat test-timeout.t >> test-slow-timeout.t
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   792
  $ rt --timeout=1 --slowtimeout=3 test-timeout.t test-slow-timeout.t
27393
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
   793
  st
32474
c2b7fb580794 tests: hint how to run slow tests when rejecting
Kyle Lippincott <spectral@google.com>
parents: 32316
diff changeset
   794
  Skipped test-slow-timeout.t: missing feature: allow slow tests (use --allow-slow-tests)
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   795
  Failed test-timeout.t: timed out
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   796
  # Ran 1 tests, 1 skipped, 1 failed.
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   797
  python hash seed: * (glob)
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   798
  [1]
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   799
  $ rt --timeout=1 --slowtimeout=3 \
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   800
  > test-timeout.t test-slow-timeout.t --allow-slow-tests
27393
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
   801
  .t
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   802
  Failed test-timeout.t: timed out
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   803
  # Ran 2 tests, 0 skipped, 1 failed.
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   804
  python hash seed: * (glob)
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   805
  [1]
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   806
  $ rm test-timeout.t test-slow-timeout.t
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   807
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   808
test for --time
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   809
==================
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   810
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   811
  $ rt test-success.t --time
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   812
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   813
  # Ran 1 tests, 0 skipped, 0 failed.
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   814
  # Producing time report
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   815
  start   end     cuser   csys    real      Test
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   816
  \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   test-success.t (re)
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   817
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   818
test for --time with --job enabled
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   819
====================================
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   820
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   821
  $ rt test-success.t --time --jobs 2
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   822
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   823
  # Ran 1 tests, 0 skipped, 0 failed.
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   824
  # Producing time report
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   825
  start   end     cuser   csys    real      Test
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   826
  \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   test-success.t (re)
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   827
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   828
Skips
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   829
================
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   830
  $ cat > test-skip.t <<EOF
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   831
  >   $ echo xyzzy
22045
769198c6a62d run-tests: add #require to abort full test
Matt Mackall <mpm@selenic.com>
parents: 22044
diff changeset
   832
  > #require false
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   833
  > EOF
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   834
  $ rt --nodiff
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   835
  !.s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   836
  Skipped test-skip.t: missing feature: nail clipper
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   837
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   838
  # Ran 2 tests, 1 skipped, 1 failed.
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   839
  python hash seed: * (glob)
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   840
  [1]
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   841
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   842
  $ rt --keyword xyzzy
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   843
  .s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   844
  Skipped test-skip.t: missing feature: nail clipper
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   845
  # Ran 2 tests, 2 skipped, 0 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   846
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   847
Skips with xml
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   848
  $ rt --keyword xyzzy \
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   849
  >  --xunit=xunit.xml
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   850
  .s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   851
  Skipped test-skip.t: missing feature: nail clipper
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   852
  # Ran 2 tests, 2 skipped, 0 failed.
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   853
  $ cat xunit.xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   854
  <?xml version="1.0" encoding="utf-8"?>
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   855
  <testsuite errors="0" failures="0" name="run-tests" skipped="2" tests="2">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   856
    <testcase name="test-success.t" time="*"/> (glob)
32715
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
   857
    <testcase name="test-skip.t">
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
   858
      <skipped>
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
   859
  <![CDATA[missing feature: nail clipper]]>    </skipped>
a4d0e816a672 run-tests: add information about skipped tests to XUnit output
Siddharth Agarwal <sid0@fb.com>
parents: 32714
diff changeset
   860
    </testcase>
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   861
  </testsuite>
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   862
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   863
Missing skips or blacklisted skips don't count as executed:
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   864
  $ echo test-failure.t > blacklist
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   865
  $ rt --blacklist=blacklist --json\
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   866
  >   test-failure.t test-bogus.t
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   867
  ss
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   868
  Skipped test-bogus.t: Doesn't exist
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   869
  Skipped test-failure.t: blacklisted
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   870
  # Ran 0 tests, 2 skipped, 0 failed.
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   871
  $ cat report.json
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   872
  testreport ={
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   873
      "test-bogus.t": {
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   874
          "result": "skip"
29199
daff05dcd184 run-tests: handle json.dumps divergence
timeless <timeless@mozdev.org>
parents: 29173
diff changeset
   875
      },
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   876
      "test-failure.t": {
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   877
          "result": "skip"
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   878
      }
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   879
  } (no-eol)
29173
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   880
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   881
Whitelist trumps blacklist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   882
  $ echo test-failure.t > whitelist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   883
  $ rt --blacklist=blacklist --whitelist=whitelist --json\
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   884
  >   test-failure.t test-bogus.t
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   885
  s
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   886
  --- $TESTTMP/test-failure.t
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   887
  +++ $TESTTMP/test-failure.t.err
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   888
  @@ -1,5 +1,5 @@
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   889
     $ echo babar
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   890
  -  rataxes
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   891
  +  babar
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   892
   This is a noop statement so that
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   893
   this test is still more bytes than success.
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   894
   pad pad pad pad............................................................
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   895
  
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   896
  ERROR: test-failure.t output changed
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   897
  !
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   898
  Skipped test-bogus.t: Doesn't exist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   899
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   900
  # Ran 1 tests, 1 skipped, 1 failed.
29173
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   901
  python hash seed: * (glob)
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   902
  [1]
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   903
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   904
test for --json
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   905
==================
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   906
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   907
  $ rt --json
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   908
  
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   909
  --- $TESTTMP/test-failure.t
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   910
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   911
  @@ -1,5 +1,5 @@
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   912
     $ echo babar
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   913
  -  rataxes
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   914
  +  babar
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   915
   This is a noop statement so that
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   916
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   917
   pad pad pad pad............................................................
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   918
  
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   919
  ERROR: test-failure.t output changed
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   920
  !.s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   921
  Skipped test-skip.t: missing feature: nail clipper
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   922
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   923
  # Ran 2 tests, 1 skipped, 1 failed.
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   924
  python hash seed: * (glob)
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   925
  [1]
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   926
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   927
  $ cat report.json
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   928
  testreport ={
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   929
      "test-failure.t": [\{] (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   930
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   931
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   932
          "diff": "---.+\+\+\+.+", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   933
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   934
          "result": "failure", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   935
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   936
          "time": "\s*[\d\.]{4,5}" (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   937
      }, ? (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   938
      "test-skip.t": {
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   939
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   940
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   941
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   942
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   943
          "result": "skip", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   944
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   945
          "time": "\s*[\d\.]{4,5}" (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   946
      }, ? (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   947
      "test-success.t": [\{] (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   948
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   949
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   950
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   951
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   952
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   953
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   954
          "time": "\s*[\d\.]{4,5}" (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   955
      }
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   956
  } (no-eol)
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   957
--json with --outputdir
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   958
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   959
  $ rm report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   960
  $ rm -r output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   961
  $ mkdir output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   962
  $ rt --json --outputdir output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   963
  
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   964
  --- $TESTTMP/test-failure.t
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   965
  +++ $TESTTMP/output/test-failure.t.err
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   966
  @@ -1,5 +1,5 @@
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   967
     $ echo babar
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   968
  -  rataxes
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   969
  +  babar
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   970
   This is a noop statement so that
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   971
   this test is still more bytes than success.
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   972
   pad pad pad pad............................................................
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   973
  
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   974
  ERROR: test-failure.t output changed
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   975
  !.s
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   976
  Skipped test-skip.t: missing feature: nail clipper
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   977
  Failed test-failure.t: output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
   978
  # Ran 2 tests, 1 skipped, 1 failed.
32718
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   979
  python hash seed: * (glob)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   980
  [1]
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   981
  $ f report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   982
  report.json: file not found
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   983
  $ cat output/report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   984
  testreport ={
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   985
      "test-failure.t": [\{] (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   986
          "csys": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   987
          "cuser": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   988
          "diff": "---.+\+\+\+.+", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   989
          "end": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   990
          "result": "failure", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   991
          "start": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   992
          "time": "\s*[\d\.]{4,5}" (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   993
      }, ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   994
      "test-skip.t": {
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   995
          "csys": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   996
          "cuser": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   997
          "diff": "", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   998
          "end": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
   999
          "result": "skip", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1000
          "start": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1001
          "time": "\s*[\d\.]{4,5}" (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1002
      }, ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1003
      "test-success.t": [\{] (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1004
          "csys": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1005
          "cuser": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1006
          "diff": "", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1007
          "end": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1008
          "result": "success", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1009
          "start": "\s*[\d\.]{4,5}", ? (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1010
          "time": "\s*[\d\.]{4,5}" (re)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1011
      }
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1012
  } (no-eol)
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1013
  $ ls -a output
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1014
  .
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1015
  ..
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1016
  .testtimes
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1017
  report.json
232875623c27 run-tests: write JSON reports to output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32717
diff changeset
  1018
  test-failure.t.err
22579
36a940d82f88 test: protect the run-tests.py --json test behind an hghave rule
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22486
diff changeset
  1019
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1020
Test that failed test accepted through interactive are properly reported:
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1021
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1022
  $ cp test-failure.t backup
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1023
  $ echo y | rt --json -i
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1024
  
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1025
  --- $TESTTMP/test-failure.t
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1026
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
  1027
  @@ -1,5 +1,5 @@
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1028
     $ echo babar
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1029
  -  rataxes
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1030
  +  babar
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1031
   This is a noop statement so that
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1032
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
  1033
   pad pad pad pad............................................................
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1034
  Accept this change? [n] ..s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
  1035
  Skipped test-skip.t: missing feature: nail clipper
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1036
  # Ran 2 tests, 1 skipped, 0 failed.
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1037
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1038
  $ cat report.json
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1039
  testreport ={
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1040
      "test-failure.t": [\{] (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1041
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1042
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1043
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1044
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1045
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1046
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1047
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1048
      }, ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1049
      "test-skip.t": {
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1050
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1051
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1052
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1053
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1054
          "result": "skip", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1055
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1056
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1057
      }, ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1058
      "test-success.t": [\{] (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1059
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1060
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
  1061
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1062
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1063
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
  1064
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1065
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1066
      }
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1067
  } (no-eol)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1068
  $ mv backup test-failure.t
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
  1069
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1070
backslash on end of line with glob matching is handled properly
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1071
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1072
  $ cat > test-glob-backslash.t << EOF
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1073
  >   $ echo 'foo bar \\'
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1074
  >   foo * \ (glob)
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1075
  > EOF
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1076
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1077
  $ rt test-glob-backslash.t
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1078
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1079
  # Ran 1 tests, 0 skipped, 0 failed.
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1080
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1081
  $ rm -f test-glob-backslash.t
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
  1082
31673
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1083
Test globbing of local IP addresses
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29280
diff changeset
  1084
  $ echo 172.16.18.1
31673
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1085
  $LOCALIP (glob)
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1086
  $ echo dead:beef::1
6a2959acae1a runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
Jun Wu <quark@fb.com>
parents: 29518
diff changeset
  1087
  $LOCALIP (glob)
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29280
diff changeset
  1088
25728
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1089
Test reusability for third party tools
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1090
======================================
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1091
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1092
  $ mkdir "$TESTTMP"/anothertests
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1093
  $ cd "$TESTTMP"/anothertests
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1094
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1095
test that `run-tests.py` can execute hghave, even if it runs not in
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1096
Mercurial source tree.
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1097
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1098
  $ cat > test-hghave.t <<EOF
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1099
  > #require true
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1100
  >   $ echo foo
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1101
  >   foo
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1102
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1103
  $ rt test-hghave.t
25728
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
  1104
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1105
  # Ran 1 tests, 0 skipped, 0 failed.
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1106
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1107
test that RUNTESTDIR refers the directory, in which `run-tests.py` now
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1108
running is placed.
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1109
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1110
  $ cat > test-runtestdir.t <<EOF
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1111
  > - $TESTDIR, in which test-run-tests.t is placed
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1112
  > - \$TESTDIR, in which test-runtestdir.t is placed (expanded at runtime)
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1113
  > - \$RUNTESTDIR, in which run-tests.py is placed (expanded at runtime)
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1114
  > 
27057
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1115
  > #if windows
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1116
  >   $ test "\$TESTDIR" = "$TESTTMP\anothertests"
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1117
  > #else
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1118
  >   $ test "\$TESTDIR" = "$TESTTMP"/anothertests
27057
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
  1119
  > #endif
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1120
  >   $ test "\$RUNTESTDIR" = "$TESTDIR"
32937
7fe1f9785c75 tests: sed away python #! in test-run-tests.t to avoid some upcoming insanity
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
  1121
  >   $ head -n 3 "\$RUNTESTDIR"/../contrib/check-code.py | sed 's@.!.*python@#!USRBINENVPY@'
7fe1f9785c75 tests: sed away python #! in test-run-tests.t to avoid some upcoming insanity
Augie Fackler <augie@google.com>
parents: 32720
diff changeset
  1122
  >   #!USRBINENVPY
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1123
  >   #
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1124
  >   # check-code - a style and portability checker for Mercurial
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1125
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1126
  $ rt test-runtestdir.t
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
  1127
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1128
  # Ran 1 tests, 0 skipped, 0 failed.
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1129
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1130
#if execbit
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1131
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1132
test that TESTDIR is referred in PATH
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1133
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1134
  $ cat > custom-command.sh <<EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1135
  > #!/bin/sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1136
  > echo "hello world"
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1137
  > EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1138
  $ chmod +x custom-command.sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1139
  $ cat > test-testdir-path.t <<EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1140
  >   $ custom-command.sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1141
  >   hello world
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1142
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1143
  $ rt test-testdir-path.t
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1144
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1145
  # Ran 1 tests, 0 skipped, 0 failed.
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1146
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
  1147
#endif
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1148
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1149
test support for --allow-slow-tests
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1150
  $ cat > test-very-slow-test.t <<EOF
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1151
  > #require slow
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1152
  >   $ echo pass
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1153
  >   pass
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1154
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
  1155
  $ rt test-very-slow-test.t
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1156
  s
32474
c2b7fb580794 tests: hint how to run slow tests when rejecting
Kyle Lippincott <spectral@google.com>
parents: 32316
diff changeset
  1157
  Skipped test-very-slow-test.t: missing feature: allow slow tests (use --allow-slow-tests)
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1158
  # Ran 0 tests, 1 skipped, 0 failed.
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
  1159
  $ rt $HGTEST_RUN_TESTS_PURE --allow-slow-tests test-very-slow-test.t
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
  1160
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1161
  # Ran 1 tests, 0 skipped, 0 failed.
28180
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1162
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1163
support for running a test outside the current directory
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1164
  $ mkdir nonlocal
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1165
  $ cat > nonlocal/test-is-not-here.t << EOF
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1166
  >   $ echo pass
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1167
  >   pass
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1168
  > EOF
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1169
  $ rt nonlocal/test-is-not-here.t
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
  1170
  .
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1171
  # Ran 1 tests, 0 skipped, 0 failed.
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1172
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1173
support for bisecting failed tests automatically
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1174
  $ hg init bisect
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1175
  $ cd bisect
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1176
  $ cat >> test-bisect.t <<EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1177
  >   $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1178
  >   pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1179
  > EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1180
  $ hg add test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1181
  $ hg ci -m 'good'
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1182
  $ cat >> test-bisect.t <<EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1183
  >   $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1184
  >   fail
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1185
  > EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1186
  $ hg ci -m 'bad'
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1187
  $ rt --known-good-rev=0 test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1188
  
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1189
  --- $TESTTMP/anothertests/bisect/test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1190
  +++ $TESTTMP/anothertests/bisect/test-bisect.t.err
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1191
  @@ -1,4 +1,4 @@
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1192
     $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1193
     pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1194
     $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1195
  -  fail
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1196
  +  pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1197
  
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1198
  ERROR: test-bisect.t output changed
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1199
  !
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1200
  Failed test-bisect.t: output changed
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1201
  test-bisect.t broken by 72cbf122d116 (bad)
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1202
  # Ran 1 tests, 0 skipped, 1 failed.
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1203
  python hash seed: * (glob)
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
  1204
  [1]
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1205
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1206
  $ cd ..
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1207
32622
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1208
Test a broken #if statement doesn't break run-tests threading.
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1209
==============================================================
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1210
  $ mkdir broken
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1211
  $ cd broken
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1212
  $ cat > test-broken.t <<EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1213
  > true
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1214
  > #if notarealhghavefeature
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1215
  >   $ false
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1216
  > #endif
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1217
  > EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1218
  $ for f in 1 2 3 4 ; do
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1219
  > cat > test-works-$f.t <<EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1220
  > This is test case $f
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1221
  >   $ sleep 1
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1222
  > EOF
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1223
  > done
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1224
  $ rt -j 2
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1225
  ....
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1226
  # Ran 5 tests, 0 skipped, 0 failed.
32622
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1227
  skipped: unknown feature: notarealhghavefeature
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1228
  
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1229
  $ cd ..
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1230
  $ rm -rf broken
931bb962e0eb tests: fix run-tests when there's a bad #if in a test
Augie Fackler <augie@google.com>
parents: 32474
diff changeset
  1231
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1232
Test cases in .t files
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1233
======================
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1234
  $ mkdir cases
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1235
  $ cd cases
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1236
  $ cat > test-cases-abc.t <<'EOF'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1237
  > #testcases A B C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1238
  >   $ V=B
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1239
  > #if A
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1240
  >   $ V=A
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1241
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1242
  > #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1243
  >   $ V=C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1244
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1245
  >   $ echo $V | sed 's/A/C/'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1246
  >   C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1247
  > #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1248
  >   $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1249
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1250
  > #if A
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1251
  >   $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1252
  >   [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1253
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1254
  > #if no-C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1255
  >   $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1256
  >   [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1257
  > #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1258
  >   $ [ $V = D ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1259
  >   [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1260
  > EOF
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1261
  $ rt
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1262
  .
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1263
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1264
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t.B.err
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1265
  @@ -7,7 +7,7 @@
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1266
     $ V=C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1267
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1268
     $ echo $V | sed 's/A/C/'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1269
  -  C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1270
  +  B
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1271
   #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1272
     $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1273
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1274
  
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1275
  ERROR: test-cases-abc.t (case B) output changed
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1276
  !.
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1277
  Failed test-cases-abc.t (case B): output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1278
  # Ran 3 tests, 0 skipped, 1 failed.
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1279
  python hash seed: * (glob)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1280
  [1]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1281
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1282
--restart works
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1283
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1284
  $ rt --restart
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1285
  
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1286
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1287
  +++ $TESTTMP/anothertests/cases/test-cases-abc.t.B.err
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1288
  @@ -7,7 +7,7 @@
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1289
     $ V=C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1290
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1291
     $ echo $V | sed 's/A/C/'
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1292
  -  C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1293
  +  B
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1294
   #if C
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1295
     $ [ $V = C ]
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1296
   #endif
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1297
  
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1298
  ERROR: test-cases-abc.t (case B) output changed
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1299
  !.
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1300
  Failed test-cases-abc.t (case B): output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1301
  # Ran 2 tests, 0 skipped, 1 failed.
32316
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1302
  python hash seed: * (glob)
7340465bd788 run-tests: support multiple cases in .t test
Jun Wu <quark@fb.com>
parents: 31829
diff changeset
  1303
  [1]
32720
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1304
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1305
--restart works with outputdir
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1306
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1307
  $ mkdir output
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1308
  $ mv test-cases-abc.t.B.err output
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1309
  $ rt --restart --outputdir output
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1310
  
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1311
  --- $TESTTMP/anothertests/cases/test-cases-abc.t
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1312
  +++ $TESTTMP/anothertests/cases/output/test-cases-abc.t.B.err
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1313
  @@ -7,7 +7,7 @@
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1314
     $ V=C
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1315
   #endif
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1316
     $ echo $V | sed 's/A/C/'
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1317
  -  C
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1318
  +  B
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1319
   #if C
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1320
     $ [ $V = C ]
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1321
   #endif
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1322
  
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1323
  ERROR: test-cases-abc.t (case B) output changed
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1324
  !.
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1325
  Failed test-cases-abc.t (case B): output changed
32942
5af78c524f34 tests: remove support for warned tests
Gregory Szorc <gregory.szorc@gmail.com>
parents: 32937
diff changeset
  1326
  # Ran 2 tests, 0 skipped, 1 failed.
32720
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1327
  python hash seed: * (glob)
0cd641bfbf57 run-tests: make --restart work with output dir
Siddharth Agarwal <sid0@fb.com>
parents: 32718
diff changeset
  1328
  [1]