tests/test-run-tests.t
author Augie Fackler <augie@google.com>
Wed, 12 Apr 2017 11:23:55 -0700
branchstable
changeset 32050 77eaf9539499
parent 29518 348b2b9da703
child 31673 6a2959acae1a
permissions -rw-r--r--
dispatch: protect against malicious 'hg serve --stdio' invocations (sec) Some shared-ssh installations assume that 'hg serve --stdio' is a safe command to run for minimally trusted users. Unfortunately, the messy implementation of argument parsing here meant that trying to access a repo named '--debugger' would give the user a pdb prompt, thereby sidestepping any hoped-for sandboxing. Serving repositories over HTTP(S) is unaffected. We're not currently hardening any subcommands other than 'serve'. If your service exposes other commands to users with arbitrary repository names, it is imperative that you defend against repository names of '--debugger' and anything starting with '--config'. The read-only mode of hg-ssh stopped working because it provided its hook configuration to "hg serve --stdio" via --config parameter. This is banned for security reasons now. This patch switches it to directly call ui.setconfig(). If your custom hosting infrastructure relies on passing --config to "hg serve --stdio", you'll need to find a different way to get that configuration into Mercurial, either by using ui.setconfig() as hg-ssh does in this patch, or by placing an hgrc file someplace where Mercurial will read it. mitrandir@fb.com provided some extra fixes for the dispatch code and for hg-ssh in places that I overlooked.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     1
This file tests the behavior of run-tests.py itself.
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     2
24960
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     3
Avoid interference from actual test env:
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     4
29220
054bcdad6042 tests: refactor run-tests helpers
timeless <timeless@mozdev.org>
parents: 29199
diff changeset
     5
  $ . "$TESTDIR/helper-runtests.sh"
24960
65e8dac7b016 test-run-test: unset run-test specific environment variables
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
     6
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
     7
Smoke test with install
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     8
============
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
     9
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    10
  $ run-tests.py $HGTEST_RUN_TESTS_PURE -l
21732
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    11
  
b6192ea7f7fd test: introduce test-run-tests.t
Pierre-Yves David <pierre-yves.david@fb.com>
parents:
diff changeset
    12
  # Ran 0 tests, 0 skipped, 0 warned, 0 failed.
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    13
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    14
Define a helper to avoid the install step
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    15
=============
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    16
  $ rt()
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    17
  > {
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    18
  >     run-tests.py --with-hg=`which hg` "$@"
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    19
  > }
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    20
28037
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    21
error paths
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    22
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    23
#if symlink
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    24
  $ ln -s `which true` hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    25
  $ run-tests.py --with-hg=./hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    26
  warning: --with-hg should specify an hg script
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    27
  
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    28
  # Ran 0 tests, 0 skipped, 0 warned, 0 failed.
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    29
  $ rm hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    30
#endif
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    31
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    32
#if execbit
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    33
  $ touch hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    34
  $ run-tests.py --with-hg=./hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    35
  Usage: run-tests.py [options] [tests]
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    36
  
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    37
  run-tests.py: error: --with-hg must specify an executable hg script
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    38
  [2]
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    39
  $ rm hg
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    40
#endif
54c896f8bb79 run-tests: warn about symlinks to non hg scripts
timeless <timeless@mozdev.org>
parents: 27927
diff changeset
    41
28812
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    42
an empty test
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    43
=======================
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    44
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    45
  $ touch test-empty.t
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    46
  $ rt
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    47
  .
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    48
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    49
  $ rm test-empty.t
f1de5a612a74 run-tests: handle empty tests
timeless <timeless@mozdev.org>
parents: 28701
diff changeset
    50
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    51
a succesful test
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    52
=======================
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    53
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    54
  $ cat > test-success.t << EOF
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    55
  >   $ echo babar
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    56
  >   babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    57
  >   $ echo xyzzy
28701
3bce3d2fd727 run-tests: make _processoutput picky about optional globs
timeless <timeless@mozdev.org>
parents: 28619
diff changeset
    58
  >   never*happens (glob) (?)
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
    59
  >   xyzzy
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25098
diff changeset
    60
  >   nor this (?)
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    61
  >   $ printf 'abc\ndef\nxyz\n'
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    62
  >   123 (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    63
  >   abc
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    64
  >   def (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    65
  >   456 (?)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
    66
  >   xyz
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    67
  > EOF
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    68
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
    69
  $ rt
21734
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    70
  .
34e0bd895afc test-run-tests.t: test running a passing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21732
diff changeset
    71
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    72
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    73
failing test
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    74
==================
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
    75
28619
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    76
test churn with globs
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    77
  $ cat > test-failure.t <<EOF
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    78
  >   $ echo "bar-baz"; echo "bar-bad"
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    79
  >   bar*bad (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    80
  >   bar*baz (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    81
  > EOF
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    82
  $ rt test-failure.t
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    83
  
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    84
  --- $TESTTMP/test-failure.t
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    85
  +++ $TESTTMP/test-failure.t.err
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    86
  @@ -1,3 +1,3 @@
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    87
     $ echo "bar-baz"; echo "bar-bad"
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    88
  +  bar*baz (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    89
     bar*bad (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    90
  -  bar*baz (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    91
  
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    92
  ERROR: test-failure.t output changed
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    93
  !
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    94
  Failed test-failure.t: output changed
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    95
  # Ran 1 tests, 0 skipped, 0 warned, 1 failed.
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    96
  python hash seed: * (glob)
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    97
  [1]
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    98
695c666f42ff tests: ensure run-tests handles multiple lines of churn
timeless <timeless@mozdev.org>
parents: 28616
diff changeset
    99
basic failing test
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   100
  $ cat > test-failure.t << EOF
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   101
  >   $ echo babar
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   102
  >   rataxes
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   103
  > This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   104
  > this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   105
  > pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   106
  > pad pad pad pad............................................................
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   107
  > EOF
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   108
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   109
  >>> fh = open('test-failure-unicode.t', 'wb')
25054
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   110
  >>> fh.write(u'  $ echo babar\u03b1\n'.encode('utf-8')) and None
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   111
  >>> fh.write(u'  l\u03b5\u03b5t\n'.encode('utf-8')) and None
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   112
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   113
  $ rt
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   114
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   115
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   116
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   117
  @@ -1,5 +1,5 @@
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   118
     $ echo babar
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   119
  -  rataxes
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   120
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   121
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   122
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   123
   pad pad pad pad............................................................
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   124
  
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   125
  ERROR: test-failure.t output changed
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   126
  !.
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   127
  --- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   128
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   129
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   130
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   131
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   132
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   133
  
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   134
  ERROR: test-failure-unicode.t output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   135
  !
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   136
  Failed test-failure.t: output changed
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   137
  Failed test-failure-unicode.t: output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   138
  # Ran 3 tests, 0 skipped, 0 warned, 2 failed.
21738
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   139
  python hash seed: * (glob)
7ec3b32b98bb test-run-tests.t: test running a failing test
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21734
diff changeset
   140
  [1]
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   141
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   142
test --xunit support
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   143
  $ rt --xunit=xunit.xml
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   144
  
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   145
  --- $TESTTMP/test-failure.t
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   146
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   147
  @@ -1,5 +1,5 @@
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   148
     $ echo babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   149
  -  rataxes
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   150
  +  babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   151
   This is a noop statement so that
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   152
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   153
   pad pad pad pad............................................................
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   154
  
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   155
  ERROR: test-failure.t output changed
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   156
  !.
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   157
  --- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   158
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   159
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   160
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   161
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   162
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   163
  
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   164
  ERROR: test-failure-unicode.t output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   165
  !
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   166
  Failed test-failure.t: output changed
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   167
  Failed test-failure-unicode.t: output changed
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   168
  # Ran 3 tests, 0 skipped, 0 warned, 2 failed.
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   169
  python hash seed: * (glob)
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   170
  [1]
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   171
  $ cat xunit.xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   172
  <?xml version="1.0" encoding="utf-8"?>
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   173
  <testsuite errors="0" failures="2" name="run-tests" skipped="0" tests="3">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   174
    <testcase name="test-success.t" time="*"/> (glob)
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   175
    <testcase name="test-failure-unicode.t" time="*"> (glob)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   176
  <![CDATA[--- $TESTTMP/test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   177
  +++ $TESTTMP/test-failure-unicode.t.err
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   178
  @@ -1,2 +1,2 @@
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   179
     $ echo babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   180
  -  l\xce\xb5\xce\xb5t (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   181
  +  babar\xce\xb1 (esc)
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   182
  ]]>  </testcase>
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   183
    <testcase name="test-failure.t" time="*"> (glob)
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   184
  <![CDATA[--- $TESTTMP/test-failure.t
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   185
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   186
  @@ -1,5 +1,5 @@
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   187
     $ echo babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   188
  -  rataxes
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   189
  +  babar
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   190
   This is a noop statement so that
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   191
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   192
   pad pad pad pad............................................................
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   193
  ]]>  </testcase>
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   194
  </testsuite>
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   195
29280
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   196
  $ cat .testtimes
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   197
  test-failure-unicode.t * (glob)
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   198
  test-failure.t * (glob)
6c97b9739f9d tests: add run-test .testtimes basic testing
timeless <timeless@mozdev.org>
parents: 29221
diff changeset
   199
  test-success.t * (glob)
24500
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   200
  $ rm test-failure-unicode.t
7b0a20cd8c95 run-tests: explicitly handle unicode when writing xunit file
Gregory Szorc <gregory.szorc@gmail.com>
parents: 23935
diff changeset
   201
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   202
test for --retest
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   203
====================
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   204
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   205
  $ rt --retest
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   206
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   207
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   208
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   209
  @@ -1,5 +1,5 @@
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   210
     $ echo babar
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   211
  -  rataxes
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   212
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   213
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   214
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   215
   pad pad pad pad............................................................
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   216
  
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   217
  ERROR: test-failure.t output changed
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   218
  !
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   219
  Failed test-failure.t: output changed
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   220
  # Ran 2 tests, 1 skipped, 0 warned, 1 failed.
21741
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   221
  python hash seed: * (glob)
02cd29156d5d test-run-tests.t: tests the --retest option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21738
diff changeset
   222
  [1]
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   223
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   224
Selecting Tests To Run
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   225
======================
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   226
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   227
successful
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   228
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   229
  $ rt test-success.t
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   230
  .
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   231
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   232
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   233
success w/ keyword
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   234
  $ rt -k xyzzy
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   235
  .
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   236
  # Ran 2 tests, 1 skipped, 0 warned, 0 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   237
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   238
failed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   239
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   240
  $ rt test-failure.t
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   241
  
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   242
  --- $TESTTMP/test-failure.t
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   243
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   244
  @@ -1,5 +1,5 @@
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   245
     $ echo babar
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   246
  -  rataxes
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   247
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   248
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   249
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   250
   pad pad pad pad............................................................
21742
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   251
  
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   252
  ERROR: test-failure.t output changed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   253
  !
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   254
  Failed test-failure.t: output changed
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   255
  # Ran 1 tests, 0 skipped, 0 warned, 1 failed.
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   256
  python hash seed: * (glob)
66f91859822b test-run-tests.t: test selection of testfile from the command line
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21741
diff changeset
   257
  [1]
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   258
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   259
failure w/ keyword
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   260
  $ rt -k rataxes
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   261
  
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   262
  --- $TESTTMP/test-failure.t
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   263
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   264
  @@ -1,5 +1,5 @@
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   265
     $ echo babar
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   266
  -  rataxes
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   267
  +  babar
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   268
   This is a noop statement so that
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   269
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   270
   pad pad pad pad............................................................
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   271
  
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   272
  ERROR: test-failure.t output changed
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   273
  !
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   274
  Failed test-failure.t: output changed
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   275
  # Ran 2 tests, 1 skipped, 0 warned, 1 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   276
  python hash seed: * (glob)
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   277
  [1]
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   278
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   279
Verify that when a process fails to start we show a useful message
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   280
==================================================================
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   281
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   282
  $ cat > test-serve-fail.t <<EOF
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   283
  >   $ echo 'abort: child process failed to start blah'
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   284
  > EOF
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   285
  $ rt test-serve-fail.t
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   286
  
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   287
  ERROR: test-serve-fail.t output changed
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   288
  !
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   289
  Failed test-serve-fail.t: server failed to start (HGPORT=*) (glob)
27567
54bc0e884385 run-tests: avoid double counting server fails
timeless <timeless@mozdev.org>
parents: 27564
diff changeset
   290
  # Ran 1 tests, 0 skipped, 0 warned, 1 failed.
22840
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   291
  python hash seed: * (glob)
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   292
  [1]
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   293
  $ rm test-serve-fail.t
ba5c635921e1 test-run-tests: add a test for detection of failure to start a server
Augie Fackler <raf@durin42.com>
parents: 22838
diff changeset
   294
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   295
Verify that we can try other ports
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   296
===================================
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   297
  $ hg init inuse
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   298
  $ hg serve -R inuse -p $HGPORT -d --pid-file=blocks.pid
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   299
  $ cat blocks.pid >> $DAEMON_PIDS
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   300
  $ cat > test-serve-inuse.t <<EOF
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   301
  >   $ hg serve -R `pwd`/inuse -p \$HGPORT -d --pid-file=hg.pid
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   302
  >   $ cat hg.pid >> \$DAEMON_PIDS
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   303
  > EOF
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   304
  $ rt test-serve-inuse.t
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   305
  .
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   306
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   307
  $ rm test-serve-inuse.t
29221
73f58eb14684 test-run-tests: clean up inuse server eagerly
timeless <timeless@mozdev.org>
parents: 29220
diff changeset
   308
  $ killdaemons.py $DAEMON_PIDS
73f58eb14684 test-run-tests: clean up inuse server eagerly
timeless <timeless@mozdev.org>
parents: 29220
diff changeset
   309
  $ rm $DAEMON_PIDS
27602
67aa88e00fc7 run-tests: fix get port to try differing ports
timeless <timeless@mozdev.org>
parents: 27567
diff changeset
   310
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   311
Running In Debug Mode
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   312
======================
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   313
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   314
  $ rt --debug 2>&1 | grep -v pwd
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   315
  + echo *SALT* 0 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   316
  *SALT* 0 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   317
  + echo babar
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   318
  babar
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   319
  + echo *SALT* 6 0 (glob)
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   320
  *SALT* 6 0 (glob)
27509
6ff90d618d72 test-run-tests: glob away a --debug run difference on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 27395
diff changeset
   321
  *+ echo *SALT* 0 0 (glob)
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   322
  *SALT* 0 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   323
  + echo babar
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   324
  babar
23676
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   325
  + echo *SALT* 2 0 (glob)
e8b09f920fe6 tests: adapt glob pattern to fix test with NetBSD's sh(1) (issue4484)
Thomas Klausner <tk@giga.or.at>
parents: 23348
diff changeset
   326
  *SALT* 2 0 (glob)
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   327
  + echo xyzzy
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   328
  xyzzy
25388
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25098
diff changeset
   329
  + echo *SALT* 6 0 (glob)
6025cac3d02f tests: add (?) flag for optional lines
Matt Mackall <mpm@selenic.com>
parents: 25098
diff changeset
   330
  *SALT* 6 0 (glob)
28317
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   331
  + printf *abc\ndef\nxyz\n* (glob)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   332
  abc
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   333
  def
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   334
  xyz
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   335
  + echo *SALT* 12 0 (glob)
8de70574be2c run-tests: defer leftover (?) cleanup until after all output is exhausted
Matt Harbison <matt_harbison@yahoo.com>
parents: 28316
diff changeset
   336
  *SALT* 12 0 (glob)
21743
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   337
  .
d72bea858cbd test-run-tests.t: test --debug option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21742
diff changeset
   338
  # Ran 2 tests, 0 skipped, 0 warned, 0 failed.
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   339
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   340
Parallel runs
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   341
==============
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   342
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   343
(duplicate the failing test to get predictable output)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   344
  $ cp test-failure.t test-failure-copy.t
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   345
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   346
  $ rt --jobs 2 test-failure*.t -n
23107
5459b30aa498 tests: silence output race in test-run-tests.t
Matt Mackall <mpm@selenic.com>
parents: 22840
diff changeset
   347
  !!
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   348
  Failed test-failure*.t: output changed (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   349
  Failed test-failure*.t: output changed (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   350
  # Ran 2 tests, 0 skipped, 0 warned, 2 failed.
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   351
  python hash seed: * (glob)
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   352
  [1]
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   353
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   354
failures in parallel with --first should only print one failure
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   355
  >>> f = open('test-nothing.t', 'w')
25054
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   356
  >>> f.write('foo\n' * 1024) and None
af5a778f8e2e test-run-tests.t: work around file.write() returning an int
Augie Fackler <augie@google.com>
parents: 24990
diff changeset
   357
  >>> f.write('  $ sleep 1') and None
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   358
  $ rt --jobs 2 --first
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   359
  
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   360
  --- $TESTTMP/test-failure*.t (glob)
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   361
  +++ $TESTTMP/test-failure*.t.err (glob)
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   362
  @@ -1,5 +1,5 @@
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   363
     $ echo babar
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   364
  -  rataxes
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   365
  +  babar
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   366
   This is a noop statement so that
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   367
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   368
   pad pad pad pad............................................................
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   369
  
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   370
  Failed test-failure*.t: output changed (glob)
24507
a0668a587c04 run-tests: wait for test threads after first error
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
   371
  Failed test-nothing.t: output changed
a0668a587c04 run-tests: wait for test threads after first error
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24500
diff changeset
   372
  # Ran 2 tests, 0 skipped, 0 warned, 2 failed.
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   373
  python hash seed: * (glob)
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   374
  [1]
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   375
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   376
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   377
(delete the duplicated test file)
22838
9a20f53e436f run-tests: handle --jobs and --first gracefully
Augie Fackler <raf@durin42.com>
parents: 22579
diff changeset
   378
  $ rm test-failure-copy.t test-nothing.t
21744
395e29928db2 test-run-tests.t: tests --jobs option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21743
diff changeset
   379
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   380
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   381
Interactive run
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   382
===============
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   383
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   384
(backup the failing test)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   385
  $ cp test-failure.t backup
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   386
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   387
Refuse the fix
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   388
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   389
  $ echo 'n' | rt -i
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   390
  
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   391
  --- $TESTTMP/test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   392
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   393
  @@ -1,5 +1,5 @@
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   394
     $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   395
  -  rataxes
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   396
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   397
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   398
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   399
   pad pad pad pad............................................................
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21756
diff changeset
   400
  Accept this change? [n] 
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   401
  ERROR: test-failure.t output changed
21763
84cd5ee787ed run-tests: hold iolock across diff/prompt when interactive
Matt Mackall <mpm@selenic.com>
parents: 21756
diff changeset
   402
  !.
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   403
  Failed test-failure.t: output changed
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   404
  # Ran 2 tests, 0 skipped, 0 warned, 1 failed.
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   405
  python hash seed: * (glob)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   406
  [1]
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   407
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   408
  $ cat test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   409
    $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   410
    rataxes
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   411
  This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   412
  this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   413
  pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   414
  pad pad pad pad............................................................
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   415
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   416
Interactive with custom view
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   417
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   418
  $ echo 'n' | rt -i --view echo
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   419
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err (glob)
22361
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   420
  Accept this change? [n]* (glob)
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   421
  ERROR: test-failure.t output changed
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   422
  !.
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   423
  Failed test-failure.t: output changed
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   424
  # Ran 2 tests, 0 skipped, 0 warned, 1 failed.
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   425
  python hash seed: * (glob)
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   426
  [1]
eb6adf750954 run-tests: make --interactive work with --view
Gregory Szorc <gregory.szorc@gmail.com>
parents: 22292
diff changeset
   427
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   428
View the fix
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   429
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   430
  $ echo 'y' | rt --view echo
23348
bbe56e07e07a tests: fix globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 23245
diff changeset
   431
  $TESTTMP/test-failure.t $TESTTMP/test-failure.t.err (glob)
22108
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   432
  
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   433
  ERROR: test-failure.t output changed
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   434
  !.
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   435
  Failed test-failure.t: output changed
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   436
  # Ran 2 tests, 0 skipped, 0 warned, 1 failed.
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   437
  python hash seed: * (glob)
ccae572c5d09 test-run-tests: test --view
Matt Mackall <mpm@selenic.com>
parents: 22107
diff changeset
   438
  [1]
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   439
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   440
Accept the fix
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   441
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   442
  $ echo "  $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   443
  $ echo "  saved backup bundle to \$TESTTMP/foo.hg" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   444
  $ echo "  $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   445
  $ echo "  saved backup bundle to \$TESTTMP/foo.hg (glob)" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   446
  $ echo "  $ echo 'saved backup bundle to \$TESTTMP/foo.hg'" >> test-failure.t
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   447
  $ echo "  saved backup bundle to \$TESTTMP/*.hg (glob)" >> test-failure.t
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   448
  $ echo 'y' | rt -i 2>&1
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   449
  
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   450
  --- $TESTTMP/test-failure.t
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   451
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   452
  @@ -1,11 +1,11 @@
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   453
     $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   454
  -  rataxes
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   455
  +  babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   456
   This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   457
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   458
   pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   459
   pad pad pad pad............................................................
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   460
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   461
  -  saved backup bundle to $TESTTMP/foo.hg
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   462
  +  saved backup bundle to $TESTTMP/foo.hg* (glob)
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   463
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   464
     saved backup bundle to $TESTTMP/foo.hg* (glob)
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   465
     $ echo 'saved backup bundle to $TESTTMP/foo.hg'
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   466
  Accept this change? [n] ..
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   467
  # Ran 2 tests, 0 skipped, 0 warned, 0 failed.
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   468
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   469
  $ sed -e 's,(glob)$,&<,g' test-failure.t
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   470
    $ echo babar
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   471
    babar
21995
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   472
  This is a noop statement so that
20bfcb484554 test-run-tests.t: add extra data to tests for keyword tests
Augie Fackler <raf@durin42.com>
parents: 21977
diff changeset
   473
  this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   474
  pad pad pad pad............................................................
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   475
  pad pad pad pad............................................................
23728
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   476
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   477
    saved backup bundle to $TESTTMP/foo.hg (glob)<
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   478
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   479
    saved backup bundle to $TESTTMP/foo.hg (glob)<
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   480
    $ echo 'saved backup bundle to $TESTTMP/foo.hg'
31d3f973d079 run-tests: automatically add (glob) to "saved backup bundle to" lines
Mads Kiilerich <madski@unity3d.com>
parents: 23676
diff changeset
   481
    saved backup bundle to $TESTTMP/*.hg (glob)<
21755
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   482
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   483
(reinstall)
9e9288b9463a test-run-tests.t: test the --interactive option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21744
diff changeset
   484
  $ mv backup test-failure.t
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   485
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   486
No Diff
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   487
===============
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   488
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   489
  $ rt --nodiff
21756
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   490
  !.
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   491
  Failed test-failure.t: output changed
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   492
  # Ran 2 tests, 0 skipped, 0 warned, 1 failed.
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   493
  python hash seed: * (glob)
f5fd1580a649 test-run-tests.t: test the --nodiff option
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 21755
diff changeset
   494
  [1]
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   495
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   496
test --tmpdir support
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   497
  $ rt --tmpdir=$TESTTMP/keep test-success.t
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   498
  
26432
39577d4520ab test-run-tests: add globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 26422
diff changeset
   499
  Keeping testtmp dir: $TESTTMP/keep/child1/test-success.t (glob)
39577d4520ab test-run-tests: add globs for Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 26422
diff changeset
   500
  Keeping threadtmp dir: $TESTTMP/keep/child1  (glob)
26422
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   501
  .
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   502
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
41436beaf463 run-tests: report paths saved by --keep-tmpdir
timeless@mozdev.org
parents: 26158
diff changeset
   503
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   504
timeouts
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   505
========
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   506
  $ cat > test-timeout.t <<EOF
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   507
  >   $ sleep 2
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   508
  >   $ echo pass
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   509
  >   pass
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   510
  > EOF
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   511
  > echo '#require slow' > test-slow-timeout.t
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   512
  > cat test-timeout.t >> test-slow-timeout.t
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   513
  $ rt --timeout=1 --slowtimeout=3 test-timeout.t test-slow-timeout.t
27393
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
   514
  st
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   515
  Skipped test-slow-timeout.t: missing feature: allow slow tests
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   516
  Failed test-timeout.t: timed out
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   517
  # Ran 1 tests, 1 skipped, 0 warned, 1 failed.
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   518
  python hash seed: * (glob)
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   519
  [1]
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   520
  $ rt --timeout=1 --slowtimeout=3 \
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   521
  > test-timeout.t test-slow-timeout.t --allow-slow-tests
27393
a40b623e6380 run-tests: report timeouts in a less alarming fashion
Matt Mackall <mpm@selenic.com>
parents: 27141
diff changeset
   522
  .t
27141
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   523
  Failed test-timeout.t: timed out
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   524
  # Ran 2 tests, 0 skipped, 0 warned, 1 failed.
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   525
  python hash seed: * (glob)
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   526
  [1]
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   527
  $ rm test-timeout.t test-slow-timeout.t
a4e3dec3010e run-tests: add --slowtimeout and use it for slow tests
timeless <timeless@mozdev.org>
parents: 27057
diff changeset
   528
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   529
test for --time
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   530
==================
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   531
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   532
  $ rt test-success.t --time
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   533
  .
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   534
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   535
  # Producing time report
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   536
  start   end     cuser   csys    real      Test
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   537
  \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   test-success.t (re)
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   538
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   539
test for --time with --job enabled
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   540
====================================
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   541
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   542
  $ rt test-success.t --time --jobs 2
21977
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   543
  .
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   544
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
4ca4e1572022 run-tests: '--time' option provide more details to Linux users
anuraggoel <anurag.dsps@gmail.com>
parents: 21926
diff changeset
   545
  # Producing time report
25098
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   546
  start   end     cuser   csys    real      Test
bf84ab53c2fd run-tests: include 'start' and 'end' in --time output
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25097
diff changeset
   547
  \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   \s*[\d\.]{5}   test-success.t (re)
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   548
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   549
Skips
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   550
================
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   551
  $ cat > test-skip.t <<EOF
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   552
  >   $ echo xyzzy
22045
769198c6a62d run-tests: add #require to abort full test
Matt Mackall <mpm@selenic.com>
parents: 22044
diff changeset
   553
  > #require false
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   554
  > EOF
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   555
  $ rt --nodiff
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   556
  !.s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   557
  Skipped test-skip.t: missing feature: nail clipper
21996
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   558
  Failed test-failure.t: output changed
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   559
  # Ran 2 tests, 1 skipped, 0 warned, 1 failed.
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   560
  python hash seed: * (glob)
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   561
  [1]
947553944e92 test-run-tests.t: add tests for skips
Augie Fackler <raf@durin42.com>
parents: 21995
diff changeset
   562
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   563
  $ rt --keyword xyzzy
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   564
  .s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   565
  Skipped test-skip.t: missing feature: nail clipper
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   566
  # Ran 2 tests, 2 skipped, 0 warned, 0 failed.
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   567
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   568
Skips with xml
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   569
  $ rt --keyword xyzzy \
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   570
  >  --xunit=xunit.xml
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   571
  .s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   572
  Skipped test-skip.t: missing feature: nail clipper
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   573
  # Ran 2 tests, 2 skipped, 0 warned, 0 failed.
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   574
  $ cat xunit.xml
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   575
  <?xml version="1.0" encoding="utf-8"?>
22107
3b5cf39ffcc4 run-tests: don't show 'i' for tests that don't match a keyword
Matt Mackall <mpm@selenic.com>
parents: 22045
diff changeset
   576
  <testsuite errors="0" failures="0" name="run-tests" skipped="2" tests="2">
22044
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   577
    <testcase name="test-success.t" time="*"/> (glob)
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   578
  </testsuite>
a06172e85fd4 run-tests: add support for xunit test reports
Augie Fackler <raf@durin42.com>
parents: 21997
diff changeset
   579
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   580
Missing skips or blacklisted skips don't count as executed:
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   581
  $ echo test-failure.t > blacklist
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   582
  $ rt --blacklist=blacklist --json\
21997
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   583
  >   test-failure.t test-bogus.t
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   584
  ss
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   585
  Skipped test-bogus.t: Doesn't exist
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   586
  Skipped test-failure.t: blacklisted
93c3b3f55d59 run-tests: fix test result counts with --keyword specified or skips occurring
Augie Fackler <raf@durin42.com>
parents: 21996
diff changeset
   587
  # Ran 0 tests, 2 skipped, 0 warned, 0 failed.
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   588
  $ cat report.json
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   589
  testreport ={
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   590
      "test-bogus.t": {
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   591
          "result": "skip"
29199
daff05dcd184 run-tests: handle json.dumps divergence
timeless <timeless@mozdev.org>
parents: 29173
diff changeset
   592
      },
27927
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   593
      "test-failure.t": {
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   594
          "result": "skip"
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   595
      }
0de4dfc9af0c run-tests: fix crash when --json and --blacklist are both used (issue5050)
Laurent Charignon <lcharignon@fb.com>
parents: 27686
diff changeset
   596
  } (no-eol)
29173
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   597
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   598
Whitelist trumps blacklist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   599
  $ echo test-failure.t > whitelist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   600
  $ rt --blacklist=blacklist --whitelist=whitelist --json\
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   601
  >   test-failure.t test-bogus.t
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   602
  s
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   603
  --- $TESTTMP/test-failure.t
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   604
  +++ $TESTTMP/test-failure.t.err
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   605
  @@ -1,5 +1,5 @@
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   606
     $ echo babar
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   607
  -  rataxes
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   608
  +  babar
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   609
   This is a noop statement so that
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   610
   this test is still more bytes than success.
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   611
   pad pad pad pad............................................................
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   612
  
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   613
  ERROR: test-failure.t output changed
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   614
  !
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   615
  Skipped test-bogus.t: Doesn't exist
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   616
  Failed test-failure.t: output changed
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   617
  # Ran 1 tests, 1 skipped, 0 warned, 1 failed.
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   618
  python hash seed: * (glob)
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   619
  [1]
fa9fae9a9372 tests: add coverage for run-tests.py --whitelist
timeless <timeless@mozdev.org>
parents: 28812
diff changeset
   620
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   621
test for --json
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   622
==================
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   623
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   624
  $ rt --json
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   625
  
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   626
  --- $TESTTMP/test-failure.t
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   627
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   628
  @@ -1,5 +1,5 @@
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   629
     $ echo babar
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   630
  -  rataxes
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   631
  +  babar
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   632
   This is a noop statement so that
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   633
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   634
   pad pad pad pad............................................................
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   635
  
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   636
  ERROR: test-failure.t output changed
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   637
  !.s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   638
  Skipped test-skip.t: missing feature: nail clipper
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   639
  Failed test-failure.t: output changed
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   640
  # Ran 2 tests, 1 skipped, 0 warned, 1 failed.
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   641
  python hash seed: * (glob)
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   642
  [1]
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   643
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   644
  $ cat report.json
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   645
  testreport ={
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   646
      "test-failure.t": [\{] (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   647
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   648
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   649
          "diff": "---.+\+\+\+.+", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   650
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   651
          "result": "failure", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   652
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   653
          "time": "\s*[\d\.]{4,5}" (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   654
      }, ? (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   655
      "test-skip.t": {
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   656
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   657
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   658
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   659
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   660
          "result": "skip", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   661
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   662
          "time": "\s*[\d\.]{4,5}" (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   663
      }, ? (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   664
      "test-success.t": [\{] (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   665
          "csys": "\s*[\d\.]{4,5}", ? (re)
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   666
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   667
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   668
          "end": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   669
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   670
          "start": "\s*[\d\.]{4,5}", ? (re)
23245
d33a90cb2a32 test-run-tests: accept more levels of precision and trailing ws (issue4440)
Augie Fackler <raf@durin42.com>
parents: 23107
diff changeset
   671
          "time": "\s*[\d\.]{4,5}" (re)
22391
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   672
      }
c42e69268f5b run-tests: added '--json' functionality to store test result in json file
anuraggoel <anurag.dsps@gmail.com>
parents: 22361
diff changeset
   673
  } (no-eol)
22579
36a940d82f88 test: protect the run-tests.py --json test behind an hghave rule
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 22486
diff changeset
   674
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   675
Test that failed test accepted through interactive are properly reported:
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   676
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   677
  $ cp test-failure.t backup
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   678
  $ echo y | rt --json -i
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   679
  
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   680
  --- $TESTTMP/test-failure.t
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   681
  +++ $TESTTMP/test-failure.t.err
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   682
  @@ -1,5 +1,5 @@
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   683
     $ echo babar
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   684
  -  rataxes
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   685
  +  babar
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   686
   This is a noop statement so that
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   687
   this test is still more bytes than success.
28316
c0cecc153d25 test-run-tests: pad the failure test to preserve the run order
Matt Harbison <matt_harbison@yahoo.com>
parents: 28180
diff changeset
   688
   pad pad pad pad............................................................
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   689
  Accept this change? [n] ..s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   690
  Skipped test-skip.t: missing feature: nail clipper
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   691
  # Ran 2 tests, 1 skipped, 0 warned, 0 failed.
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   692
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   693
  $ cat report.json
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   694
  testreport ={
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   695
      "test-failure.t": [\{] (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   696
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   697
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   698
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   699
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   700
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   701
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   702
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   703
      }, ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   704
      "test-skip.t": {
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   705
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   706
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   707
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   708
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   709
          "result": "skip", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   710
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   711
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   712
      }, ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   713
      "test-success.t": [\{] (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   714
          "csys": "\s*[\d\.]{4,5}", ? (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   715
          "cuser": "\s*[\d\.]{4,5}", ? (re)
27686
df142b77b4c2 run-tests: add 'diff' entry in json report
Laurent Charignon <lcharignon@fb.com>
parents: 27602
diff changeset
   716
          "diff": "", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   717
          "end": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   718
          "result": "success", ? (re)
25097
a4fce7905721 run-tests: track start and end time of tests
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 25054
diff changeset
   719
          "start": "\s*[\d\.]{4,5}", ? (re)
24979
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   720
          "time": "\s*[\d\.]{4,5}" (re)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   721
      }
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   722
  } (no-eol)
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   723
  $ mv backup test-failure.t
f44db7343be9 run-test: add a test for json output when -i is used
Pierre-Yves David <pierre-yves.david@fb.com>
parents: 24811
diff changeset
   724
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   725
backslash on end of line with glob matching is handled properly
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   726
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   727
  $ cat > test-glob-backslash.t << EOF
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   728
  >   $ echo 'foo bar \\'
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   729
  >   foo * \ (glob)
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   730
  > EOF
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   731
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   732
  $ rt test-glob-backslash.t
24811
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   733
  .
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   734
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   735
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   736
  $ rm -f test-glob-backslash.t
a2dcf460e141 run-tests: don't error when glob matched line ends with backslash
Gregory Szorc <gregory.szorc@gmail.com>
parents: 24507
diff changeset
   737
29518
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29280
diff changeset
   738
Test globbing of 127.0.0.1
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29280
diff changeset
   739
  $ echo 172.16.18.1
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29280
diff changeset
   740
  127.0.0.1 (glob)
348b2b9da703 run-tests: add support for using 127.0.0.1 as a glob
Augie Fackler <raf@durin42.com>
parents: 29280
diff changeset
   741
25728
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   742
Test reusability for third party tools
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   743
======================================
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   744
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   745
  $ mkdir "$TESTTMP"/anothertests
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   746
  $ cd "$TESTTMP"/anothertests
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   747
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   748
test that `run-tests.py` can execute hghave, even if it runs not in
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   749
Mercurial source tree.
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   750
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   751
  $ cat > test-hghave.t <<EOF
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   752
  > #require true
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   753
  >   $ echo foo
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   754
  >   foo
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   755
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
   756
  $ rt test-hghave.t
25728
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   757
  .
905c32321cfb run-tests.py: execute hghave by the path relative to run-tests.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25472
diff changeset
   758
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   759
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   760
test that RUNTESTDIR refers the directory, in which `run-tests.py` now
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   761
running is placed.
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   762
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   763
  $ cat > test-runtestdir.t <<EOF
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   764
  > - $TESTDIR, in which test-run-tests.t is placed
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   765
  > - \$TESTDIR, in which test-runtestdir.t is placed (expanded at runtime)
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   766
  > - \$RUNTESTDIR, in which run-tests.py is placed (expanded at runtime)
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   767
  > 
27057
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
   768
  > #if windows
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
   769
  >   $ test "\$TESTDIR" = "$TESTTMP\anothertests"
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
   770
  > #else
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   771
  >   $ test "\$TESTDIR" = "$TESTTMP"/anothertests
27057
c281c20ad0ad test-run-tests: conditionalize the $TESTDIR check for Windows separator
Matt Harbison <matt_harbison@yahoo.com>
parents: 26432
diff changeset
   772
  > #endif
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   773
  >   $ test "\$RUNTESTDIR" = "$TESTDIR"
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   774
  >   $ head -n 3 "\$RUNTESTDIR"/../contrib/check-code.py
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   775
  >   #!/usr/bin/env python
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   776
  >   #
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   777
  >   # check-code - a style and portability checker for Mercurial
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   778
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
   779
  $ rt test-runtestdir.t
25729
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   780
  .
57dfadc4f46c run-tests.py: add RUNTESTDIR to refer `tests` of Mercurial
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25728
diff changeset
   781
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   782
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   783
#if execbit
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   784
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   785
test that TESTDIR is referred in PATH
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   786
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   787
  $ cat > custom-command.sh <<EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   788
  > #!/bin/sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   789
  > echo "hello world"
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   790
  > EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   791
  $ chmod +x custom-command.sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   792
  $ cat > test-testdir-path.t <<EOF
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   793
  >   $ custom-command.sh
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   794
  >   hello world
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   795
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
   796
  $ rt test-testdir-path.t
25730
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   797
  .
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   798
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   799
c380d5273e91 run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 25729
diff changeset
   800
#endif
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   801
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   802
test support for --allow-slow-tests
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   803
  $ cat > test-very-slow-test.t <<EOF
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   804
  > #require slow
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   805
  >   $ echo pass
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   806
  >   pass
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   807
  > EOF
28616
ca38d993b1b6 tests: remove obsolete uses of HGTEST_RUN_TESTS_PURE
timeless <timeless@mozdev.org>
parents: 28596
diff changeset
   808
  $ rt test-very-slow-test.t
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   809
  s
27564
80b53082a353 run-tests: report missing feature for skipped tests
timeless <timeless@mozdev.org>
parents: 27509
diff changeset
   810
  Skipped test-very-slow-test.t: missing feature: allow slow tests
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   811
  # Ran 0 tests, 1 skipped, 0 warned, 0 failed.
27395
0622d6e134fb tests: avoid duplicate install steps in test-run-tests
Matt Mackall <mpm@selenic.com>
parents: 27393
diff changeset
   812
  $ rt $HGTEST_RUN_TESTS_PURE --allow-slow-tests test-very-slow-test.t
26109
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   813
  .
bad09bd22b6a run-tests: add support for marking tests as very slow
Augie Fackler <augie@google.com>
parents: 25730
diff changeset
   814
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
28180
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   815
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   816
support for running a test outside the current directory
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   817
  $ mkdir nonlocal
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   818
  $ cat > nonlocal/test-is-not-here.t << EOF
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   819
  >   $ echo pass
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   820
  >   pass
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   821
  > EOF
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   822
  $ rt nonlocal/test-is-not-here.t
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   823
  .
2836a43c7722 run-tests: allow run-tests.py to run tests outside current directory
David R. MacIver <david@drmaciver.com>
parents: 28126
diff changeset
   824
  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
28596
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   825
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   826
support for bisecting failed tests automatically
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   827
  $ hg init bisect
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   828
  $ cd bisect
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   829
  $ cat >> test-bisect.t <<EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   830
  >   $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   831
  >   pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   832
  > EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   833
  $ hg add test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   834
  $ hg ci -m 'good'
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   835
  $ cat >> test-bisect.t <<EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   836
  >   $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   837
  >   fail
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   838
  > EOF
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   839
  $ hg ci -m 'bad'
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   840
  $ rt --known-good-rev=0 test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   841
  
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   842
  --- $TESTTMP/anothertests/bisect/test-bisect.t
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   843
  +++ $TESTTMP/anothertests/bisect/test-bisect.t.err
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   844
  @@ -1,4 +1,4 @@
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   845
     $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   846
     pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   847
     $ echo pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   848
  -  fail
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   849
  +  pass
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   850
  
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   851
  ERROR: test-bisect.t output changed
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   852
  !
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   853
  Failed test-bisect.t: output changed
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   854
  test-bisect.t broken by 72cbf122d116 (bad)
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   855
  # Ran 1 tests, 0 skipped, 0 warned, 1 failed.
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   856
  python hash seed: * (glob)
9949950664cd run-tests: add support for automatically bisecting test failures
Augie Fackler <augie@google.com>
parents: 28317
diff changeset
   857
  [1]