run-tests: factor options out of run() We were only utilizing 2 parts of options. These have been converted to named arguments.

convert: mercurial source: convert global tags only - not local tags Mercurial tags can be local (tag -l, stored in .hg/localtags) or global (normal tags, tracked in .hgtags) ... or extensions can add other kind of tags. Convert would take all tags (except "tip"), not just the ones from .hgtags, and put them into .hgtags. Instead, convert only the global tags that come from .hgtags.

run-tests: refactor temporary directories in Test We no longer need cleanup() because setUp() and tearDown() cover it. We move tempdir creation into setUp() and always delete in tearDown()

run-tests: move _executetests into TestSuite

run-tests: capture execution times in TestResult TestResult has facilities for recording when tests start and stop. It makes sense to move execution time recording into TestResult. In addition, output generation is being moved into TestResult, a class that has a concept of an output buffer (and a lock).

run-tests: move outputtimes() into unittest runner class The unittest runner now does all the printing. Move outputtimes() there.

run-tests: refactor how times are recorded We now record times in Test.run(), which is a more appropriate place for it.

run-tests: eliminate Test._result The result of test execution is now reported on a TestResult. We no longer need to store a tuple of the result.

run-tests: replace Test.ignore() with raise IgnoreTest

run-tests: replace Test.skip() with raise SkipTest All the result functions now raise an Exception. We no longer need these functions and we can inline the raise.

run-tests: remove some legacy code around result handling unittest handles all results now. We no longer need to manually keep track of results.

run-tests: merge MercurialTest into Test Now that we execute all tests via unittest, the MercurialTest wrapper is not necessary. This patches moves the logic from MercurialTest into Test and makes Test a child of unittest.TestCase.

check-code: check for consistent usage of the websub filter in hgweb templates The check-code tool now expects the "desc" keyword to be followed by the "websub" filter, with the following exceptions: a) It has no filters at all, e.g. a changeset description in the raw style templates or the repository description in the summary page. b) It is followed by the "firstline" filter, e.g. the first line of the changeset description is displayed as a summary or title.

hgweb: apply websub filter to the changeset description in rss and atom feeds For example, this is useful for linking from the feed reader to a bug tracker. This follows the existing pattern used within the hgweb templates. With the exception of the raw style, all usages of the changeset "desc" keyword are now followed by either the "firstline" filter or the "websub" filter. When "websub" is used, it always follows the "escape" filter.

tests: write the hgrc file for test-hgweb-commands.t with cat instead of echo To improve the readability.

tests: tests for the rss feeds This matches the existing tests for the atom feeds.

localrepo: use _matchstatus of context objects Now that we have the machinery in place, we call the context method for changing the match object in the case of comparing the working directory with its parent. This removes the last of the hard-coded workingctx knowledge in localrepo paving the way for us to remove localrepo.status completely.

workingctx: override _matchstatus for parentworking case This patch encapsulate the logic for changing the match.bad function when comparing against the working directory's parent. Future patches will remove more of the 'if ... else' blocks in localrepo.status that test for this working directory parent case.

basectx: add _matchstatus method for factoring out last of parentworking logic This patch paves the way to allow a workingctx to override the match object with a custom 'bad' method for cases where status is sent a directory pattern.

workingctx: use inheritance for _buildstatus while keeping the fastpath This patch maintains the fast path for workingctx which is to not build a manifest if the working directory is being compared to its parent since, in this case, we can just copy the parent manifest.

localrepo: remove unneeded unpacking of r This unpacking is unneeded now because previous patches have removed the need for this method to modify each of these variables in favor of passing the whole set around to pre/post hook methods of the corresponding context object.

localrepo: call _poststatus hook instead of hardcoding symlink knowledge

workingctx: add _poststatus method to call _filtersuspectsymlink With this patch, we are one step closer to removing 'if ... else' logic in localrepo.status for testing if the context is the working directory or not. Future patches will replace those blocks of code with a call to the context's _poststatus hook so that each context object will do the right thing depending on the situation.

context: add a no-op _poststatus method This patch adds a private _poststatus method so that certain contexts, such as workingctx, can add custom post-processing to status.

localrepo: call _prestatus hook instead of hardcoding dirstate knowledge

workingctx: add _prestatus method to call _dirstatestatus With this patch, we are one step closer to removing 'if ... else' logic in localrepo.status for testing if the context is the working directory or not. Future patches will replace those blocks of code with a call to the context's _prestatus hook so that each context object will do the right thing depending on the situation.

context: add a no-op _prestatus method This patch adds a private _prestatus method so that certain contexts, such as workingctx, can add custom pre-processing to status.

localrepo: use _buildstatus from context

context: add _buildstatus method This method is a copy of localstatus.status's core logic. Later patches will clean up some of the dense coditionals in the for loop.

localrepo: remove workingctx logic already moved to context

localrepo: move the order of local variables This movement is a small step in getting rid of the 'if ... else' logic for testing the current working directory with its parent. Previously, the deleted, unknown, and ignored variables were set in a combination of before an 'if ... else' block and within the block. This moves the variables to be set outside the loop in one common place.

localrepo: factor out _manifestmatch logic for workingctx

localrepo: use _manifestmatches instead of duplicating logic

basectx: add _manifestmatches method This method is a duplicate of localrepo.mfmatches and sets the stage for factoring localrepo.status into a context method that will be customizable based on inheritance and object type.

localrepo: always set the variable r in status method This is a small patch to help streamline keeping tracking of the list of files for status in a variable already called 'r' ('s' is for subrepos in this method). We now move the setting of it out of an 'if' block so that we can later refactor more code into the context objects.

run-tests: execute tests via unittest Now that unittest mode is functionally equivalent to the default mode, we switch the default execution mode to unittest and remove the choice of different execution modes.

run-tests: implement Test.__str__ unittest sometimes does a str(test) and the default implementation doesn't do the right thing. So we override it.

run-tests: make failure reporting in unittest mode equivalent to default Unlike unittest's defaults, our result formatter does not print stack traces. Here, we change TestResult.addFailure() to be compatible with the existing/default execution mode.

run-tests: ignore failures from rmtree rmtree() may fail under certain conditions. We ignore failures at the individual test level because they can interfere with test execution. Furthermore, we'll reattempt deletion in the high-level test runner after all tests have finished. Note that this patch is not a code refactor like most of the patches before it. This change logically makes sense given the execution behavior of the tests.

run-tests: abort tests after first failure in unittest mode There is an execution mode on run-tests.py that stops after the first failure. unittest mode was previously not obeying this option. This patch fixes that.

run-tests: print compatible output from TextTestRunner Our unittest TextTestRunner now prints output similar to the output from the default execution mode - the output people expect from run-tests.py.

run-tests: don't print results in unittest mode A subsequent patch will have unittest print tests for us. Don't perform old-style printing when in unittest mode.

run-tests: move errpath deletion to setUp()

run-tests: kill daemons during tearDown()

run-tests: move err file writing to tearDown()

run-tests: refactor testtmp testtmp is now a member variable of our test class. It's value is computed during instance creation and the directory is managed via the lifetime of the test object.

run-tests: store skipped state on Test As code is being moved between functions, we need a member variable to hold skipped state.

run-tests: move some functionality to Test.tearDown()

run-tests: store last result in Test._result Subsequent patches move post-test actions into tearDown(). This requires tearDown() to have access to the last result.

run-tests: report test duration in Test.tearDown()

run-tests: store test return code and output in Test instance A subsequent patch that will introduce tearDown() requires test execution state be stored in the instance.

run-tests: keep track of test start and stop in MercurialTest.run() This brings run() more compatible with unittest.TestCase.run()

run-tests: keep track of test execution state in Test This patch starts a mini series of moving functionality into the newly-established setUp() and tearDown() methods.

run-tests: support setUp() and tearDown() in TestCase wrapper unittest.TestCase.run() calls setUp() and tearDown() during run(). We emulate that implementation.

run-tests: fail tests by raising an exception When in unittest mode, Test.run() will now raise for all non-success cases. This makes it behave like TestCase.run().

run-tests: record warnings by raising WarnTest We continue the conversion of recording test results by raising exceptions.

run-tests: record ignored tests by raising IgnoreTest

run-tests: record skips by raising SkipTest The unittest way of recording a skipped test is to raise an exception (at least with modern unittest implementation). We change Test to raise a SkipTest when operating in unittest mode. This does prevent some "tear down" activities from running in unittest mode. This will be fixed in subsequent patches. Since unittest mode is experimental, this should be OK.

run-tests: implement TestCase.run() Simply wrapping TestCase.run() is not sufficient for robust results reporting because unittest in Python 2.4 does not know about things like skipped tests and reports them as success or failures instead of skips. We will reimplement TestCase.run() with knowledge and semantics present in modern Python releases.

run-tests: don't print progress from Test when in unittest mode unittest does its own printing of progress indicators as part of TestResult. So, there is no need to print them when running in unittest mode. This will fix the double output of progress indicators that had been occurring in unittest mode.

run-tests: define a custom TestSuite that uses _executetests() We now have a custom unittest.TestSuite implementation that uses _executetests() and thus knows how to execute tests concurrently. Running tests in --unittest mode will use this TestSuite. Since the TestSuite handles concurrency, the warnings around --jobs and --loop have been removed.

run-tests: pass an optional TestResult into _executetests() If the result is passed, we execute tests in the unittest way. A subsequent patch will actually do this.

run-tests: pass Test instances into TestRunner._executetests() Test instances (not paths) are passed into _executetests(). This means the logic for instantiating Test instances has been moved outside of _executetests(). This was done because an upcoming patch will reuse the logic in _executetests(). As part of this change, test counts are no longer managed by _executetests().

run-tests: define cleanup() on unittest class We need a cleanup() on our unittest TestCase class to conform with the execution API. This will eventually be replaced by tearDown().

run-tests: rename Test._test to Test.name A lot of the attributes in Test could probably be renamed. This one is needed for an upcoming patch.

run-tests: teach Test instances whether they are in unittest mode Tests executing in unittest mode behave a little differently. For example, they report their results to a TestResult rather than just printing. This patch paves the way for better integration of Test into the unittest framework.

run-tests: teach unittest about warned results

run-tests: document how success code is handled

run-tests: teach unittest about ignored tests

run-tests: teach unittest about skipped tests

run-tests: define custom result and runner classes for unittest We need to teach unittest about some custom result types. This will require some custom classes. This patch creates a skeleton for them.

run-tests: start to report test results against TestResult Previously, our unittest wrapper didn't report results properly. We now properly report failures. We had to rename the local variable to prevent "t" from being overwritten in the local scope.

run-tests: give unittest wrapper a short description This will make the output contain something useful rather than the default repr() for the class.

run-tests: initial support for running tests with unittest The unittest package in Python's standard library provides an almost universal mechanism for defining and running tests. This patch starts the process of making run-tests.py talk to it. The main benefit of speaking unittest is that this will enable Mercurial's tests to be more easily consumed by other testing tools, like nose. This is useful for 3rd party extensions having their own test infrastructure, for example. Running tests in unittest mode will not result in completely sane behavior until the unittest mode is made the default execution mode. Expect things like double printing of output until support stabilizes.

localrepo: reverse contexts in status This is a slight tweak to how localrepo.status calculates what files have changed. By forcing a changectx to be first operator and anything not a changectx to be the second operator, we can later exploit this to allow refactoring the status operation as a method of a context object. Furthermore, this change will speed up 'hg diff --reverse' when used with the working directory because the code will now hit a fast path without needing to calculate an unneeded second manifest.

test: split test-largefile.t in multiple file The `test-largefiles.t` unified test is significantly longer (about 30%) than any other tests in the mercurial test suite. As a result, its is alway the last test my test runner is waiting for at the end of a run. In practice, this means that `test-largefile.t` is wasting half a minute of my life every times I'm running the mercurial test suites. This probably mean more a few cumulated day by now. I've finally decided to split it up in multiple smaller tests to bring it back in reasonable length. This changeset extracts independent test cases in two files. One dedicated to wire protocole testing, and another one dedicated to all other tests that could be independently extracted. No test case were haltered in the making of this changeset. Various timing available below. All timing have been done on a with 90 jobs on a 64 cores machine. Similar result are shown on firefly (20 jobs on 12 core). General timing of the whole run -------------------------------- We see a 25% real time improvement for no significant cpu time impact. Before split: real 2m1.149s user 58m4.662s sys 11m28.563s After split: real 1m31.977s user 57m45.993s sys 11m33.634s Last test to finish (using run-test.py --time) ---------------------------------------------- test-largefile.t is now finishing at the same time than other slow tests. Before split: Time Test 119.280 test-largefiles.t 93.995 test-mq.t 89.897 test-subrepo.t 86.920 test-glog.t 85.508 test-rename-merge2.t 83.594 test-revset.t 79.824 test-keyword.t 78.077 test-mq-header-date.t After split: Time Test 90.414 test-mq.t 88.594 test-largefiles.t 85.363 test-subrepo.t 81.059 test-glog.t 78.927 test-rename-merge2.t 78.021 test-revset.t 77.777 test-command-template.t Timing of largefile test themself ----------------------------------- Running only tests prefixed with "test-largefiles". No significant change in cumulated time. Before: Time Test 58.673 test-largefiles.t 2.931 test-largefiles-cache.t 0.583 test-largefiles-small-disk.t After: Time Test 31.754 test-largefiles.t 17.460 test-largefiles-misc.t 8.888 test-largefiles-wireproto.t 2.864 test-largefiles-cache.t 0.580 test-largefiles-small-disk.t

mq: use the editor gotten by "getcommiteditor()" instead of "ui.edit()" (qrefresh/qfold) This patch also replaces "editor = False" by "editor = cmdutil.getcommiteditor()", because: - it allows to hook commit message determination easily, even in the case without "--edit" - it avoids regression (or additional care) around saving "last-message.txt", even if MQ's "newcommit()" changes its implementation logic from "localrepository.commit" to "localrepository.commitctx" with "memctx" in the future to save commit message into "last-messge.txt" with "memctx", "editor" should be valid function.

mq: fold the code paths to invoke editor into specific logic (qrefresh/qfold) This factoring simplifies the succeeding patch to switch from "ui.edit()" to "getcommiteditor()" for qrefresh/qfold.

mq: use the editor gotten by "getcommiteditor()" instead of "ui.edit()" (qnew) This patch also replaces "editor = False" by "editor = cmdutil.getcommiteditor()", because: - the latter allows to hook commit message determination easily, even in the case without "--edit" - the latter can avoid regression (or additional care) around saving "last-message.txt", even if MQ's "newcommit()" changes its implementation logic from "localrepository.commit" to "localrepository.commitctx" with "memctx" in the future to save commit message into "last-messge.txt" with "memctx", "editor" should be valid function.

mq: fold the code path to invoke editor into specific logic (qnew) This factoring simplifies the succeeding patch to switch from "ui.edit()" to "getcommiteditor()" for qnew.

cmdutil: enhance "getcommiteditor()" for specific usages in MQ This patch introduces "finishdesc" and "extramsg" arguments into "getcommiteditor()" for specific usages in MQ. "finishdesc" will be used to treat the commit message as "[mq]; patch-file-name" (default MQ commit message), if it is left as empty, instead of aborting commit process. "extramsg" will be used to show the line below: HG: Leave message empty to use default message instead of: HG: Leave message empty to abort commit

tag: use the editor gotten by "getcommiteditor()" instead of "ui.edit()" This patch also enhances "test-tag.t", because "hg tag" hasn't been explicitly tested around editor invocation and "--edit" option.

import: use "getcommiteditor()" instead of explicit editor choice This patch also enhances "test-import-bypass.t" and "test-import.t", because "hg import" hasn't been explicitly tested around editor invocation and "--edit" option. This patch explicitly tests below: - with "--bypass" (regardless of "--edit"): - not invoked, if the patch contains the commit message - not invoked, if the commit message is explicitly specified - invoked, otherwise (just adding comment) - without "--bypass": - with "--edit": - not invoked, if "--no-commit" is not specified - invoked, otherwise - without "--edit": - not invoked, if the patch contains the commit message - not invoked, if "--message" or "--logfile" is specified (just adding comment) - invoked, otherwise

graft: use "getcommiteditor()" instead of explicit editor choice This patch also enhances "test-graft.t", because "hg graft" hasn't been explicitly tested around editor invocation and "--edit" option.

amend: use "getcommiteditor()" instead of explicit editor choice This patch doesn't change any tests like as preceding patches, because editor invocation is already tested in "test-commit-amend.t".

commit: use "getcommiteditor()" instead of explicit editor choice This patch also eliminates "forceeditor" no more referred. This patch doesn't change any tests like as preceding patches, because editor invocation is already tested well.

backout: use "getcommiteditor()" instead of explicit editor choice This patch also enhances "test-backout.t", because "hg backout" hasn't been explicitly tested around editor invocation and '--edit' option.

backout: avoid redundant message examination Before this patch, "hg backout" examines "--message" and "--logfile" options explicitly. But this examination is redundant, because "commitfunc()" can receive the result of same examination by "cmdutil.logmessage()" in "cmdutil.commit()" through "message" argument. This patch avoids redundant message examination by "message" examination in "commitfunc()".

transplant: use "getcommiteditor()" instead of explicit editor choice This patch also enhances "test-transplant.t", because "hg transplant" hasn't been explicitly tested around editor invocation and "--edit" option.

rebase: use "getcommiteditor()" instead of explicit editor choice This patch also enhances "test-rebase-scenario-global.t", because "hg rebase" hasn't been explicitly tested around editor invocation and "--edit" option. In the other hand, this patch doesn't enhance tests in "hg rebase --collapse" case, because it is already tested in "test-rebase-collapse.t".

histedit: use "getcommiteditor()" instead of explicit editor choice for "--continue" This omits (redundant) adding "\n' to "message", because: - empty line is inserted by "commitforceeditor", if editor is invoked - tail white-spaces are stripped at storing into chaneglog, otherwise This patch also enhances "test-histedit-edit.t", because "hg histedit" hasn't been explicitly tested around editor invocation and "--continue" option.

histedit: use "getcommiteditor()" instead of explicit editor choice for "fold" This patch doesn't change any tests like as preceding patches, because editor invocation is already tested in "test-histedit-fold.t".

histedit: use the editor gotten by "getcommiteditor()" for "message" Before this patch, "hg histedit" for "message" uses "ui.edit()" for commit message editing. It shows original commit message, but not detail about the target revision: status of each modified/added/removed files, for example. This patch uses the editor gotten by "getcommiteditor()" instead of "ui.edit()" for "message" In "test-histedit-edit.t", this patch omits "fixbundle" invocation, because it prevents from confirming the "HG: added f" line in commit message by filtering " added " lines. Omiting "fixbundle" invocation causes that the exit code of "hg histedit" appears as one of command line: in this case, "hg histedit" is aborted by (expected) exception raising.

fetch: use "getcommiteditor()" instead of explicit editor choice This patch also enhances "test-fetch.t", because "hg fetch" hasn't been explicitly tested around editor invocation and '--edit' option.

cmdutil: introduce "getcommiteditor()" to simplify code paths to choose editor "getcommiteditor()" can simplify code paths to choose commit editor according to '--edit' option as below: before: editor = cmdutil.commiteditor # or editor = None/False if opts.get('edit'): editor = cmdutil.commitforceeditor after: editor = cmdutil.getcommiteditor(**opts) "getcommiteditor()" accepts option arguments not in "opts" style but in "**opts" style, because some code paths want to invoke it with just explicit "edit=True" argument (building dictionary is redundant).

update: when deactivating a bookmark, print a message This helps prevent user confusion when innocent-seeming commands like 'hg update -C .' are run.

doc: improves merge-tools example with an overwrite of existing config We highlight the fact that a user hgrc can overwrite the system wide preconfiguration. As other benefit we show priority value other than 1 and shows vimdiff configuration is usually directly available. This is valuable as vimdiff is (surprisingly) a common request from user.

help: add additional information in the merge-tools section of config help The merge tool configuration in general has always been a confusing topic for user. We add mention of: - possible pre-existing configuration, - way to look at it, - the more detailed help topic about how merge-tools configuration works. This should help users find they way.

subrepo: use subrepo shortid method to generate subrepo diverged promptchoice We were always using only the first 12 characters of the subrepo revision id when generating the "subrepo diverged" promptchoice. This is not necessarily correct for non mercurial subrepos.

subrepo: add shortid() method to subrepo classes This method takes an "id" (e.g. a revision id) and returns a "short" version (e.g. a short revision id). This will be used on the next revision to fix a small bug in the way that the text on the promptchoice shown when a subrepo diverges is generated.

localrepo: call _dirstatestatus instead of duplicating logic

workingctx: call _dirstatestatus in status Rip out the call from workingctx.status to localrepo.status.

context: add private _dirstatestatus method This patch is a step forward in getting rid of needing to check 'parentworking' throughout the status method. Eventually, we will use the power of inheritance to do the correct thing when comparing the working directory with its parent. This method is mostly a copy from localrepo.status. The custom status method of workingctx will eventually be absorbed by the refactoring of localrepo.status to context.status but unfortunately we can't do it in one step.

committablectx: move status to workingctx This method was accidentally placed into the committablectx class. It contains logic for querying the dirstate so we move it to the correct class.

localrepo: factor out parentworking logic for comparing files We will temporarily call a private method of the context class while we are in the process of removing the need of having localrepo.status.

localrepo: improve dirstate.normal lookup by cacheing the function

localrepo: move symlink logic to workingctx

merge: separate worker functions for batch remove and batch get The old code had one function that could do 2 different things. First, is was called a bunch of times to do one thing. Next, it was called a bunch of times to do the other thing. That gave unnecessary complexity and a dispatch overhead. Having separate functions is "obviously" better than having a function that can do two things, depending on its parameters. It also prepares the code for the next refactorings.

merge: change debug logging - test output changes but no real changes Preparing for action list split-up, making sure the final change don't have any test changes. The patch moves debug statements around without really changing anything. Arguably, it temporarily makes the code worse. The only justification is that it makes it easier to review the test changes ... and in the end the big change will not change test output at all. The changes to test output are due to changes in the ordering of debug output. That is mainly because we now do the debug logging for files when we actually process them. Files are also processed in a slightly different but still correct order. It is now primarily ordered by action type, secondarily by filename. The patch introduces some redundancy. Some of it will be removed again, some of it will in the end help code readability and efficiency. It is possible that we later on could introduce a "process this action list and do some logging and progress reporting and apply this function". The "preserving X for resolve" debug statements will only have single space indentation. It will no longer have a leading single space indented "f: msg -> m" message. Having this message double indented would thus no longer make sense. The bid actions will temporarily be sorted using a custom sort key that happens to match the sort order the simplified code will have in the end.

merge: move constant assignments a bit and use them more

merge: change priority / ordering of merge actions The ordering of actions matters. Normal file system semantics is that files have to be removed before a directory with the same name can be created. Before the first ordering key was to have 'r' and 'f' actions come first, secondary key was the filename. Because of future refactorings we want to consistently have all action types (with a sensible priority) as separate first keys. Grouped by action type, we sort by filename. Not processing in strict filename order could give worse performance, especially on spinning disks. That is however primarily an issue in the cases where "all" actions are of the same kind and will be grouped together anyway.

run-tests: fix --interactive option ``TTest._testpath`` never existed. This must be the remain of some contributors side refactoring.

run-test: fix AttributeError in the --interactive prompt One must choose between ``"y yes".split()`` and ``('y', 'yes')``. I choose the later. The feature still crash when you answer "yes" to use it. But at least, the prompt itself works.

run-test: restore the -i prompt by write .err before checking if it exists The `--interactive` flag workis by overwriting the original test file by its `.err` version. So we need to write it before calling `self.fail`. Otherwise the `.err` file does not exists and `--interactive` is ignored. We can move that block code around because it is dedicated to write changed output and we moves it in the try-except dedicated to handling changed output. Note that the flog is still badly broken after this change. But at least it crash instead of being ignored.

run-tests: move _gethgpath() into TestRunner

run-tests: move string escaping to TTest With this patch, TTest is almost fully self-contained and extractable. Only logging functions remain outside of its class.

run-tests: add some docstrings

run-tests: move createhgrc into Test

run-tests: move SKIPPED_PREFIX and FAILED_PREFIX into TTest

run-tests: move SKIPPED_STATUS into Test class