hg: obtain lock when creating share from pooled repo (issue5104) There are race conditions between clients performing a shared clone to pooled storage: 1) Clients race to create the new shared repo in the pool directory 2) 1 client is seeding the repo in the pool directory and another goes to share it before it is fully cloned We prevent these race conditions by obtaining a lock in the pool directory that is derived from the name of the repo we will be accessing. To test this, a simple generic "lockdelay" extension has been added. The extension inserts an optional, configurable delay before or after lock acquisition. In the test, we delay 2 seconds after lock acquisition in the first process and 1 second before lock acquisition in the 2nd process. This means the first process has 1s to obtain the lock. There is a race condition here. If we encounter it in the wild, we could change the dummy extension to wait on the lock file to appear instead of relying on timing. But that's more complicated. Let's see what happens first.

doc: remove deprecated option from synopsis of command help Before this patch, deprecated options below are used in synopsis of command help, even though they aren't listed up as available options by default. These might confuse readers. - -n (no-op, now) of strip - -a/--active of branches - -f/--force of merge

i18n-pt_BR: synchronized with 535f2900d078

merge with stable

merge with i18n

run-tests: fix Python 3 incompatibilities At one point run-tests.py and test-run-tests.t worked and passed under Python 3.5. Various changes to run-tests.py over the past several months appear to have broken Python 3.5 compatibility. This patch implements various fixes (all related to str/bytes type coercion) to make run-tests.py and test-run-tests.t mostly work again. There are still a few failures in test-run-tests.t due to issues importing mercurial.* modules. But at least run-tests.py seems to work under 3.5 again.

hghave: use print function For Python 3 compatibility.

changelog: remove redundant parentheses You don't need to surround returned tuples with parens.

changegroup: use changelog.readfiles We have a dedicated function to get just the list of files in a changelog entry. Use it. This will presumably speed up changegroup application since we're no longer decoding the entire changelog entry. But I didn't measure the impact.

rebase: remove experimental option from 'rebase' config section Changeset f0e9f38d250f introduced a guard against case where obsolete changesets are included in the rebase in a way this will result in divergence (because rebase create new successors for changeset which already have successors). In the same go a 'rebase.allowdivergence' option was introduced to control that behavior. We rename this config option to 'experimental.allowdivergence' for multiple reasons: * First this behavior is attached to changeset evolution, a feature still experimental. * Second, there was no 'rebase' section in config before we introduced this option. I would like to avoid proliferation of micro config section and therefore would like to avoid the creation of this new section just for an experimental feature. * Third, this guard (warning the user about a history rewriting operation that will create divergence) will very likely be generalised to all history rewriting operations, making this not rebase specific. * Finally, because this will likely be a general guard present a bit everywhere in the UI we'll likely end up with something better than a config option to control this behavior, so having the current config option living in experimental will allow us make it disappear in the future. So we banish this config option back to the experimental section where it belongs, killing the newly born 'rebase' config section in the process.

testing: allow Hypothesis tests to disable extensions Doing this required the introduction of a mechanism for keeping track of more general config in the test. At present this is only used for extensions but it could be used more widely (e.g. to control specific extension behaviour) This greatly simplifies the extension management logic by introducing a general notion of config, which we maintain ourselves and pass to HG on every invocation. This results in significantly less error prone test generation, and also allows us to turn extensions off as well as on. The logic that used an environment variable to rerun the tests with an extension disabled now just edits the test file (in a fresh copy) to remove these --config command line flags.

keepalive: remove useless parentheses around exception type

chgserver: add a structure for confighash and mtimehash confighash and mtimehash are often used together. This patch adds a simple structure called hashstate to store them. hashstate also has a handly method called fromui to calculate the hashes from a ui object.

chgserver: add utilities to calculate mtimehash mtimehash is designed to detect file changes. These files include: - single file extensions (__init__.py for complex extensions) - mercurial/__version__.py - python (sys.executable) mtimehash only uses stat to check files so it's fast but not 100% accurate. However it should be good enough for our use case. For chgserver, once mtimehash changes, the server is considered outdated immediately and should no longer provide service.

tests: rename 'test-module-import.t' into 'test-check-module-import.t' This test is checking our source code to ensure style and correct behavior (eg: no cycle). Current convention is that such tests starts with 'test-check-' so we flock this on back with the others.

pull: deactivate a bookmark not matching with the destination of the update Before this patch, "hg pull -u" with a target doesn't deactivate a current active bookmark, which doesn't match with the explicit destination of the update, even though bare "hg update" does so. A "target" can be provided through: - option --rev ANOTHER - option --branch ANOTHER - source URL#ANOTHER

pull: activate a bookmark matching with the destination of the update (BC) Before this patch, "hg pull -u" with a target doesn't activate a bookmark, which matches with the explicit destination of the update, even though bare "hg update" does so. A "target" can be provided through: - option --rev BOOKMARK - source URL#BOOKMARK

revset: define "pat" variable unconditionally in subrepo() It's a source of UnboundLocalError to define and use local variables conditionally. As getstring() always returns a str, "pat" can be initialized to None.

revset: drop translation markers from error messages of internal _matchfiles They are a sort of debug messages, which should never be visible to end users.

templatekw: switch ctx of list expression to rev of {parents} (BC) This is the same semantics as revset() introduced at e4609ec959f8. Before this patch, {parents} provided nothing useful in new-style template. For example, '{parents % "{parent}"}' generated cryptic string like "rev12345node0123abcdef...". This patch drops {parent} variable since it was useless. We can get a revision number by '{parents % "{rev}"}'.

commands: add postincoming explicit brev argument (API) Before this patch, postincoming() initializes 'brev' with 'checkout', but this isn't useful to activate/deactivate bookmark after updating, because 'checkout' is not a string actually specified at command line, but an already node-nized byte sequence. This patch adds postincoming() explicit 'brev' argument, and makes 'pull()' pass appropriate value. This patch adds 'brev' argument instead of 'brev=None', because 'brev=None' isn't reasonable value if checkout is not None.

hgweb: add index template to json/map This template allows showing the list of all repos in an hgweb instance (in hgwebdir mode) as json. The test has "lastchange" globbed because hgweb uses here file modification time and not the last commit time.

merge: fix error message Obvious copy-and-paste error

destutil: show message about other branch heads, even if on a closed head Before this patch, bare "hg update" displays message below, if there is at least one non-closed branch head other than current parent: 1. 'XX other heads for branch "BRANCH"' message, if current parent is on a non-closed branch head This suggests user to invoke "hg heads" or so for merging them. 2. no message, if current parent is on a closed branch head At this patch, bare "hg update" might choose closed branch head as update destination, and it causes this situation easily. 3. no message, otherwise (= current parent isn't on any branch head) 'XX other heads for branch "BRANCH"' should be displayed also in #2 case above, because user might overlook other non-closed branch heads. This patch gets a list of all branch heads regardless of closed-ness of it, and uses it (= 'allheads') to distinguish #1/#2 from #3 above.

repoview: discard filtered changelog if index isn't shared with unfiltered Before this patch, revisions rollbacked at failure of previous transaction might be visible at subsequent operations unintentionally, if repoview object is reused even after failure of transaction: e.g. command server and HTTP server are typical cases. 'repoview' uses the tuple of values below of unfiltered changelog as "the key" to examine validity of filtered changelog cache. - length - tip node - filtered revisions (as hashed value) - '_delayed' field 'repoview' compares between "the key" of unfiltered changelog at previous caching and now, and reuses filtered changelog cache if no change is detected. But this comparison indicates only that there is no change between unfiltered 'repo.changelog' at last caching and now, but not that filtered changelog cache is valid for current unfiltered one. 'repoview' uses "shallow copy" of unfiltered changelog to create filtered changelog cache. In this case, 'index' buffer of unfiltered changelog is also referred by filtered changelog. At failure of transaction, unfiltered changelog itself is invalidated (= un-referred) on the 'repo' side (see 0a7610758c42 also). But 'index' of it still contains revisions to be rollbacked at this failure, and is referred by filtered changelog. Therefore, even if there is no change between unfiltered 'repo.changelog' at last caching and now, steps below makes rollbacked revisions visible via filtered changelog unintentionally. 1. instantiate unfiltered changelog as 'repo.changelog' (call it CL1) 2. make filtered (= shallow copy of) CL1 (call it FCL1) 3. cache FCL1 with "the key" of CL1 4. revisions are appended to 'index', which is shared by CL1 and FCL1 5. invalidate 'repo.changelog' (= CL1) at failure of transaction 6. instantiate 'repo.changelog' again at next operation (call it CL2) CL2 doesn't have revisions added at (4), because it is instantiated from '00changelog.i', which isn't changed while failed transaction. 7. compare between "the key" of CL1 and CL2 8. FCL1 cached at (3) is reused, because comparison at (7) doesn't detect change between CL1 at (1) and CL2 9. revisions rollbacked at (5) are visible via FCL1 unintentionally, because FCL1 still refers 'index' changed at (4) The root cause of this issue is that there is no examination about validity of filtered changelog cache against current unfiltered one. This patch discards filtered changelog cache, if its 'index' object isn't shared with unfiltered one. BTW, at the time of this patch, redundant truncation of '00changelog.i' at failure of transaction (see 0a7610758c42 for detail) often prevents "hg serve" from making already rollbacked revisions visible, because updating timestamps of '00changelog.i' by truncation makes "hg serve" discard old repoview object with invalid filtered changelog cache. This is reason why this issue is overlooked before this patch, even though test-bundle2-exchange.t has tests in similar situation: failure of "hg push" via HTTP by pretxnclose hook on server side doesn't prevent subsequent commands from looking up outgoing revisions correctly. But timestamp on the filesystem doesn't have enough resolution for recent computation power, and it can't be assumed that this avoidance always works as expected. Therefore, without this patch, this issue might appear occasionally.

chgserver: make _renewui load repo and command line configs Before this patch, there is no way to load repo config in chgserver. This patch revised _renewui to let it load repo config and take command line flags passed into consideration. The _renewui logic is not ideal at present because it's very tricky to know what should copy and what should not from the old ui object to the new one. This is partially because the current ui and config object design is not ideal. In the future, we may want to avoid all ui or config copies.

dispatch: add wd parameter to _getlocal Before this patch, _getlocal uses os.getcwd() to locate repo in current dir. chgserver needs it to load repo config and has to do chdir twice: the first is to set current directory and the second is to redo the side effect (in case hg --cwd some/relative/path, chdir will be called again in dispatch later), which is not pretty. This patch adds an optional wd parameter to make it possible to specify wd without chdir (and its side effect).

chgserver: add utilities to calculate confighash confighash is the hash of sensitive config items like [extensions], and sensitive environment variables like HG*, LD_*, etc. The config items can come from global, user, repo config, and command line flags. For chgserver, it is designed that once confighash changes, the server is not qualified to serve its client and should redirect the client to a new server. The server does not need to exit in this case, since it can still be valid (have a matched confighash) to serve other chg clients.

chg: detect chg started by chg Sometimes people may create a symbol link from hg to chg, or write a wrapper script named hg calling chg. Without $HG and $CHGHG set, this will lead to chg executes itself causing deadlock. The user will notice chg hangs for some time and aborts with a timed out message, without knowing the root cause and how to solve it. This patch sets a dummy environment variable before executing hg to detect this situation, and print a fatal message with some possible solutions. CHGINTERNALMARK is set by chg client to detect the situation that chg is started by chg. It is temporary and should be dropped to avoid possible side effects.

chg: fallback to original hg for some unsupported commands or flags There are some known unsupported commands or flags for chg, such as hg serve -d and hg foo --time. This patch detects these situations and transparently fall back to the original hg. So the users won't bother remembering what chg can and cannot do by themselves. The current detection is not 100% accurate since we do not have an equivalent command line parser in C. But it tries not to cause false positives that prevents people from using chg for legit cases. In the future we may want to implement a more accurate "unsupported" check server-side.