revlog: use raw revision for rawsize When writing the revlog-ng index, the third field is len(rawtext). See revlog._addrevision: textlen = len(rawtext) .... e = (offset_type(offset, flags), l, textlen, base, link, p1r, p2r, node) self.index.insert(-1, e) Therefore, revlog.index[rev][2] returned by revlog.rawsize should be len(rawtext), where "rawtext" is revlog.revision(raw=True). Unfortunately it's hard to add a test for this code path because "if l >= 0" catches most cases.

revsetlang: enable optimization of 'x + y' expression It's been disabled since 4d1e56b29a91, but it can be enabled now as the ordering requirement is resolved at analyze().

repair: use rawvfs when copying extra store files If we use the normal vfs, store encoding will be applied when we .join() the path to be copied. This results in attempting to copy a file that (likely) doesn't exist. Using the rawvfs operates on the raw file path, which is returned by vfs.readdir(). Users at Mozilla are encountering this, as I've instructed them to run `hg debugupgraderepo` to upgrade to generaldelta. While Mercurial shouldn't deposit any files under .hg/store that require encoding, it is possible for e.g. .DS_Store files to be created by the operating system.

tests: add test demonstrating buggy path handling `hg debugupgraderepo` is currently buggy with regards to path handling when copying files in .hg/store/. Specifically, it applies the store filename encoding to paths instead of operating on raw files. This commit adds a test demonstrating the buggy behavior.

repair: iterate store files deterministically An upcoming test will add a 2nd file. Since readdir() is non-deterministic, add a sorted() to make traversal deterministic.

zstd: vendor python-zstandard 0.8.0 Commit 81e1f5bbf1fc54808649562d3ed829730765c540 from https://github.com/indygreg/python-zstandard is imported without modifications (other than removing unwanted files). Updates relevant to Mercurial include: * Support for multi-threaded compression (we can use this for bundle and wire protocol compression). * APIs for batch compression and decompression operations using multiple threads and optimal memory allocation mechanism. (Can be useful for revlog perf improvements.) * A ``BufferWithSegments`` type that models a single memory buffer containing N discrete items of known lengths. This type can be used for very efficient 0-copy data operations. # no-check-commit

commands: update help for "unbundle" Similar to the recent change to "bundle," this command no longer just deals with "changegroup" data.

commands: update help for "bundle" We now have a dedicated help topic to describe bundle specification strings. Let's update `hg bundle`'s documentation to reflect its existence. While I was hear, I also tweaked some wording which I felt was out of date and needed tweaking. Specifically, `hg bundle` no longer just deals with "changegroup" data: it can also generate files that have non-changegroup data.

help: document bundle specifications I softly formalized the concept of a "bundle specification" a while ago when I was working on clone bundles and stream clone bundles and wanted a more robust way to define what exactly is in a bundle file. The concept has existed for a while. Since it is part of the clone bundles feature and exposed to the user via the "-t" argument to `hg bundle`, it is something we need to support for the long haul. After the 4.1 release, I heard a few people comment that they didn't realize you could generate zstd bundles with `hg bundle`. I'm partially to blame for not documenting it in bundle's docstring. Additionally, I added a hacky, experimental feature for controlling the compression level of bundles in 76104a4899ad. As the commit message says, I went with a quick and dirty solution out of time constraints. Furthermore, I wanted to eventually store this configuration in the "bundlespec" so it could be made more flexible. Given: a) bundlespecs are here to stay b) we don't have great documentation over what they are, despite being a user-facing feature c) the list of available compression engines and their behavior isn't exposed d) we need an extensible place to modify behavior of compression engines I want to move forward with formalizing bundlespecs as a user-facing feature. This commit does that by introducing a "bundlespec" help page. Leaning on the just-added compression engine documentation and API, the topic also conveniently lists available compression engines and details about them. This makes features like zstd bundle compression more discoverable. e.g. you can now `hg help -k zstd` and it lists the "bundlespec" topic.

util: document bundle compression An upcoming patch will add support for documenting bundle specifications in more detail. As part of this, we'd like to enumerate available bundle compression formats. In order to do this, we need to provide the help mechanism a dict of names and objects with docstrings. This patch adds docstrings to compengine.bundletype and adds a function for retrieving a dict of them. The code is not yet used.

tests: store ETag when using --headeronly Previously, --headeronly would prevent --twice from working because the ETag wasn't stored when --headeronly was used. This feels like a bug. That feeling is reaffirmed by the fact that this change doesn't regress any tests.

hgweb: extract path traversal checking into standalone function A common exploit in web applications that access paths is to insert path separator strings like ".." to try to get the server to serve up files it shouldn't. We have code for detecting this in staticfile(). A subsequent commit will need to perform this test as well. Since this is security code, let's factor the check so we don't have to reinvent the wheel.

hgweb: use context manager for file I/O

tags: rename "head" to "node" where we don't care Followup to 5eb4d206202b (tags: extract fnode retrieval into its own function, 2017-03-28) in which the "for head in head" became "for head in nodes".

manifest: update comment to be about bytearray Looks like a leftover from 2a18e9e6ca43 (py3: use bytearray() instead of array('c', ...) constructions, 2017-03-12).

check-code: fix "covert" typo

hgweb: rename linerangelog.js as followlines.js So that the file name matches both the feature name and user facing vocabulary (e.g. the revset function).

hgweb: rely on a specific class to change cursor type in followlines UI The previous CSS rule would also apply in pages where followlines UI was not available (e.g. "changeset" view at /rev/<node>/). We insert a "followlines-select" class in JavaScript on actually selectable lines and restrict the CSS selector to use it.

hgweb: use a function expression for the install listener of followlines UI We define the listener of document's "DOMContentLoaded" inline in registration and use a function expression (anonymous) with everything inside. This makes it clearer that this file is not a library of JavaScript functions but rather an executable script. (Most of changes consists of reindenting the "followlinesBox" function, so mostly white space changes.)

formatter: use templatefilters.json() Now _jsonifyobj() is identical to templatefilters.json(paranoid=False).

templatefilters: use list comprehension in json() Not important, but the code slightly looks better.

templatefilters: unroll handling of None/False/True It doesn't make sense to use a dict here.

templatefilters: drop callable support from json() This backs out ae5447de4c11. A callable should be evaluated beforehand by templater.runsymbol().

ui: use bytes IO and convert EOL manually in ui.editor() Text IO sucks on Python 3 as it must be a unicode stream. We could introduce a wrapper that converts unicode back to bytes, but it wouldn't be simple to handle offsets transparently from/to underlying IOBase API. Fortunately, we don't need to process huge text files, so let's stick to bytes IO and convert EOL in memory.

util: add helper to convert between LF and native EOL See the next patch for why.

util: extract pure tolf/tocrlf() functions from eol extension This can be used for EOL conversion of text files.

pycompat: provide bytes os.linesep

pycompat: introduce identity function as a compat stub I was sometimes too lazy to use 'str' instead of 'lambda a: a'. Let's add a named function for that purpose.

test-blackbox: glob away quoting differences on Windows Windows uses double quotes in these places.

test-subrepo: update output for Windows

test-serve: disable unfixable tests on Windows These tests would run if hghave.has_serve() were enabled on Windows. Windows has no issue allowing an unpriviledged process to open port 13, so it doesn't abort. The other tests are related to how MSYS tries to be helpful and converts Unix constructs to the Windows equivalent. There isn't any way to disable this behavior, though it supposedly doesn't happen if the exe is linked against the MSYS library.

test-serve: kill daemons before deleting the access and error logs On Windows, `rm` will fail with 'Permission denied' if another process has it open. It looks like the rollback test was missing the kill entirely.

test-serve: make the 'listening at *' lines optional The daemonized serve process doesn't print these lines out (see 448d0c452140). I was able to get it to with the following hack: diff --git a/mercurial/win32.py b/mercurial/win32.py --- a/mercurial/win32.py +++ b/mercurial/win32.py @@ -418,6 +418,11 @@ return str(ppid) def spawndetached(args): + + import subprocess + return subprocess.Popen(args, cwd=pycompat.getcwd(), env=encoding.environ, + creationflags=subprocess.CREATE_NEW_PROCESS_GROUP).pid + # No standard library function really spawns a fully detached # process under win32 because they allocate pipes or other objects # to handle standard streams communications. Passing these objects However, MSYS translates --prefixes starting with '/' to 'C:/MinGW/msys/1.0', which changes the output. The output isn't so important that I want to spend a bunch of time on this, and risk breaking some subtle behavior of `hg serve -d` with the more complicated code.

test-http: update output for Windows The http test simply wasn't updated in 161ab32b44a1 for Windows. It looks like the https test meant to glob away the error message in 3e2d8120528b, but forgot the '*', and was subsequently removed in 408f2202bd80.

tests: quote paths in shell script hooks Without the quoting, MSYS will remove the '\' directory separators, and the repo can't be opened.

tests: add globs for Windows

show: new extension for displaying various repository data Currently, Mercurial has a number of commands to show information. And, there are features coming down the pipe that will introduce more commands for showing information. Currently, when introducing a new class of data or a view that we wish to expose to the user, the strategy is to introduce a new command or overload an existing command, sometimes both. For example, there is a desire to formalize the wip/smartlog/underway/mine functionality that many have devised. There is also a desire to introduce a "topics" concept. Others would like views of "the current stack." In the current model, we'd need a new command for wip/smartlog/etc (that behaves a lot like a pre-defined alias of `hg log`). For topics, we'd likely overload `hg topic[s]` to both display and manipulate topics. Adding new commands for every pre-defined query doesn't scale well and pollutes `hg help`. Overloading commands to perform read-only and write operations is arguably an UX anti-pattern: while having all functionality for a given concept in one command is nice, having a single command doing multiple discrete operations is not. Furthermore, a user may be surprised that a command they thought was read-only actually changes something. We discussed this at the Mercurial 4.0 Sprint in Paris and decided that having a single command where we could hang pre-defined views of various data would be a good idea. Having such a command would: * Help prevent an explosion of new query-related commands * Create a clear separation between read and write operations (mitigates footguns) * Avoids overloading the meaning of commands that manipulate data (bookmark, tag, branch, etc) (while we can't take away the existing behavior for BC reasons, we now won't introduce this behavior on new commands) * Allows users to discover informational views more easily by aggregating them in a single location * Lowers the barrier to creating the new views (since the barrier to creating a top-level command is relatively high) So, this commit introduces the `hg show` command via the "show" extension. This command accepts a positional argument of the "view" to show. New views can be registered with a decorator. To prove it works, we implement the "bookmarks" view, which shows a table of bookmarks and their associated nodes. We introduce a new style to hold everything used by `hg show`. For our initial bookmarks view, the output varies from `hg bookmarks`: * Padding is performed in the template itself as opposed to Python * Revision integers are not shown * shortest() is used to display a 5 character node by default (as opposed to static 12 characters) I chose to implement the "bookmarks" view first because it is simple and shouldn't invite too much bikeshedding that detracts from the evaluation of `hg show` itself. But there is an important point to consider: we now have 2 ways to show a list of bookmarks. I'm not a fan of introducing multiple ways to do very similar things. So it might be worth discussing how we wish to tackle this issue for bookmarks, tags, branches, MQ series, etc. I also made the choice of explicitly declaring the default show template not part of the standard BC guarantees. History has shown that we make mistakes and poor choices with output formatting but can't fix these mistakes later because random tools are parsing output and we don't want to break these tools. Optimizing for human consumption is one of my goals for `hg show`. So, by not covering the formatting as part of BC, the barrier to future change is much lower and humans benefit. There are some improvements that can be made to formatting. For example, we don't yet use label() in the templates. We obviously want this for color. But I'm not sure if we should reuse the existing log.* labels or invent new ones. I figure we can punt that to a follow-up. At the aforementioned Sprint, we discussed and discarded various alternatives to `hg show`. We considered making `hg log <view>` perform this behavior. The main reason we can't do this is because a positional argument to `hg log` can be a file path and if there is a conflict between a path name and a view name, behavior is ambiguous. We could have introduced `hg log --view` or similar, but we felt that required too much typing (we don't want to require a command flag to show a view) and wasn't very discoverable. Furthermore, `hg log` is optimized for showing changelog data and there are things that `hg display` could display that aren't changelog centric. There were concerns about using "show" as the command name. Some users already have a "show" alias that is similar to `hg export`. There were also concerns that Git users adapted to `git show` would be confused by `hg show`'s different behavior. The main difference here is `git show` prints an `hg export` like view of the current commit by default and `hg show` requires an argument. `git show` can also display any Git object. `git show` does not support displaying more complex views: just single objects. If we implemented `hg show <hash>` or `hg show <identifier>`, `hg show` would be a superset of `git show`. Although, I'm hesitant to do that at this time because I view `hg show` as a higher-level querying command and there are namespace collisions between valid identifiers and registered views. There is also a prefix collision with `hg showconfig`, which is an alias of `hg config`. We also considered `hg view`, but that is already used by the "hgk" extension. `hg display` was also proposed at one point. It has a prefix collision with `hg diff`. General consensus was "show" or "view" are the best verbs. And since "view" was taken, "show" was chosen. There are a number of inline TODOs in this patch. Some of these represent decisions yet to be made. Others represent features requiring non-trivial complexity. Rather than bloat the patch or invite additional bikeshedding, I figured I'd document future enhancements via TODO so we can get a minimal implmentation landed. Something is better than nothing.

test-revlog-raw: remove duplicated option

test-revlog-raw: fix "genbits" implementation The "genbits" implementation is actually incorrect. This patch fixes it. A good "genbits" implementation should pass the below assertion: n = 3 # or other number l = list(genbits(n)) assert 2**(n*2) == len(set((l[i]<<n)+l[i+1] for i in range(len(l)-1))) An assertion is added to make sure "genbits" won't work unexpectedly.

verify: fix length check According to the document added above, we should check L1 == L2, and the only way to get L1 in all cases is to call "rawsize()", and the only way to get L2 is to call "revision(raw=True)". Therefore the fix. Meanwhile there are still a lot of things about flagprocessor broken in revlog.py. Tests will be added after revlog.py gets fixed.

verify: document corner cases It seems a good idea to list all kinds of "surprises" and expected behavior to make the upcoming changes easier to understand. Note: the comment added does not reflect the actual behavior of the current code.

Added signature for changeset ed5b25874d99

Added tag 4.1.2 for changeset ed5b25874d99

hgweb: expose a followlines UI in filerevision view In filerevision view (/file/<rev>/<fname>) we add some event listeners on mouse clicks of <span> elements in the <pre class="sourcelines"> block. Those listeners will capture a range of lines selected between two mouse clicks and a box inviting to follow the history of selected lines will then show up. Selected lines (i.e. the block of lines) get a CSS class which make them highlighted. Selection can be cancelled (and restarted) by either clicking on the cancel ("x") button in the invite box or clicking on any other source line. Also clicking twice on the same line will abort the selection and reset event listeners to restart the process. As a first step, this action is only advertised by the "cursor: cell" CSS rule on source lines elements as any other mechanisms would make the code significantly more complicated. This might be improved later. All JavaScript code lives in a new "linerangelog.js" file, sourced in filerevision template (only in "paper" style for now).

shelve: move ui.quiet manipulations to configoverride

revlog: add a fast path for revision(raw=False) If cache hit and flags are empty, no flag processor runs and "text" equals to "rawtext". So we check flags, and return rawtext. This resolves performance issue introduced by a previous patch.

revlog: make _addrevision only accept rawtext All 3 users of _addrevision use raw: - addrevision: passing rawtext to _addrevision - addgroup: passing rawtext and raw=True to _addrevision - clone: passing rawtext to _addrevision There is no real user using _addrevision(raw=False). On the other hand, _addrevision is low-level code dealing with raw revlog deltas and rawtexts. It should not transform rawtext to non-raw text. This patch removes the "raw" parameter from "_addrevision", and does some rename and doc change to make it clear that "_addrevision" expects rawtext. Archeology shows 2df983125d37 added "raw" flag to "_addrevision", follow-ups e12c0fa1f65b and c1b7b2285522 seem to make the flag unnecessary. test-revlog-raw.py no longer complains.

revlog: use raw revisions in clone test-revlog-raw.py now shows "clone test passed", but there is more to fix.

revlog: use raw revisions in revdiff See the added comment. revdiff is meant to output the raw delta that will be written to revlog. It should use raw. test-revlog-raw.py now shows "addgroupcopy test passed", but there is more to fix.

revlog: use raw content when building delta Using external content provided by flagprocessor when building revlog delta is wrong, because deltas are applied to raw contents in revlog. This patch fixes the above issue by adding "raw=True". test-revlog-raw.py now shows "local test passed", but there is more to fix.

revlog: fix _cache usage in revision() As documented at revlog.__init__, revlog._cache stores raw text. The current read and write usage of "_cache" in revlog.revision lacks of raw=True check. This patch fixes that by adding check about raw, and storing rawtext explicitly in _cache. Note: it may slow down cache hit code path when raw=False and flags=0. That performance issue will be fixed in a later patch. test-revlog-raw now points us to a new problem.

revlog: rename some "text"s to "rawtext" This makes code easier to understand. "_addrevision" is left untouched - it will be changed in a later patch.

revlog: clarify flagprocessor documentation The words "text", "newtext", "bool" could be confusing. Use explicit "text" or "rawtext" and document more about the "bool".

revlog: add a stronger test for raw processing There are some issues about revlog raw processing (flag processor). The test is relatively strong covering many cases. It will verify fixes.

hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.

hook: provide hook type information to external hook The python hooks have access to the hook type information. There is not reason for external hook to not be aware of it too. For the record my use case is to make sure a hook script is configured for the right type.

hook: use 'htype' in 'hook' Same rational as for 'runhooks', we fix the naming in another function.

hook: use 'htype' in 'runhooks' Same rational as for '_pythonhook', 'htype' is more accurate and less error prone. We just fixed an error from the 'name'/'hname' confusion and this should prevent them in the future.

hook: fix name used in untrusted message The name used in the message we issue when a hook is untrusted was using "name" which is actually the hook type and not the name of the hook.

hook: use "htype" as variable name in _pythonhook We rename 'name' to 'htype' because it fits the variable content better. Multiple python hooks already use 'htype' as a name for the argument. This makes the difference with "hname" clearer and the code less error prone.

run-tests: auto-replace 'TXNID' output Hooks related to the transaction are aware of the transaction id. By definition this txn-id is unique and different for each transaction. As a result it can never be predicted in test and always needs matching. As a result, touching any like with this data is annoying. We solve the problem once and for all by installing an automatic replacement. In test, this will now show as: TXNID=TXN:$ID$

largefiles: use readasstandin() to read hex hash directly from filectx BTW, C implementation of hexdigest() for SHA-1/256/512 returns hex hash in lower case, and doctest in Python standard hashlib assumes that, too. But it isn't explicitly described in API document or so. Therefore, we can't assume that hexdigest() always returns hex hash in lower case, for any hash algorithms, on any Python runtimes and versions. From point of view of that, it is reasonable for portability that 40800668e019 applies lower() on hex hash in overridefilemerge(). But on the other hand, in largefiles extension, there are still many code paths comparing between hex hashes or storing hex hash into standin file, without lower(). Switching to hash algorithm other than SHA-1 may be good chance to clarify our policy about hexdigest()-ed hash value string. - assume that hexdigest() always returns hex hash in lower case, or - apply lower() on hex hash in appropriate layers to ensure lower-case-ness of it for portability

largefiles: remove unused readstandin() Now, there is no client of readstandin().

largefiles: make copytostore() accept only changectx as the 2nd argument (API) As the name describes, the 2nd argument 'revorctx' of copytostore() can accept non-changectx value, for historical reason, But, since 10561eb97c7f, copyalltostore(), the only one copytostore() client in Mercurial source tree, always passes changectx as 'revorctx'. Therefore, it is reasonable to make copytostore() accept only changectx as the 2nd argument, now.

largefiles: remove unused keyword argument of copytostore() (API) AFAIK, 'uploaded' argument of copytostore() (or copytocache(), before renaming at c65f5b6e26d4) has been never used both on caller and callee sides, since official release of bundled largefiles extension.

largefiles: add copytostore() fstandin argument to replace readstandin() (API) copyalltostore(), only one caller of copytostore(), already knows standin file name of the target largefile. Therefore, passing it to copytostore() is more efficient than calculating it in copytostore() or readstandin().

largefiles: replace readstandin() by readasstandin() These code paths already (or should, for efficiency at repetition) know the target changectx and path of standin file.

largefiles: introduce readasstandin() to read hex hash from given filectx This will be used to centralize and encapsulate the logic to read hash from given (filectx of) standin file. readstandin() isn't suitable for this purpose, because there are some code paths, which want to read hex hash directly from filectx.

rebase: abort hg pull --rebase if rebase.requiredest is set (issue5514) Previously, the pull would succeed, but the subsequent rebase would fail due to the rebase.requiredest flag. Now abort earlier with a more useful error message.

rebase: demonstrate behavior with requiredest and pull --rebase

rebase: allow destination-free continue and abort (issue5513)

rebase: test to show brokenness with requiredest As shown in issue5513, --continue is broken when destination is required. This adds a patch that demonstates this silly behavior, which will be fixed in a future patch.

rebase: move destination test to new test file We'll be adding a lot more tests here, so it makes sense to have this in its own file now.

templatefilter: add support for 'long' to json() When disabling the '#requires serve' check in test-hgwebdir.t and running it on Windows, several 500 errors popped up when querying '?style=json', with the following in the error log: File "...\\mercurial\\templater.py", line 393, in runfilter "keyword '%s'") % (filt.func_name, dt)) Abort: template filter 'json' is not compatible with keyword 'lastchange' The swallowed exception at that point was: File "...\\mercurial\\templatefilters.py", line 242, in json raise TypeError('cannot encode type %s' % obj.__class__.__name__) TypeError: cannot encode type long This corresponds to 'lastchange' being populated by hgweb.common.get_stat(), which uses os.stat().st_mtime. os.stat_float_times() is being disabled in util, so the type for the times is 'long' on Windows, and 'int' on Linux.

hgweb: prefix line id by ctx shortnode in filelog when patches are shown When "patch" query parameter is present in requests to filelog view, line ids in patches diff are no longer unique in the page since several patches are shown on the same page. We now prefix line id by changeset shortnode when several patches are displayed in the same page to have unique line ids overall.

test-check-code: prevent files being added to the root directory Adding new files in the root directory is probably a mistake, and is usually discouraged [1]. The test catches it to avoid mistakes like [2]. Modify the test if files need to be added in the root. [1]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-July/086442.html [2]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-March/095836.html

sslutil: clarify internal documentation I ran into this python issue with an incomplete certificate chain on Windows recently, and this is the clarification that came from that experimenting. The comment I left on the bug tracker [1] with a reference to the CPython code [2] indicates that the original problem I had is a different bug, but happened to be mentioned under issue20916 on the Python bug tracker. [1] https://bz.mercurial-scm.org/show_bug.cgi?id=5313#c7 [2] https://hg.python.org/cpython/file/v2.7.12/Modules/_ssl.c#l628

unionrepo: avoid unnecessary node -> rev conversion

bundlerepo: avoid unnecessary node -> rev conversion

revlog: avoid unnecessary node -> rev conversion

check-code: detect r.revision(r.node(rev)) revlog.revision takes either node or rev, but taking a rev is more efficient, because converting rev to node is just a seek and read. That's cheaper than converting node to rev, which may require O(n) walk in revlog index for the first times, and then triggering building the radix tree index. Even with the radix tree built, rev -> node is still faster than node -> rev because the radix tree requires more jumps in memory. So r.revision(r.node(rev)) should be changed to r.revision(rev). This patch adds a check-code rule to detect that.

hardlink: check directory's st_dev when copying files Previously, when copying a file, copyfiles will compare src's st_dev with dirname(dst)'s st_dev, to decide whether to enable hardlink or not. That could have issues on Linux's overlayfs, where stating directories could result in different st_dev from st_dev of stating files, even if both the directories and the files exist in the overlay's upperdir. This patch fixes it by checking dirname(src) instead. It's more consistent because we are checking directories for both src and dest. That fixes test-hardlinks.t running on common Docker setups.

hardlink: duplicate hardlink detection for copying files and directories A later patch will change one of them so they diverge.

hardlink: extract topic text logic of copyfiles The topic text shows whether it's "linking" or "copying", based on "hardlink" value. The function is extracted so a later patch can reuse it.

tests: add check to make sure summary works on Python 2

color: replace str() with pycompat.bytestr()

diff: slice over bytes to make sure conditions work normally Both of this are part of generating `hg diff` on python 3.

minirst: remove redundant _admonitions set As Yuya pointed out during a review a month ago, _admonitions and _admonitiontitles are largely redundant. With the last commit, they are exactly redundant. So, remove _admonitions and use _admonitiontitles.keys() instead.

minirst: remove "admonition" from _admonitions The "admonition" rst primitive is split into "specific" admonitions ("attention," "caution," etc) and the "generic" admonition ("admonition"). For more, see http://docutils.sourceforge.net/docs/ref/rst/directives.html#admonitions The _admonitions set and keys of the _admonitiontitles dict overlap exactly except _admonitions has an "admonition" entry. Nowhere in Mercurial is the "admonition" admonition directive used. Even if it were, it doesn't have a title, so it wouldn't be rendered correctly. So, let's remove "admonition" from the set of recognized admonition directives.

minirst: reindent _admonitiontitles I don't like the verical indent. While I was here, I cleaned up some whitespace and added a trailing comma on the last element.

tags: extract filenode filtering into its own function We'll also need to reuse this logic so we extract it into its own function. We document some of the logic in the process.

tags: extract tags computation from fnodes into its own function I'm about to introduce code that needs to perform such computation on "arbitrary" nodes. The logic is extracted into its own function for reuse.

tags: only return 'alltags' in 'findglobaltags' This is minor update along the way. We simplify the 'findglobaltags' function to only return the tags. Since no existing data is reused, we know that all tags returned are global and we can let the caller get that information if it cares about it.

tags: make argument 'tagtype' optional in '_updatetags' This is the next step from the previous changesets, we are now ready to use this function in a simpler way.

tags: reorder argument of '_updatetags' We move all arguments related to tagtype to the end, together. This will allow us to make these arguments optional and reuse of this logic for callers that do not care about the tag types.

tags: do not feed dictionaries to 'findglobaltags' The code asserts that these dictionary are empty. So we can be more explicit and have the function return the dictionaries directly.

tags: extract fnode retrieval into its own function My main goal here is to be able to reuse this logic easily. As a side effect this important logic is now insulated and the code is clearer.

hgweb: fix diff hunks filtering by line range in webutil.diffs() The previous clause for filter out a diff hunk was too restrictive. We need to consider the following cases (assuming linerange=(lb, ub) and the @s2,l2 hunkrange): <-(s2)--------(s2+l2)-> <-(lb)---(ub)-> <-(lb)---(ub)-> <-(lb)---(ub)-> previously on the first and last situations were considered. In test-hgweb-filelog.t, add a couple of lines at the beginning of file "b" so that the line range we will follow does not start at the beginning of file. This covers the change in aforementioned diff hunk filter clause.

summary: display obsolete state of parents Extend the "parent: " lines in summary to display "(obsolete)" when the parent is obsolete.

templates: add "changeset.obsolete" label in command line style Following respective change in cmdutil.changeset_printer.

templates: shorten definition of changeset labels in command-line style We'll add more labels and the line is already quite long, so let's define a variable to hold all evolution "troubles" labels.

templates: use separate() to build changeset labels in command-line style

templatekw: add an "obsolete" keyword Definition is the same as the one in evolve extension.

cmdutil: add a "changeset.obsolete" label in changeset_printer Until now there were no label to highlight obsolete changesets in log output, only evolution troubles (unstable, bumped, divergent) are supported. We add a "changeset.obsolete" label on changeset entries produced by changeset_printer so that obsolete changesets can be highlighted in log output. This is useful because, unless using a graph log where obsolete changesets have a 'x' marker, there's no way to identify obsolete changesets. And even in graph mode, when working directory's parent is obsolete, we get a '@' marker and we do not see it as obsolete.

fileset: perform membership test against set for status queries Previously, fileset functions operating on status items performed membership tests against a list of items. When there are thousands of items having a specific status, that test can be extremely slow. Changing the membership test to a set makes this operation substantially faster. On the mozilla-central repo: $ hg files -r d14cac631ecc 'set:added()' before: 28.120s after: 0.860s $ hg status --change d14cac631ecc --added 0.690s

worker: flush ui buffers before running the worker a91c6275 introduces flushing ui buffers after a worker finished. If the ui was not flushed before the worker was started, fork will copy the existing buffers to the worker. This causes messages issued before the worker started to be written to the terminal for each worker. We are now flushing the ui before we start a worker and add an appropriate test which will fail before this patch.

chgserver: do not copy configs set by environment variables Config set by environment variables have a source like "$ENVNAME". They should not be copied because they will be recalculated by rcutil.rccomponents.

rcutil: extract duplicated logic to a lambda This simplifies the code a bit.

rcutil: unindent a block Since global _rccomponents is gone, the code could be simplified.

rcutil: do not cache rccomponents The function is only called once except for "hg debugconfig", where it is called twice. So there is no need to cache it. Caching it will cause issues with chgserver. Instead of dropping the cache in chgserver, it seems cleaner to just avoid the cache.

ui: rerun color.setup() once the pager has spawned to honor 'color.pagermode' Otherwise, ui.pageractive is False when color is setup in dispatch.py (without --pager=on), and this config option is ignored.

ui: defer setting pager related properties until the pager has spawned When --pager=on is given, dispatch.py spawns a pager before setting up color. If the pager failed to launch, ui.pageractive was left set to True, so color configured itself based on 'color.pagermode'. A typical MSYS setting would be 'color.mode=auto, color.pagermode=ansi'. In the failure case, this would print a warning, disable the pager, and then print the raw ANSI codes to the terminal. Care needs to be taken, because it appears that leaving ui.pageractive=True was the only thing that prevented an attempt at running the pager again from inside the command. This results in a double warning message, so pager is simply disabled on failure. The ui config settings didn't need to be moved to fix this, but it seemed like the right thing to do for consistency.

color: stop mutating the default effects map A future change will make color.setup() callable a second time when the pager is spawned, in order to honor the 'color.pagermode' setting. The problem was that when 'color.mode=auto' was resolved to 'win32' in the first pass, the default ANSI effects were overwritten, making it impossible to honor 'pagermode=ansi'. Also, the two separate maps didn't have the same keys. The symmetric difference is 'dim' and 'italic' (from ANSI), and 'bold_background' (from win32). Thus, the update left entries that didn't belong for the current mode. This bled through `hg debugcolor`, where the unsupported ANSI keys were listed in 'win32' mode. As an added bonus, this now correctly enables color with MSYS `less` for a command like this, where pager is forced on: $ hg log --config color.pagermode=ansi --pager=yes --color=auto Previously, the output was corrupted. The raw output, as seen through the ANSI blind `more.com` was: <-[-1;6mchangeset: 34840:3580d1197af9<-[-1m ... which MSYS `less -FRX` rendered as: 1;6mchangeset: 34840:3580d1197af91m ... (The two '<-' instances were actually an arrow character that TortoiseHg warned couldn't be encoded, and notepad++ translated to a single '?'.) Returning an empty map for 'ui._colormode == None' seems better that defaulting to '_effects' (since some keys are mode dependent), and is better than None, which blows up `hg debugcolor --color=never`.

pager: do not read from environment variable $PAGER is converted to the pager.pager config item. So it's no longer necessary to read $PAGER in ui.pager().

ui: simplify geteditor Now $EDITOR and $VISUAL will affect ui.editor directly. So it's no longer necessary to test them in ui.geteditor.

debugconfig: list environment variables in debug output Since we print "read config from" for config files, printing environment variables will make it more consistent.

rcutil: let environ override system configs (BC) This is BC because system configs won't be able to override $EDITOR, $PAGER. The new behavior is arguably more rational.

rcutil: add a method to convert environment variables to config items Handling config and environ priorities has been messy. Partially because we don't have config layers - you either get all configs (sys + user), or none. Ideally, environ like $EDITOR, $PAGER should be able to override the system configs "ui.editor", "pager.pager". This patch provides the ability to convert them into config items, so they can be inserted into the middle config layer between system rc and user rc.

rcutil: let rccomponents return different types of configs (API) The next patches will convert environ to raw config items, and insert the config items between systemrcpath and userrcpath. This patch teaches rccomponents to return the type information so the caller could distinguish between "path" and raw config "items".

rcutil: rename rcpath to rccomponents (API)