hgweb: build the "entries" list directly in filelog command There's no apparent reason to have this "entries" generator function that builds a list and then yields its elements in reverse order and which is only called to build the "entries" list. So just build the list directly, in reverse order. Adjust "parity" generator's offset to keep rendering the same.

convert: remove "replacecommitter" action As pointed out by Yuya, this action doesn't add much (any?) value.

ui: check EOF of getpass() response read from command-server channel readline() returns '' only when EOF is encountered, in which case, Python's getpass() raises EOFError. We should do the same to abort the session as "response expected." This bug was reported to https://bitbucket.org/tortoisehg/thg/issues/4659/

convert: config option to control Git committer actions When converting a Git repository to Mercurial at Mozilla, I encountered a scenario where I didn't want `hg convert` to automatically add the "committer: <committer>" line to commit messages. While I can hack around this by rewriting the Git commit before it is fed into `hg convert`, I figured it would be a useful knob to control. This patch introduces a config option that allows lots of control over the committer value. I initially implemented this as a single boolean flag to control whether to save the committer message. But then there was feedback that it would be useful to save the committer in extra data. While this patch doesn't implement support for saving in extra data, it does add a mechanism for extending which actions to take on the committer field. We should be able to easily add actions to save in extra data. Some of the implemented features weren't asked for. But I figured they could be useful. If nothing else they demonstrate the extensibility of this mechanism.

help: make "mergetool" an alias for "merge-tools" I've probably typed `hg help mergetool` dozens of times. I'm tired of it not working.

templatekw: force noprefix=False to insure diffstat consistency (issue4755) The result of diffstatdata should not depend on having noprefix set or not, as was reported in issue 4755. Forcing noprefix to false on call makes sure the parser receives the diff in the correct format and returns the proper result. Another way to fix this would have been to change the regular expressions in path.diffstatdata(), but that would have introduced many unecessary special cases.

check-code: reject module-level @cachefunc Module-level @cachefunc usage is risky because it can easily create a memory "leak". Let's reject it completely for now. If a valid usage comes up in the future, we can always improve the check or reconsider.

similar: remove caching from the module level To prevent Bad Things™ from happening, let's rework the logic to not use util.cachefunc.

patch: add label for coloring the similarity extended header Just like the summary says, this will colorize the: similarity index 88% line in the diff output.

patch: use opt.showsimilarity to calculate and show the similarity Tests have been added.

patch: add similarity config knob in experimental section This config knob will control whether or not to show the similarity calculation in the diff output: diff --git a/README.md b/foo.md similarity index 88% rename from README.md rename to foo.md --- a/README.md +++ b/foo.md

similar: move score function to module level Future patches will use this to report the similarity of a rename / copy in the patch output.

revset: abuse x:y syntax to specify line range of followlines() This slightly complicates the parsing (see the previous patch), but the overall result seems not bad. I keep x:, :y and : for future extension.

revset: do not transform range* operators in parsed tree This allows us to handle x:y range as a general range object. A primary user of it is followlines().

revset: add default value to getinteger() helper This seems handy.

revset: factor out getinteger() helper We have 4 revset functions that take integer arguments, and they handle their arguments in slightly different ways. This patch unifies them: - getstring() in place of getsymbol(), which is more consistent with the handling of integer revisions (both 1 and '1' are valid) - say "expects" instead of "requires" for type errors We don't need to catch TypeError since getstring() must return a string.

revset: rename rev argument of followlines() to startrev The rev argument has the same meaning as startrev of follow(), and I think startrev is more informative. followlines() is new function, we can make BC now.

help: use :hg: role and canonical name to point to revset string patterns Follows up 5dd67f0993ce. Now revisions.txt and revsets.txt has been merged, so use revisions.* as a pointer.

util: compression APIs to support revlog decompression Previously, compression engines had APIs for performing revlog compression but no mechanism to perform revlog decompression. This patch changes that. Revlog decompression is slightly more complicated than compression because in the compression case there is (currently) only a single engine that can be used at a time. However for decompression, a revlog could contain chunks from multiple compression engines. This means decompression needs to map to multiple engines and decompressors. This functionality is outside the scope of this patch. But it drives the decision for engines to declare a byte header sequence that identifies revlog data as belonging to an engine and an API for obtaining an engine from a revlog header.

crecord: add an experimental option for space key to move cursor down I really want to have an option of toggling a selection on a line and also moving cursor down as a single keystroke. It also kinda makes sense for space key to do this, because some other curses UIs in the wild do this (e.g. various file managers, htop). So I got an idea to make a config option that defaults to False for compatibility, but allows making crecord UI a lot more useful for people with big hunks. We add this an experimental option to experiment with this behavior.

perf: support multiple compression engines in perfrevlogchunks Now that the revlog has a reference to a compressor, it is possible to swap in other compression engines. So, teach `hg perfrevlogchunks` to do that. The default behavior of `hg perfrevlogchunks` is now to measure the compression performance of all compression engines implementing the revlog compressor API. This effectively adds the no-op "none" compressor and zstd (when available) into the default set. While we can't yet plug alternate compressors into revlogs, this command gives us a preview of the performance. On the mozilla-unified repository: $ hg perfrevlogchunks -c ! compress w/ none ! wall 0.115159 comb 0.110000 user 0.110000 sys 0.000000 (best of 86) ! compress w/ zlib ! wall 5.681406 comb 5.680000 user 5.680000 sys 0.000000 (best of 3) ! compress w/ zstd ! wall 2.624781 comb 2.620000 user 2.620000 sys 0.000000 (best of 4) $ hg perfrevlogchunks -m ! compress w/ none ! wall 0.124486 comb 0.120000 user 0.120000 sys 0.000000 (best of 79) ! compress w/ zlib ! wall 10.144701 comb 10.150000 user 10.150000 sys 0.000000 (best of 3) ! compress w/ zstd ! wall 4.383118 comb 4.390000 user 4.390000 sys 0.000000 (best of 3) Those numbers for zstd look promising. But they aren't the full story. For that, we'll need to look at decompression times and storage sizes. Stay tuned...

revlog: use compression engine API for compression This commit swaps in the just-added revlog compressor API into the revlog class. Instead of implementing zlib compression inline in compress(), we now store a cached-on-first-use revlog compressor on each revlog instance and invoke its "compress()" method. As part of this, revlog.compress() has been refactored a bit to use a cleaner code flow and modern formatting (e.g. avoiding parenthesis around returned tuples). On a mozilla-unified repo, here are the "compress" times for a few commands: $ hg perfrevlogchunks -c ! wall 5.772450 comb 5.780000 user 5.780000 sys 0.000000 (best of 3) ! wall 5.795158 comb 5.790000 user 5.790000 sys 0.000000 (best of 3) $ hg perfrevlogchunks -m ! wall 9.975789 comb 9.970000 user 9.970000 sys 0.000000 (best of 3) ! wall 10.019505 comb 10.010000 user 10.010000 sys 0.000000 (best of 3) Compression times did seem to slow down just a little. There are 360,210 changelog revisions and 359,342 manifest revisions. For the changelog, mean time to compress a revision increased from ~16.025us to ~16.088us. That's basically a function call or an attribute lookup. I suppose this is the price you pay for abstraction. It's so low that I'm not concerned.

util: compression APIs to support revlog compression As part of "zstd all of the things," we need to teach revlogs to use non-zlib compression formats. Because we're routing all compression via the "compression manager" and "compression engine" APIs, we need to introduction functionality there for performing revlog operations. Ideally, revlog compression and decompression operations would be implemented in terms of simple "compress" and "decompress" primitives. However, there are a few considerations that make us want to have a specialized primitive for handling revlogs: 1) Performance. Revlogs tend to do compression and especially decompression operations in batches. Any overhead for e.g. instantiating a "context" for performing an operation can be noticed. For this reason, our "revlog compressor" primitive is reusable. For zstd, we reuse the same compression "context" for multiple operations. I've measured this to have a performance impact versus constructing new contexts for each operation. 2) Specialization. By having a primitive dedicated to revlog use, we can make revlog-specific choices and leave the door open for more functionality in the future. For example, the zstd revlog compressor may one day make use of dictionary compression. A future patch will introduce a decompress() on the compressor object. The code for the zlib compressor is basically copied from revlog.compress(). Although it doesn't handle the empty input case, the null first byte case, and the 'u' prefix case. These cases will continue to be handled in revlog.py once that code is ported to use this API.

revlog: move decompress() from module to revlog class (API) Upcoming patches will convert revlogs to use the compression engine APIs to perform all things compression. The yet-to-be-introduced APIs support a persistent "compressor" object so the same object can be reused for multiple compression operations, leading to better performance. In addition, compression engines like zstd may wish to tweak compression engine state based on the revlog (e.g. per-revlog compression dictionaries). A global and shared decompress() function will shortly no longer make much sense. So, we move decompress() to be a method of the revlog class. It joins compress() there. On the mozilla-unified repo, we can measure the impact of this change on reading performance: $ hg perfrevlogchunks -c ! chunk ! wall 1.932573 comb 1.930000 user 1.900000 sys 0.030000 (best of 6) ! wall 1.955183 comb 1.960000 user 1.930000 sys 0.030000 (best of 6) ! chunk batch ! wall 1.787879 comb 1.780000 user 1.770000 sys 0.010000 (best of 6 ! wall 1.774444 comb 1.770000 user 1.750000 sys 0.020000 (best of 6) "chunk" appeared to become slower but "chunk batch" got faster. Upon further examination by running both sets multiple times, the numbers appear to converge across all runs. This tells me that there is no perceived performance impact to this refactor.

revlog: make compressed size comparisons consistent revlog.compress() compares the compressed size to the input size and throws away the compressed data if it is larger than the input. This is the correct thing to do, as storing compressed data that is larger than the input takes up more storage space and makes reading slower. However, the comparison was implemented inconsistently. For the streaming compression mode, we threw away the result if it was greater than or equal to the input size. But for the one-shot compression, we threw away the compression only if it was greater than the input size! This patch changes the comparison for the simple case so it is consistent with the streaming case. As a few tests demonstrate, this adds 1 byte to some revlog entries. This is because of an added 'u' header on the chunk. It seems somewhat wrong to increase the revlog size here. However, IMO the cost of 1 byte in storage is insignificant compared to the performance gains of avoiding decompression. This patch should invite questions around the heuristic for throwing away compressed data. For example, I'd argue we should be more liberal about rejecting compressed data, additionally doing so where the number of bytes saved fails to reach a threshold. But we can have this discussion another time.

similar: rename local variable to not collide with previous Future patches will move the score function to the module level, so let's not shadow that.

patch: add label for coloring the index extended header Just like the summary says, this will colorize the: index 3d3ba4b65e11..57274a0f46b2 100644 line in the diff output.

patch: add index line for diff output This helps highlighting in third-party diff coloring (which assumes git output) and maintains pedantic correctness with diff --git. Tests will be added at the end of the series.

patch: add config knob for displaying the index header This config knob can take an integer between 0 and 40 or a keyword ('none', 'short', 'full') to control the length of hash to output. It will display diffs with the git index header as such, diff --git a/mercurial/mdiff.py b/mercurial/mdiff.py index 112edf7..d6b52c5 100644 We'll put this in the experimental section for now.

bisect: refer directly to bisect() revset predicate in help We have specific syntax for displaying the help text for a particular revset predicate, so let's refer directly to the bisect() revset in the verbose bisect help. It seems likely that the user doesn't care about other revsets at that point, so they will probably not miss the text about the other revset predicates.