branchcache: add attributes to track which nodes are verified Half of the cost of loading branchcache comes from verifiying all the nodes it has. We don't need to verify all the nodes in all the cases. Sometimes we need to verify only a set of nodes for a set of branches. We can ignore nodes of other branches as we are not going to read them. This patch introduces two attributes to branchcache class. _verifiedbranches is a set which will tell the branches for which it's head nodes are verified. _closedverified is a boolean which will tell whether all closednodes are verified or not. Another idea which I had was to keep a set of nodes which are verified, but it will be more ugly and difficult to track things on node level. Differential Revision: https://phab.mercurial-scm.org/D6156

branchcache: make entries a private attribute Differential Revision: https://phab.mercurial-scm.org/D6155

branchcache: introduce hasbranch() This will be used to check whether a branch exists or not. This will optimized in future. Differential Revision: https://phab.mercurial-scm.org/D6154

branchmap: drop branchcache.setdefault() (API) All the callers are updated to call setdefault of branchcache.entries Differential Revision: https://phab.mercurial-scm.org/D6153

branchcache: rename itervalues() to iterheads() The itervalues() exists because branchcache() had a dict interface. Since it no longer has a dict interface, it makes sense to have better function names. If a person does not understand how branchcache stores info, it will be hard for them to guess what itervalues() does. Differential Revision: https://phab.mercurial-scm.org/D6152

branchmap: remove the dict interface from the branchcache class (API) The current branchmap computation involves reading the whole branchmap from disk, validating all the nodes even if they are not required. This leads to a lot of time on repos which have large branchmap or a lot of branches. On large repos, this can validate around 1000's of nodes. On some operations, like finding whether a branch exists or not, we don't need to validate all the nodes. Or updating heads for a single branch. Before this patch, branchcache class was having dict interface and it was hard to keep track of reads. This patch removes the dict interface. Upcoming patches will implement lazy loading and validation of data and implement better API's. Differential Revision: https://phab.mercurial-scm.org/D6151

test-template: fix stdio mode on Windows Otherwise, CBOR data would be corrupted. Spotted by Matt Harbison.

samplehgrcs: update the list of suggested extensions Back in the day, this was color and pager, both of which are now default. Churn isn't that popular, but the other four below (obviously?) are.

samplehgrcs: clarify which lines should be uncommented The original wording has confused at least one person. Hopefully it's clearer this way. https://stackoverflow.com/questions/55288177/adding-hg-strip-to-hgrc-config-file

templatefilters: add {x|cbor} filter for custom CBOR output

template: add CBOR output format The whole output is wrapped as an array just like the other serialization formats. It's an indefinite-length array since the size is unknown while encoding. Maybe we can add 'cbor-stream' (and 'pickle-stream') as needed.

memfilectx: override copysource() instead of using dummy nodeid The "_copied" property in basefilectx is used by renamed() and copysource(). committablefilectx (which memfilectx subclasses) overrides renamed() and writes it in terms of copysource() instead of _copied. That means that the nodeid part of "_copied" is memfilectx is unused. Let's instead override copysource() too so we don't need the "_copied". Differential Revision: https://phab.mercurial-scm.org/D6159

memctx: rename constructor argument "copied" to "copysource" (API) It's just the path, not the nodeid, so "copysource" seems more appropriate. Differential Revision: https://phab.mercurial-scm.org/D6158

crecord: redraw the screen when starting up chunkselector Failure to do this can cause screen corruption like: <headerline> [X] filename.cc <several blank lines> <output from previous iteration of split that happened to be here> I believe this might only happen in some terminals, and maybe only when using the "alternate screen". Regardless of the exact conditions to reproduce, it should be safe to always clear it when starting up and is probably the correct thing to do anyway :) Differential Revision: https://phab.mercurial-scm.org/D6131

crecord: redraw the screen on ctrl-L This is the normal use of Ctrl-L, so I think this is going to be what most people expect it to do. We're keeping the adjustment of what line we're scrolled to as well. I believe both to be necessary to handle otherwise inescapable situations when we've got screen corruption or edge-cases during window resizing. Differential Revision: https://phab.mercurial-scm.org/D6130

crecord: completely redraw screen when coming back from editor Differential Revision: https://phab.mercurial-scm.org/D6129

tests: glob seconds in test-upgrade-repo.t I had the test failing locally for me with diff showing `1.4s` instead of 0.0s Differential Revision: https://phab.mercurial-scm.org/D6161

store: recommend using `hg debugrebuildfncache` is fncache is corrupted Differential Revision: https://phab.mercurial-scm.org/D6160

debugsparse: abort if the repository is not sparse instead of ui.status() This is similar to what narrow extension does. Differential Revision: https://phab.mercurial-scm.org/D6149

revert: option to choose what to keep, not what to discard I know the you (the reader) are probably tired of discussing how `hg revert -i -r .` should behave and so am I. And I know I'm one of the people who argued that showing the diff from the working copy to the parent was confusing. I think it is less confusing now that we show the diff from the parent to the working copy, but I still find it confusing. I think showing the diff of hunks to keep might make it easier to understand. So that's what this patch provides an option for. One argument doing it this way is that most people seem to find `hg split` natural. I suspect that is because it shows the forward diff (from parent commit to the commit) and asks you what to put in the first commit. I think the new "keep" mode for revert (this patch) matches that. In "keep" mode, all the changes are still selected by default. That means that `hg revert -i` followed by 'A' (keep all) (or 'c' in curses) will be different from `hg revert -a`. That's mostly because that was simplest. It can also be argued that it's safest. But it can also be argued that it should be consistent with `hg revert -a`. Note that in this mode, you can edit the hunks and it will do what you expect (e.g. add new lines to your file if you added a new lines when editing). The test case shows that that works. Differential Revision: https://phab.mercurial-scm.org/D6125

patch: include newline at EOF in help text for interactive patch The lack of a newline means that some "editors" that are useful in tests, such as `echo "+new line" >> "$1"` don't work. It's obviously easy to work around it, but newline at EOF seems like a good practice anyway. Differential Revision: https://phab.mercurial-scm.org/D6124

merge with stable

Added signature for changeset 4ea21df312ec

Added tag 4.9.1 for changeset 4ea21df312ec

patch: include flag-only file changes in "special" when filtering (issue5864) This patch fix the issue5864 (or maybe issue5865 too) which occurs during split (or I should say at the time of filtering the hunks in interactive mode) where user hits a not ending loop of "no changes to record". And it's not only the case for split it will happen in every interactive case for e.g. `hg commit -i` or `hg uncommit -i` After looking into code I found that when filtering we have some notation called "special" for the file headers which doesn't contain any hunk and just contain the header (for e.g. newly added empty file or deleted file) where the user cannot change the content of operation. And I think we can put this "flag-only" file change in that same bucket of "special". But I doubt a bit about the case when a file have flag change and atleast one hunk then user won't be able to separate the flag change from hunks. Changed test file reflect the fixed behaviour. Differential Revision: https://phab.mercurial-scm.org/D6058

store: error out if fncache does not ends with a newline If fncache does not ends with a newline, chunk will not be fully consumed. It should be a bug somewhere or the fncache is corrupted if that happens. Let's error out in such cases. Differential Revision: https://phab.mercurial-scm.org/D6148

tracked: add documentation about `--import-rules` flag The documentation is inspired from the `--import-rules` flag of hg debugsparse command. Differential Revision: https://phab.mercurial-scm.org/D6150

discovery: fix embarrassing typo in slice definition The code introduced in e514799e4e07 ended up having a silly bug. The indexing selected a single item slice picking only p1. The discovery result was still correct, but the sampling was hampered, sometime leading to much more round trips being performed. Fixing this issue restore the previous sampling behavior. This fix has a negative performance impact on the pathological case the previous test has been built. # parent of this changesets ! wall 5.313884 comb 5.310000 user 5.260000 sys 0.050000 (best of 5) ! wall 6.711860 comb 6.710000 user 6.670000 sys 0.040000 (max of 5) ! wall 5.844016 comb 5.842000 user 5.784000 sys 0.058000 (avg of 5) ! wall 5.778635 comb 5.780000 user 5.740000 sys 0.040000 (median of 5) # With this changesets. ! wall 6.350879 comb 6.350000 user 6.300000 sys 0.050000 (best of 5) ! wall 6.653647 comb 6.660000 user 6.480000 sys 0.180000 (max of 5) ! wall 6.492762 comb 6.494000 user 6.414000 sys 0.080000 (avg of 5) ! wall 6.547577 comb 6.550000 user 6.490000 sys 0.060000 (median of 5) Changeset e514799e4e07 raised the question of using the "_uncheckedparentrevs" instead of the current code. So I ran comparative timing: # old code: 55919b96c02a (e514799e4e07 parent) ! wall 64.078708 comb 64.080000 user 63.160000 sys 0.920000 (best of 5) ! wall 68.296300 comb 68.290000 user 67.410000 sys 0.880000 (max of 5) ! wall 65.899075 comb 65.894000 user 65.082000 sys 0.812000 (avg of 5) ! wall 66.140286 comb 66.130000 user 65.330000 sys 0.800000 (median of 5) # buggy code: e514799e4e07 ! wall 46.605362 comb 46.610000 user 45.880000 sys 0.730000 (best of 5) ! wall 48.619659 comb 48.620000 user 47.890000 sys 0.730000 (max of 5) ! wall 47.350247 comb 47.350000 user 46.672000 sys 0.678000 (avg of 5) ! wall 46.983224 comb 46.980000 user 46.350000 sys 0.630000 (median of 5) # fixed code: e514799e4e07 with this fix ! wall 55.858460 comb 55.850000 user 55.090000 sys 0.760000 (best of 5) ! wall 59.048805 comb 59.060000 user 58.110000 sys 0.950000 (max of 5) ! wall 57.192639 comb 57.192000 user 56.350000 sys 0.842000 (avg of 5) ! wall 57.056373 comb 57.060000 user 56.160000 sys 0.900000 (median of 5) # version using uncheckedparents ! wall 56.471916 comb 56.470000 user 55.630000 sys 0.840000 (best of 5) ! wall 58.228793 comb 58.230000 user 57.600000 sys 0.630000 (max of 5) ! wall 57.377583 comb 57.378000 user 56.674000 sys 0.704000 (avg of 5) ! wall 57.008843 comb 57.010000 user 56.330000 sys 0.680000 (median of 5) So it looks like the overhead from `_uncheckedparentrevs` is not that impactful. I'll investigate this shortly. I'm almost done updating our benchmark suite with more meaningful discovery cases.

store: don't read the whole fncache in memory In large repositories with lot of files, the fncache grows more than 100 MB and reading that whole thing into memory slows things down. Let's not read the whole thing into memory. This patch changes fncache loading code to read 1 MB at once. Loading 1 MB at once saves ~1 sec on perffncacheload for our internal repository. I tried various values such as 0.5 MB, 5 MB, 10 MB but best results were produced using 1 MB as the chunksize. On a narrow clone with fncache around 40 MB, this patch saves ~0.04 seconds on average on perffncacheload. To test the code, I have coded an extension in test-fncache.t which set chunksize to 1 byte, and the test passes with that. Differential Revision: https://phab.mercurial-scm.org/D5296

record: prevent commits that don't pick up dirty subrepo changes (issue6102) This path covers interactive mode for commit, amend, and shelve, as well as the deprecated record extension. Since shelf creation uses commit without -S in the non-interactive case, aborting here should be OK. (I didn't check what happens to non interactive shelve creation if `ui.commitsubrepos=True` is set.) subrepoutil.precommit() will abort on a dirty subrepo if the config option isn't set, but the hint recommends using --subrepos to commit. Since only the commit command currently supports that option, the error has to be raised here to omit the hint. Doing the check before asking about all of the hunks in the MQ test seems like an improvement on its own. There's probably an additional check on this path that can be removed.