Yuya Nishihara <yuya@tcha.org> [Sat, 04 Apr 2015 22:30:59 +0900] rev 24662
changelog: inline revlog.__contains__ in case it is used in hot loop
Currently __contains__ is called only by "rev()" revset, but "x in cl" is a
function that is likely to be used in hot loop. revlog.__contains__ is simple
enough to duplicate to changelog, so just inline it.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Wed, 08 Apr 2015 02:56:19 +0900] rev 24661
bookmarks: show detailed status about outgoing bookmarks
Before this patch, "hg outgoing -B" shows only difference of bookmarks
between two repositories, and it isn't user friendly.
This patch shows detailed status about outgoing bookmarks at "hg
outgoing -B".
To avoid breaking backward compatibility with other tool chains, this
patch shows status, only if --verbose is specified,
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Wed, 08 Apr 2015 02:56:19 +0900] rev 24660
bookmarks: show detailed status about incoming bookmarks
Before this patch, "hg incoming -B" shows only difference of bookmarks
between two repositories, and it isn't user friendly.
This patch shows detailed status about incoming bookmarks at "hg
incoming -B".
To avoid breaking backward compatibility with other tool chains, this
patch shows status, only if --verbose is specified,
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Wed, 08 Apr 2015 02:56:19 +0900] rev 24659
bookmarks: enhance test of showing detail about incoming/outgoing bookmarks
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Wed, 08 Apr 2015 02:56:19 +0900] rev 24658
bookmarks: show outgoing bookmarks more exactly
Before this patch, "hg outgoing -B" shows only bookmarks added
locally. Then, users can't know about bookmarks below before "hg push"
execution.
- deleted locally (even though it may be added remotely from "hg pull" view)
- advanced locally
- diverged
- changed (= remote revision is unknown for local)
This patch shows such bookmarks, too.
FUJIWARA Katsunori <foozy@lares.dti.ne.jp> [Wed, 08 Apr 2015 02:56:19 +0900] rev 24657
bookmarks: show incoming bookmarks more exactly
Before this patch, "hg incoming -B" shows only bookmarks added
remotely. Then, users can't know about bookmarks below before "hg
pull" execution.
- advanced remotely
- diverged
- changed (remote revision is unknown for local)
This patch shows such bookmarks, too.
Drew Gottlieb <drgott@google.com> [Tue, 07 Apr 2015 15:16:19 -0700] rev 24656
test-manifest: add some test coverage for treemanifest
Similar to the testmanifest test case, testtreemanifest extends the base test
case but uses treemanifests instead of manifestdicts. Adding this test provides
some basic test coverage of treemanifest within the standard test suite.
Drew Gottlieb <drgott@google.com> [Tue, 07 Apr 2015 15:16:19 -0700] rev 24655
test-manifest: make manifesttest a base class that is extended
The implementation of the testmanifest test case is moved to a new base class,
which is then extended to make the testmanifest. And instead of testmanifest,
the subclass is named testmanifestdict because, well, that's what it's testing.
This refactoring makes it possible to create alternate versions of what was
formerly testmanifest, improving test coverage of different manifestdict
implementations.
Drew Gottlieb <drgott@google.com> [Tue, 07 Apr 2015 15:16:19 -0700] rev 24654
test-manifest: move parsemanifest() to be a testmanifest class method
This refactoring lets testmanifest subclasses override this method to
return different manifestdict implementations, such as treemanifest.
It is useful for later commits where the testmanifest class is moved into a
base class, and test cases that extend the base class can provide their own
parsemanifest() implementation.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 07 Apr 2015 22:31:36 -0400] rev 24653
windows: allow readpipe() to actually read data out of the pipe
It appears that the read() in readpipe() never actually ran before (in
test-ssh.t anyway). A print of the size returned from os.fstat() is 0 for every
single print output in test-ssh.t, so the data in the pipe ends up being read
later instead of when it is available. This is the same problem as Linux, as
mentioned in 331cbf088c4c.
There are several places in the Windows SSH tests where the order of local
output vs remote output differ from the other platforms. This only fixes one of
those cases (and interstingly, not the one added in order to test 331cbf088c4c),
so there is more investigation needed. However, without this patch, test-ssh.t
also has this diff:
--- c:/Users/Matt/Projects/hg/tests/test-ssh.t
+++ c:/Users/Matt/Projects/hg/tests/test-ssh.t.err
@@ -397,11 +397,11 @@
$ hg push --ssh "sh ../ssh.sh"
pushing to ssh://user@dummy/*/remote (glob)
searching for changes
- remote: Permission denied
- remote: abort: prechangegroup.hg-ssh hook failed
- remote: Permission denied
- remote: pushkey-abort: prepushkey.hg-ssh hook failed
updating 6c0482d977a3 to public failed!
+ remote: Permission denied
+ remote: abort: prechangegroup.hg-ssh hook failed
+ remote: Permission denied
+ remote: pushkey-abort: prepushkey.hg-ssh hook failed
[1]
$ cd ..
Output with this change was stable over 600+ runs of test-ssh.t. I initially
tried a background thread to read the pipe[1], but this was simpler and the test
results were exactly the same. I also tried SetNamedPipeHandleState(), but the
PIPE_NOWAIT is for compatibility with LANMAN 2.0, not for async I/O (the results
were identical though).
[1] http://eyalarubas.com/python-subproc-nonblock.html
Matt Harbison <matt_harbison@yahoo.com> [Tue, 07 Apr 2015 22:30:25 -0400] rev 24652
win32: add a method to fetch the available pipe data size
This will be used in the next patch to do nonblocking reads from the child
process, like on posix platforms. See that for why os.fstat() is insufficient.
Siddharth Agarwal <sid0@fb.com> [Tue, 07 Apr 2015 20:43:04 -0700] rev 24651
dirs._addpath: reinstate use of Py_CLEAR
I changed this to an explicit Py_DECREF + set to null in 6f0e6fa9fdd7. This was
a silly misunderstanding on my part -- for some reason I thought Py_CLEAR set
its argument to null only if its refcount reached 0. Turns out that's not
actually the case -- Py_CLEAR is just Py_DECREF + set to null with some
additional precautions around destructors that aren't relevant here.
The real bug that 6f0e6fa9fdd7 fixed was the fact that we were mutating the
string after setting it in the Python dictionary.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 06 Apr 2015 18:31:59 -0700] rev 24650
exchange: introduce a '_canusebundle2' function
This function refactors the logic that decides to use 'bundle2' during an
exchange (pull/push). This will help being consistent while transitioning from
the experimental protocol to the final frozen version.
I do not expect this function to survive on the long run when using 'bundle2'
will become a simple capability check.
This is also necessary to allow HG2Y support in an extension to ease transition
of companies using the experimental protocol in production (yeah...). Such
extension will be able to wrap this function to use the experimental protocol in
some case.
Pierre-Yves David <pierre-yves.david@fb.com> [Tue, 07 Apr 2015 16:01:32 -0700] rev 24649
bundle2: detect bundle2 stream/request on /HG2./ instead of /HG2Y/
To support more bundle2 formats, we need a wider detection of bundle2-family
streams. The various places what were explicitly detecting the full magic string
are now matching on the first three characters of it.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 06 Apr 2015 17:23:11 -0700] rev 24648
unbundle20: allow generic dispatch between unbundlers
We now take full advantage of the 'getunbundler' function by using a
'{version -> unbundler-class}' mapping. This map currently contains a single
entry but will make it easy to support more versions from an extension/the
future.
At some point, this map will probably contain bundler-class information too,
in the same fashion the packer map does. However, this is not critically required
right now so it will happen by itself when needed.
The main target is to allow HG2Y support in an extension to ease transition of
companies using the experimental protocol in production (yeah...) But I've no
doubt this will be useful when playing with a future HG21.
Drew Gottlieb <drgott@google.com> [Tue, 07 Apr 2015 15:18:52 -0700] rev 24647
treemanifest: refactor treemanifest.walk()
This refactor is a preparation for an optimization in the next commit. This
introduces a recursive element that recurses each submanifest. By using a
recursive function, the next commit can avoid walking over some subdirectories
altogether.
Drew Gottlieb <drgott@google.com> [Tue, 07 Apr 2015 15:18:52 -0700] rev 24646
manifest: move changectx.walk() to manifests
The logic of walking a manifest to yield files matching a match object is
currently being done by context, not the manifest itself. This moves the walk()
function to both manifestdict and treemanifest. This separate implementation
will also permit differing, optimized implementations for each manifest.
Matt Harbison <matt_harbison@yahoo.com> [Sun, 05 Apr 2015 15:08:55 -0400] rev 24645
subrepo: precisely identify the missing subrepo spec file
It isn't obvious which file is the problem with deep subrepos, so provide the
path. Since the parsing is done with a ctx and not a subrepo object, it isn't
possible to display a path from the root subrepo. Therefore, the path shown is
relative to cwd.
There's no test coverage for the first abort, and I couldn't figure out how to
trigger it, but it is changed for consistency.
Durham Goode <durham@fb.com> [Sun, 05 Apr 2015 12:12:02 -0700] rev 24644
graft: record intermediate grafts in extras
Previously the extra field for a graft only contained the original commit hash.
This made it impossible to use graft to copy a commit more than once, because
the extras fields did not change after the second graft.
The fix is to add an extra.intermediate-source field that records the immediate
predecessor to graft. This changes hashes for commits that have been grafted
twice, which is why the test was affected.
Durham Goode <durham@fb.com> [Sun, 05 Apr 2015 11:55:38 -0700] rev 24643
graft: allow creating sibling grafts
Previously it was impossible to graft a commit onto it's own parent (i.e. create
a copy of the commit). This is useful when wanting to create a backup of the
commit before continuing to amend it. This patch enables that behavior.
The change to the histedit test is because histedit uses graft to apply commits.
The test in question moves a commit backwards onto an ancestor. Since the graft
logic now more explicitly supports this, it knows to simply accept the incoming
changes (since they are more recent), instead of prompting.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 06 Apr 2015 16:07:18 -0700] rev 24642
unbundle20: move header parsing into the 'getunbundler' function
The dispatching will be based on the header content, so we need to move this
logic into the factory function.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 06 Apr 2015 16:04:33 -0700] rev 24641
unbundle20: retrieve unbundler instances through a factory function
To support multiple bundle2 formats, we will need a function returning
the proper unbundler according to the header. We introduce such aa
function and change the usage in the code base. The function will get
smarter in later changesets.
This is somewhat similar to the dispatching we do for 'HG10' and 'HG11'.
The main target is to allow HG2Y support in an extension to ease transition of
companies using the experimental protocol in production (yeah...) But I've no
doubt this will be useful when playing with a future HG21.
Pierre-Yves David <pierre-yves.david@fb.com> [Mon, 06 Apr 2015 15:40:12 -0700] rev 24640
bundle20: move magic string into the class
This makes it easy to create a new bundler class that inherits from
the core one. This matches the way 'changegroup' packers work.
The main target is to allow HG2Y support in an extension to ease transition of
companies using the experimental protocol in production (yeah...) But I've no
doubt this will be useful when playing with a future HG21.
Martin von Zweigbergk <martinvonz@google.com> [Tue, 07 Apr 2015 08:45:52 -0700] rev 24639
localrepo.getbundle: drop unused 'format' argument
The 'format' argument was not used even when it was added in
60ad2ea5b106 (getbundle: pass arbitrary arguments all along the call
chain, 2014-04-17).
Note that by removing the argument, if any caller did pass a named
'format' argument, we will now pass that along to exchange.getbundle()
via the kwargs. If the idea was to remove such a key, that should have
been done explicitly.
Martin von Zweigbergk <martinvonz@google.com> [Tue, 07 Apr 2015 12:35:07 -0700] rev 24638
exchange: remove check for 'format' key
When the 'kwargs' variable was added in 12f161f08d74 (bundle2: allow
pulling changegroups using bundle2, 2014-04-01), it could contain only
'bundlecaps', 'common' and 'heads', so the check for 'format' would
always be false. Since then, _pullbundle2extraprepare() has been added
for hooks, but it seems unlikely that they would a 'format' key.
Yuya Nishihara <yuya@tcha.org> [Fri, 27 Mar 2015 22:12:53 +0900] rev 24637
templates-default: do not show description or summary if empty
changeset_printer shows description only if ctx.description().strip() is not
empty. The default template should do the same way.
Drew Gottlieb <drgott@google.com> [Mon, 06 Apr 2015 10:51:53 -0700] rev 24636
treemanifest: further optimize treemanifest.matches()
The matches function was previously traversing all submanifests to look for
matching files, even though it was possible to know if a submanifest won't
contain any matches.
This change adds a visitdir function on the match object to decide quickly if
a directory should be visited when traversing. The function also decides if
_all_ subdirectories should be traversed.
Adding this logic as methods on the match object also makes the logic
modifiable by extensions, such as largefiles.
An example of a command this speeds up is running
hg status --rev .^ python/
on the Mozilla repo with the treemanifest experiment enabled.
It goes from 2.03s to 1.85s.
More improvements to speed from this change will happen when treemanifests are
lazily loaded. Because a flat manifest is still loaded and then converted
into treemanifests, speed improvements are limited.
This change has no negative effect on speed. For a worst-case example, this
command is not negatively impacted:
hg status --rev .^ 'relglob:*.js'
on the Mozilla repo. It goes from 2.83s to 2.82s.
Drew Gottlieb <drgott@google.com> [Mon, 06 Apr 2015 14:36:08 -0700] rev 24635
util: move dirs() and finddirs() from scmutil to util
An upcoming commit requires that match.py be able to call scmutil.dirs(), but
when match.py imports scmutil, a dependency cycle is created. This commit
avoids the cycle by moving dirs() and its related finddirs() function from
scmutil to util, which match.py already depends on.
Drew Gottlieb <drgott@google.com> [Mon, 06 Apr 2015 13:59:36 -0700] rev 24634
parsers: remove unused dependency on util
Parsers.py had a reference to util.sha1 which was unused. This commit removes
this reference as well as the unused import of util to simplify the dependency
graph. This is important for the next commit which actually relocates part
of a module to eliminate a cycle.
Martin von Zweigbergk <martinvonz@google.com> [Mon, 06 Apr 2015 17:03:35 -0700] rev 24633
changectx.walk: drop unnecessary call to match function
If all the files in match.files() are in the context/manifest, we
already know that the matcher will match each file.
Matt Mackall <mpm@selenic.com> [Mon, 06 Apr 2015 17:16:55 -0500] rev 24632
merge with stable
Matt Harbison <matt_harbison@yahoo.com> [Sat, 04 Apr 2015 19:06:43 -0400] rev 24631
largefiles: use the share source as the primary local store (issue4471)
The benefit of retargeting the local store to the share source is that all
shares will always have access to the largefiles any one of them commit, even if
the user cache is deleted (which is documented to be OK to do). Further, any
push into the source (and now any shares), will likewise make the largefile(s)
visible to all related repositories.
In order to maintain compatibility with existing repos, where the largefiles
would be cached only in the local share, fallback to searching the local share
if it isn't found at the share source.
The unshare command should probably be taught to copy the source store into the
store for the repo being unshared to complete the loop.
This patch changes the test like this:
@@ -159,6 +159,5 @@
$ hg share -q src share_dst --config extensions.share=
$ hg -R share_dst update -r0
getting changed largefiles
- large: largefile $HASH not available from file:///$TESTTMP\share_dst
- 0 largefiles updated, 0 removed
+ 1 largefiles updated, 0 removed
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
The issue writeup mentions pushing a largefile from a remote repo to the main
local repo, and the largefile is then not available in any shares. Since the
push doesn't cache the largefile in $USERCACHE, the trashed $USERCACHE in this
test is equivalent.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 04 Apr 2015 19:34:36 -0400] rev 24630
largefiles: use lfutil.findstorepath() when verifying a local repo
Matt Harbison <matt_harbison@yahoo.com> [Sat, 04 Apr 2015 19:31:40 -0400] rev 24629
largefiles: introduce lfutil.findstorepath()
The handful of direct uses of lfutil.storepath() merely need a single path to
read from or write to the largefile, whether or not it exists. Most callers
that care about the file existing call lfutil.findfile(), in order to fallback
from the store to the user cache.
localstore._verify() doesn't call lfutil.findfile(). This prevents redirecting
the store to the share source because the largefiles for existing repos may not
be in the source's store, so verification may fail. It can't be changed to call
findfile(), because findfile() links the file from the usercache to the local
store[1], and because it returns None instead of a path if the file doesn't
exist.
For now, this method is just a cover for lfutil.storepath(), but it will be
filled out in an upcoming patch.
[1] Maybe we shouldn't care? But on a filesystem that doesn't support
hardlinks, then verify will take a lot longer, and start to consume disk
space.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 04 Apr 2015 17:19:16 -0400] rev 24628
vfs: make it possible to pass multiple path elements to join
os.path.join(), localrepo.join() and localrepo.wjoin() allow passing multiple
path elements; vfs.join() should be as convenient.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 04 Apr 2015 15:43:00 -0400] rev 24627
largefiles: drop os.path reference in lfutil.storepath()
localrepo.join() can concatenate multiple parts on its own.
Durham Goode <durham@fb.com> [Sat, 04 Apr 2015 11:24:48 -0700] rev 24626
histedit: fix preventing strips during histedit
We were trying to prevent strips of important nodes during histedit,
but the check was actually comparing the short hashes in the rules to
the exact value the user typed in, so it only ever worked if the user
typed a 12 character hash.
Durham Goode <durham@fb.com> [Fri, 03 Apr 2015 15:18:34 -0700] rev 24625
copies: pass changectx instead of manifest to _computenonoverlap
The _computenonoverlap function takes two manifests to allow extensions to hook
in and read the manifest nodes produced by the function. The remotefilelog
extension actually needs the entire changectx instead (which includes the
manifest) so it can prefetch the subset of files necessary for a sparse checkout
(and the sparse checkout depends on which commit is being accessed, hence the
need for the changectx).
I have tests in the remotefilelog extension that cover this.
Siddharth Agarwal <sid0@fb.com> [Mon, 06 Apr 2015 10:46:44 -0700] rev 24624
dirs._addpath: don't mutate Python strings after exposing them (issue4589)
One of the rules of Python strings is that they're immutable. dirs._addpath
breaks this assumption for performance, which is fine as long as it is done
safely -- once a string is no longer internal-only it shouldn't be mutated.
Unfortunately, we weren't being safe here -- we were mutating 'key' even after
adding it to a dictionary.
This only really affects other C code that reads strings, so it's somewhat hard
to write a test for this without poking into the internal representation of the
string via ctypes or similar. There is currently no C code that reads the
output of the string, but there will likely be some soon as the bug indicates.
There's no significant difference in performance.
Bryan O'Sullivan <bryano@fb.com> [Mon, 06 Apr 2015 08:23:27 -0700] rev 24623
parsers: check for memory allocation overflows more carefully
André Sintzoff <andre.sintzoff@gmail.com> [Sat, 04 Apr 2015 11:27:15 +0200] rev 24622
parsers.c: avoid implicit conversion loses integer precision warning
This warning is raised by Apple LLVM version 6.0 (clang-600.0.57)
(based on LLVM 3.5svn) and was introduced in 670aaee7931c
Martin von Zweigbergk <martinvonz@google.com> [Sat, 04 Apr 2015 21:54:12 -0700] rev 24621
dirstate.walk: don't report same file stat multiple times
dirstate.walk() generates pairs of filename and a stat-like
object. After "hg mv foo Foo", it generates one pair for "foo" and one
for "Foo", as it should. However, on case-insensitive file systems,
when it tries to stat to get the disk state as well, it gets the same
stat result for both names. This confuses at least
scmutil._interestingfiles(), making it think that "foo" was forgotten
rather than removed. That, in turn, makes "hg addremove" add "foo"
back, resulting in both cases in the dirstate, as reported in
issue4590.
This change only takes care of the "if unknown" branch. A similar fix
should perhaps be applied to the other branch.
Pierre-Yves David <pierre-yves.david@fb.com> [Fri, 03 Apr 2015 14:41:18 -0700] rev 24620
repoview: avoid processing the same rev twice in _getstatichidden
If a rev had multiple children, it would be added to the heap multiple times. We
now ensure it is added only once.
Pierre-Yves David <pierre-yves.david@fb.com> [Fri, 03 Apr 2015 14:37:52 -0700] rev 24619
repoview: skip public parent earlier in _getstatichidden
Public changeset have nothing to offer regarding hidden changeset. Lets not add
them to the heap at all.
Pierre-Yves David <pierre-yves.david@fb.com> [Fri, 03 Apr 2015 14:36:05 -0700] rev 24618
repoview: directly skip public head in _getstatichidden
Public heads have nothing to offer regarding hidden stuff, let's skip them.
Pierre-Yves David <pierre-yves.david@fb.com> [Fri, 03 Apr 2015 14:35:53 -0700] rev 24617
repoview: simplify process in _getstatichidden
Since all children are processed before their parents, we can apply the following algorithm:
For each rev (descending order):
* If I'm still hidden, no children will block me,
* If I'm not hidden, I must remove my parent from the hidden set,
This allows us to dynamically change the set of 'hidden' revisions, dropping the
need for the 'actuallyhidden' dictionary and the 'blocked' boolean in the queue.
As before, we start iterating from all heads and stop at the first public
changesets. This ensures the hidden computation is 'O(not public())' instead of
'O(len(min(not public()):))'.
Pierre-Yves David <pierre-yves.david@fb.com> [Fri, 03 Apr 2015 14:16:50 -0700] rev 24616
repoview: use a heap in _getstatichidden
Since we want to process all non-public changesets from top to bottom, a heap
seems more appropriate. This will ensure any revision is processed after all
its children, opening the way to code simplification.
Pierre-Yves David <pierre-yves.david@fb.com> [Fri, 03 Apr 2015 13:58:12 -0700] rev 24615
repoview: update documentation of _getstatichidden
In 2f7cb6e6acdd, the function name, role and return was changed. But the
documentation was not. This fixes it.
Yuya Nishihara <yuya@tcha.org> [Sat, 04 Apr 2015 14:56:18 +0900] rev 24614
ssl: resolve symlink before checking for Apple python executable (issue4588)
test-https.t was broken at 07fafcd4bc74 if /usr/bin/pythonX.Y is used on
Mac OS X.
If python executable is not named as "python", run-tests.py creates a symlink
and hghave uses it. On the other hand, the installed hg executable knows the
real path to the system Python. Therefore, there was an inconsistency that
hghave said it was not an Apple python but hg knew it was.
Augie Fackler <augie@google.com> [Fri, 03 Apr 2015 18:19:24 -0400] rev 24613
test-subrepo-recursion: fix output on non-hardlink systems
There's a slight bug present where a topic doesn't get closed when it should.
This isn't a regression, so I've made a note in the test file.
Augie Fackler <augie@google.com> [Fri, 03 Apr 2015 23:12:21 -0400] rev 24612
test-subrepo-recursion: set progress.changedelay really high
This avoids subtopics from showing up, which were introduced by my recent
change that added progress bars to hardlink clones in some cases.
Augie Fackler <augie@google.com> [Fri, 03 Apr 2015 18:10:12 -0400] rev 24611
test-clone: fix test expectations on systems without hardlinks
Siddharth Agarwal <sid0@fb.com> [Wed, 01 Apr 2015 00:44:33 -0700] rev 24610
dirstate: use parsers.make_file_foldmap when available
This is a significant performance win on large repositories. perffilefoldmap:
On Linux/gcc, on a test repo with over 500,000 files:
before: wall 0.605021 comb 0.600000 user 0.560000 sys 0.040000 (best of 17)
after: wall 0.280530 comb 0.280000 user 0.250000 sys 0.030000 (best of 35)
On Mac OS X/clang, on a real-world repo with over 200,000 files:
before: wall 0.281103 comb 0.280000 user 0.260000 sys 0.020000 (best of 34)
after: wall 0.133622 comb 0.140000 user 0.120000 sys 0.020000 (best of 65)
This visibly impacts status times on case-insensitive file systems. On the Mac
OS X repo, status goes from 3.64 seconds to 3.50.
With the third-party hgwatchman extension [1], 'hg status' on the same repo
goes from 0.80 seconds to 0.65.
[1] https://bitbucket.org/facebook/hgwatchman
Siddharth Agarwal <sid0@fb.com> [Tue, 31 Mar 2015 23:32:27 -0700] rev 24609
parsers: add a C function to create a file foldmap
This is a hot path on case-insensitive filesystems -- it's guaranteed to be
called every time 'hg status' is run.
This is significantly faster than the equivalent Python code: see the following
patch for numbers.
Siddharth Agarwal <sid0@fb.com> [Thu, 02 Apr 2015 19:17:32 -0700] rev 24608
util.h: define an enum for normcase specs
These will be used in upcoming patches to efficiently create a dirstate
foldmap.
Siddharth Agarwal <sid0@fb.com> [Thu, 02 Apr 2015 19:13:50 -0700] rev 24607
perf: make measuring foldmap perf work again
Rev 25c1d3ca5ff6 split the foldmap into two, but I forgot to update perf for
the changes.
Siddharth Agarwal <sid0@fb.com> [Tue, 31 Mar 2015 23:22:03 -0700] rev 24606
parsers._asciitransform: also accept a fallback function
This function will be used in upcoming patches to provide a C implementation of
the function to generate the foldmap.
Siddharth Agarwal <sid0@fb.com> [Wed, 01 Apr 2015 00:38:56 -0700] rev 24605
util: add normcase spec and fallback
These will be used in upcoming patches to efficiently create a dirstate
foldmap.
Andrew Shadura <andrew@shadura.me> [Fri, 03 Apr 2015 22:44:25 +0200] rev 24604
hgk: display committer name when set by hg-git
Yuya Nishihara <yuya@tcha.org> [Sat, 14 Mar 2015 20:16:35 +0900] rev 24603
jsonchangeset: set manifest node to "null" for workingctx
Unlike changeset_printer, it does not hide the manifest field because JSON
output will be parsed by machine where explicit "null" will be more useful
than nothing.
Yuya Nishihara <yuya@tcha.org> [Sat, 14 Mar 2015 20:15:40 +0900] rev 24602
jsonchangeset: set rev and node to "null" for workingctx
Yuya Nishihara <yuya@tcha.org> [Fri, 03 Apr 2015 21:36:39 +0900] rev 24601
templater: tell hggettext to collect help of template functions
Martin von Zweigbergk <martinvonz@google.com> [Tue, 10 Mar 2015 09:57:42 -0700] rev 24600
treemanifest: disable readdelta optimization
When tree manifests are stored with one revlog per directory and
loaded lazily, it's unclear how much readdelta will help. If only a
few files change, then only a small part of the full manifest will be
loaded, and the delta chains should also be shorter for tree
manifests. Therefore, let's disable readdelta for tree manifests for
now.
Laurent Charignon <lcharignon@fb.com> [Mon, 30 Mar 2015 15:38:24 -0700] rev 24599
phases: make two functions private for phase computation
Siddharth Agarwal <sid0@fb.com> [Wed, 01 Apr 2015 00:31:41 -0700] rev 24598
windows: define normcase spec and fallback
These will be used in upcoming patches to efficiently create a dirstate
foldmap.
Siddharth Agarwal <sid0@fb.com> [Wed, 01 Apr 2015 00:30:41 -0700] rev 24597
encoding.upper: factor out fallback code
This will be used as the fallback function on Windows.
Siddharth Agarwal <sid0@fb.com> [Wed, 01 Apr 2015 00:29:22 -0700] rev 24596
cygwin: define normcase spec and fallback
These will be used in upcoming patches to efficiently create a dirstate
foldmap.
The Cygwin normcase behavior is more complicated than just a simple lowercasing
or uppercasing. That's why we specify 'other'.
Siddharth Agarwal <sid0@fb.com> [Tue, 31 Mar 2015 23:30:19 -0700] rev 24595
darwin: define normcase spec and fallback
These will be used in upcoming patches to efficiently create a dirstate
foldmap.
Siddharth Agarwal <sid0@fb.com> [Wed, 01 Apr 2015 00:26:07 -0700] rev 24594
posix: define normcase spec and fallback
These will be used in upcoming patches to efficiently create a dirstate
foldmap.
Siddharth Agarwal <sid0@fb.com> [Wed, 01 Apr 2015 00:21:10 -0700] rev 24593
encoding: define an enum that specifies what normcase does to ASCII strings
For C code we don't want to pay the cost of calling into a Python function for
the common case of ASCII filenames. However, while on most POSIX platforms we
normalize filenames by lowercasing them, on Windows we uppercase them. We
define an enum here indicating the direction that filenames should be
normalized as. Some platforms (notably Cygwin) have more complicated
normalization behavior -- we add a case for that too.
In upcoming patches we'll also define a fallback function that is called if the
string has non-ASCII bytes.
This enum will be replicated in the C code to make foldmaps. There's
unfortunately no nice way to avoid that -- we can't have encoding import
parsers because of import cycles. One way might be to have parsers import
encoding, but accessing Python modules from C code is just awkward.
The name 'normcasespecs' was chosen to indicate that this is merely an integer
that specifies a behavior, not a function. The name was pluralized since in
upcoming patches we'll introduce 'normcasespec' which will be one of these
values.
Matt Mackall <mpm@selenic.com> [Thu, 02 Apr 2015 16:51:00 -0500] rev 24592
merge with stable
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 01 Apr 2015 22:24:03 -0700] rev 24591
json: implement {help} template
We should consider add HTML rendering of the RST into the response as a
follow-up. I attempted to do this, but there was an empty array
returned by the rstdoc() template function. Not sure what's going on.
Will deal with it later.
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 01 Apr 2015 22:16:05 -0700] rev 24590
json: implement {helptopics} template
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 01 Apr 2015 22:04:03 -0700] rev 24589
json: implement {manifest} template
Property naming was borrowed from `hg files -Tjson`.
We omit branch because, again, representation of branches in this
template is wonky.
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 31 Mar 2015 22:53:48 -0700] rev 24588
json: implement {shortlog} and {changelog} templates
These are the same dispatch function under the hood. The only difference
is the default number of entries to render and the template to use. So
it makes sense to use a shared template.
Format for {changelistentry} is similar to {changeset}. However, there
are differences to argument names and their values preventing us from
(easily) using the same template. (Perhaps there is room to consolidate
the templates as a follow-up.)
We're currently not recording some data in {changelistentry} that exists
in {changeset}. This includes the branch name. This should be added in
a follow-up. For now, something is better than nothing.
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 01 Apr 2015 20:23:58 -0700] rev 24587
help: populate template functions via docstrings
We do this for revsets, template keywrods, and template filters. Now we
do it for template functions as well.
Gregory Szorc <gregory.szorc@gmail.com> [Wed, 01 Apr 2015 20:19:43 -0700] rev 24586
templater: add consistent docstrings to functions
The content of "hg help templating" is largely derived from docstrings
on functions providing functionality. Template functions are the long
holdout.
Prepare for generating them dynamically by defining docstrings for all
template functions.
There are numerous ways these docs could be improved. Right now, the
help output simply shows function names and arguments. So literally
any accurate data is better than what is there now.
Yuya Nishihara <yuya@tcha.org> [Sat, 14 Mar 2015 17:33:22 +0900] rev 24585
changeset_printer: hide manifest node for workingctx
Because workingctx has no manifest, it makes sense to hide "manifest:" row
completely.
Yuya Nishihara <yuya@tcha.org> [Sat, 14 Mar 2015 20:01:30 +0900] rev 24584
changeset_printer: display p1rev:p1node with "+" suffix for workingctx
Still templater can't handle workingctx, which will be fixed later.
Yuya Nishihara <yuya@tcha.org> [Sat, 14 Mar 2015 17:29:48 +0900] rev 24583
changeset_printer: handle workingctx in _meaningful_parentrevs()
Yuya Nishihara <yuya@tcha.org> [Sat, 14 Mar 2015 19:38:59 +0900] rev 24582
scmutil: add function to help handling workingctx in arithmetic operation
It's unfortunate that workingctx revision is None, which doesn't work well in
arithmetic operation or comparison. This function is trivial but will be used
in several places.
Mathias De Maré <mathias.demare@gmail.com> [Thu, 02 Apr 2015 08:18:33 +0200] rev 24581
tests: handle deleted .hg directory (git 2.2.0 and higher) (issue4585)
In git 2.2.0 and higher, removing files and directories is changed:
removing an object that does not exist returns success rather than failure.
As a result, even though .hg/hgrc does not exist, success is returned
and the .hg/ directory is removed.
To handle this correctly, use 'rm -rf' to allow successful removing
for all git versions.
The exact changeset where this was introduced in git:
1054af7d04aef64378d69a0496b45cdbf6a0bef2
wrapper.c: remove/unlink_or_warn: simplify, treat ENOENT as success
Yuya Nishihara <yuya@tcha.org> [Thu, 02 Apr 2015 21:29:05 +0900] rev 24580
hgweb: resurrect <span> tag on diffline to fix rendering in monoblue style
It was removed at 606a3bf82e30 as a useless tag, but it is necessary to
apply "div.diff pre span" style.
http://selenic.com/repo/hg/rev/606a3bf82e30?style=monoblue
Matt Mackall <mpm@selenic.com> [Wed, 01 Apr 2015 20:38:36 -0500] rev 24579
merge with stable
Siddharth Agarwal <sid0@fb.com> [Tue, 31 Mar 2015 15:22:09 -0700] rev 24578
encoding: use parsers.asciiupper when available
This is used on Windows and Cygwin, and the gains from this are expected to be
similar to what was seen in 80f2b63dd83a.
Siddharth Agarwal <sid0@fb.com> [Tue, 31 Mar 2015 13:46:21 -0700] rev 24577
parsers: introduce an asciiupper function
Siddharth Agarwal <sid0@fb.com> [Tue, 31 Mar 2015 10:28:17 -0700] rev 24576
parsers: make _asciilower a generic _asciitransform function
We can now pass in whatever table we like. For example, an upcoming patch will
introduce asciiupper.
Siddharth Agarwal <sid0@fb.com> [Wed, 01 Apr 2015 13:58:51 -0700] rev 24575
parsers._asciilower: use an explicit return object
No functional change, but this will make upcoming patches cleaner.
Siddharth Agarwal <sid0@fb.com> [Tue, 31 Mar 2015 10:25:29 -0700] rev 24574
parsers: factor out most of asciilower into an internal function
We're going to reuse this in upcoming patches.
The change to Py_ssize_t is necessary because parsers.c doesn't define
PY_SSIZE_T_CLEAN. That macro changes the behavior of PyArg_ParseTuple but not
PyBytes_GET_SIZE.
Martin von Zweigbergk <martinvonz@google.com> [Tue, 31 Mar 2015 14:01:33 -0700] rev 24573
manifestv2: add support for writing new manifest format
If .hg/requires has 'manifestv2', the manifest will be written using
the new format.
Martin von Zweigbergk <martinvonz@google.com> [Fri, 27 Mar 2015 22:26:41 -0700] rev 24572
manifestv2: add support for reading new manifest format
The new manifest format is designed to be smaller, in particular to
produce smaller deltas. It stores hashes in binary and puts the hash
on a new line (for smaller deltas). It also uses stem compression to
save space for long paths. The format has room for metadata, but
that's there only for future-proofing. The parser thus accepts any
metadata and throws it away. For more information, see
http://mercurial.selenic.com/wiki/ManifestV2Plan.
The current manifest format doesn't allow an empty filename, so we use
an empty filename on the first line to tell a manifest of the new
format from the old. Since we still never write manifests in the new
format, the added code is unused, but it is tested by
test-manifest.py.
Martin von Zweigbergk <martinvonz@google.com> [Tue, 31 Mar 2015 22:45:45 -0700] rev 24571
manifestv2: set requires at repo creation time
While it should be safe to switch to the new manifest format on an
existing repo, let's keep it simple for now and make the configuration
have any effect only at repo creation time. If the configuration is
enabled then (at repo creation), we add an entry to requires and read
that instead of the configuration from then on.
Martin von Zweigbergk <martinvonz@google.com> [Tue, 31 Mar 2015 15:06:55 -0700] rev 24570
test-manifest: extract constants for binary hashes
The binary hashes are used quite frequently, so let's extract
constants for them so we don't have to repeat binascii.unhexlify() so
often.
Martin von Zweigbergk <martinvonz@google.com> [Tue, 31 Mar 2015 14:46:05 -0700] rev 24569
test-manifest: create constant for empty manifest
For symmetry with manifest v2, once we add that, let's extract a
constant for the empty v1 manifest.
Yuya Nishihara <yuya@tcha.org> [Mon, 30 Mar 2015 23:25:55 +0900] rev 24568
patchbomb: factor out scmutil.revrange() calls
This allows us to access the calculated revs in patchbomb() function.
Yuya Nishihara <yuya@tcha.org> [Tue, 31 Mar 2015 00:52:17 +0900] rev 24567
patchbomb: return outgoing revs as a smartset
This helps to factor out scmutil.revrange() calls from _getpatches() and
_getoutgoing(). In future patches, a smartset will be passed to _getpatches().
Yuya Nishihara <yuya@tcha.org> [Mon, 30 Mar 2015 23:54:29 +0900] rev 24566
templatefilters: add "upper" and "lower" for case conversion
Typically it will be used in patchbomb's flag template, which will be
implemented by future patches.
Durham Goode <durham@fb.com> [Wed, 01 Apr 2015 12:50:10 -0700] rev 24565
repoview: improve compute staticblockers perf
Previously we would compute the repoview's static blockers by finding all the
children of hidden commits that were not hidden. This was O(number of commits
since first hidden change) since 'children' requires walking every commit from
tip until the first hidden change.
The new algorithm walks all heads down until it sees a public commit. This makes
the computation O(number of draft) commits, which is much faster in large
repositories with a large number of commits and a low number of drafts.
On a large repo with 1000+ obsolete markers and the earliest draft commit around
tip~200000, this improves computehidden perf by 200x (2s to 0.01s).
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 31 Mar 2015 22:29:12 -0700] rev 24564
hgweb: add phase to {changeset} template
It's pretty surprising phase wasn't part of this template call already.
We now expose {phase} to the {changeset} template and we expose this
data to JSON.
This brings JSON output in line with the output from `hg log -Tjson`.
The lone exception is hweb doesn't print the numeric rev. As has been
stated previously, I don't believe hgweb should be exposing these
unstable identifiers. (We can add them later if we really want them.)
There is still work to bring hgweb in parity with --verbose and
--debug output from the CLI.
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 31 Mar 2015 22:35:12 -0700] rev 24563
json: implement {changeset} template
Output only contains basic changeset information for the moment. The
format is compatible with `hg log -Tjson`.
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 31 Mar 2015 21:49:10 -0700] rev 24562
test-hgweb-json: fix URL for file revision tests
Likely a copy and paste fail.
Siddharth Agarwal <sid0@fb.com> [Tue, 31 Mar 2015 19:34:37 -0700] rev 24561
dirstate._normalize: don't construct dirfoldmap if not necessary
Constructing the dirfoldmap is expensive, so if there's a hit in the
filefoldmap, don't construct the directory foldmap.
This helps with cases like 'hg add foo' where foo is already tracked: for a
large repository, the operation goes from 1.5 seconds to 1.2 (which is still
way too much, but that's a matter for another day.)
Siddharth Agarwal <sid0@fb.com> [Tue, 31 Mar 2015 19:29:39 -0700] rev 24560
dirstate.walk: don't keep track of normalized files in parallel
Rev 2bb13f2b778c changed the semantics of the work list to store (normalized,
non-normalized) pairs. All the tuple creation and destruction hurts perf: on a
large repo on OS X, 'hg status' went from 3.62 seconds to 3.78.
It also is unnecessary in most cases:
- it is clearly unnecessary on case-sensitive filesystems.
- it is also unnecessary when filenames have been read off of disk rather than
being supplied by the user.
The only case where the non-normalized case is required at all is when the file
is unknown.
To eliminate most of the perf cost, keep trace of whether the directory needs
to be normalized at all with a boolean called 'alreadynormed'. Pay the cost of
directory normalization only when necessary.
For the above large repo, 'hg status' goes to 3.63 seconds.
Siddharth Agarwal <sid0@fb.com> [Tue, 31 Mar 2015 19:18:27 -0700] rev 24559
dirstate.walk: factor out directory traversal
This function will be used in upcoming patches.
Matt Mackall <mpm@selenic.com> [Wed, 01 Apr 2015 13:27:56 -0500] rev 24558
Added signature for changeset 2e2e9a0750f9
Matt Mackall <mpm@selenic.com> [Wed, 01 Apr 2015 13:27:42 -0500] rev 24557
Added tag 3.3.3 for changeset 2e2e9a0750f9
Wagner Bruna <wbruna@softwareexpress.com.br> [Tue, 31 Mar 2015 20:20:17 -0300] rev 24556
i18n-pt_BR: synchronized with d09262d6ec23
Matt Mackall <mpm@selenic.com> [Tue, 31 Mar 2015 18:09:21 -0500] rev 24555
tests: fix py2.4 glob for devel warnings
Matt Mackall <mpm@selenic.com> [Tue, 31 Mar 2015 17:49:46 -0500] rev 24554
merge with stable
Siddharth Agarwal <sid0@fb.com> [Tue, 31 Mar 2015 15:41:02 -0700] rev 24553
dirstate: fix order of initializing nf vs f
Result of a bad merge.
Drew Gottlieb <drgott@google.com> [Mon, 30 Mar 2015 18:10:59 -0700] rev 24552
treemanifest: make treemanifest.matches() faster
By converting treemanifest.matches() into a recursively additivie operation,
it becomes O(n).
The old matches function made a copy of the entire manifest and deleted
files that didn't match. With tree manifests, this was an O(n log n) operation
because del() was O(log n).
This change speeds up the command
"hg status --rev .^ 'relglob:*.js'
on the Mozilla repo, now taking 2.53s, down from 3.51s.
Drew Gottlieb <drgott@google.com> [Mon, 30 Mar 2015 17:21:49 -0700] rev 24551
treemanifest: add treemanifest._isempty()
During operations that involve building up a new manifest tree, it will be
useful to be able to quickly check if a submanifest is empty, and if so, to
avoid including it in the final tree. Doing this check lets us avoid creating
treemanifest structures that contain any empty submanifests.
Drew Gottlieb <drgott@google.com> [Fri, 27 Mar 2015 13:16:13 -0700] rev 24550
treemanifest: remove treemanifest._intersectfiles()
In preparation for the optimization in the following commit, this commit
removes treemanifest.matches()'s call to _intersectfiles(), and removes
_intersectfiles() itself since it's unused at this point.
Drew Gottlieb <drgott@google.com> [Mon, 30 Mar 2015 11:58:39 -0700] rev 24549
manifest: add some tests for manifest.matches()
There were no tests for the various code paths in manifestdict.matches(), so I
added some. This also adds a more complex testing manifest so that any bugs
relating to traversal of directories are more likely to be caught.
Matt Harbison <matt_harbison@yahoo.com> [Tue, 31 Mar 2015 17:42:46 -0400] rev 24548
forget: cleanup the output for an inexact case match on icasefs
Previously, specifying a file name but not matching the dirstate case yielded
the following, even though the file was actually removed:
$ hg forget capsdir1/capsdir/abc.txt
not removing capsdir\a.txt: file is already untracked
removing CapsDir\A.txt
[1]
This change doesn't appear to cause any extra filesystem accesses, even if a
nonexistant file is specified.
If a directory is specified without a case match, it is (and was previously)
still silently ignored.
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 30 Mar 2015 21:37:24 -0700] rev 24547
json: implement {branches} template
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 31 Mar 2015 14:54:56 -0700] rev 24546
json: implement {bookmarks} template
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 31 Mar 2015 14:52:21 -0700] rev 24545
json: implement {tags} template
Tags is pretty easy to implement. Let's start there.
The output is slightly different from `hg tags -Tjson`. For reference,
the CLI has the following output:
[
{
"node": "e2049974f9a23176c2addb61d8f5b86e0d620490",
"rev": 29880,
"tag": "tip",
"type": ""
},
...
]
Our output has the format:
{
"node": "0aeb19ea57a6d223bacddda3871cb78f24b06510",
"tags": [
{
"node": "e2049974f9a23176c2addb61d8f5b86e0d620490",
"tag": "tag1",
"date": [1427775457.0, 25200]
},
...
]
}
"rev" is omitted because it isn't a reliable identifier. We shouldn't
be exposing them in web APIs and giving the impression it remotely
resembles a stable identifier. Perhaps we could one day hide this behind
a config option (it might be useful to expose when running servers
locally).
The "type" of the tag isn't defined because this information isn't yet
exposed to the hgweb templater (it could be in a follow-up) and because
it is questionable whether different types should be exposed at all.
(Should the web interface really be exposing "local" tags?)
We use an object for the outer type instead of Array for a few reasons.
First, it is extensible. If we ever need to throw more global properties
into the output, we can do that without breaking backwards compatibility
(property additions should be backwards compatible). Second, uniformity
in web APIs is nice. Having everything return objects seems much saner than
a mix of array and object. Third, there are security issues with arrays
in older browsers. The JSON web services world almost never uses arrays
as the main type for this reason.
Another possibly controversial part about this patch is how dates are
defined. While JSON has a Date type, it is based on the JavaScript Date
type, which is widely considered a pile of garbage. It is a non-starter
for this reason.
Many of Mercurial's built-in date filters drop seconds resolution. So
that's a non-starter as well, since we want the API to be lossless where
possible. rfc3339date, rfc822date, isodatesec, and date are all lossless.
However, they each require the client to perform string parsing on top of
JSON decoding. While date parsing libraries are pretty ubiquitous, some
languages don't have them out of the box. However, pretty much every
programming language can deal with UNIX timestamps (which are just
integers or floats). So, we choose to use Mercurial's internal date
representation, which in JSON is modeled as float seconds since UNIX
epoch and an integer timezone offset from UTC (keep in mind
JavaScript/JSON models all "Numbers" as double prevision floating point
numbers, so there isn't a difference between ints and floats in JSON).
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 30 Mar 2015 20:15:03 -0700] rev 24544
templates: add a stub template for json
Many have long wanted hgweb to emit a common machine readable output.
We start the process by defining a stub json template.
Right now, each endpoint returns a stub "not yet implemented" string.
Individual templates will be implemented in subsequent patches.
Basic tests for templates have been included. Coverage isn't perfect,
but it is better than nothing.
Gregory Szorc <gregory.szorc@gmail.com> [Mon, 30 Mar 2015 20:56:54 -0700] rev 24543
get-with-headers: support parsing and pretty printing JSON
Upcoming patches will add support for JSON output from hgweb.
Because JSON output from the templater is hard to read and because it
is easy to introduce malformed JSON, we introduce a JSON processing
mode to get-with-headers.py that will parse and pretty print JSON
from HTTP responses. This will make tests easier to read and write
and it will ensure hgweb is emitting well-formed JSON.