setdiscovery: precompute children revisions to avoid quadratic lookup
Moving away from dagutil a few commits ago introduced quadratic
behavior when resolving children revisions during discovery.
This commit introduces a precompute step of the children revisions
to avoid the bad behavior.
I believe the new code should have near identical performance to
what dagutil was doing before. Behavior is still slightly different
because we take into account filtered revisions. But this change was
made when we moved off dagutil.
I added a comment about multiple invocations of this function
redundantly calculating the children revisions. I believe this
potentially undesirable behavior was present when we used dagutil,
as the call to inverse() previously in this function created a new
object and required computing children on every invocation. I thought
we should document the potential for a performance issue rather than
let it go undocumented.
Differential Revision: https://phab.mercurial-scm.org/D4326
--- a/mercurial/setdiscovery.py Fri Aug 17 19:24:36 2018 +0000
+++ b/mercurial/setdiscovery.py Fri Aug 17 19:35:24 2018 +0000
@@ -117,13 +117,33 @@
# update from heads
revsheads = set(repo.revs('heads(%ld)', revs))
_updatesample(revs, revsheads, sample, repo.changelog.parentrevs)
+
# update from roots
revsroots = set(repo.revs('roots(%ld)', revs))
- # TODO this is quadratic
- parentfn = lambda rev: repo.changelog.children(repo.changelog.node(rev))
+ # _updatesample() essentially does interaction over revisions to look up
+ # their children. This lookup is expensive and doing it in a loop is
+ # quadratic. We precompute the children for all relevant revisions and
+ # make the lookup in _updatesample() a simple dict lookup.
+ #
+ # Because this function can be called multiple times during discovery, we
+ # may still perform redundant work and there is room to optimize this by
+ # keeping a persistent cache of children across invocations.
+ children = {}
- _updatesample(revs, revsroots, sample, parentfn)
+ parentrevs = repo.changelog.parentrevs
+ for rev in repo.changelog.revs(start=min(revsroots)):
+ # Always ensure revision has an entry so we don't need to worry about
+ # missing keys.
+ children.setdefault(rev, [])
+
+ for prev in parentrevs(rev):
+ if prev == nullrev:
+ continue
+
+ children.setdefault(prev, []).append(rev)
+
+ _updatesample(revs, revsroots, sample, children.__getitem__)
assert sample
sample = _limitsample(sample, size)
if len(sample) < size: