revset: speed up '_matchfiles'
authorPierre-Yves David <pierre-yves.david@fb.com>
Wed, 18 Nov 2015 23:23:03 -0800
changeset 27028 f92053df8f0b
parent 27027 a01ecbcfaf84
child 27029 8279c5d116a0
revset: speed up '_matchfiles' File matching is done by applying the matcher to all elements in the 'file' field of all changesets in the repository. This requires to read/parse all changesets in the repository and do a lot of matching. However about 1/3 of the time of the function is used to create 'changectx' object and retrieve their 'file' field. This is far too much overhead so we are skipping the changectx layer and directly access the data from the changelog. This provide use significant speed up: repository: mozilla central 252524 revisions command: hg perfrevset '_matchfiles("p:browser")' Before: 15.899687s After: 10.011705s Slowdown is even more significant if you have a lot of namespace that slowdown lookup. The time is now spent with this approximate repartition: Matcher: 20% regexp matching: 10% changelog.read: 80% reading revision: 60% checking hash: 15% decompression: 15% reading chunk: 30% changelog parsing: 20% decoding to local: 10% The next easy win is probably to have more of the changelog stack implemented using the CPython api.
mercurial/revset.py
--- a/mercurial/revset.py	Wed Nov 18 15:46:45 2015 -0800
+++ b/mercurial/revset.py	Wed Nov 18 23:23:03 2015 -0800
@@ -1164,8 +1164,16 @@
     m = matchmod.match(repo.root, repo.getcwd(), pats, include=inc,
                        exclude=exc, ctx=repo[rev], default=default)
 
+    # This directly read the changelog data as creating changectx for all
+    # revisions is quite expensive.
+    getchangeset = repo.changelog.read
+    wdirrev = node.wdirrev
     def matches(x):
-        for f in repo[x].files():
+        if x == wdirrev:
+            files = repo[x].files()
+        else:
+            files = getchangeset(x)[3]
+        for f in files:
             if m(f):
                 return True
         return False