largefiles: more safe handling of interruptions while updating modifications stable
authorMads Kiilerich <madski@unity3d.com>
Sun, 16 Oct 2016 02:29:45 +0200
branchstable
changeset 30190 56b930238036
parent 30189 4999c12c526b
child 30191 328545c7d8a1
largefiles: more safe handling of interruptions while updating modifications Largefiles are fragile with the design where dirstate and lfdirstate must be kept in sync. To be less fragile, mark all clean largefiles as unsure ("normallookup") before updating standins. After standins have been updated and we know exactly which largefile standins actually was changed, mark the unchanged largefiles back to clean ("normal"). This will make the failure mode more safe. If interrupted, the next command will continue to perform extra hashing of all largefiles. That will do that all largefiles that are out of sync with their standin will be marked dirty and they will show up in status and can be cleaned with update --clean.
hgext/largefiles/overrides.py
tests/test-largefiles-update.t
--- a/hgext/largefiles/overrides.py	Sun Oct 16 02:26:38 2016 +0200
+++ b/hgext/largefiles/overrides.py	Sun Oct 16 02:29:45 2016 +0200
@@ -1390,7 +1390,8 @@
         lfdirstate = lfutil.openlfdirstate(repo.ui, repo)
         unsure, s = lfdirstate.status(matchmod.always(repo.root,
                                                     repo.getcwd()),
-                                      [], False, False, False)
+                                      [], False, True, False)
+        oldclean = set(s.clean)
         pctx = repo['.']
         for lfile in unsure + s.modified:
             lfileabs = repo.wvfs.join(lfile)
@@ -1402,9 +1403,13 @@
                                 lfutil.getexecutable(lfileabs))
             if (standin in pctx and
                 lfhash == lfutil.readstandin(repo, lfile, '.')):
-                lfdirstate.normal(lfile)
+                oldclean.add(lfile)
         for lfile in s.added:
             lfutil.updatestandin(repo, lfutil.standin(lfile))
+        # mark all clean largefiles as dirty, just in case the update gets
+        # interrupted before largefiles and lfdirstate are synchronized
+        for lfile in oldclean:
+            lfdirstate.normallookup(lfile)
         lfdirstate.write()
 
         oldstandins = lfutil.getstandinsstate(repo)
@@ -1413,6 +1418,13 @@
 
         newstandins = lfutil.getstandinsstate(repo)
         filelist = lfutil.getlfilestoupdate(oldstandins, newstandins)
+
+        # to avoid leaving all largefiles as dirty and thus rehash them, mark
+        # all the ones that didn't change as clean
+        for lfile in oldclean.difference(filelist):
+            lfdirstate.normal(lfile)
+        lfdirstate.write()
+
         if branchmerge or force or partial:
             filelist.extend(s.deleted + s.removed)
 
--- a/tests/test-largefiles-update.t	Sun Oct 16 02:26:38 2016 +0200
+++ b/tests/test-largefiles-update.t	Sun Oct 16 02:29:45 2016 +0200
@@ -732,16 +732,16 @@
 
 #endif
 
-Test a fatal error interrupting an update. lfdirstate doesn't realize that
-.hglf has been updated while the largefile hasn't. Status thus shows a clean
-state ... but rebuilding lfdirstate and checking all hashes reveals it isn't
-clean.
+Test a fatal error interrupting an update. Verify that status report dirty
+files correctly after an interrupted update. Also verify that checking all
+hashes reveals it isn't clean.
 
 Start with clean dirstates:
   $ hg up -qcr "8^"
   $ sleep 1
   $ hg st
-Update standins without updating largefiles:
+Update standins without updating largefiles - large1 is modified and largeX is
+added:
   $ cat << EOF > ../crashupdatelfiles.py
   > import hgext.largefiles.lfutil
   > def getlfilestoupdate(oldstandins, newstandins):
@@ -750,29 +750,31 @@
   > EOF
   $ hg up -Cr "8" --config extensions.crashupdatelfiles=../crashupdatelfiles.py
   [7]
-Check large1 content and status:
-BUG: largeX is R and large1 is not M and update does nothing
+Check large1 content and status ... and that update will undo modifications:
+BUG: large is R
   $ cat large1
   large1 in #3
   $ hg st
+  M large1
   R largeX
   $ hg up -Cr .
-  0 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  getting changed largefiles
+  1 largefiles updated, 0 removed
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ cat large1
+  manually modified before 'hg transplant --continue'
   $ hg st
   R largeX
-Force largefiles rehashing and check again - revealing modifications that
-update now can remove:
+Force largefiles rehashing and check again - which makes it realize that largeX
+not has been removed but just doesn't exist:
   $ rm .hg/largefiles/dirstate
   $ hg st
-  M large1
   ! largeX
   $ hg up -Cr .
   getting changed largefiles
-  2 largefiles updated, 0 removed
-  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  1 largefiles updated, 0 removed
+  0 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ hg st
-  $ cat large1
-  manually modified before 'hg transplant --continue'
 
   $ cd ..