context: write dirstate out explicitly after marking files as clean To detect change of a file without redundant comparison of file content, dirstate recognizes a file as certainly clean, if: (1) it is already known as "normal", (2) dirstate entry for it has valid (= not "-1") timestamp, and (3) mode, size and timestamp of it on the filesystem are as same as ones expected in dirstate This works as expected in many cases, but doesn't in the corner case that changing a file keeps mode, size and timestamp of it on the filesystem. The timetable below shows steps in one of typical such situations: ---- ----------------------------------- ---------------- timestamp of "f" ---------------- dirstate file- time action mem file system ---- ----------------------------------- ---- ----- ----- N -1 *** - make file "f" clean N - execute 'hg foobar' - instantiate 'dirstate' -1 -1 - 'dirstate.normal("f")' N -1 (e.g. via dirty check) - change "f", but keep size N N+1 - release wlock - 'dirstate.write()' N N - 'hg status' shows "f" as "clean" N N N ---- ----------------------------------- ---- ----- ----- The most important point is that 'dirstate.write()' is executed at N+1 or later. This causes writing dirstate timestamp N of "f" out successfully. If it is executed at N, 'parsers.pack_dirstate()' replaces timestamp N with "-1" before actual writing dirstate out. Occasional test failure for unexpected file status is typical example of this corner case. Batch execution with small working directory is finished in no time, and rarely satisfies condition (2) above. This issue can occur in cases below; - 'hg revert --rev REV' for revisions other than the parent - failure of 'merge.update()' before 'merge.recordupdates()' The root cause of this issue is that files are changed without flushing in-memory dirstate changes via 'repo.commit()' (even though omitting 'dirstate.normallookup()' on changed files also causes this issue). To detect changes of files correctly, this patch writes in-memory dirstate changes out explicitly after marking files as clean in 'workingctx._checklookup()', which is invoked via 'repo.status()'. After this change, timetable is changed as below: ---- ----------------------------------- ---------------- timestamp of "f" ---------------- dirstate file- time action mem file system ---- ----------------------------------- ---- ----- ----- N -1 *** - make file "f" clean N - execute 'hg foobar' - instantiate 'dirstate' -1 -1 - 'dirstate.normal("f")' N -1 (e.g. via dirty check) ----------------------------------- ---- ----- ----- - 'dirsttate.write()' -1 -1 ----------------------------------- ---- ----- ----- - change "f", but keep size N N+1 - release wlock - 'dirstate.write()' -1 -1 - 'hg status' -1 -1 N ---- ----------------------------------- ---- ----- ----- To reproduce this issue in tests certainly, this patch emulates some timing critical actions as below: - timestamp of "f" in '.hg/dirstate' is -1 at the beginning 'hg debugrebuildstate' before command invocation ensures it. - make file "f" clean at N - change "f" at N 'touch -t 200001010000' before and after command invocation changes mtime of "f" to "2000-01-01 00:00" (= N). - invoke 'dirstate.write()' via 'repo.status()' at N 'fakedirstatewritetime.py' forces 'pack_dirstate()' to use "2000-01-01 00:00" as "now", only if 'pack_dirstate()' is invoked via 'workingctx._checklookup()'. - invoke 'dirstate.write()' via releasing wlock at N+1 (or "not at N") 'pack_dirstate()' via releasing wlock uses actual timestamp at runtime as "now", and it should be different from the "2000-01-01 00:00" of "f". BTW, this patch also changes 'test-largefiles-misc.t', because adding 'dirstate.write()' makes recent dirstate changes visible to external process.

     1

  $ cat <<EOF > merge

     2

  > import sys, os

     3

  > 

     4

  > try:

     5

  >     import msvcrt

     6

  >     msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)

     7

  >     msvcrt.setmode(sys.stderr.fileno(), os.O_BINARY)

     8

  > except ImportError:

     9

  >     pass

    10

  > 

    11

  > print "merging for", os.path.basename(sys.argv[1])

    12

  > EOF

    13

  $ HGMERGE="python ../merge"; export HGMERGE

    14

    15

  $ hg init t

    16

  $ cd t

    17

  $ echo This is file a1 > a

    18

  $ hg add a

    19

  $ hg commit -m "commit #0"

    20

  $ echo This is file b1 > b

    21

  $ hg add b

    22

  $ hg commit -m "commit #1"

    23

    24

  $ hg update 0

    25

  0 files updated, 0 files merged, 1 files removed, 0 files unresolved

    26

    27

Test interrupted updates by exploiting our non-handling of directory collisions

    28

    29

  $ mkdir b

    30

  $ hg up

    31

  abort: *: '$TESTTMP/t/b' (glob)

    32

  [255]

    33

  $ hg ci

    34

  abort: last update was interrupted

    35

  (use 'hg update' to get a consistent checkout)

    36

  [255]

    37

  $ hg sum

    38

  parent: 0:538afb845929 

    39

   commit #0

    40

  branch: default

    41

  commit: (interrupted update)

    42

  update: 1 new changesets (update)

    43

  phases: 2 draft

    44

  $ rmdir b

    45

  $ hg up

    46

  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

    47

  $ hg sum

    48

  parent: 1:b8bb4a988f25 tip

    49

   commit #1

    50

  branch: default

    51

  commit: (clean)

    52

  update: (current)

    53

  phases: 2 draft

    54

    55

Prepare a basic merge

    56

    57

  $ hg up 0

    58

  0 files updated, 0 files merged, 1 files removed, 0 files unresolved

    59

  $ echo This is file c1 > c

    60

  $ hg add c

    61

  $ hg commit -m "commit #2"

    62

  created new head

    63

  $ echo This is file b1 > b

    64

no merges expected

    65

  $ hg merge -P 1

    66

  changeset:   1:b8bb4a988f25

    67

  user:        test

    68

  date:        Thu Jan 01 00:00:00 1970 +0000

    69

  summary:     commit #1

    70

    71

  $ hg merge 1

    72

  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

    73

  (branch merge, don't forget to commit)

    74

  $ hg diff --nodates

    75

  diff -r 49035e18a8e6 b

    76

  --- /dev/null

    77

  +++ b/b

    78

  @@ -0,0 +1,1 @@

    79

  +This is file b1

    80

  $ hg status

    81

  M b

    82

  $ cd ..; rm -r t

    83

    84

  $ hg init t

    85

  $ cd t

    86

  $ echo This is file a1 > a

    87

  $ hg add a

    88

  $ hg commit -m "commit #0"

    89

  $ echo This is file b1 > b

    90

  $ hg add b

    91

  $ hg commit -m "commit #1"

    92

    93

  $ hg update 0

    94

  0 files updated, 0 files merged, 1 files removed, 0 files unresolved

    95

  $ echo This is file c1 > c

    96

  $ hg add c

    97

  $ hg commit -m "commit #2"

    98

  created new head

    99

  $ echo This is file b2 > b

   100

merge should fail

   101

  $ hg merge 1

   102

  b: untracked file differs

   103

  abort: untracked files in working directory differ from files in requested revision

   104

  [255]

   105

merge of b expected

   106

  $ hg merge -f 1

   107

  merging b

   108

  merging for b

   109

  0 files updated, 1 files merged, 0 files removed, 0 files unresolved

   110

  (branch merge, don't forget to commit)

   111

  $ hg diff --nodates

   112

  diff -r 49035e18a8e6 b

   113

  --- /dev/null

   114

  +++ b/b

   115

  @@ -0,0 +1,1 @@

   116

  +This is file b2

   117

  $ hg status

   118

  M b

   119

  $ cd ..; rm -r t

   120

   121

  $ hg init t

   122

  $ cd t

   123

  $ echo This is file a1 > a

   124

  $ hg add a

   125

  $ hg commit -m "commit #0"

   126

  $ echo This is file b1 > b

   127

  $ hg add b

   128

  $ hg commit -m "commit #1"

   129

  $ echo This is file b22 > b

   130

  $ hg commit -m "commit #2"

   131

  $ hg update 1

   132

  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

   133

  $ echo This is file c1 > c

   134

  $ hg add c

   135

  $ hg commit -m "commit #3"

   136

  created new head

   137

   138

Contents of b should be "this is file b1"

   139

  $ cat b

   140

  This is file b1

   141

   142

  $ echo This is file b22 > b

   143

merge fails

   144

  $ hg merge 2

   145

  abort: uncommitted changes

   146

  (use 'hg status' to list changes)

   147

  [255]

   148

merge expected!

   149

  $ hg merge -f 2

   150

  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

   151

  (branch merge, don't forget to commit)

   152

  $ hg diff --nodates

   153

  diff -r 85de557015a8 b

   154

  --- a/b

   155

  +++ b/b

   156

  @@ -1,1 +1,1 @@

   157

  -This is file b1

   158

  +This is file b22

   159

  $ hg status

   160

  M b

   161

  $ cd ..; rm -r t

   162

   163

  $ hg init t

   164

  $ cd t

   165

  $ echo This is file a1 > a

   166

  $ hg add a

   167

  $ hg commit -m "commit #0"

   168

  $ echo This is file b1 > b

   169

  $ hg add b

   170

  $ hg commit -m "commit #1"

   171

  $ echo This is file b22 > b

   172

  $ hg commit -m "commit #2"

   173

  $ hg update 1

   174

  1 files updated, 0 files merged, 0 files removed, 0 files unresolved

   175

  $ echo This is file c1 > c

   176

  $ hg add c

   177

  $ hg commit -m "commit #3"

   178

  created new head

   179

  $ echo This is file b33 > b

   180

merge of b should fail

   181

  $ hg merge 2

   182

  abort: uncommitted changes

   183

  (use 'hg status' to list changes)

   184

  [255]

   185

merge of b expected

   186

  $ hg merge -f 2

   187

  merging b

   188

  merging for b

   189

  0 files updated, 1 files merged, 0 files removed, 0 files unresolved

   190

  (branch merge, don't forget to commit)

   191

  $ hg diff --nodates

   192

  diff -r 85de557015a8 b

   193

  --- a/b

   194

  +++ b/b

   195

  @@ -1,1 +1,1 @@

   196

  -This is file b1

   197

  +This is file b33

   198

  $ hg status

   199

  M b

   200

   201

Test for issue2364

   202

   203

  $ hg up -qC .

   204

  $ hg rm b

   205

  $ hg ci -md

   206

  $ hg revert -r -2 b

   207

  $ hg up -q -- -2

   208

   209

Test that updated files are treated as "modified", when

   210

'merge.update()' is aborted before 'merge.recordupdates()' (= parents

   211

aren't changed), even if none of mode, size and timestamp of them

   212

isn't changed on the filesystem (see also issue4583).

   213

   214

  $ cat > $TESTTMP/abort.py <<EOF

   215

  > # emulate aborting before "recordupdates()". in this case, files

   216

  > # are changed without updating dirstate

   217

  > from mercurial import extensions, merge, util

   218

  > def applyupdates(orig, *args, **kwargs):

   219

  >     orig(*args, **kwargs)

   220

  >     raise util.Abort('intentional aborting')

   221

  > def extsetup(ui):

   222

  >     extensions.wrapfunction(merge, "applyupdates", applyupdates)

   223

  > EOF

   224

   225

  $ cat >> .hg/hgrc <<EOF

   226

  > [fakedirstatewritetime]

   227

  > # emulate invoking dirstate.write() via repo.status()

   228

  > # at 2000-01-01 00:00

   229

  > fakenow = 200001010000

   230

  > EOF

   231

   232

(file gotten from other revision)

   233

   234

  $ hg update -q -C 2

   235

  $ echo 'THIS IS FILE B5' > b

   236

  $ hg commit -m 'commit #5'

   237

   238

  $ hg update -q -C 3

   239

  $ cat b

   240

  This is file b1

   241

  $ touch -t 200001010000 b

   242

  $ hg debugrebuildstate

   243

   244

  $ cat >> .hg/hgrc <<EOF

   245

  > [extensions]

   246

  > fakedirstatewritetime = $TESTDIR/fakedirstatewritetime.py

   247

  > abort = $TESTTMP/abort.py

   248

  > EOF

   249

  $ hg merge 5

   250

  abort: intentional aborting

   251

  [255]

   252

  $ cat >> .hg/hgrc <<EOF

   253

  > [extensions]

   254

  > fakedirstatewritetime = !

   255

  > abort = !

   256

  > EOF

   257

   258

  $ cat b

   259

  THIS IS FILE B5

   260

  $ touch -t 200001010000 b

   261

  $ hg status -A b

   262

  M b

   263

   264

(file merged from other revision)

   265

   266

  $ hg update -q -C 3

   267

  $ echo 'this is file b6' > b

   268

  $ hg commit -m 'commit #6'

   269

  created new head

   270

   271

  $ cat b

   272

  this is file b6

   273

  $ touch -t 200001010000 b

   274

  $ hg debugrebuildstate

   275

   276

  $ cat >> .hg/hgrc <<EOF

   277

  > [extensions]

   278

  > fakedirstatewritetime = $TESTDIR/fakedirstatewritetime.py

   279

  > abort = $TESTTMP/abort.py

   280

  > EOF

   281

  $ hg merge --tool internal:other 5

   282

  abort: intentional aborting

   283

  [255]

   284

  $ cat >> .hg/hgrc <<EOF

   285

  > [extensions]

   286

  > fakedirstatewritetime = !

   287

  > abort = !

   288

  > EOF

   289

   290

  $ cat b

   291

  THIS IS FILE B5

   292

  $ touch -t 200001010000 b

   293

  $ hg status -A b

   294

  M b

   295

   296

  $ cd ..