json: implement {bookmarks} template

json: implement {tags} template Tags is pretty easy to implement. Let's start there. The output is slightly different from `hg tags -Tjson`. For reference, the CLI has the following output: [ { "node": "e2049974f9a23176c2addb61d8f5b86e0d620490", "rev": 29880, "tag": "tip", "type": "" }, ... ] Our output has the format: { "node": "0aeb19ea57a6d223bacddda3871cb78f24b06510", "tags": [ { "node": "e2049974f9a23176c2addb61d8f5b86e0d620490", "tag": "tag1", "date": [1427775457.0, 25200] }, ... ] } "rev" is omitted because it isn't a reliable identifier. We shouldn't be exposing them in web APIs and giving the impression it remotely resembles a stable identifier. Perhaps we could one day hide this behind a config option (it might be useful to expose when running servers locally). The "type" of the tag isn't defined because this information isn't yet exposed to the hgweb templater (it could be in a follow-up) and because it is questionable whether different types should be exposed at all. (Should the web interface really be exposing "local" tags?) We use an object for the outer type instead of Array for a few reasons. First, it is extensible. If we ever need to throw more global properties into the output, we can do that without breaking backwards compatibility (property additions should be backwards compatible). Second, uniformity in web APIs is nice. Having everything return objects seems much saner than a mix of array and object. Third, there are security issues with arrays in older browsers. The JSON web services world almost never uses arrays as the main type for this reason. Another possibly controversial part about this patch is how dates are defined. While JSON has a Date type, it is based on the JavaScript Date type, which is widely considered a pile of garbage. It is a non-starter for this reason. Many of Mercurial's built-in date filters drop seconds resolution. So that's a non-starter as well, since we want the API to be lossless where possible. rfc3339date, rfc822date, isodatesec, and date are all lossless. However, they each require the client to perform string parsing on top of JSON decoding. While date parsing libraries are pretty ubiquitous, some languages don't have them out of the box. However, pretty much every programming language can deal with UNIX timestamps (which are just integers or floats). So, we choose to use Mercurial's internal date representation, which in JSON is modeled as float seconds since UNIX epoch and an integer timezone offset from UTC (keep in mind JavaScript/JSON models all "Numbers" as double prevision floating point numbers, so there isn't a difference between ints and floats in JSON).

templates: add a stub template for json Many have long wanted hgweb to emit a common machine readable output. We start the process by defining a stub json template. Right now, each endpoint returns a stub "not yet implemented" string. Individual templates will be implemented in subsequent patches. Basic tests for templates have been included. Coverage isn't perfect, but it is better than nothing.

get-with-headers: support parsing and pretty printing JSON Upcoming patches will add support for JSON output from hgweb. Because JSON output from the templater is hard to read and because it is easy to introduce malformed JSON, we introduce a JSON processing mode to get-with-headers.py that will parse and pretty print JSON from HTTP responses. This will make tests easier to read and write and it will ensure hgweb is emitting well-formed JSON.

merge with stable

dirstate.walk: use the file foldmap to normalize Computing the set of directories in the dirstate is expensive. It turns out that it isn't necessary for operations like 'hg status' at all. Why? Consider the file 'foo/bar' on disk, which is represented in the dirstate as 'FOO/BAR'. On 'hg status', we'd walk down the directory tree, coming across 'foo' first. Before: we'd normalize 'foo' to 'FOO', then add 'FOO' to our visited stack. We'd then visit 'FOO', finding the file 'bar'. We'd normalize 'FOO/bar' to 'FOO/BAR', then add it to the results dict. After: we wouldn't normalize 'foo' at all. We'd add it to our visited stack, then visit 'foo', finding the file 'bar'. We'd normalize 'foo/bar' to 'FOO/BAR', then add it to the results dict. So whether we normalize intermediate directories or not actually makes no difference in most cases. The only case where normalization matters at all is if a file is replaced with a directory with the same case-folded name. In that case we can do a relatively cheap file normalization instead and still get away with not computing the set of directories. This is a nice boost in status performance. On OS X with case-insensitive HFS+, for a large repo with over 200,000 files, this brings down 'hg status' from 4.00 seconds to 3.62.

dirstate: split the foldmap into separate ones for files and directories Computing the set of directories in the dirstate can be pretty expensive. For 'hg status' without arguments, it turns out we actually never need to figure out the right case for directories in the foldmap. (An upcoming patch explains why.) This patch splits up the directory and file maps into separate ones, allowing for the subsequent optimization in status.

dirstate: introduce function to normalize just filenames This will be used in upcoming patches to stop generating the set of directories in many common cases.

dirstate: factor out code to discover normalized path In upcoming patches we're going to reuse this code. The storemap is currently always the foldmap, but will vary in future patches.

dirstate: don't require exact case when adding dirs on icasefs (issue4578) We don't require it when adding files on a case insensitive filesystem, so don't require it to add directories for consistency. The problem with the previous code was that _walkexplicit() was only returning the normalized directory. The file(s) in the directory are then appended, and passed to the matcher. But if the user asks for 'capsdir1/capsdir', the matcher will not accept 'CapsDir1/CapsDir/AbC.txt', and the name is dropped. Matching based on the non-normalized name is required. If not normalizing, skip the extra string building for efficiency. '.' is replaced with '' so that the path being tested when no file is specified, isn't prefixed with './' (and therefore fail the match).