mod_http_upload_external/README.markdown
author Matthew Wild <mwild1@gmail.com>
Sun, 21 Oct 2018 14:54:19 +0100
changeset 3362 e49660ba3161
parent 3193 57332ea0c1c7
child 3363 3d01ab6b1186
permissions -rw-r--r--
mod_http_upload_external: Improve implementation docs, including v2 details
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
2338
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
     1
---
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
     2
description: HTTP File Upload (external service)
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
     3
labels: 'Stage-Alpha'
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
     4
---
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
     5
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
     6
Introduction
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
     7
============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
     8
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
     9
This module implements [XEP-0363], which lets clients upload files
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    10
over HTTP to an external web server.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    11
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    12
This module generates URLs that are signed using a HMAC. Any web service that can authenticate
2827
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2338
diff changeset
    13
these URLs can be used. 
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2338
diff changeset
    14
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2338
diff changeset
    15
Implementations
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2338
diff changeset
    16
---------------
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2338
diff changeset
    17
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2338
diff changeset
    18
* [PHP implementation](https://hg.prosody.im/prosody-modules/raw-file/tip/mod_http_upload_external/share.php)
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2338
diff changeset
    19
* [Python3+Flask implementation](https://github.com/horazont/xmpp-http-upload)
3172
73a610c3c7a9 mod_http_external: Link to prosody-filer (Go implementation)
Matthew Wild <mwild1@gmail.com>
parents: 2827
diff changeset
    20
* [Go implementation, Prosody Filer](https://github.com/ThomasLeister/prosody-filer)
3193
57332ea0c1c7 mod_http_upload_external/README: Add Perl implementation by Holger to list
Kim Alvefur <zash@zash.se>
parents: 3172
diff changeset
    21
* [Perl implementation for nginx](https://github.com/weiss/ngx_http_upload)
2827
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2338
diff changeset
    22
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2338
diff changeset
    23
To implement your own service compatible with this module, check out the implementation notes below 
f14bea5da323 mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents: 2338
diff changeset
    24
(and if you publish your implementation - let us know!).
2338
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    25
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    26
Configuration
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    27
=============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    28
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    29
Add `"http_upload_external"` to modules_enabled in your global section, or under the host(s) you wish
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    30
to use it on.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    31
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    32
External URL
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    33
------------
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    34
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    35
You need to provide the path to the external service. Ensure it ends with '/'.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    36
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    37
For example, to use the PHP implementation linked above, you might set it to:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    38
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    39
``` {.lua}
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    40
http_upload_external_base_url = "https://your.example.com/path/to/share.php/"
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    41
```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    42
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    43
Secret
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    44
------
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    45
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    46
Set a long and unpredictable string as your secret. This is so the upload service can verify that
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    47
the upload comes from mod_http_upload_external, and random strangers can't upload to your server.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    48
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    49
``` {.lua}
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    50
http_upload_external_secret = "this is a secret string!"
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    51
```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    52
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    53
You need to set exactly the same secret string in your external service.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    54
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    55
Limits
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    56
------
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    57
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    58
A maximum file size can be set by:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    59
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    60
``` {.lua}
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    61
http_upload_external_file_size_limit = 123 -- bytes
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    62
```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    63
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    64
Default is 100MB (100\*1024\*1024).
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    65
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    66
Compatibility
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    67
=============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    68
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    69
Works with Prosody 0.9.x and later.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    70
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    71
Implementation
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    72
==============
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    73
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    74
To implement your own external service that is compatible with this module, you need to expose a
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    75
simple API that allows the HTTP GET, HEAD and PUT methods on arbitrary URLs located on your service.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    76
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    77
For example, if http_upload_external_base_url is set to `https://example.com/upload/` then your service
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    78
might receive the following requests:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    79
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    80
Upload a new file:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    81
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    82
```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    83
PUT https://example.com/upload/foo/bar.jpg?v=49e9309ff543ace93d25be90635ba8e9965c4f23fc885b2d86c947a5d59e55b2
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    84
```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    85
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    86
Recipient checks the file size and other headers:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    87
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    88
```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    89
HEAD https://example.com/upload/foo/bar.jpg
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    90
```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    91
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    92
Recipient downloads the file:
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    93
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    94
```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    95
GET https://example.com/upload/foo/bar.jpg
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    96
```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    97
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    98
The only tricky logic is in validation of the PUT request. Firstly, don't overwrite existing files (return 409 Conflict).
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
    99
3362
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   100
Then you need to validate the auth token.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   101
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   102
### Validating the auth token
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   103
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   104
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   105
| Version | Supports                                                                                                |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   106
|:--------|:--------------------------------------------------------------------------------------------------------|
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   107
| v       | Validates only filename and size. Does not support file type restrictions by the XMPP server.           |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   108
| v2      | Validates the filename, size and MIME type. This allows the server to implement MIME type restrictions. |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   109
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   110
It is probable that a future v3 will be specified that allows carrying information about the uploader identity, allowing
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   111
the implementation of per-user quotas and limits.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   112
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   113
Implementations may implement one or more versions of the protocol simultaneously. The XMPP server generates the URLs and ultimately selects which version will be used.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   114
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   115
#### Version 1 (v)
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   116
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   117
The token will be in the URL query parameter 'v'. If it is absent, fail with 403 Forbidden.
2338
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
   118
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
   119
Calculate the expected auth token by reading the value of the Content-Length header of the PUT request. E.g. for a 1MB file
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
   120
will have a Content-Length of '1048576'. Append this to the uploaded file name, separated by a space (0x20) character.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
   121
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
   122
For the above example, you would end up with the following string: "foo/bar.jpg 1048576"
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
   123
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
   124
The auth token is a SHA256 HMAC of this string, using the configured secret as the key. E.g.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
   125
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
   126
```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
   127
calculated_auth_token = hmac_sha256("foo/bar.jpg 1048576", "secret string")
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
   128
```
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
   129
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
   130
If this is not equal to the 'v' parameter provided in the upload URL, reject the upload with 403 Forbidden.
c728b2f77c7c mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff changeset
   131
3362
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   132
**Security note:** When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   133
comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   134
as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   135
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   136
#### Version 2 (v2)
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   137
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   138
The token will be in the URL query parameter 'v2'. If it is absent, fail with 403 Forbidden.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   139
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   140
| Input         | Example     |Read from                                                            |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   141
|:--------------|:------------|:--------------------------------------------------------------------|
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   142
|`file_path`    | foo/bar.jpg | The URL of the PUT request, with the service's base prefix removed. |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   143
|`content_size` | 1048576     | Content-Size header                                                 |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   144
|`content_type` | image/jpeg  | Content-Type header                                                 |
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   145
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   146
The parameters should be joined into a single string, separated by NUL bytes (`\0`):
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   147
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   148
```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   149
  signed_string = ( file_path + '\0' + content_size + '\0' + content_type )
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   150
```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   151
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   152
```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   153
  signed_string = "foo/bar.jpg\01048576\0image/jpeg"
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   154
```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   155
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   156
The expected auth token is the SHA256 HMAC of this string, using the configured secret key as the key. E.g.:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   157
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   158
```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   159
calculated_auth_token = hmac_sha256(signed_string, "secret string")
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   160
```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   161
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   162
If this is not equal to the 'v2' parameter provided in the upload URL, reject the upload with 403 Forbidden.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   163
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   164
**Security note:** When comparing `calculated_auth_token` with the token provided in the URL, you must use a constant-time string
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   165
comparison, otherwise an attacker may be able to discover your secret key. Most languages/environments provide such a function, such
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   166
as `hash_equals()` in PHP, `hmac.compare_digest()` in Python, or `ConstantTimeCompare()` from `crypto/subtle` in Go.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   167
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   168
### Security considerations
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   169
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   170
#### HTTPS
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   171
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   172
All uploads and downloads should only be over HTTPS. The security of the served content is protected only
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   173
by the uniqueness present in the URLs themselves, and not using HTTPS may leak the URLs and contents to third-parties.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   174
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   175
Implementations should consider including HSTS and HPKP headers, with consent of the administrator.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   176
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   177
#### MIME types
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   178
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   179
If the upload Content-Type header matches any of the following MIME types, it MUST be preserved and included in the Content-Type
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   180
of any GET requests made to download the file:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   181
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   182
- `image/*`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   183
- `video/*`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   184
- `audio/*`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   185
- `text/plain`
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   186
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   187
It is recommended that other MIME types are preserved, but served with the addition of the following header:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   188
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   189
```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   190
Content-Disposition: attachment
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   191
```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   192
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   193
This prevents the browser interpreting scripts and other resources that may potentially be malicious.
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   194
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   195
Some browsers may also benefit from explicitly telling them not to try guessing the type of a file:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   196
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   197
```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   198
X-Content-Type-Options "nosniff"
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   199
```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   200
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   201
#### Security headers
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   202
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   203
The following headers should be included to provide additional sandboxing of resources, considering the uploaded
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   204
content is not understood or trusted by the upload service:
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   205
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   206
```
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   207
Content-Security-Policy: "default-src 'none'"
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   208
X-Content-Security-Policy: "default-src 'none'"
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   209
X-WebKit-CSP: "default-src 'none'"
e49660ba3161 mod_http_upload_external: Improve implementation docs, including v2 details
Matthew Wild <mwild1@gmail.com>
parents: 3193
diff changeset
   210
```