author | Kim Alvefur <zash@zash.se> |
Wed, 18 Jul 2018 23:44:09 +0200 | |
changeset 3193 | 57332ea0c1c7 |
parent 3172 | 73a610c3c7a9 |
child 3362 | e49660ba3161 |
permissions | -rw-r--r-- |
2338
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
1 |
--- |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
2 |
description: HTTP File Upload (external service) |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
3 |
labels: 'Stage-Alpha' |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
4 |
--- |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
5 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
6 |
Introduction |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
7 |
============ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
8 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
9 |
This module implements [XEP-0363], which lets clients upload files |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
10 |
over HTTP to an external web server. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
11 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
12 |
This module generates URLs that are signed using a HMAC. Any web service that can authenticate |
2827
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2338
diff
changeset
|
13 |
these URLs can be used. |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2338
diff
changeset
|
14 |
|
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2338
diff
changeset
|
15 |
Implementations |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2338
diff
changeset
|
16 |
--------------- |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2338
diff
changeset
|
17 |
|
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2338
diff
changeset
|
18 |
* [PHP implementation](https://hg.prosody.im/prosody-modules/raw-file/tip/mod_http_upload_external/share.php) |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2338
diff
changeset
|
19 |
* [Python3+Flask implementation](https://github.com/horazont/xmpp-http-upload) |
3172
73a610c3c7a9
mod_http_external: Link to prosody-filer (Go implementation)
Matthew Wild <mwild1@gmail.com>
parents:
2827
diff
changeset
|
20 |
* [Go implementation, Prosody Filer](https://github.com/ThomasLeister/prosody-filer) |
3193
57332ea0c1c7
mod_http_upload_external/README: Add Perl implementation by Holger to list
Kim Alvefur <zash@zash.se>
parents:
3172
diff
changeset
|
21 |
* [Perl implementation for nginx](https://github.com/weiss/ngx_http_upload) |
2827
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2338
diff
changeset
|
22 |
|
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2338
diff
changeset
|
23 |
To implement your own service compatible with this module, check out the implementation notes below |
f14bea5da323
mod_http_upload_external: add Python service implementation
Jonas Wielicki <jonas@wielicki.name>
parents:
2338
diff
changeset
|
24 |
(and if you publish your implementation - let us know!). |
2338
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
25 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
26 |
Configuration |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
27 |
============= |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
28 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
29 |
Add `"http_upload_external"` to modules_enabled in your global section, or under the host(s) you wish |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
30 |
to use it on. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
31 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
32 |
External URL |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
33 |
------------ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
34 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
35 |
You need to provide the path to the external service. Ensure it ends with '/'. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
36 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
37 |
For example, to use the PHP implementation linked above, you might set it to: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
38 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
39 |
``` {.lua} |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
40 |
http_upload_external_base_url = "https://your.example.com/path/to/share.php/" |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
41 |
``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
42 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
43 |
Secret |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
44 |
------ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
45 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
46 |
Set a long and unpredictable string as your secret. This is so the upload service can verify that |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
47 |
the upload comes from mod_http_upload_external, and random strangers can't upload to your server. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
48 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
49 |
``` {.lua} |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
50 |
http_upload_external_secret = "this is a secret string!" |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
51 |
``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
52 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
53 |
You need to set exactly the same secret string in your external service. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
54 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
55 |
Limits |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
56 |
------ |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
57 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
58 |
A maximum file size can be set by: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
59 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
60 |
``` {.lua} |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
61 |
http_upload_external_file_size_limit = 123 -- bytes |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
62 |
``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
63 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
64 |
Default is 100MB (100\*1024\*1024). |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
65 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
66 |
Compatibility |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
67 |
============= |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
68 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
69 |
Works with Prosody 0.9.x and later. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
70 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
71 |
Implementation |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
72 |
============== |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
73 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
74 |
To implement your own external service that is compatible with this module, you need to expose a |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
75 |
simple API that allows the HTTP GET, HEAD and PUT methods on arbitrary URLs located on your service. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
76 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
77 |
For example, if http_upload_external_base_url is set to `https://example.com/upload/` then your service |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
78 |
might receive the following requests: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
79 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
80 |
Upload a new file: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
81 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
82 |
``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
83 |
PUT https://example.com/upload/foo/bar.jpg?v=49e9309ff543ace93d25be90635ba8e9965c4f23fc885b2d86c947a5d59e55b2 |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
84 |
``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
85 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
86 |
Recipient checks the file size and other headers: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
87 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
88 |
``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
89 |
HEAD https://example.com/upload/foo/bar.jpg |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
90 |
``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
91 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
92 |
Recipient downloads the file: |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
93 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
94 |
``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
95 |
GET https://example.com/upload/foo/bar.jpg |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
96 |
``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
97 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
98 |
The only tricky logic is in validation of the PUT request. Firstly, don't overwrite existing files (return 409 Conflict). |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
99 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
100 |
Then you need to validate the auth token. This will be in the URL query parameter 'v'. If it is absent, fail with 403 Forbidden. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
101 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
102 |
Calculate the expected auth token by reading the value of the Content-Length header of the PUT request. E.g. for a 1MB file |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
103 |
will have a Content-Length of '1048576'. Append this to the uploaded file name, separated by a space (0x20) character. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
104 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
105 |
For the above example, you would end up with the following string: "foo/bar.jpg 1048576" |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
106 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
107 |
The auth token is a SHA256 HMAC of this string, using the configured secret as the key. E.g. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
108 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
109 |
``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
110 |
calculated_auth_token = hmac_sha256("foo/bar.jpg 1048576", "secret string") |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
111 |
``` |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
112 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
113 |
If this is not equal to the 'v' parameter provided in the upload URL, reject the upload with 403 Forbidden. |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
114 |
|
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
115 |
Note: your language/environment may provide a function for doing a constant-time comparison of these, to guard against |
c728b2f77c7c
mod_http_upload_external: Add README
Matthew Wild <mwild1@gmail.com>
parents:
diff
changeset
|
116 |
timing attacks that may be used to discover the secret key. |