rickub
nandi/frqpublic⑂ Fork 0
⑂ 6089101
Branches
claude/add-remote-repository-47d12e claude/ci-image-modal-deploy-f0f1d9 claude/clojure-nim-migration-bff603 main View all branches → View all tags →
Commits
⬇ Clone ▾
git clone https://git.rickub.com/nandi/frq.git
git clone ssh://git@rickub.com/nandi/frq.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

Carrying the upload across the origin boundary

I said this needed a header from freeq and nothing else. That was
right about the page and wrong about the problem: there is no client
fix, but the page is served by our own server, and a same-origin
request never meets CORS at all.

So `/api/v1/upload` is this server's path now. `tools/webserve.py`
serves the bundle -- which is all `python3 -m http.server` was doing --
and relays that one path to freeq from a process the same-origin rule
does not apply to. The browser is not circumvented; it is asked a
different question, and the honest answer is yes.

Deliberately narrow: one path, one method, one upstream fixed in the
file rather than taken from the request, no cookie or Authorization
forwarded, and freeq's own refusals passed through with their reasons
-- a 502 in their place would lose the sentence saying what was wrong
with the picture. It is also not the last word: one
`Access-Control-Allow-Origin: *` on a public unauthenticated endpoint
retires the whole arrangement.

`just run web`, the Dockerfile and `.modal/web/app.py` all serve with
it, so the laptop and the deployment behave the same. `nim/web/test/
serve.py` covers it against a stub upstream and joins `just test web`;
the relay was also run against the real irc.freeq.at, which answered
through it with its own JSON.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nandi committed 2026-09-20T13:06:01-07:00 Browse files
6089101 parent: b8610a1
modified .modal/web/Dockerfile +2 -1
@@ -58,4 +58,5 @@ FROM python:3.13-slim
5858 COPY --from=build /src/flutter/build/web /srv/web
5959
6060 EXPOSE 8000
61-CMD ["python3", "-m", "http.server", "8000", "--directory", "/srv/web"]
61+COPY --from=build /src/tools/webserve.py /srv/webserve.py
62+CMD ["python3", "/srv/webserve.py", "8000", "/srv/web"]
@@ -58,4 +58,5 @@ FROM python:3.13-slim
58 COPY --from=build /src/flutter/build/web /srv/web58 COPY --from=build /src/flutter/build/web /srv/web
59 59
60 EXPOSE 800060 EXPOSE 8000
61-CMD ["python3", "-m", "http.server", "8000", "--directory", "/srv/web"]61+COPY --from=build /src/tools/webserve.py /srv/webserve.py
62+CMD ["python3", "/srv/webserve.py", "8000", "/srv/web"]
modified .modal/web/README.md +9 -1
@@ -11,7 +11,15 @@ four constants and a `Popen`.
1111 Unlike `dev`, this container builds nothing. rickub builds the image
1212 -- `Dockerfile` here, two stages, the second one just the bundle and
1313 a python -- and pushes it to `registry.rickub.com`; this deploys that
14-exact tag. So the thing served is the thing that was built and
14+exact tag.
15+
16+The python is `tools/webserve.py` rather than `http.server`, and it
17+serves the bundle and relays exactly one path. freeq's media endpoint
18+allows one origin -- its own -- so an upload posted from the page is
19+accepted and its answer withheld, and the URL naming the picture never
20+arrives. Posted to this server it is same-origin, and the relay makes
21+the cross-origin request from a process the rule does not apply to.
22+`FRQ_API_ORIGIN` names the freeq to relay to. So the thing served is the thing that was built and
1523 tested, and a deploy is a pull rather than a compile. The workflow is
1624 `.rickub/workflows/web.yml`.
1725
@@ -11,7 +11,15 @@ four constants and a `Popen`.
11 Unlike `dev`, this container builds nothing. rickub builds the image11 Unlike `dev`, this container builds nothing. rickub builds the image
12 -- `Dockerfile` here, two stages, the second one just the bundle and12 -- `Dockerfile` here, two stages, the second one just the bundle and
13 a python -- and pushes it to `registry.rickub.com`; this deploys that13 a python -- and pushes it to `registry.rickub.com`; this deploys that
14-exact tag. So the thing served is the thing that was built and14+exact tag.
15+
16+The python is `tools/webserve.py` rather than `http.server`, and it
17+serves the bundle and relays exactly one path. freeq's media endpoint
18+allows one origin -- its own -- so an upload posted from the page is
19+accepted and its answer withheld, and the URL naming the picture never
20+arrives. Posted to this server it is same-origin, and the relay makes
21+the cross-origin request from a process the rule does not apply to.
22+`FRQ_API_ORIGIN` names the freeq to relay to. So the thing served is the thing that was built and
15 tested, and a deploy is a pull rather than a compile. The workflow is23 tested, and a deploy is a pull rather than a compile. The workflow is
16 `.rickub/workflows/web.yml`.24 `.rickub/workflows/web.yml`.
17 25
modified .modal/web/app.py +1 -1
@@ -57,7 +57,7 @@ image = modal.Image.from_registry(IMAGE or "python:3.13-slim")
5757 app = modal.App("frq-web", image=image)
5858
5959 PORT = 8000
60-SERVE = f"python3 -m http.server {PORT} --directory /srv/web"
60+SERVE = f"python3 /srv/webserve.py {PORT} /srv/web"
6161
6262
6363 @app.function(cpu=1, memory=1024, timeout=3600, min_containers=1)
@@ -57,7 +57,7 @@ image = modal.Image.from_registry(IMAGE or "python:3.13-slim")
57 app = modal.App("frq-web", image=image)57 app = modal.App("frq-web", image=image)
58 58
59 PORT = 800059 PORT = 8000
60-SERVE = f"python3 -m http.server {PORT} --directory /srv/web"60+SERVE = f"python3 /srv/webserve.py {PORT} /srv/web"
61 61
62 62
63 @app.function(cpu=1, memory=1024, timeout=3600, min_containers=1)63 @app.function(cpu=1, memory=1024, timeout=3600, min_containers=1)
modified flutter/web/frq_host.js +21 -20
@@ -90,26 +90,25 @@
9090 // freeq's media endpoint as the multipart form it wants. The URL that
9191 // comes back goes in the line — that is how a picture travels on IRC.
9292 //
93- // From a browser this is blocked, and the shape of the block is worth
94- // writing down because it is not the one it looks like. freeq does send
95- // CORS headers -- `vary: origin`, an allow-methods and an allow-headers --
96- // and answers with `access-control-allow-origin: https://irc.freeq.at` for
97- // exactly one origin: its own. It is an allowlist, and we are not on it,
98- // for this build or the deployed one. Being added is somebody else's
99- // decision, the same one as the broker's `return_to` list.
93+ // Posted to this server rather than to freeq, and the reason is CORS.
10094 //
101- // Sending no `Authorization` header is what makes it a *simple* request,
102- // which means no preflight -- so the POST is not stopped, only the answer
103- // is. The picture does upload; the URL naming it is withheld, and a URL
104- // nobody can read is a picture nobody can see. The endpoint wants no auth
105- // for a public upload (it says "No file provided", not "Unauthorized"),
106- // and no credentials are sent, which is deliberate: a request without
107- // them can be allowed by a plain `*`, where `credentials: "include"`
108- // would oblige the server to name this origin specifically. The smaller
109- // ask is the one more likely to be granted.
95+ // freeq's media endpoint answers `access-control-allow-origin` for exactly
96+ // one origin -- its own -- so a POST straight from here is sent, accepted,
97+ // and its reply withheld: the picture uploads and the URL naming it never
98+ // arrives. A URL nobody can read is a picture nobody can see. Nothing on
99+ // the page can get around that; an `<img>` shows a cross-origin picture
100+ // because displaying is not reading, and an upload is nothing but reading
101+ // the answer.
110102 //
111- // Written the way it will work rather than left out -- the same POST from
112- // the desktop build has no such limit.
103+ // `/api/v1/upload` on our own origin is not a circumvention, it is a
104+ // different request: same-origin, so no preflight and no allowlist.
105+ // `tools/webserve.py` relays it, from a process the rule does not apply
106+ // to. One `Access-Control-Allow-Origin: *` on freeq's side would retire
107+ // the whole arrangement -- and that is now the small ask, because this
108+ // sends no credentials.
109+ //
110+ // Straight to freeq when the page is already on it, where the relay would
111+ // be a detour through nothing.
113112 function pickAndUpload(want) {
114113 var input = document.createElement("input");
115114 input.type = "file";
@@ -129,8 +128,10 @@
129128 form.append("did", want.did);
130129 if (want.channel) form.append("channel", want.channel);
131130 form.append("file", file, file.name || "picture.png");
132- fetch("https://" + want.host + "/api/v1/upload",
133- {method: "POST", body: form})
131+ var where = window.location.host === want.host
132+ ? "https://" + want.host + "/api/v1/upload"
133+ : "/api/v1/upload";
134+ fetch(where, {method: "POST", body: form})
134135 .then(function (r) { return r.text().then(function (t) {
135136 return {ok: r.ok, status: r.status, body: t}; }); })
136137 .then(function (r) {
@@ -90,26 +90,25 @@
90 // freeq's media endpoint as the multipart form it wants. The URL that90 // freeq's media endpoint as the multipart form it wants. The URL that
91 // comes back goes in the line — that is how a picture travels on IRC.91 // comes back goes in the line — that is how a picture travels on IRC.
92 //92 //
93- // From a browser this is blocked, and the shape of the block is worth93+ // Posted to this server rather than to freeq, and the reason is CORS.
94- // writing down because it is not the one it looks like. freeq does send
95- // CORS headers -- `vary: origin`, an allow-methods and an allow-headers --
96- // and answers with `access-control-allow-origin: https://irc.freeq.at` for
97- // exactly one origin: its own. It is an allowlist, and we are not on it,
98- // for this build or the deployed one. Being added is somebody else's
99- // decision, the same one as the broker's `return_to` list.
100 //94 //
101- // Sending no `Authorization` header is what makes it a *simple* request,95+ // freeq's media endpoint answers `access-control-allow-origin` for exactly
102- // which means no preflight -- so the POST is not stopped, only the answer96+ // one origin -- its own -- so a POST straight from here is sent, accepted,
103- // is. The picture does upload; the URL naming it is withheld, and a URL97+ // and its reply withheld: the picture uploads and the URL naming it never
104- // nobody can read is a picture nobody can see. The endpoint wants no auth98+ // arrives. A URL nobody can read is a picture nobody can see. Nothing on
105- // for a public upload (it says "No file provided", not "Unauthorized"),99+ // the page can get around that; an `<img>` shows a cross-origin picture
106- // and no credentials are sent, which is deliberate: a request without100+ // because displaying is not reading, and an upload is nothing but reading
107- // them can be allowed by a plain `*`, where `credentials: "include"`101+ // the answer.
108- // would oblige the server to name this origin specifically. The smaller
109- // ask is the one more likely to be granted.
110 //102 //
111- // Written the way it will work rather than left out -- the same POST from103+ // `/api/v1/upload` on our own origin is not a circumvention, it is a
112- // the desktop build has no such limit.104+ // different request: same-origin, so no preflight and no allowlist.
105+ // `tools/webserve.py` relays it, from a process the rule does not apply
106+ // to. One `Access-Control-Allow-Origin: *` on freeq's side would retire
107+ // the whole arrangement -- and that is now the small ask, because this
108+ // sends no credentials.
109+ //
110+ // Straight to freeq when the page is already on it, where the relay would
111+ // be a detour through nothing.
113 function pickAndUpload(want) {112 function pickAndUpload(want) {
114 var input = document.createElement("input");113 var input = document.createElement("input");
115 input.type = "file";114 input.type = "file";
@@ -129,8 +128,10 @@
129 form.append("did", want.did);128 form.append("did", want.did);
130 if (want.channel) form.append("channel", want.channel);129 if (want.channel) form.append("channel", want.channel);
131 form.append("file", file, file.name || "picture.png");130 form.append("file", file, file.name || "picture.png");
132- fetch("https://" + want.host + "/api/v1/upload",131+ var where = window.location.host === want.host
133- {method: "POST", body: form})132+ ? "https://" + want.host + "/api/v1/upload"
133+ : "/api/v1/upload";
134+ fetch(where, {method: "POST", body: form})
134 .then(function (r) { return r.text().then(function (t) {135 .then(function (r) { return r.text().then(function (t) {
135 return {ok: r.ok, status: r.status, body: t}; }); })136 return {ok: r.ok, status: r.status, body: t}; }); })
136 .then(function (r) {137 .then(function (r) {
modified flutter/web/frq_oauth.js +7 -0
@@ -192,6 +192,12 @@
192192 async function whoami(pds, token) {
193193 const r = await getWithDpop(sessionUrl(pds), token, '');
194194 if (r.status !== 200) {
195+ // Said twice on purpose. The throw reaches the reader as the app's own
196+ // warning; this puts the same words in the console, which is where a
197+ // refusal is looked at -- and where, until it was printed, there was a
198+ // bare `401 (Unauthorized)` and no way to tell an expired token from a
199+ // wrong one.
200+ console.warn('frq: getSession refused', r.status, r.body);
195201 throw new Error('The PDS would not accept the token (' + r.status +
196202 '): ' + r.body);
197203 }
@@ -350,6 +356,7 @@
350356 // The refresh token is gone or was revoked, and no retry will bring it
351357 // back. Clearing the session is what turns "signed in, and nothing
352358 // works" into a sign-in button.
359+ console.warn('frq: refresh refused', r.status, r.body);
353360 forget();
354361 throw new Error('The session has expired; sign in again (' +
355362 r.status + '): ' + r.body);
@@ -192,6 +192,12 @@
192 async function whoami(pds, token) {192 async function whoami(pds, token) {
193 const r = await getWithDpop(sessionUrl(pds), token, '');193 const r = await getWithDpop(sessionUrl(pds), token, '');
194 if (r.status !== 200) {194 if (r.status !== 200) {
195+ // Said twice on purpose. The throw reaches the reader as the app's own
196+ // warning; this puts the same words in the console, which is where a
197+ // refusal is looked at -- and where, until it was printed, there was a
198+ // bare `401 (Unauthorized)` and no way to tell an expired token from a
199+ // wrong one.
200+ console.warn('frq: getSession refused', r.status, r.body);
195 throw new Error('The PDS would not accept the token (' + r.status +201 throw new Error('The PDS would not accept the token (' + r.status +
196 '): ' + r.body);202 '): ' + r.body);
197 }203 }
@@ -350,6 +356,7 @@
350 // The refresh token is gone or was revoked, and no retry will bring it356 // The refresh token is gone or was revoked, and no retry will bring it
351 // back. Clearing the session is what turns "signed in, and nothing357 // back. Clearing the session is what turns "signed in, and nothing
352 // works" into a sign-in button.358 // works" into a sign-in button.
359+ console.warn('frq: refresh refused', r.status, r.body);
353 forget();360 forget();
354 throw new Error('The session has expired; sign in again (' +361 throw new Error('The session has expired; sign in again (' +
355 r.status + '): ' + r.body);362 r.status + '): ' + r.body);
modified justfile +3 -3
@@ -77,8 +77,7 @@ run target="desktop":
7777 case "{{target}}" in
7878 desktop) just _flutter desktop run ;;
7979 web) just _web-bundle
80- echo "serving build/web on http://localhost:8000"
81- exec python3 -m http.server 8000 --directory build/web ;;
80+ exec python3 tools/webserve.py 8000 build/web ;;
8281 *) echo "usage: just run [desktop|web]" >&2; exit 1 ;;
8382 esac
8483
@@ -110,7 +109,8 @@ test suite="all" *args:
110109 nim) just _nim-test "$@" ;;
111110 web) just _nim-js
112111 node nim/web/test/smoke.js build/web/frq_core.js
113- exec node nim/web/test/session.js ;;
112+ node nim/web/test/session.js
113+ exec python3 nim/web/test/serve.py ;;
114114 dart) just _nim-lib
115115 exec "{{tc}}" exec -- bash -c \
116116 'cd dart/frq_core && dart pub get && dart test -r expanded' ;;
@@ -77,8 +77,7 @@ run target="desktop":
77 case "{{target}}" in77 case "{{target}}" in
78 desktop) just _flutter desktop run ;;78 desktop) just _flutter desktop run ;;
79 web) just _web-bundle79 web) just _web-bundle
80- echo "serving build/web on http://localhost:8000"80+ exec python3 tools/webserve.py 8000 build/web ;;
81- exec python3 -m http.server 8000 --directory build/web ;;
82 *) echo "usage: just run [desktop|web]" >&2; exit 1 ;;81 *) echo "usage: just run [desktop|web]" >&2; exit 1 ;;
83 esac82 esac
84 83
@@ -110,7 +109,8 @@ test suite="all" *args:
110 nim) just _nim-test "$@" ;;109 nim) just _nim-test "$@" ;;
111 web) just _nim-js110 web) just _nim-js
112 node nim/web/test/smoke.js build/web/frq_core.js111 node nim/web/test/smoke.js build/web/frq_core.js
113- exec node nim/web/test/session.js ;;112+ node nim/web/test/session.js
113+ exec python3 nim/web/test/serve.py ;;
114 dart) just _nim-lib114 dart) just _nim-lib
115 exec "{{tc}}" exec -- bash -c \115 exec "{{tc}}" exec -- bash -c \
116 'cd dart/frq_core && dart pub get && dart test -r expanded' ;;116 'cd dart/frq_core && dart pub get && dart test -r expanded' ;;
added nim/web/test/serve.py +129 -0
new file mode 100644
@@ -0,0 +1,129 @@
1+#!/usr/bin/env python3
2+"""The server that carries the upload across the origin boundary.
3+
4+ python3 nim/web/test/serve.py
5+
6+Offline: the upstream is a stub in this process, so nothing is uploaded
7+anywhere. What is checked is the part with teeth -- that it relays the one
8+path and is not a proxy for anything else.
9+"""
10+import json
11+import os
12+import subprocess
13+import sys
14+import threading
15+import time
16+import urllib.error
17+import urllib.request
18+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
19+
20+ROOT = os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(
21+ os.path.abspath(__file__)))))
22+seen = {}
23+
24+
25+class Upstream(BaseHTTPRequestHandler):
26+ def do_POST(self):
27+ n = int(self.headers.get("Content-Length") or 0)
28+ seen["path"] = self.path
29+ seen["body"] = self.rfile.read(n)
30+ seen["type"] = self.headers.get("Content-Type")
31+ seen["cookie"] = self.headers.get("Cookie")
32+ if b"boom" in seen["body"]:
33+ out = json.dumps({"error": "Bad Request"}).encode()
34+ self.send_response(400)
35+ else:
36+ out = json.dumps({"url": "https://media.example/p.png"}).encode()
37+ self.send_response(200)
38+ self.send_header("Content-Type", "application/json")
39+ self.send_header("Content-Length", str(len(out)))
40+ self.end_headers()
41+ self.wfile.write(out)
42+
43+ def log_message(self, *a):
44+ pass
45+
46+
47+def post(url, body, headers=None):
48+ req = urllib.request.Request(url, data=body, method="POST",
49+ headers=headers or {})
50+ try:
51+ with urllib.request.urlopen(req, timeout=10) as r:
52+ return r.status, r.read()
53+ except urllib.error.HTTPError as e:
54+ return e.code, e.read()
55+
56+
57+def main():
58+ up = ThreadingHTTPServer(("127.0.0.1", 0), Upstream)
59+ threading.Thread(target=up.serve_forever, daemon=True).start()
60+ origin = "http://127.0.0.1:%d" % up.server_address[1]
61+
62+ www = os.path.join(ROOT, "nim", "web", "test")
63+ env = dict(os.environ, FRQ_API_ORIGIN=origin)
64+ proc = subprocess.Popen(
65+ [sys.executable, os.path.join(ROOT, "tools", "webserve.py"),
66+ "8732", www],
67+ env=env, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
68+ base = "http://127.0.0.1:8732"
69+ try:
70+ for _ in range(50):
71+ try:
72+ urllib.request.urlopen(base + "/serve.py", timeout=1).read()
73+ break
74+ except OSError:
75+ time.sleep(0.1)
76+
77+ ok = lambda s: print(" ok " + s)
78+
79+ # Still a file server: the bundle is the reason it exists.
80+ with urllib.request.urlopen(base + "/serve.py", timeout=5) as r:
81+ assert b"across the origin boundary" in r.read()
82+ ok("it still serves the bundle")
83+
84+ # The relay, with the body and its multipart boundary intact --
85+ # without the Content-Type the body is unreadable at the far end.
86+ status, body = post(base + "/api/v1/upload", b"--b\r\npicture\r\n--b--",
87+ {"Content-Type": "multipart/form-data; boundary=b"})
88+ assert status == 200, status
89+ assert json.loads(body)["url"] == "https://media.example/p.png"
90+ assert seen["path"] == "/api/v1/upload"
91+ assert seen["body"] == b"--b\r\npicture\r\n--b--"
92+ assert seen["type"] == "multipart/form-data; boundary=b"
93+ ok("an upload reaches the far end whole, and the URL comes back")
94+
95+ # A refusal is freeq's to explain. A 502 in its place would lose the
96+ # sentence saying what was wrong with the file.
97+ status, body = post(base + "/api/v1/upload", b"boom",
98+ {"Content-Type": "multipart/form-data; boundary=b"})
99+ assert status == 400, status
100+ assert json.loads(body)["error"] == "Bad Request"
101+ ok("and a refusal keeps its reason")
102+
103+ # Nothing this server was trusted with is handed on.
104+ post(base + "/api/v1/upload", b"x",
105+ {"Content-Type": "text/plain", "Cookie": "session=secret"})
106+ assert seen["cookie"] is None, seen["cookie"]
107+ ok("a cookie of ours is not forwarded")
108+
109+ # One path. Not a proxy.
110+ status, _ = post(base + "/api/v1/search", b"x")
111+ assert status == 404, status
112+ status, _ = post(base + "/", b"x")
113+ assert status == 404, status
114+ ok("and no other path is relayed at all")
115+
116+ # The endpoint's own cap, refused before it crosses this machine.
117+ status, _ = post(base + "/api/v1/upload", b"x" * (12 * 1024 * 1024 + 1),
118+ {"Content-Type": "multipart/form-data; boundary=b"})
119+ assert status == 413, status
120+ ok("something over the limit is turned down here")
121+
122+ print("all ok")
123+ finally:
124+ proc.terminate()
125+ up.shutdown()
126+
127+
128+if __name__ == "__main__":
129+ main()
new file mode 100644
@@ -0,0 +1,129 @@
1+#!/usr/bin/env python3
2+"""The server that carries the upload across the origin boundary.
3+
4+ python3 nim/web/test/serve.py
5+
6+Offline: the upstream is a stub in this process, so nothing is uploaded
7+anywhere. What is checked is the part with teeth -- that it relays the one
8+path and is not a proxy for anything else.
9+"""
10+import json
11+import os
12+import subprocess
13+import sys
14+import threading
15+import time
16+import urllib.error
17+import urllib.request
18+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
19+
20+ROOT = os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(
21+ os.path.abspath(__file__)))))
22+seen = {}
23+
24+
25+class Upstream(BaseHTTPRequestHandler):
26+ def do_POST(self):
27+ n = int(self.headers.get("Content-Length") or 0)
28+ seen["path"] = self.path
29+ seen["body"] = self.rfile.read(n)
30+ seen["type"] = self.headers.get("Content-Type")
31+ seen["cookie"] = self.headers.get("Cookie")
32+ if b"boom" in seen["body"]:
33+ out = json.dumps({"error": "Bad Request"}).encode()
34+ self.send_response(400)
35+ else:
36+ out = json.dumps({"url": "https://media.example/p.png"}).encode()
37+ self.send_response(200)
38+ self.send_header("Content-Type", "application/json")
39+ self.send_header("Content-Length", str(len(out)))
40+ self.end_headers()
41+ self.wfile.write(out)
42+
43+ def log_message(self, *a):
44+ pass
45+
46+
47+def post(url, body, headers=None):
48+ req = urllib.request.Request(url, data=body, method="POST",
49+ headers=headers or {})
50+ try:
51+ with urllib.request.urlopen(req, timeout=10) as r:
52+ return r.status, r.read()
53+ except urllib.error.HTTPError as e:
54+ return e.code, e.read()
55+
56+
57+def main():
58+ up = ThreadingHTTPServer(("127.0.0.1", 0), Upstream)
59+ threading.Thread(target=up.serve_forever, daemon=True).start()
60+ origin = "http://127.0.0.1:%d" % up.server_address[1]
61+
62+ www = os.path.join(ROOT, "nim", "web", "test")
63+ env = dict(os.environ, FRQ_API_ORIGIN=origin)
64+ proc = subprocess.Popen(
65+ [sys.executable, os.path.join(ROOT, "tools", "webserve.py"),
66+ "8732", www],
67+ env=env, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
68+ base = "http://127.0.0.1:8732"
69+ try:
70+ for _ in range(50):
71+ try:
72+ urllib.request.urlopen(base + "/serve.py", timeout=1).read()
73+ break
74+ except OSError:
75+ time.sleep(0.1)
76+
77+ ok = lambda s: print(" ok " + s)
78+
79+ # Still a file server: the bundle is the reason it exists.
80+ with urllib.request.urlopen(base + "/serve.py", timeout=5) as r:
81+ assert b"across the origin boundary" in r.read()
82+ ok("it still serves the bundle")
83+
84+ # The relay, with the body and its multipart boundary intact --
85+ # without the Content-Type the body is unreadable at the far end.
86+ status, body = post(base + "/api/v1/upload", b"--b\r\npicture\r\n--b--",
87+ {"Content-Type": "multipart/form-data; boundary=b"})
88+ assert status == 200, status
89+ assert json.loads(body)["url"] == "https://media.example/p.png"
90+ assert seen["path"] == "/api/v1/upload"
91+ assert seen["body"] == b"--b\r\npicture\r\n--b--"
92+ assert seen["type"] == "multipart/form-data; boundary=b"
93+ ok("an upload reaches the far end whole, and the URL comes back")
94+
95+ # A refusal is freeq's to explain. A 502 in its place would lose the
96+ # sentence saying what was wrong with the file.
97+ status, body = post(base + "/api/v1/upload", b"boom",
98+ {"Content-Type": "multipart/form-data; boundary=b"})
99+ assert status == 400, status
100+ assert json.loads(body)["error"] == "Bad Request"
101+ ok("and a refusal keeps its reason")
102+
103+ # Nothing this server was trusted with is handed on.
104+ post(base + "/api/v1/upload", b"x",
105+ {"Content-Type": "text/plain", "Cookie": "session=secret"})
106+ assert seen["cookie"] is None, seen["cookie"]
107+ ok("a cookie of ours is not forwarded")
108+
109+ # One path. Not a proxy.
110+ status, _ = post(base + "/api/v1/search", b"x")
111+ assert status == 404, status
112+ status, _ = post(base + "/", b"x")
113+ assert status == 404, status
114+ ok("and no other path is relayed at all")
115+
116+ # The endpoint's own cap, refused before it crosses this machine.
117+ status, _ = post(base + "/api/v1/upload", b"x" * (12 * 1024 * 1024 + 1),
118+ {"Content-Type": "multipart/form-data; boundary=b"})
119+ assert status == 413, status
120+ ok("something over the limit is turned down here")
121+
122+ print("all ok")
123+ finally:
124+ proc.terminate()
125+ up.shutdown()
126+
127+
128+if __name__ == "__main__":
129+ main()
added tools/webserve.py +113 -0
new file mode 100644
@@ -0,0 +1,113 @@
1+#!/usr/bin/env python3
2+"""Serve the web bundle, and relay the one request a browser may not make.
3+
4+`python3 -m http.server` did this until the upload needed to work. freeq's
5+media endpoint answers `access-control-allow-origin` for exactly one origin
6+-- its own -- so a POST from this page is sent, accepted, and its reply
7+withheld by the browser. The upload happens; the URL naming it does not
8+come back, and a URL nobody can read is a picture nobody can see.
9+
10+Nothing on the page can get around that: an `<img>` displays a
11+cross-origin picture without permission because displaying is not reading,
12+and the whole point of an upload is reading the answer.
13+
14+So it is not done from the page. `/api/v1/upload` here is this server's own
15+path, which makes it same-origin, which means no preflight, no allowlist and
16+no CORS at all -- the browser is not being circumvented, it is being told
17+the truth. This process makes the cross-origin request, where the rule does
18+not apply, and hands back what came.
19+
20+Two things this is deliberately not. It is not a general proxy: one path,
21+one method, one upstream, fixed here rather than taken from the request, so
22+it cannot be pointed at anything. And it is not a permanent answer -- one
23+`Access-Control-Allow-Origin: *` on a public, unauthenticated endpoint would
24+retire it, and that ask is now the small one because the page sends no
25+credentials.
26+
27+ python3 tools/webserve.py [PORT] [DIRECTORY]
28+
29+`FRQ_API_ORIGIN` names the freeq to relay to, for anyone running their own.
30+"""
31+
32+import os
33+import sys
34+import urllib.error
35+import urllib.request
36+from functools import partial
37+from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
38+
39+UPSTREAM = os.environ.get("FRQ_API_ORIGIN", "https://irc.freeq.at").rstrip("/")
40+UPLOAD = "/api/v1/upload"
41+
42+# The endpoint's own cap. Refused here rather than after twelve megabytes
43+# have crossed this machine on their way to being turned down.
44+LIMIT = 12 * 1024 * 1024
45+
46+
47+class Handler(SimpleHTTPRequestHandler):
48+ def do_POST(self):
49+ if self.path.split("?")[0] != UPLOAD:
50+ self.send_error(404, "Not Found")
51+ return
52+
53+ length = int(self.headers.get("Content-Length") or 0)
54+ if length <= 0 or length > LIMIT:
55+ # Drained before it is refused, and the draining is the point.
56+ # Answering 413 and closing on a sender still writing gives it a
57+ # broken pipe instead of the sentence explaining why -- so the
58+ # bytes are read and dropped, up to a bound past which nobody is
59+ # owed an explanation.
60+ if 0 < length <= LIMIT * 4:
61+ left = length
62+ while left > 0:
63+ chunk = self.rfile.read(min(left, 65536))
64+ if not chunk:
65+ break
66+ left -= len(chunk)
67+ else:
68+ self.close_connection = True
69+ self.send_error(413, "Body too large")
70+ return
71+
72+ # `Content-Type` carries the multipart boundary, so the body is
73+ # unreadable without it. Nothing else is forwarded: no cookies, no
74+ # Authorization, nothing this server was trusted with.
75+ req = urllib.request.Request(
76+ UPSTREAM + UPLOAD,
77+ data=self.rfile.read(length),
78+ method="POST",
79+ headers={"Content-Type": self.headers.get("Content-Type", "")},
80+ )
81+ try:
82+ with urllib.request.urlopen(req, timeout=120) as r:
83+ status, body, kind = r.status, r.read(), r.headers.get_content_type()
84+ except urllib.error.HTTPError as e:
85+ # freeq's own refusal, passed through with its reason. Turning
86+ # every one of these into a 502 would lose the message that says
87+ # what was wrong with the file.
88+ status, body, kind = e.code, e.read(), "application/json"
89+ except OSError as e:
90+ status, body, kind = 502, str(e).encode(), "text/plain"
91+
92+ self.send_response(status)
93+ self.send_header("Content-Type", kind)
94+ self.send_header("Content-Length", str(len(body)))
95+ self.end_headers()
96+ self.wfile.write(body)
97+
98+ def log_message(self, fmt, *args):
99+ sys.stderr.write("%s %s\n" % (self.address_string(), fmt % args))
100+
101+
102+def main() -> None:
103+ port = int(sys.argv[1]) if len(sys.argv) > 1 else 8000
104+ directory = sys.argv[2] if len(sys.argv) > 2 else "build/web"
105+ handler = partial(Handler, directory=directory)
106+ srv = ThreadingHTTPServer(("", port), handler)
107+ print(f"serving {directory} on http://localhost:{port}"
108+ f" ({UPLOAD} relays to {UPSTREAM})", flush=True)
109+ srv.serve_forever()
110+
111+
112+if __name__ == "__main__":
113+ main()
new file mode 100644
@@ -0,0 +1,113 @@
1+#!/usr/bin/env python3
2+"""Serve the web bundle, and relay the one request a browser may not make.
3+
4+`python3 -m http.server` did this until the upload needed to work. freeq's
5+media endpoint answers `access-control-allow-origin` for exactly one origin
6+-- its own -- so a POST from this page is sent, accepted, and its reply
7+withheld by the browser. The upload happens; the URL naming it does not
8+come back, and a URL nobody can read is a picture nobody can see.
9+
10+Nothing on the page can get around that: an `<img>` displays a
11+cross-origin picture without permission because displaying is not reading,
12+and the whole point of an upload is reading the answer.
13+
14+So it is not done from the page. `/api/v1/upload` here is this server's own
15+path, which makes it same-origin, which means no preflight, no allowlist and
16+no CORS at all -- the browser is not being circumvented, it is being told
17+the truth. This process makes the cross-origin request, where the rule does
18+not apply, and hands back what came.
19+
20+Two things this is deliberately not. It is not a general proxy: one path,
21+one method, one upstream, fixed here rather than taken from the request, so
22+it cannot be pointed at anything. And it is not a permanent answer -- one
23+`Access-Control-Allow-Origin: *` on a public, unauthenticated endpoint would
24+retire it, and that ask is now the small one because the page sends no
25+credentials.
26+
27+ python3 tools/webserve.py [PORT] [DIRECTORY]
28+
29+`FRQ_API_ORIGIN` names the freeq to relay to, for anyone running their own.
30+"""
31+
32+import os
33+import sys
34+import urllib.error
35+import urllib.request
36+from functools import partial
37+from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
38+
39+UPSTREAM = os.environ.get("FRQ_API_ORIGIN", "https://irc.freeq.at").rstrip("/")
40+UPLOAD = "/api/v1/upload"
41+
42+# The endpoint's own cap. Refused here rather than after twelve megabytes
43+# have crossed this machine on their way to being turned down.
44+LIMIT = 12 * 1024 * 1024
45+
46+
47+class Handler(SimpleHTTPRequestHandler):
48+ def do_POST(self):
49+ if self.path.split("?")[0] != UPLOAD:
50+ self.send_error(404, "Not Found")
51+ return
52+
53+ length = int(self.headers.get("Content-Length") or 0)
54+ if length <= 0 or length > LIMIT:
55+ # Drained before it is refused, and the draining is the point.
56+ # Answering 413 and closing on a sender still writing gives it a
57+ # broken pipe instead of the sentence explaining why -- so the
58+ # bytes are read and dropped, up to a bound past which nobody is
59+ # owed an explanation.
60+ if 0 < length <= LIMIT * 4:
61+ left = length
62+ while left > 0:
63+ chunk = self.rfile.read(min(left, 65536))
64+ if not chunk:
65+ break
66+ left -= len(chunk)
67+ else:
68+ self.close_connection = True
69+ self.send_error(413, "Body too large")
70+ return
71+
72+ # `Content-Type` carries the multipart boundary, so the body is
73+ # unreadable without it. Nothing else is forwarded: no cookies, no
74+ # Authorization, nothing this server was trusted with.
75+ req = urllib.request.Request(
76+ UPSTREAM + UPLOAD,
77+ data=self.rfile.read(length),
78+ method="POST",
79+ headers={"Content-Type": self.headers.get("Content-Type", "")},
80+ )
81+ try:
82+ with urllib.request.urlopen(req, timeout=120) as r:
83+ status, body, kind = r.status, r.read(), r.headers.get_content_type()
84+ except urllib.error.HTTPError as e:
85+ # freeq's own refusal, passed through with its reason. Turning
86+ # every one of these into a 502 would lose the message that says
87+ # what was wrong with the file.
88+ status, body, kind = e.code, e.read(), "application/json"
89+ except OSError as e:
90+ status, body, kind = 502, str(e).encode(), "text/plain"
91+
92+ self.send_response(status)
93+ self.send_header("Content-Type", kind)
94+ self.send_header("Content-Length", str(len(body)))
95+ self.end_headers()
96+ self.wfile.write(body)
97+
98+ def log_message(self, fmt, *args):
99+ sys.stderr.write("%s %s\n" % (self.address_string(), fmt % args))
100+
101+
102+def main() -> None:
103+ port = int(sys.argv[1]) if len(sys.argv) > 1 else 8000
104+ directory = sys.argv[2] if len(sys.argv) > 2 else "build/web"
105+ handler = partial(Handler, directory=directory)
106+ srv = ThreadingHTTPServer(("", port), handler)
107+ print(f"serving {directory} on http://localhost:{port}"
108+ f" ({UPLOAD} relays to {UPSTREAM})", flush=True)
109+ srv.serve_forever()
110+
111+
112+if __name__ == "__main__":
113+ main()