nandi/frq · 20b5f50

nandi/frqpublic Fork

⑂ 20b5f50

One address for every reader, so ask freeq once

Hitting freeq's rate limit from the web client, and the shape of this relay
is why. Every reader's previews leave from this one process, so freeq sees
a single address for all of them and counts against a single budget -- and
nothing was cached, so one page load asked for a preview per link on
screen, every time it was opened. One reader scrolling a link-heavy backlog
could spend the budget for everybody, and what that looks like on someone
else's screen is cards quietly turning into 429s.

So the answers are kept, in two places for two different reasons. In this
process, where the point is that all those readers share one table and
therefore one request; and in the browser with `max-age`, where the point
is that a reload and a tab restore ask this server nothing at all. An
OpenGraph title is not news: a day here and an hour there are both far
inside how often one changes and far outside how often a backlog is
re-read.

Failures are kept too, for a minute. A host that is down is down for the
next reader as well, and retrying it per page load is how one dead link
becomes the thing spending the budget. A 429 is the exception and is never
stored -- it is not what the link is, it is what freeq thought of us a
moment ago, and keeping it would turn one exhausted budget into a day of
blank cards for a link that was fine.

Bounded at 2048, dropping the oldest third when it fills: this process is
long-lived, and a cache that only grows is a slower leak rather than not
one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

nandi committed 2026-09-20T18:55:51-07:00 Browse files

20b5f50 parent: ff28020

modified tools/webserve.py +92 -1 Viewed

@@ -40,6 +40,8 @@ the small one because the page sends no credentials.
40	40
41	41	import os
42	42	import sys
	43	+import threading
	44	+import time
43	45	import urllib.error
44	46	import urllib.request
45	47	from functools import partial
@@ -53,6 +55,60 @@ OG = "/api/v1/og"
53	55	# have crossed this machine on their way to being turned down.
54	56	LIMIT = 12 * 1024 * 1024
55	57
	58	+# What a link turned out to be, kept for a while.
	59	+#
	60	+# This is not a speed optimisation, it is a fairness one. Every reader's
	61	+# previews leave from this one process, so freeq sees one address for all of
	62	+# them and rate-limits accordingly -- and uncached, a single page load asks
	63	+# for a preview per link on screen, every time it is opened. One reader
	64	+# scrolling a link-heavy backlog could spend the budget for everybody, and
	65	+# what that looks like on the screen is somebody else's previews quietly
	66	+# turning into 429s.
	67	+#
	68	+# An OpenGraph title is not news. A day is far inside how often one changes
	69	+# and far outside how often a backlog is re-read, which is the pair of
	70	+# numbers that matters -- and a stale card is a smaller wrong answer than an
	71	+# absent one.
	72	+#
	73	+# Bounded, because this process is long-lived and a cache that only grows is
	74	+# a slower leak rather than not one. At the cap the oldest third goes: enough
	75	+# that eviction is rare rather than continuous, and `dict` preserves
	76	+# insertion order, so "oldest" needs no bookkeeping of its own.
	77	+#
	78	+# Failures are cached too, for much less time. A host that is down is down
	79	+# for the next reader a minute from now, and retrying it per page load is
	80	+# how one dead link in a backlog becomes the thing spending the budget.
	81	+OG_TTL = 24 * 60 * 60
	82	+OG_FAIL_TTL = 60
	83	+OG_MAX = 2048
	84	+
	85	+_og_cache: dict[str, tuple[float, int, bytes, str]] = {}
	86	+_og_lock = threading.Lock()
	87	+
	88	+
	89	+def og_cached(query):
	90	+ """The stored answer for this query, or None."""
	91	+ with _og_lock:
	92	+ hit = _og_cache.get(query)
	93	+ if hit is None:
	94	+ return None
	95	+ expires, status, body, kind = hit
	96	+ if expires < time.monotonic():
	97	+ # Expired rather than absent: dropped here so a URL nobody asks
	98	+ # for again does not sit in the table until the cap evicts it.
	99	+ del _og_cache[query]
	100	+ return None
	101	+ return status, body, kind
	102	+
	103	+
	104	+def og_remember(query, status, body, kind):
	105	+ ttl = OG_TTL if status == 200 else OG_FAIL_TTL
	106	+ with _og_lock:
	107	+ if len(_og_cache) >= OG_MAX:
	108	+ for stale in list(_og_cache)[: OG_MAX // 3]:
	109	+ del _og_cache[stale]
	110	+ _og_cache[query] = (time.monotonic() + ttl, status, body, kind)
	111	+
56	112
57	113	class Handler(SimpleHTTPRequestHandler):
58	114	# `http.server` answers HTTP/1.0 and closes the connection after every
@@ -89,7 +145,20 @@ class Handler(SimpleHTTPRequestHandler):
89	145	no body. A round trip per file, against a fix that does not arrive.
90	146	"""
91	147	path = self.path.split("?")[0]
92		- if path not in (UPLOAD, OG):
	148	+ if path == OG:
	149	+ # The one thing here that may be held without revalidating. A
	150	+ # bundle file is a name whose contents change on every deploy;
	151	+ # a preview is a URL's OpenGraph tags, which are not about this
	152	+ # client at all and do not change because we shipped. An hour in
	153	+ # the browser is a reload, a tab restore and a second visit that
	154	+ # ask this server nothing -- and this server is where every
	155	+ # reader's requests converge into the one address freeq counts.
	156	+ #
	157	+ # Shorter than the table's own day, deliberately: what a browser
	158	+ # holds cannot be dropped when it turns out to be wrong, and an
	159	+ # hour is the length of a sitting rather than of a habit.
	160	+ self.send_header("Cache-Control", "max-age=3600")
	161	+ elif path != UPLOAD:
93	162	self.send_header("Cache-Control", "no-cache")
94	163	SimpleHTTPRequestHandler.end_headers(self)
95	164
@@ -108,6 +177,21 @@ class Handler(SimpleHTTPRequestHandler):
108	177	self.send_error(400, "url is required")
109	178	return
110	179
	180	+ # Answered from the table where it is there, which is most of the
	181	+ # time: a backlog is re-read far more often than a page's OpenGraph
	182	+ # tags change. The query string is the key rather than the decoded
	183	+ # URL -- it is what would be sent upstream, so two requests sharing
	184	+ # it share an answer by construction.
	185	+ hit = og_cached(query)
	186	+ if hit is not None:
	187	+ status, body, kind = hit
	188	+ self.send_response(status)
	189	+ self.send_header("Content-Type", kind)
	190	+ self.send_header("Content-Length", str(len(body)))
	191	+ self.end_headers()
	192	+ self.wfile.write(body)
	193	+ return
	194	+
111	195	# Nothing of the caller's is forwarded -- no cookies, no
112	196	# Authorization, nothing this server was trusted with. The query is
113	197	# already encoded by whoever built it and is passed through as it
@@ -128,6 +212,13 @@ class Handler(SimpleHTTPRequestHandler):
128	212	except OSError as e:
129	213	status, body, kind = 502, str(e).encode(), "text/plain"
130	214
	215	+ # A 429 is never stored. It is not what this link is, it is what
	216	+ # freeq thought of us a moment ago -- and keeping it would turn one
	217	+ # exhausted budget into a day of blank cards for a link that was
	218	+ # fine.
	219	+ if status != 429:
	220	+ og_remember(query, status, body, kind)
	221	+
131	222	self.send_response(status)
132	223	self.send_header("Content-Type", kind)
133	224	self.send_header("Content-Length", str(len(body)))

  import os
  import sys
++import threading
++import time
  import urllib.error
  import urllib.request
  from functools import partial
  # have crossed this machine on their way to being turned down.
  LIMIT = 12 * 1024 * 1024
++# What a link turned out to be, kept for a while.
++#
++# This is not a speed optimisation, it is a fairness one. Every reader's
++# previews leave from this one process, so freeq sees one address for all of
++# them and rate-limits accordingly -- and uncached, a single page load asks
++# for a preview per link on screen, every time it is opened. One reader
++# scrolling a link-heavy backlog could spend the budget for everybody, and
++# what that looks like on the screen is somebody else's previews quietly
++# turning into 429s.
++#
++# An OpenGraph title is not news. A day is far inside how often one changes
++# and far outside how often a backlog is re-read, which is the pair of
++# numbers that matters -- and a stale card is a smaller wrong answer than an
++# absent one.
++#
++# Bounded, because this process is long-lived and a cache that only grows is
++# a slower leak rather than not one. At the cap the oldest third goes: enough
++# that eviction is rare rather than continuous, and `dict` preserves
++# insertion order, so "oldest" needs no bookkeeping of its own.
++#
++# Failures are cached too, for much less time. A host that is down is down
++# for the next reader a minute from now, and retrying it per page load is
++# how one dead link in a backlog becomes the thing spending the budget.
++OG_TTL = 24 * 60 * 60
++OG_FAIL_TTL = 60
++OG_MAX = 2048
++
++_og_cache: dict[str, tuple[float, int, bytes, str]] = {}
++_og_lock = threading.Lock()
++
++
++def og_cached(query):
++    """The stored answer for this query, or None."""
++    with _og_lock:
++        hit = _og_cache.get(query)
++        if hit is None:
++            return None
++        expires, status, body, kind = hit
++        if expires < time.monotonic():
++            # Expired rather than absent: dropped here so a URL nobody asks
++            # for again does not sit in the table until the cap evicts it.
++            del _og_cache[query]
++            return None
++        return status, body, kind
++
++
++def og_remember(query, status, body, kind):
++    ttl = OG_TTL if status == 200 else OG_FAIL_TTL
++    with _og_lock:
++        if len(_og_cache) >= OG_MAX:
++            for stale in list(_og_cache)[: OG_MAX // 3]:
++                del _og_cache[stale]
++        _og_cache[query] = (time.monotonic() + ttl, status, body, kind)
++
  class Handler(SimpleHTTPRequestHandler):
      # `http.server` answers HTTP/1.0 and closes the connection after every
          no body. A round trip per file, against a fix that does not arrive.
          """
          path = self.path.split("?")[0]
--        if path not in (UPLOAD, OG):
++        if path == OG:
++            # The one thing here that may be held without revalidating. A
++            # bundle file is a name whose contents change on every deploy;
++            # a preview is a URL's OpenGraph tags, which are not about this
++            # client at all and do not change because we shipped. An hour in
++            # the browser is a reload, a tab restore and a second visit that
++            # ask this server nothing -- and this server is where every
++            # reader's requests converge into the one address freeq counts.
++            #
++            # Shorter than the table's own day, deliberately: what a browser
++            # holds cannot be dropped when it turns out to be wrong, and an
++            # hour is the length of a sitting rather than of a habit.
++            self.send_header("Cache-Control", "max-age=3600")
++        elif path != UPLOAD:
              self.send_header("Cache-Control", "no-cache")
          SimpleHTTPRequestHandler.end_headers(self)
              self.send_error(400, "url is required")
              return
++        # Answered from the table where it is there, which is most of the
++        # time: a backlog is re-read far more often than a page's OpenGraph
++        # tags change. The query string is the key rather than the decoded
++        # URL -- it is what would be sent upstream, so two requests sharing
++        # it share an answer by construction.
++        hit = og_cached(query)
++        if hit is not None:
++            status, body, kind = hit
++            self.send_response(status)
++            self.send_header("Content-Type", kind)
++            self.send_header("Content-Length", str(len(body)))
++            self.end_headers()
++            self.wfile.write(body)
++            return
++
          # Nothing of the caller's is forwarded -- no cookies, no
          # Authorization, nothing this server was trusted with. The query is
          # already encoded by whoever built it and is passed through as it
          except OSError as e:
              status, body, kind = 502, str(e).encode(), "text/plain"
++        # A 429 is never stored. It is not what this link is, it is what
++        # freeq thought of us a moment ago -- and keeping it would turn one
++        # exhausted budget into a day of blank cards for a link that was
++        # fine.
++        if status != 429:
++            og_remember(query, status, body, kind)
++
          self.send_response(status)
          self.send_header("Content-Type", kind)
          self.send_header("Content-Length", str(len(body)))