rickub
nandi/gleanpublic⑂ Fork 0
⑂ b3f6ba3
Branches
claude/modal-container-min-76527a main View all branches → View all tags →
Commits
⬇ Clone ▾
git clone https://git.rickub.com/nandi/glean.git
git clone ssh://git@rickub.com/nandi/glean.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

Split databases by domainUnverified

Julien Robert committed 2026-04-23T11:08:19+02:00 Browse files
b3f6ba3 parent: 238f0f3
modified .gitignore +1 -2
@@ -23,8 +23,7 @@ go.work.sum
2323 .env
2424
2525 # database
26-*.db
27-*.db-*
26+*.db*
2827
2928 # tailwind
3029 static/output.css
@@ -23,8 +23,7 @@ go.work.sum
23 .env23 .env
24 24
25 # database25 # database
26-*.db26+*.db*
27-*.db-*
28 27
29 # tailwind28 # tailwind
30 static/output.css29 static/output.css
modified docs/specs.md +24 -11
@@ -13,7 +13,7 @@ The core idea: your RSS subscriptions are a strong signal about your interests.
1313 | Layer | Technology |
1414 | ---------------- | --------------------------------------------------------------- |
1515 | Backend | Go |
16-| Database | SQLite (via `mattn/go-sqlite3`) |
16+| Database | SQLite (3 files: users, articles, recs via `mattn/go-sqlite3`) |
1717 | Frontend | htmx + TailwindCSS |
1818 | Auth | AT Protocol OAuth / DID resolution (configurable PLC directory) |
1919 | AT Protocol role | AppView for `at.glean.*` lexicons |
@@ -475,7 +475,19 @@ Glean runs as a single Go binary that fills three roles: **AppView** (indexing `
475475
476476 ## 6. Database Schema (SQLite)
477477
478-### 6.1 Users
478+Glean uses three separate SQLite database files to reduce write-lock contention. Each is opened with its own connection pool:
479+
480+| File | Contents | ATTACH alias |
481+|------|----------|-------------|
482+| `<base>_users` | Users, follows, OAuth | `main` (primary) |
483+| `<base>_articles` | Feeds, subscriptions, articles, read state, likes, annotations | `articles` |
484+| `<base>_recs` | Similarity scores, impressions, dismissals, signal weights | `recs` |
485+
486+The users connection pool uses a custom SQLite driver with a `ConnectHook` that ATTACHes the articles and recs databases on every new connection. This allows the cluster engine to run cross-database queries using schema prefixes (`articles.subscriptions`, `recs.user_similarity`, `main.follows`).
487+
488+The articles and recs connections are independent pools used by the service layer for single-database operations.
489+
490+### 6.1 Users (`<base>_users`)
479491
480492 ```sql
481493 CREATE TABLE users (
@@ -488,7 +500,7 @@ CREATE TABLE users (
488500 );
489501 ```
490502
491-### 6.2 Feed Subscriptions
503+### 6.2 Feed Subscriptions (`<base>_articles`)
492504
493505 Indexed from `at.glean.subscription` records on user PDS.
494506
@@ -509,7 +521,7 @@ CREATE INDEX idx_subscriptions_feed ON subscriptions(feed_url);
509521 CREATE INDEX idx_subscriptions_user ON subscriptions(user_did);
510522 ```
511523
512-### 6.3 Feeds
524+### 6.3 Feeds (`<base>_articles`)
513525
514526 Master list of all known RSS feeds.
515527
@@ -533,7 +545,7 @@ CREATE TABLE feeds (
533545 );
534546 ```
535547
536-### 6.4 Articles
548+### 6.4 Articles (`<base>_articles`)
537549
538550 Fetched from RSS feeds. Only fetched for feeds that have local subscribers.
539551
@@ -557,7 +569,7 @@ CREATE INDEX idx_articles_feed ON articles(feed_url);
557569 CREATE INDEX idx_articles_published ON articles(published DESC);
558570 ```
559571
560-### 6.5 Read State
572+### 6.5 Read State (`<base>_articles`)
561573
562574 ```sql
563575 CREATE TABLE read_state (
@@ -571,7 +583,7 @@ CREATE TABLE read_state (
571583 CREATE INDEX idx_read_state_unread ON read_state(user_did, is_read) WHERE is_read = 0;
572584 ```
573585
574-### 6.6 Annotations, Likes
586+### 6.6 Annotations, Likes (`<base>_articles`)
575587
576588 Local mirror of AT Protocol lexicon records for fast querying.
577589
@@ -602,7 +614,7 @@ CREATE TABLE likes (
602614 );
603615 ```
604616
605-### 6.7 Cluster Precomputation
617+### 6.7 Cluster Precomputation (`<base>_recs`)
606618
607619 Stores precomputed similarity data to avoid recalculating on every request.
608620
@@ -627,7 +639,7 @@ CREATE TABLE user_similarity (
627639 );
628640 ```
629641
630-### 6.8 Follows
642+### 6.8 Follows (`<base>_users`)
631643
632644 Tracks follow relationships between users (from `app.bsky.graph.follow` and `sh.tangled.graph.follow` records).
633645
@@ -645,7 +657,7 @@ CREATE INDEX idx_follows_user ON follows(user_did);
645657 CREATE INDEX idx_follows_target ON follows(target_did);
646658 ```
647659
648-### 6.9 OAuth Storage
660+### 6.9 OAuth Storage (`<base>_users`)
649661
650662 ```sql
651663 CREATE TABLE oauth_auth_requests (
@@ -899,7 +911,8 @@ glean/
899911 │ │ ├── sync.go # PDS record reconciliation
900912 │ │ └── xrpc.go # XRPC query handlers (AppView endpoints)
901913 │ ├── db/
902-│ │ ├── db.go # SQLite connection, migrations
914+│ │ ├── db.go # SQLite connection, single-DB schema
915+│ │ ├── multi.go # Multi-DB setup with ATTACH for cross-database queries
903916 │ │ ├── user.go # User queries
904917 │ │ ├── feed.go # Feed + subscription queries
905918 │ │ ├── article.go # Article queries
@@ -13,7 +13,7 @@ The core idea: your RSS subscriptions are a strong signal about your interests.
13 | Layer | Technology |13 | Layer | Technology |
14 | ---------------- | --------------------------------------------------------------- |14 | ---------------- | --------------------------------------------------------------- |
15 | Backend | Go |15 | Backend | Go |
16-| Database | SQLite (via `mattn/go-sqlite3`) |16+| Database | SQLite (3 files: users, articles, recs via `mattn/go-sqlite3`) |
17 | Frontend | htmx + TailwindCSS |17 | Frontend | htmx + TailwindCSS |
18 | Auth | AT Protocol OAuth / DID resolution (configurable PLC directory) |18 | Auth | AT Protocol OAuth / DID resolution (configurable PLC directory) |
19 | AT Protocol role | AppView for `at.glean.*` lexicons |19 | AT Protocol role | AppView for `at.glean.*` lexicons |
@@ -475,7 +475,19 @@ Glean runs as a single Go binary that fills three roles: **AppView** (indexing `
475 475
476 ## 6. Database Schema (SQLite)476 ## 6. Database Schema (SQLite)
477 477
478-### 6.1 Users478+Glean uses three separate SQLite database files to reduce write-lock contention. Each is opened with its own connection pool:
479+
480+| File | Contents | ATTACH alias |
481+|------|----------|-------------|
482+| `<base>_users` | Users, follows, OAuth | `main` (primary) |
483+| `<base>_articles` | Feeds, subscriptions, articles, read state, likes, annotations | `articles` |
484+| `<base>_recs` | Similarity scores, impressions, dismissals, signal weights | `recs` |
485+
486+The users connection pool uses a custom SQLite driver with a `ConnectHook` that ATTACHes the articles and recs databases on every new connection. This allows the cluster engine to run cross-database queries using schema prefixes (`articles.subscriptions`, `recs.user_similarity`, `main.follows`).
487+
488+The articles and recs connections are independent pools used by the service layer for single-database operations.
489+
490+### 6.1 Users (`<base>_users`)
479 491
480 ```sql492 ```sql
481 CREATE TABLE users (493 CREATE TABLE users (
@@ -488,7 +500,7 @@ CREATE TABLE users (
488 );500 );
489 ```501 ```
490 502
491-### 6.2 Feed Subscriptions503+### 6.2 Feed Subscriptions (`<base>_articles`)
492 504
493 Indexed from `at.glean.subscription` records on user PDS.505 Indexed from `at.glean.subscription` records on user PDS.
494 506
@@ -509,7 +521,7 @@ CREATE INDEX idx_subscriptions_feed ON subscriptions(feed_url);
509 CREATE INDEX idx_subscriptions_user ON subscriptions(user_did);521 CREATE INDEX idx_subscriptions_user ON subscriptions(user_did);
510 ```522 ```
511 523
512-### 6.3 Feeds524+### 6.3 Feeds (`<base>_articles`)
513 525
514 Master list of all known RSS feeds.526 Master list of all known RSS feeds.
515 527
@@ -533,7 +545,7 @@ CREATE TABLE feeds (
533 );545 );
534 ```546 ```
535 547
536-### 6.4 Articles548+### 6.4 Articles (`<base>_articles`)
537 549
538 Fetched from RSS feeds. Only fetched for feeds that have local subscribers.550 Fetched from RSS feeds. Only fetched for feeds that have local subscribers.
539 551
@@ -557,7 +569,7 @@ CREATE INDEX idx_articles_feed ON articles(feed_url);
557 CREATE INDEX idx_articles_published ON articles(published DESC);569 CREATE INDEX idx_articles_published ON articles(published DESC);
558 ```570 ```
559 571
560-### 6.5 Read State572+### 6.5 Read State (`<base>_articles`)
561 573
562 ```sql574 ```sql
563 CREATE TABLE read_state (575 CREATE TABLE read_state (
@@ -571,7 +583,7 @@ CREATE TABLE read_state (
571 CREATE INDEX idx_read_state_unread ON read_state(user_did, is_read) WHERE is_read = 0;583 CREATE INDEX idx_read_state_unread ON read_state(user_did, is_read) WHERE is_read = 0;
572 ```584 ```
573 585
574-### 6.6 Annotations, Likes586+### 6.6 Annotations, Likes (`<base>_articles`)
575 587
576 Local mirror of AT Protocol lexicon records for fast querying.588 Local mirror of AT Protocol lexicon records for fast querying.
577 589
@@ -602,7 +614,7 @@ CREATE TABLE likes (
602 );614 );
603 ```615 ```
604 616
605-### 6.7 Cluster Precomputation617+### 6.7 Cluster Precomputation (`<base>_recs`)
606 618
607 Stores precomputed similarity data to avoid recalculating on every request.619 Stores precomputed similarity data to avoid recalculating on every request.
608 620
@@ -627,7 +639,7 @@ CREATE TABLE user_similarity (
627 );639 );
628 ```640 ```
629 641
630-### 6.8 Follows642+### 6.8 Follows (`<base>_users`)
631 643
632 Tracks follow relationships between users (from `app.bsky.graph.follow` and `sh.tangled.graph.follow` records).644 Tracks follow relationships between users (from `app.bsky.graph.follow` and `sh.tangled.graph.follow` records).
633 645
@@ -645,7 +657,7 @@ CREATE INDEX idx_follows_user ON follows(user_did);
645 CREATE INDEX idx_follows_target ON follows(target_did);657 CREATE INDEX idx_follows_target ON follows(target_did);
646 ```658 ```
647 659
648-### 6.9 OAuth Storage660+### 6.9 OAuth Storage (`<base>_users`)
649 661
650 ```sql662 ```sql
651 CREATE TABLE oauth_auth_requests (663 CREATE TABLE oauth_auth_requests (
@@ -899,7 +911,8 @@ glean/
899 │ │ ├── sync.go # PDS record reconciliation911 │ │ ├── sync.go # PDS record reconciliation
900 │ │ └── xrpc.go # XRPC query handlers (AppView endpoints)912 │ │ └── xrpc.go # XRPC query handlers (AppView endpoints)
901 │ ├── db/913 │ ├── db/
902-│ │ ├── db.go # SQLite connection, migrations914+│ │ ├── db.go # SQLite connection, single-DB schema
915+│ │ ├── multi.go # Multi-DB setup with ATTACH for cross-database queries
903 │ │ ├── user.go # User queries916 │ │ ├── user.go # User queries
904 │ │ ├── feed.go # Feed + subscription queries917 │ │ ├── feed.go # Feed + subscription queries
905 │ │ ├── article.go # Article queries918 │ │ ├── article.go # Article queries
modified internal/atproto/stream_handler.go +24 -30
@@ -12,12 +12,13 @@ import (
1212 )
1313
1414 type StreamDBHandler struct {
15- db *db.DB
16- logger *slog.Logger
15+ articles *db.DB
16+ users *db.DB
17+ logger *slog.Logger
1718 }
1819
19-func NewStreamDBHandler(database *db.DB, logger *slog.Logger) *StreamDBHandler {
20- return &StreamDBHandler{db: database, logger: logger}
20+func NewStreamDBHandler(articles, users *db.DB, logger *slog.Logger) *StreamDBHandler {
21+ return &StreamDBHandler{articles: articles, users: users, logger: logger}
2122 }
2223
2324 func (h *StreamDBHandler) Handle(ctx context.Context, event *Event) error {
@@ -49,17 +50,17 @@ func (h *StreamDBHandler) handleSubscription(ctx context.Context, event *Event)
4950 return nil
5051 }
5152
52- existing, err := h.db.GetSubscription(ctx, event.DID, rec.FeedURL)
53+ existing, err := h.articles.GetSubscription(ctx, event.DID, rec.FeedURL)
5354 if err == nil && existing != nil {
5455 if !existing.URI.Valid || existing.URI.String == "" {
55- return h.db.UpdateSubscriptionURI(ctx, event.DID, rec.FeedURL, event.URI, event.CID)
56+ return h.articles.UpdateSubscriptionURI(ctx, event.DID, rec.FeedURL, event.URI, event.CID)
5657 }
5758 return nil
5859 }
5960
6061 f := &db.Feed{FeedURL: rec.FeedURL, Title: db.NullStr(rec.Title)}
61- _ = h.db.UpsertFeed(ctx, f)
62- err = h.db.CreateSubscription(ctx, event.DID, rec.FeedURL, rec.Title, rec.Category, event.URI, event.CID)
62+ _ = h.articles.UpsertFeed(ctx, f)
63+ err = h.articles.CreateSubscription(ctx, event.DID, rec.FeedURL, rec.Title, rec.Category, event.URI, event.CID)
6364 if errors.Is(err, db.ErrDuplicateSubscription) {
6465 return nil
6566 }
@@ -70,9 +71,9 @@ func (h *StreamDBHandler) handleSubscription(ctx context.Context, event *Event)
7071 if !ok {
7172 return nil
7273 }
73- sub, err := h.db.GetSubscriptionByURI(ctx, event.DID, event.URI)
74+ sub, err := h.articles.GetSubscriptionByURI(ctx, event.DID, event.URI)
7475 if err == nil && sub != nil {
75- return h.db.DeleteSubscription(ctx, event.DID, sub.FeedURL)
76+ return h.articles.DeleteSubscription(ctx, event.DID, sub.FeedURL)
7677 }
7778 _ = parsed
7879 }
@@ -90,13 +91,13 @@ func (h *StreamDBHandler) handleLike(ctx context.Context, event *Event) error {
9091 return nil
9192 }
9293
93- exists, err := h.db.HasLiked(ctx, event.DID, rec.FeedURL, rec.ArticleURL)
94+ exists, err := h.articles.HasLiked(ctx, event.DID, rec.FeedURL, rec.ArticleURL)
9495 if err != nil || exists {
9596 return nil
9697 }
9798
9899 t, _ := time.Parse(time.RFC3339, rec.CreatedAt)
99- err = h.db.CreateLike(ctx, &db.Like{
100+ err = h.articles.CreateLike(ctx, &db.Like{
100101 URI: event.URI,
101102 AuthorDID: event.DID,
102103 FeedURL: rec.FeedURL,
@@ -110,7 +111,7 @@ func (h *StreamDBHandler) handleLike(ctx context.Context, event *Event) error {
110111 return err
111112
112113 case "delete":
113- return h.db.DeleteLike(ctx, event.URI)
114+ return h.articles.DeleteLike(ctx, event.URI)
114115 }
115116 return nil
116117 }
@@ -141,10 +142,10 @@ func (h *StreamDBHandler) handleAnnotation(ctx context.Context, event *Event) er
141142 if rec.Rating > 0 {
142143 a.Rating = sql.NullInt64{Int64: int64(rec.Rating), Valid: true}
143144 }
144- return h.db.CreateAnnotation(ctx, a)
145+ return h.articles.CreateAnnotation(ctx, a)
145146
146147 case "delete":
147- return h.db.DeleteAnnotation(ctx, event.URI)
148+ return h.articles.DeleteAnnotation(ctx, event.URI)
148149 }
149150 return nil
150151 }
@@ -159,10 +160,10 @@ func (h *StreamDBHandler) handleFollow(ctx context.Context, event *Event) error
159160 if rec.Subject == "" {
160161 return nil
161162 }
162- return h.db.UpsertFollow(ctx, event.DID, rec.Subject, event.URI, event.CID)
163+ return h.users.UpsertFollow(ctx, event.DID, rec.Subject, event.URI, event.CID)
163164
164165 case "delete":
165- return h.db.DeleteFollowByURI(ctx, event.URI)
166+ return h.users.DeleteFollowByURI(ctx, event.URI)
166167 }
167168 return nil
168169 }
@@ -194,11 +195,9 @@ func (h *StreamDBHandler) handleMarginNote(ctx context.Context, event *Event) er
194195 CreatedAt: sql.NullTime{Time: t, Valid: true},
195196 CID: sql.NullString{String: event.CID, Valid: event.CID != ""},
196197 }
197- return h.db.CreateAnnotation(ctx, a)
198+ return h.articles.CreateAnnotation(ctx, a)
198199
199200 case "delete":
200- // TODO: I actually don't think we should delete an annotation on Glean if deleted from Margin
201- // return h.db.DeleteAnnotation(ctx, event.URI)
202201 }
203202 return nil
204203 }
@@ -214,10 +213,10 @@ func (h *StreamDBHandler) handleSkyreaderSubscription(ctx context.Context, event
214213 return nil
215214 }
216215
217- existing, err := h.db.GetSubscription(ctx, event.DID, rec.FeedURL)
216+ existing, err := h.articles.GetSubscription(ctx, event.DID, rec.FeedURL)
218217 if err == nil && existing != nil {
219218 if !existing.URI.Valid || existing.URI.String == "" {
220- return h.db.UpdateSubscriptionURI(ctx, event.DID, rec.FeedURL, event.URI, event.CID)
219+ return h.articles.UpdateSubscriptionURI(ctx, event.DID, rec.FeedURL, event.URI, event.CID)
221220 }
222221 return nil
223222 }
@@ -227,25 +226,20 @@ func (h *StreamDBHandler) handleSkyreaderSubscription(ctx context.Context, event
227226 Title: db.NullStr(rec.Title),
228227 SiteURL: db.NullStr(rec.SiteURL),
229228 }
230- _ = h.db.UpsertFeed(ctx, f)
231- err = h.db.CreateSubscription(ctx, event.DID, rec.FeedURL, rec.Title, rec.Category, event.URI, event.CID)
229+ _ = h.articles.UpsertFeed(ctx, f)
230+ err = h.articles.CreateSubscription(ctx, event.DID, rec.FeedURL, rec.Title, rec.Category, event.URI, event.CID)
232231 if errors.Is(err, db.ErrDuplicateSubscription) {
233232 return nil
234233 }
235234 return err
236235
237236 case "delete":
238- // TODO: I actually don't think we should delete an subscription on Glean if deleted from Skyreader
239- // sub, err := h.db.GetSubscriptionByURI(ctx, event.DID, event.URI)
240- // if err == nil && sub != nil {
241- // return h.db.DeleteSubscription(ctx, event.DID, sub.FeedURL)
242- // }
243237 }
244238 return nil
245239 }
246240
247241 func (h *StreamDBHandler) resolveFeedURL(ctx context.Context, articleURL string) string {
248- article, err := h.db.GetArticleByURL(ctx, articleURL)
242+ article, err := h.articles.GetArticleByURL(ctx, articleURL)
249243 if err != nil {
250244 return ""
251245 }
@@ -12,12 +12,13 @@ import (
12 )12 )
13 13
14 type StreamDBHandler struct {14 type StreamDBHandler struct {
15- db *db.DB15+ articles *db.DB
16- logger *slog.Logger16+ users *db.DB
17+ logger *slog.Logger
17 }18 }
18 19
19-func NewStreamDBHandler(database *db.DB, logger *slog.Logger) *StreamDBHandler {20+func NewStreamDBHandler(articles, users *db.DB, logger *slog.Logger) *StreamDBHandler {
20- return &StreamDBHandler{db: database, logger: logger}21+ return &StreamDBHandler{articles: articles, users: users, logger: logger}
21 }22 }
22 23
23 func (h *StreamDBHandler) Handle(ctx context.Context, event *Event) error {24 func (h *StreamDBHandler) Handle(ctx context.Context, event *Event) error {
@@ -49,17 +50,17 @@ func (h *StreamDBHandler) handleSubscription(ctx context.Context, event *Event)
49 return nil50 return nil
50 }51 }
51 52
52- existing, err := h.db.GetSubscription(ctx, event.DID, rec.FeedURL)53+ existing, err := h.articles.GetSubscription(ctx, event.DID, rec.FeedURL)
53 if err == nil && existing != nil {54 if err == nil && existing != nil {
54 if !existing.URI.Valid || existing.URI.String == "" {55 if !existing.URI.Valid || existing.URI.String == "" {
55- return h.db.UpdateSubscriptionURI(ctx, event.DID, rec.FeedURL, event.URI, event.CID)56+ return h.articles.UpdateSubscriptionURI(ctx, event.DID, rec.FeedURL, event.URI, event.CID)
56 }57 }
57 return nil58 return nil
58 }59 }
59 60
60 f := &db.Feed{FeedURL: rec.FeedURL, Title: db.NullStr(rec.Title)}61 f := &db.Feed{FeedURL: rec.FeedURL, Title: db.NullStr(rec.Title)}
61- _ = h.db.UpsertFeed(ctx, f)62+ _ = h.articles.UpsertFeed(ctx, f)
62- err = h.db.CreateSubscription(ctx, event.DID, rec.FeedURL, rec.Title, rec.Category, event.URI, event.CID)63+ err = h.articles.CreateSubscription(ctx, event.DID, rec.FeedURL, rec.Title, rec.Category, event.URI, event.CID)
63 if errors.Is(err, db.ErrDuplicateSubscription) {64 if errors.Is(err, db.ErrDuplicateSubscription) {
64 return nil65 return nil
65 }66 }
@@ -70,9 +71,9 @@ func (h *StreamDBHandler) handleSubscription(ctx context.Context, event *Event)
70 if !ok {71 if !ok {
71 return nil72 return nil
72 }73 }
73- sub, err := h.db.GetSubscriptionByURI(ctx, event.DID, event.URI)74+ sub, err := h.articles.GetSubscriptionByURI(ctx, event.DID, event.URI)
74 if err == nil && sub != nil {75 if err == nil && sub != nil {
75- return h.db.DeleteSubscription(ctx, event.DID, sub.FeedURL)76+ return h.articles.DeleteSubscription(ctx, event.DID, sub.FeedURL)
76 }77 }
77 _ = parsed78 _ = parsed
78 }79 }
@@ -90,13 +91,13 @@ func (h *StreamDBHandler) handleLike(ctx context.Context, event *Event) error {
90 return nil91 return nil
91 }92 }
92 93
93- exists, err := h.db.HasLiked(ctx, event.DID, rec.FeedURL, rec.ArticleURL)94+ exists, err := h.articles.HasLiked(ctx, event.DID, rec.FeedURL, rec.ArticleURL)
94 if err != nil || exists {95 if err != nil || exists {
95 return nil96 return nil
96 }97 }
97 98
98 t, _ := time.Parse(time.RFC3339, rec.CreatedAt)99 t, _ := time.Parse(time.RFC3339, rec.CreatedAt)
99- err = h.db.CreateLike(ctx, &db.Like{100+ err = h.articles.CreateLike(ctx, &db.Like{
100 URI: event.URI,101 URI: event.URI,
101 AuthorDID: event.DID,102 AuthorDID: event.DID,
102 FeedURL: rec.FeedURL,103 FeedURL: rec.FeedURL,
@@ -110,7 +111,7 @@ func (h *StreamDBHandler) handleLike(ctx context.Context, event *Event) error {
110 return err111 return err
111 112
112 case "delete":113 case "delete":
113- return h.db.DeleteLike(ctx, event.URI)114+ return h.articles.DeleteLike(ctx, event.URI)
114 }115 }
115 return nil116 return nil
116 }117 }
@@ -141,10 +142,10 @@ func (h *StreamDBHandler) handleAnnotation(ctx context.Context, event *Event) er
141 if rec.Rating > 0 {142 if rec.Rating > 0 {
142 a.Rating = sql.NullInt64{Int64: int64(rec.Rating), Valid: true}143 a.Rating = sql.NullInt64{Int64: int64(rec.Rating), Valid: true}
143 }144 }
144- return h.db.CreateAnnotation(ctx, a)145+ return h.articles.CreateAnnotation(ctx, a)
145 146
146 case "delete":147 case "delete":
147- return h.db.DeleteAnnotation(ctx, event.URI)148+ return h.articles.DeleteAnnotation(ctx, event.URI)
148 }149 }
149 return nil150 return nil
150 }151 }
@@ -159,10 +160,10 @@ func (h *StreamDBHandler) handleFollow(ctx context.Context, event *Event) error
159 if rec.Subject == "" {160 if rec.Subject == "" {
160 return nil161 return nil
161 }162 }
162- return h.db.UpsertFollow(ctx, event.DID, rec.Subject, event.URI, event.CID)163+ return h.users.UpsertFollow(ctx, event.DID, rec.Subject, event.URI, event.CID)
163 164
164 case "delete":165 case "delete":
165- return h.db.DeleteFollowByURI(ctx, event.URI)166+ return h.users.DeleteFollowByURI(ctx, event.URI)
166 }167 }
167 return nil168 return nil
168 }169 }
@@ -194,11 +195,9 @@ func (h *StreamDBHandler) handleMarginNote(ctx context.Context, event *Event) er
194 CreatedAt: sql.NullTime{Time: t, Valid: true},195 CreatedAt: sql.NullTime{Time: t, Valid: true},
195 CID: sql.NullString{String: event.CID, Valid: event.CID != ""},196 CID: sql.NullString{String: event.CID, Valid: event.CID != ""},
196 }197 }
197- return h.db.CreateAnnotation(ctx, a)198+ return h.articles.CreateAnnotation(ctx, a)
198 199
199 case "delete":200 case "delete":
200- // TODO: I actually don't think we should delete an annotation on Glean if deleted from Margin
201- // return h.db.DeleteAnnotation(ctx, event.URI)
202 }201 }
203 return nil202 return nil
204 }203 }
@@ -214,10 +213,10 @@ func (h *StreamDBHandler) handleSkyreaderSubscription(ctx context.Context, event
214 return nil213 return nil
215 }214 }
216 215
217- existing, err := h.db.GetSubscription(ctx, event.DID, rec.FeedURL)216+ existing, err := h.articles.GetSubscription(ctx, event.DID, rec.FeedURL)
218 if err == nil && existing != nil {217 if err == nil && existing != nil {
219 if !existing.URI.Valid || existing.URI.String == "" {218 if !existing.URI.Valid || existing.URI.String == "" {
220- return h.db.UpdateSubscriptionURI(ctx, event.DID, rec.FeedURL, event.URI, event.CID)219+ return h.articles.UpdateSubscriptionURI(ctx, event.DID, rec.FeedURL, event.URI, event.CID)
221 }220 }
222 return nil221 return nil
223 }222 }
@@ -227,25 +226,20 @@ func (h *StreamDBHandler) handleSkyreaderSubscription(ctx context.Context, event
227 Title: db.NullStr(rec.Title),226 Title: db.NullStr(rec.Title),
228 SiteURL: db.NullStr(rec.SiteURL),227 SiteURL: db.NullStr(rec.SiteURL),
229 }228 }
230- _ = h.db.UpsertFeed(ctx, f)229+ _ = h.articles.UpsertFeed(ctx, f)
231- err = h.db.CreateSubscription(ctx, event.DID, rec.FeedURL, rec.Title, rec.Category, event.URI, event.CID)230+ err = h.articles.CreateSubscription(ctx, event.DID, rec.FeedURL, rec.Title, rec.Category, event.URI, event.CID)
232 if errors.Is(err, db.ErrDuplicateSubscription) {231 if errors.Is(err, db.ErrDuplicateSubscription) {
233 return nil232 return nil
234 }233 }
235 return err234 return err
236 235
237 case "delete":236 case "delete":
238- // TODO: I actually don't think we should delete an subscription on Glean if deleted from Skyreader
239- // sub, err := h.db.GetSubscriptionByURI(ctx, event.DID, event.URI)
240- // if err == nil && sub != nil {
241- // return h.db.DeleteSubscription(ctx, event.DID, sub.FeedURL)
242- // }
243 }237 }
244 return nil238 return nil
245 }239 }
246 240
247 func (h *StreamDBHandler) resolveFeedURL(ctx context.Context, articleURL string) string {241 func (h *StreamDBHandler) resolveFeedURL(ctx context.Context, articleURL string) string {
248- article, err := h.db.GetArticleByURL(ctx, articleURL)242+ article, err := h.articles.GetArticleByURL(ctx, articleURL)
249 if err != nil {243 if err != nil {
250 return ""244 return ""
251 }245 }
modified internal/atproto/sync.go +23 -22
@@ -22,13 +22,14 @@ import (
2222 )
2323
2424 type Sync struct {
25- db *db.DB
26- client *Client
27- logger *slog.Logger
25+ articles *db.DB
26+ users *db.DB
27+ client *Client
28+ logger *slog.Logger
2829 }
2930
30-func NewSync(database *db.DB, client *Client, logger *slog.Logger) *Sync {
31- return &Sync{db: database, client: client, logger: logger}
31+func NewSync(articles, users *db.DB, client *Client, logger *slog.Logger) *Sync {
32+ return &Sync{articles: articles, users: users, client: client, logger: logger}
3233 }
3334
3435 func (s *Sync) Run(ctx context.Context, userDID string) error {
@@ -91,17 +92,17 @@ func (s *Sync) reconcileSubscription(ctx context.Context, userDID, uri, cid stri
9192 }
9293
9394 f := &db.Feed{FeedURL: rec.FeedURL, Title: db.NullStr(rec.Title)}
94- _ = s.db.UpsertFeed(ctx, f)
95+ _ = s.articles.UpsertFeed(ctx, f)
9596
96- existing, err := s.db.GetSubscription(ctx, userDID, rec.FeedURL)
97+ existing, err := s.articles.GetSubscription(ctx, userDID, rec.FeedURL)
9798 if err == nil && existing != nil {
9899 if !existing.URI.Valid || existing.URI.String == "" {
99- return s.db.UpdateSubscriptionURI(ctx, userDID, rec.FeedURL, uri, cid)
100+ return s.articles.UpdateSubscriptionURI(ctx, userDID, rec.FeedURL, uri, cid)
100101 }
101102 return nil
102103 }
103104
104- err = s.db.CreateSubscription(ctx, userDID, rec.FeedURL, rec.Title, rec.Category, uri, cid)
105+ err = s.articles.CreateSubscription(ctx, userDID, rec.FeedURL, rec.Title, rec.Category, uri, cid)
105106 if errors.Is(err, db.ErrDuplicateSubscription) {
106107 return nil
107108 }
@@ -119,17 +120,17 @@ func (s *Sync) reconcileSkyreaderSubscription(ctx context.Context, userDID, uri,
119120 }
120121
121122 f := &db.Feed{FeedURL: rec.FeedURL, Title: db.NullStr(rec.Title), SiteURL: db.NullStr(rec.SiteURL)}
122- _ = s.db.UpsertFeed(ctx, f)
123+ _ = s.articles.UpsertFeed(ctx, f)
123124
124- existing, err := s.db.GetSubscription(ctx, userDID, rec.FeedURL)
125+ existing, err := s.articles.GetSubscription(ctx, userDID, rec.FeedURL)
125126 if err == nil && existing != nil {
126127 if !existing.URI.Valid || existing.URI.String == "" {
127- return s.db.UpdateSubscriptionURI(ctx, userDID, rec.FeedURL, uri, cid)
128+ return s.articles.UpdateSubscriptionURI(ctx, userDID, rec.FeedURL, uri, cid)
128129 }
129130 return nil
130131 }
131132
132- err = s.db.CreateSubscription(ctx, userDID, rec.FeedURL, rec.Title, "", uri, cid)
133+ err = s.articles.CreateSubscription(ctx, userDID, rec.FeedURL, rec.Title, "", uri, cid)
133134 if errors.Is(err, db.ErrDuplicateSubscription) {
134135 return nil
135136 }
@@ -146,7 +147,7 @@ func (s *Sync) reconcileLike(ctx context.Context, userDID, uri, cid string, valu
146147 return nil
147148 }
148149
149- exists, err := s.db.HasLiked(ctx, userDID, rec.FeedURL, rec.ArticleURL)
150+ exists, err := s.articles.HasLiked(ctx, userDID, rec.FeedURL, rec.ArticleURL)
150151 if err != nil {
151152 return err
152153 }
@@ -163,7 +164,7 @@ func (s *Sync) reconcileLike(ctx context.Context, userDID, uri, cid string, valu
163164 CreatedAt: db.NullTime(t),
164165 CID: db.NullStr(cid),
165166 }
166- err = s.db.CreateLike(ctx, like)
167+ err = s.articles.CreateLike(ctx, like)
167168 if errors.Is(err, db.ErrDuplicateLike) {
168169 return nil
169170 }
@@ -180,7 +181,7 @@ func (s *Sync) reconcileAnnotation(ctx context.Context, userDID, uri, cid string
180181 return nil
181182 }
182183
183- exists, err := s.db.AnnotationExists(ctx, uri)
184+ exists, err := s.articles.AnnotationExists(ctx, uri)
184185 if err != nil || exists {
185186 return err
186187 }
@@ -200,7 +201,7 @@ func (s *Sync) reconcileAnnotation(ctx context.Context, userDID, uri, cid string
200201 if rec.Rating > 0 {
201202 a.Rating = db.NullInt(int64(rec.Rating))
202203 }
203- return s.db.CreateAnnotation(ctx, a)
204+ return s.articles.CreateAnnotation(ctx, a)
204205 }
205206
206207 func (s *Sync) reconcileMarginNote(ctx context.Context, userDID, uri, cid string, value json.RawMessage) error {
@@ -214,13 +215,13 @@ func (s *Sync) reconcileMarginNote(ctx context.Context, userDID, uri, cid string
214215 return nil
215216 }
216217
217- exists, err := s.db.AnnotationExists(ctx, uri)
218+ exists, err := s.articles.AnnotationExists(ctx, uri)
218219 if err != nil || exists {
219220 return err
220221 }
221222
222223 feedURL := ""
223- if article, err := s.db.GetArticleByURL(ctx, articleURL); err == nil {
224+ if article, err := s.articles.GetArticleByURL(ctx, articleURL); err == nil {
224225 feedURL = article.FeedURL
225226 }
226227
@@ -236,7 +237,7 @@ func (s *Sync) reconcileMarginNote(ctx context.Context, userDID, uri, cid string
236237 CreatedAt: db.NullTime(t),
237238 CID: db.NullStr(cid),
238239 }
239- return s.db.CreateAnnotation(ctx, a)
240+ return s.articles.CreateAnnotation(ctx, a)
240241 }
241242
242243 func (s *Sync) syncFollows(ctx context.Context, userDID string) error {
@@ -274,7 +275,7 @@ func (s *Sync) syncFollows(ctx context.Context, userDID string) error {
274275 avatarURL = avatar
275276 }
276277
277- s.db.CreateUser(ctx, rec.Subject, handle, displayName, avatarURL)
278+ s.users.CreateUser(ctx, rec.Subject, handle, displayName, avatarURL)
278279 }
279280
280281 if next == "" || len(records) == 0 {
@@ -288,5 +289,5 @@ func (s *Sync) syncFollows(ctx context.Context, userDID string) error {
288289 return nil
289290 }
290291
291- return s.db.SyncFollows(ctx, userDID, activeFollows)
292+ return s.users.SyncFollows(ctx, userDID, activeFollows)
292293 }
@@ -22,13 +22,14 @@ import (
22 )22 )
23 23
24 type Sync struct {24 type Sync struct {
25- db *db.DB25+ articles *db.DB
26- client *Client26+ users *db.DB
27- logger *slog.Logger27+ client *Client
28+ logger *slog.Logger
28 }29 }
29 30
30-func NewSync(database *db.DB, client *Client, logger *slog.Logger) *Sync {31+func NewSync(articles, users *db.DB, client *Client, logger *slog.Logger) *Sync {
31- return &Sync{db: database, client: client, logger: logger}32+ return &Sync{articles: articles, users: users, client: client, logger: logger}
32 }33 }
33 34
34 func (s *Sync) Run(ctx context.Context, userDID string) error {35 func (s *Sync) Run(ctx context.Context, userDID string) error {
@@ -91,17 +92,17 @@ func (s *Sync) reconcileSubscription(ctx context.Context, userDID, uri, cid stri
91 }92 }
92 93
93 f := &db.Feed{FeedURL: rec.FeedURL, Title: db.NullStr(rec.Title)}94 f := &db.Feed{FeedURL: rec.FeedURL, Title: db.NullStr(rec.Title)}
94- _ = s.db.UpsertFeed(ctx, f)95+ _ = s.articles.UpsertFeed(ctx, f)
95 96
96- existing, err := s.db.GetSubscription(ctx, userDID, rec.FeedURL)97+ existing, err := s.articles.GetSubscription(ctx, userDID, rec.FeedURL)
97 if err == nil && existing != nil {98 if err == nil && existing != nil {
98 if !existing.URI.Valid || existing.URI.String == "" {99 if !existing.URI.Valid || existing.URI.String == "" {
99- return s.db.UpdateSubscriptionURI(ctx, userDID, rec.FeedURL, uri, cid)100+ return s.articles.UpdateSubscriptionURI(ctx, userDID, rec.FeedURL, uri, cid)
100 }101 }
101 return nil102 return nil
102 }103 }
103 104
104- err = s.db.CreateSubscription(ctx, userDID, rec.FeedURL, rec.Title, rec.Category, uri, cid)105+ err = s.articles.CreateSubscription(ctx, userDID, rec.FeedURL, rec.Title, rec.Category, uri, cid)
105 if errors.Is(err, db.ErrDuplicateSubscription) {106 if errors.Is(err, db.ErrDuplicateSubscription) {
106 return nil107 return nil
107 }108 }
@@ -119,17 +120,17 @@ func (s *Sync) reconcileSkyreaderSubscription(ctx context.Context, userDID, uri,
119 }120 }
120 121
121 f := &db.Feed{FeedURL: rec.FeedURL, Title: db.NullStr(rec.Title), SiteURL: db.NullStr(rec.SiteURL)}122 f := &db.Feed{FeedURL: rec.FeedURL, Title: db.NullStr(rec.Title), SiteURL: db.NullStr(rec.SiteURL)}
122- _ = s.db.UpsertFeed(ctx, f)123+ _ = s.articles.UpsertFeed(ctx, f)
123 124
124- existing, err := s.db.GetSubscription(ctx, userDID, rec.FeedURL)125+ existing, err := s.articles.GetSubscription(ctx, userDID, rec.FeedURL)
125 if err == nil && existing != nil {126 if err == nil && existing != nil {
126 if !existing.URI.Valid || existing.URI.String == "" {127 if !existing.URI.Valid || existing.URI.String == "" {
127- return s.db.UpdateSubscriptionURI(ctx, userDID, rec.FeedURL, uri, cid)128+ return s.articles.UpdateSubscriptionURI(ctx, userDID, rec.FeedURL, uri, cid)
128 }129 }
129 return nil130 return nil
130 }131 }
131 132
132- err = s.db.CreateSubscription(ctx, userDID, rec.FeedURL, rec.Title, "", uri, cid)133+ err = s.articles.CreateSubscription(ctx, userDID, rec.FeedURL, rec.Title, "", uri, cid)
133 if errors.Is(err, db.ErrDuplicateSubscription) {134 if errors.Is(err, db.ErrDuplicateSubscription) {
134 return nil135 return nil
135 }136 }
@@ -146,7 +147,7 @@ func (s *Sync) reconcileLike(ctx context.Context, userDID, uri, cid string, valu
146 return nil147 return nil
147 }148 }
148 149
149- exists, err := s.db.HasLiked(ctx, userDID, rec.FeedURL, rec.ArticleURL)150+ exists, err := s.articles.HasLiked(ctx, userDID, rec.FeedURL, rec.ArticleURL)
150 if err != nil {151 if err != nil {
151 return err152 return err
152 }153 }
@@ -163,7 +164,7 @@ func (s *Sync) reconcileLike(ctx context.Context, userDID, uri, cid string, valu
163 CreatedAt: db.NullTime(t),164 CreatedAt: db.NullTime(t),
164 CID: db.NullStr(cid),165 CID: db.NullStr(cid),
165 }166 }
166- err = s.db.CreateLike(ctx, like)167+ err = s.articles.CreateLike(ctx, like)
167 if errors.Is(err, db.ErrDuplicateLike) {168 if errors.Is(err, db.ErrDuplicateLike) {
168 return nil169 return nil
169 }170 }
@@ -180,7 +181,7 @@ func (s *Sync) reconcileAnnotation(ctx context.Context, userDID, uri, cid string
180 return nil181 return nil
181 }182 }
182 183
183- exists, err := s.db.AnnotationExists(ctx, uri)184+ exists, err := s.articles.AnnotationExists(ctx, uri)
184 if err != nil || exists {185 if err != nil || exists {
185 return err186 return err
186 }187 }
@@ -200,7 +201,7 @@ func (s *Sync) reconcileAnnotation(ctx context.Context, userDID, uri, cid string
200 if rec.Rating > 0 {201 if rec.Rating > 0 {
201 a.Rating = db.NullInt(int64(rec.Rating))202 a.Rating = db.NullInt(int64(rec.Rating))
202 }203 }
203- return s.db.CreateAnnotation(ctx, a)204+ return s.articles.CreateAnnotation(ctx, a)
204 }205 }
205 206
206 func (s *Sync) reconcileMarginNote(ctx context.Context, userDID, uri, cid string, value json.RawMessage) error {207 func (s *Sync) reconcileMarginNote(ctx context.Context, userDID, uri, cid string, value json.RawMessage) error {
@@ -214,13 +215,13 @@ func (s *Sync) reconcileMarginNote(ctx context.Context, userDID, uri, cid string
214 return nil215 return nil
215 }216 }
216 217
217- exists, err := s.db.AnnotationExists(ctx, uri)218+ exists, err := s.articles.AnnotationExists(ctx, uri)
218 if err != nil || exists {219 if err != nil || exists {
219 return err220 return err
220 }221 }
221 222
222 feedURL := ""223 feedURL := ""
223- if article, err := s.db.GetArticleByURL(ctx, articleURL); err == nil {224+ if article, err := s.articles.GetArticleByURL(ctx, articleURL); err == nil {
224 feedURL = article.FeedURL225 feedURL = article.FeedURL
225 }226 }
226 227
@@ -236,7 +237,7 @@ func (s *Sync) reconcileMarginNote(ctx context.Context, userDID, uri, cid string
236 CreatedAt: db.NullTime(t),237 CreatedAt: db.NullTime(t),
237 CID: db.NullStr(cid),238 CID: db.NullStr(cid),
238 }239 }
239- return s.db.CreateAnnotation(ctx, a)240+ return s.articles.CreateAnnotation(ctx, a)
240 }241 }
241 242
242 func (s *Sync) syncFollows(ctx context.Context, userDID string) error {243 func (s *Sync) syncFollows(ctx context.Context, userDID string) error {
@@ -274,7 +275,7 @@ func (s *Sync) syncFollows(ctx context.Context, userDID string) error {
274 avatarURL = avatar275 avatarURL = avatar
275 }276 }
276 277
277- s.db.CreateUser(ctx, rec.Subject, handle, displayName, avatarURL)278+ s.users.CreateUser(ctx, rec.Subject, handle, displayName, avatarURL)
278 }279 }
279 280
280 if next == "" || len(records) == 0 {281 if next == "" || len(records) == 0 {
@@ -288,5 +289,5 @@ func (s *Sync) syncFollows(ctx context.Context, userDID string) error {
288 return nil289 return nil
289 }290 }
290 291
291- return s.db.SyncFollows(ctx, userDID, activeFollows)292+ return s.users.SyncFollows(ctx, userDID, activeFollows)
292 }293 }
modified internal/cluster/dismiss.go +8 -8
@@ -12,7 +12,7 @@ type Impression struct {
1212
1313 func (e *Engine) DismissFeed(ctx context.Context, userDID, feedURL, reason string) error {
1414 _, err := e.db.ExecContext(ctx, `
15- INSERT INTO dismissed_recommendations (user_did, target_type, target_id, reason)
15+ INSERT INTO recs.dismissed_recommendations (user_did, target_type, target_id, reason)
1616 VALUES (?, 'feed', ?, ?)
1717 ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP
1818 `, userDID, feedURL, reason)
@@ -21,7 +21,7 @@ func (e *Engine) DismissFeed(ctx context.Context, userDID, feedURL, reason strin
2121
2222 func (e *Engine) DismissArticle(ctx context.Context, userDID, articleURL, reason string) error {
2323 _, err := e.db.ExecContext(ctx, `
24- INSERT INTO dismissed_recommendations (user_did, target_type, target_id, reason)
24+ INSERT INTO recs.dismissed_recommendations (user_did, target_type, target_id, reason)
2525 VALUES (?, 'article', ?, ?)
2626 ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP
2727 `, userDID, articleURL, reason)
@@ -37,7 +37,7 @@ func (e *Engine) RecordImpressions(ctx context.Context, userDID string, impressi
3737
3838 for _, imp := range impressions {
3939 _, err := tx.ExecContext(ctx, `
40- INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count)
40+ INSERT INTO recs.recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count)
4141 VALUES (?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, 1)
4242 ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET
4343 last_shown_at = CURRENT_TIMESTAMP,
@@ -52,7 +52,7 @@ func (e *Engine) RecordImpressions(ctx context.Context, userDID string, impressi
5252
5353 func (e *Engine) MarkImpressionActed(ctx context.Context, userDID, targetType, targetID string) error {
5454 _, err := e.db.ExecContext(ctx, `
55- UPDATE recommendation_impressions SET acted = 1
55+ UPDATE recs.recommendation_impressions SET acted = 1
5656 WHERE user_did = ? AND target_type = ? AND target_id = ?
5757 `, userDID, targetType, targetID)
5858 return err
@@ -62,9 +62,9 @@ func (e *Engine) AutoDismissStale(ctx context.Context, minShownCount int, maxAge
6262 cutoff := time.Now().AddDate(0, 0, -maxAgeDays).Format(time.RFC3339)
6363
6464 _, err := e.db.ExecContext(ctx, `
65- INSERT OR IGNORE INTO dismissed_recommendations (user_did, target_type, target_id, reason, dismissed_at)
65+ INSERT OR IGNORE INTO recs.dismissed_recommendations (user_did, target_type, target_id, reason, dismissed_at)
6666 SELECT user_did, target_type, target_id, 'auto_stale', CURRENT_TIMESTAMP
67- FROM recommendation_impressions
67+ FROM recs.recommendation_impressions
6868 WHERE acted = 0
6969 AND shown_count >= ?
7070 AND first_shown_at < ?
@@ -75,8 +75,8 @@ func (e *Engine) AutoDismissStale(ctx context.Context, minShownCount int, maxAge
7575 func (e *Engine) IsFeedDismissed(ctx context.Context, userDID, feedURL string) (bool, error) {
7676 var count int
7777 err := e.db.QueryRowContext(ctx, `
78- SELECT COUNT(1) FROM dismissed_recommendations
78+ SELECT COUNT(1) FROM recs.dismissed_recommendations
7979 WHERE user_did = ? AND target_type = 'feed' AND target_id = ?
8080 `, userDID, feedURL).Scan(&count)
8181 return count > 0, err
82-}
82+}
\ No newline at end of file
@@ -12,7 +12,7 @@ type Impression struct {
12 12
13 func (e *Engine) DismissFeed(ctx context.Context, userDID, feedURL, reason string) error {13 func (e *Engine) DismissFeed(ctx context.Context, userDID, feedURL, reason string) error {
14 _, err := e.db.ExecContext(ctx, `14 _, err := e.db.ExecContext(ctx, `
15- INSERT INTO dismissed_recommendations (user_did, target_type, target_id, reason)15+ INSERT INTO recs.dismissed_recommendations (user_did, target_type, target_id, reason)
16 VALUES (?, 'feed', ?, ?)16 VALUES (?, 'feed', ?, ?)
17 ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP17 ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP
18 `, userDID, feedURL, reason)18 `, userDID, feedURL, reason)
@@ -21,7 +21,7 @@ func (e *Engine) DismissFeed(ctx context.Context, userDID, feedURL, reason strin
21 21
22 func (e *Engine) DismissArticle(ctx context.Context, userDID, articleURL, reason string) error {22 func (e *Engine) DismissArticle(ctx context.Context, userDID, articleURL, reason string) error {
23 _, err := e.db.ExecContext(ctx, `23 _, err := e.db.ExecContext(ctx, `
24- INSERT INTO dismissed_recommendations (user_did, target_type, target_id, reason)24+ INSERT INTO recs.dismissed_recommendations (user_did, target_type, target_id, reason)
25 VALUES (?, 'article', ?, ?)25 VALUES (?, 'article', ?, ?)
26 ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP26 ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP
27 `, userDID, articleURL, reason)27 `, userDID, articleURL, reason)
@@ -37,7 +37,7 @@ func (e *Engine) RecordImpressions(ctx context.Context, userDID string, impressi
37 37
38 for _, imp := range impressions {38 for _, imp := range impressions {
39 _, err := tx.ExecContext(ctx, `39 _, err := tx.ExecContext(ctx, `
40- INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count)40+ INSERT INTO recs.recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count)
41 VALUES (?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, 1)41 VALUES (?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, 1)
42 ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET42 ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET
43 last_shown_at = CURRENT_TIMESTAMP,43 last_shown_at = CURRENT_TIMESTAMP,
@@ -52,7 +52,7 @@ func (e *Engine) RecordImpressions(ctx context.Context, userDID string, impressi
52 52
53 func (e *Engine) MarkImpressionActed(ctx context.Context, userDID, targetType, targetID string) error {53 func (e *Engine) MarkImpressionActed(ctx context.Context, userDID, targetType, targetID string) error {
54 _, err := e.db.ExecContext(ctx, `54 _, err := e.db.ExecContext(ctx, `
55- UPDATE recommendation_impressions SET acted = 155+ UPDATE recs.recommendation_impressions SET acted = 1
56 WHERE user_did = ? AND target_type = ? AND target_id = ?56 WHERE user_did = ? AND target_type = ? AND target_id = ?
57 `, userDID, targetType, targetID)57 `, userDID, targetType, targetID)
58 return err58 return err
@@ -62,9 +62,9 @@ func (e *Engine) AutoDismissStale(ctx context.Context, minShownCount int, maxAge
62 cutoff := time.Now().AddDate(0, 0, -maxAgeDays).Format(time.RFC3339)62 cutoff := time.Now().AddDate(0, 0, -maxAgeDays).Format(time.RFC3339)
63 63
64 _, err := e.db.ExecContext(ctx, `64 _, err := e.db.ExecContext(ctx, `
65- INSERT OR IGNORE INTO dismissed_recommendations (user_did, target_type, target_id, reason, dismissed_at)65+ INSERT OR IGNORE INTO recs.dismissed_recommendations (user_did, target_type, target_id, reason, dismissed_at)
66 SELECT user_did, target_type, target_id, 'auto_stale', CURRENT_TIMESTAMP66 SELECT user_did, target_type, target_id, 'auto_stale', CURRENT_TIMESTAMP
67- FROM recommendation_impressions67+ FROM recs.recommendation_impressions
68 WHERE acted = 068 WHERE acted = 0
69 AND shown_count >= ?69 AND shown_count >= ?
70 AND first_shown_at < ?70 AND first_shown_at < ?
@@ -75,8 +75,8 @@ func (e *Engine) AutoDismissStale(ctx context.Context, minShownCount int, maxAge
75 func (e *Engine) IsFeedDismissed(ctx context.Context, userDID, feedURL string) (bool, error) {75 func (e *Engine) IsFeedDismissed(ctx context.Context, userDID, feedURL string) (bool, error) {
76 var count int76 var count int
77 err := e.db.QueryRowContext(ctx, `77 err := e.db.QueryRowContext(ctx, `
78- SELECT COUNT(1) FROM dismissed_recommendations78+ SELECT COUNT(1) FROM recs.dismissed_recommendations
79 WHERE user_did = ? AND target_type = 'feed' AND target_id = ?79 WHERE user_did = ? AND target_type = 'feed' AND target_id = ?
80 `, userDID, feedURL).Scan(&count)80 `, userDID, feedURL).Scan(&count)
81 return count > 0, err81 return count > 0, err
82-}82+}
\ No newline at end of file\ No newline at end of file
modified internal/cluster/jaccard.go +39 -39
@@ -42,20 +42,20 @@ func (e *Engine) ComputeFeedSimilarity(ctx context.Context) error {
4242 }
4343 defer func() { _ = tx.Rollback() }()
4444
45- if _, err := tx.ExecContext(ctx, `DELETE FROM feed_similarity`); err != nil {
45+ if _, err := tx.ExecContext(ctx, `DELETE FROM recs.feed_similarity`); err != nil {
4646 return err
4747 }
4848
4949 _, err = tx.ExecContext(ctx, `
50- INSERT INTO feed_similarity (feed_a, feed_b, jaccard)
50+ INSERT INTO recs.feed_similarity (feed_a, feed_b, jaccard)
5151 SELECT
5252 s1.feed_url,
5353 s2.feed_url,
5454 CAST(COUNT(*) AS REAL) / (f1.subscriber_count + f2.subscriber_count - CAST(COUNT(*) AS REAL))
55- FROM subscriptions s1
56- JOIN subscriptions s2 ON s1.user_did = s2.user_did AND s1.feed_url < s2.feed_url
57- JOIN feeds f1 ON f1.feed_url = s1.feed_url
58- JOIN feeds f2 ON f2.feed_url = s2.feed_url
55+ FROM articles.subscriptions s1
56+ JOIN articles.subscriptions s2 ON s1.user_did = s2.user_did AND s1.feed_url < s2.feed_url
57+ JOIN articles.feeds f1 ON f1.feed_url = s1.feed_url
58+ JOIN articles.feeds f2 ON f2.feed_url = s2.feed_url
5959 GROUP BY s1.feed_url, s2.feed_url
6060 `)
6161 if err != nil {
@@ -82,7 +82,7 @@ func (e *Engine) computeDescriptionSimilarity(ctx context.Context, tx *sql.Tx) e
8282 INSERT INTO _feed_words (feed_url, word)
8383 WITH feed_tokens AS (
8484 SELECT feed_url, LOWER(TRIM(value)) AS word
85- FROM feeds,
85+ FROM articles.feeds,
8686 json_each('["' || REPLACE(LOWER(COALESCE(description, '')), ' ', '","') || '"]')
8787 WHERE description IS NOT NULL AND description != ''
8888 )
@@ -140,7 +140,7 @@ func (e *Engine) computeDescriptionSimilarity(ctx context.Context, tx *sql.Tx) e
140140 }
141141
142142 descInsert := `
143- INSERT OR IGNORE INTO feed_similarity (feed_a, feed_b, jaccard)
143+ INSERT OR IGNORE INTO recs.feed_similarity (feed_a, feed_b, jaccard)
144144 SELECT feed_a, feed_b, 0 FROM _word_overlap
145145 `
146146 if _, err := tx.ExecContext(ctx, descInsert); err != nil {
@@ -148,16 +148,16 @@ func (e *Engine) computeDescriptionSimilarity(ctx context.Context, tx *sql.Tx) e
148148 }
149149
150150 descUpdate := fmt.Sprintf(`
151- UPDATE feed_similarity SET
151+ UPDATE recs.feed_similarity SET
152152 jaccard = jaccard + %g * CAST(_word_overlap.common AS REAL) / NULLIF(
153- (SELECT cnt FROM _feed_word_counts WHERE feed_url = feed_similarity.feed_a) +
154- (SELECT cnt FROM _feed_word_counts WHERE feed_url = feed_similarity.feed_b) -
153+ (SELECT cnt FROM _feed_word_counts WHERE feed_url = recs.feed_similarity.feed_a) +
154+ (SELECT cnt FROM _feed_word_counts WHERE feed_url = recs.feed_similarity.feed_b) -
155155 CAST(_word_overlap.common AS REAL),
156156 0
157157 )
158158 FROM _word_overlap
159- WHERE feed_similarity.feed_a = _word_overlap.feed_a
160- AND feed_similarity.feed_b = _word_overlap.feed_b
159+ WHERE recs.feed_similarity.feed_a = _word_overlap.feed_a
160+ AND recs.feed_similarity.feed_b = _word_overlap.feed_b
161161 `, e.config.DescriptionWeight)
162162
163163 if _, err := tx.ExecContext(ctx, descUpdate); err != nil {
@@ -174,23 +174,23 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
174174 }
175175 defer func() { _ = tx.Rollback() }()
176176
177- if _, err := tx.ExecContext(ctx, `DELETE FROM user_similarity`); err != nil {
177+ if _, err := tx.ExecContext(ctx, `DELETE FROM recs.user_similarity`); err != nil {
178178 return err
179179 }
180180
181181 _, err = tx.ExecContext(ctx, `
182- INSERT INTO user_similarity (user_a, user_b, jaccard, common_feeds)
182+ INSERT INTO recs.user_similarity (user_a, user_b, jaccard, common_feeds)
183183 SELECT
184184 s1.user_did,
185185 s2.user_did,
186186 CAST(COUNT(*) AS REAL) / (
187- (SELECT COUNT(*) FROM subscriptions WHERE user_did = s1.user_did) +
188- (SELECT COUNT(*) FROM subscriptions WHERE user_did = s2.user_did) -
187+ (SELECT COUNT(*) FROM articles.subscriptions WHERE user_did = s1.user_did) +
188+ (SELECT COUNT(*) FROM articles.subscriptions WHERE user_did = s2.user_did) -
189189 CAST(COUNT(*) AS REAL)
190190 ),
191191 COUNT(*)
192- FROM subscriptions s1
193- JOIN subscriptions s2 ON s1.feed_url = s2.feed_url AND s1.user_did < s2.user_did
192+ FROM articles.subscriptions s1
193+ JOIN articles.subscriptions s2 ON s1.feed_url = s2.feed_url AND s1.user_did < s2.user_did
194194 GROUP BY s1.user_did, s2.user_did
195195 `)
196196 if err != nil {
@@ -207,7 +207,7 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
207207 }
208208 if _, err := tx.ExecContext(ctx, `
209209 INSERT INTO _likes_count (author_did, cnt)
210- SELECT author_did, COUNT(*) FROM likes GROUP BY author_did
210+ SELECT author_did, COUNT(*) FROM articles.likes GROUP BY author_did
211211 `); err != nil {
212212 return err
213213 }
@@ -227,8 +227,8 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
227227 EXP(-0.023 * CAST(julianday('now') - julianday(l1.created_at) AS REAL))
228228 * EXP(-0.023 * CAST(julianday('now') - julianday(l2.created_at) AS REAL))
229229 ) AS INTEGER)
230- FROM likes l1
231- JOIN likes l2 ON l1.feed_url = l2.feed_url AND l1.article_url = l2.article_url
230+ FROM articles.likes l1
231+ JOIN articles.likes l2 ON l1.feed_url = l2.feed_url AND l1.article_url = l2.article_url
232232 AND l1.author_did < l2.author_did
233233 WHERE l1.created_at IS NOT NULL AND l2.created_at IS NOT NULL
234234 GROUP BY l1.author_did, l2.author_did
@@ -237,17 +237,17 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
237237 }
238238
239239 likesUpdate := fmt.Sprintf(`
240- UPDATE user_similarity SET
240+ UPDATE recs.user_similarity SET
241241 jaccard = jaccard + %g * CAST(_likes_overlap.common AS REAL) / NULLIF(
242- (SELECT cnt FROM _likes_count WHERE author_did = user_similarity.user_a) +
243- (SELECT cnt FROM _likes_count WHERE author_did = user_similarity.user_b) -
242+ (SELECT cnt FROM _likes_count WHERE author_did = recs.user_similarity.user_a) +
243+ (SELECT cnt FROM _likes_count WHERE author_did = recs.user_similarity.user_b) -
244244 CAST(_likes_overlap.common AS REAL),
245245 0
246246 ),
247247 common_likes = _likes_overlap.common
248248 FROM _likes_overlap
249- WHERE user_similarity.user_a = _likes_overlap.user_a
250- AND user_similarity.user_b = _likes_overlap.user_b
249+ WHERE recs.user_similarity.user_a = _likes_overlap.user_a
250+ AND recs.user_similarity.user_b = _likes_overlap.user_b
251251 `, e.config.LikesWeight)
252252
253253 if _, err := tx.ExecContext(ctx, likesUpdate); err != nil {
@@ -255,7 +255,7 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
255255 }
256256
257257 likesInsert := fmt.Sprintf(`
258- INSERT INTO user_similarity (user_a, user_b, jaccard, common_feeds, common_likes)
258+ INSERT INTO recs.user_similarity (user_a, user_b, jaccard, common_feeds, common_likes)
259259 SELECT sub.user_a, sub.user_b, sub.jaccard, 0, sub.common
260260 FROM (
261261 SELECT
@@ -289,7 +289,7 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
289289 _, err = tx.ExecContext(ctx, `
290290 INSERT INTO _tag_overlap (user_a, user_b, common)
291291 WITH user_tags AS (
292- SELECT author_did, TRIM(value) AS tag FROM annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]')
292+ SELECT author_did, TRIM(value) AS tag FROM articles.annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]')
293293 WHERE tags IS NOT NULL AND tags != ''
294294 )
295295 SELECT t1.author_did, t2.author_did, COUNT(DISTINCT t1.tag)
@@ -312,7 +312,7 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
312312 if _, err := tx.ExecContext(ctx, `
313313 INSERT INTO _tag_count (author_did, cnt)
314314 WITH user_tags AS (
315- SELECT author_did, TRIM(value) AS tag FROM annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]')
315+ SELECT author_did, TRIM(value) AS tag FROM articles.annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]')
316316 WHERE tags IS NOT NULL AND tags != ''
317317 )
318318 SELECT author_did, COUNT(DISTINCT tag) FROM user_tags GROUP BY author_did
@@ -321,7 +321,7 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
321321 }
322322
323323 _, err = tx.ExecContext(ctx, `
324- INSERT OR IGNORE INTO user_similarity (user_a, user_b, jaccard, common_feeds, common_tags)
324+ INSERT OR IGNORE INTO recs.user_similarity (user_a, user_b, jaccard, common_feeds, common_tags)
325325 SELECT user_a, user_b, 0, 0, 0 FROM _tag_overlap
326326 `)
327327 if err != nil {
@@ -329,17 +329,17 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
329329 }
330330
331331 tagsUpdate := fmt.Sprintf(`
332- UPDATE user_similarity SET
332+ UPDATE recs.user_similarity SET
333333 jaccard = jaccard + %g * CAST(_tag_overlap.common AS REAL) / NULLIF(
334- (SELECT cnt FROM _tag_count WHERE author_did = user_similarity.user_a) +
335- (SELECT cnt FROM _tag_count WHERE author_did = user_similarity.user_b) -
334+ (SELECT cnt FROM _tag_count WHERE author_did = recs.user_similarity.user_a) +
335+ (SELECT cnt FROM _tag_count WHERE author_did = recs.user_similarity.user_b) -
336336 CAST(_tag_overlap.common AS REAL),
337337 0
338338 ),
339339 common_tags = _tag_overlap.common
340340 FROM _tag_overlap
341- WHERE user_similarity.user_a = _tag_overlap.user_a
342- AND user_similarity.user_b = _tag_overlap.user_b
341+ WHERE recs.user_similarity.user_a = _tag_overlap.user_a
342+ AND recs.user_similarity.user_b = _tag_overlap.user_b
343343 `, e.config.TagsWeight)
344344
345345 if _, err := tx.ExecContext(ctx, tagsUpdate); err != nil {
@@ -347,13 +347,13 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
347347 }
348348
349349 followQuery := fmt.Sprintf(`
350- INSERT INTO user_similarity (user_a, user_b, jaccard, common_feeds, common_likes, common_tags)
350+ INSERT INTO recs.user_similarity (user_a, user_b, jaccard, common_feeds, common_likes, common_tags)
351351 SELECT
352352 MIN(f.user_did, f.target_did),
353353 MAX(f.user_did, f.target_did),
354354 %g,
355355 0, 0, 0
356- FROM follows f
356+ FROM main.follows f
357357 WHERE f.user_did != f.target_did
358358 GROUP BY MIN(f.user_did, f.target_did), MAX(f.user_did, f.target_did)
359359 ON CONFLICT(user_a, user_b) DO UPDATE SET
@@ -366,4 +366,4 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
366366
367367 e.logger.Info("user similarity computed")
368368 return tx.Commit()
369-}
369+}
\ No newline at end of file
@@ -42,20 +42,20 @@ func (e *Engine) ComputeFeedSimilarity(ctx context.Context) error {
42 }42 }
43 defer func() { _ = tx.Rollback() }()43 defer func() { _ = tx.Rollback() }()
44 44
45- if _, err := tx.ExecContext(ctx, `DELETE FROM feed_similarity`); err != nil {45+ if _, err := tx.ExecContext(ctx, `DELETE FROM recs.feed_similarity`); err != nil {
46 return err46 return err
47 }47 }
48 48
49 _, err = tx.ExecContext(ctx, `49 _, err = tx.ExecContext(ctx, `
50- INSERT INTO feed_similarity (feed_a, feed_b, jaccard)50+ INSERT INTO recs.feed_similarity (feed_a, feed_b, jaccard)
51 SELECT51 SELECT
52 s1.feed_url,52 s1.feed_url,
53 s2.feed_url,53 s2.feed_url,
54 CAST(COUNT(*) AS REAL) / (f1.subscriber_count + f2.subscriber_count - CAST(COUNT(*) AS REAL))54 CAST(COUNT(*) AS REAL) / (f1.subscriber_count + f2.subscriber_count - CAST(COUNT(*) AS REAL))
55- FROM subscriptions s155+ FROM articles.subscriptions s1
56- JOIN subscriptions s2 ON s1.user_did = s2.user_did AND s1.feed_url < s2.feed_url56+ JOIN articles.subscriptions s2 ON s1.user_did = s2.user_did AND s1.feed_url < s2.feed_url
57- JOIN feeds f1 ON f1.feed_url = s1.feed_url57+ JOIN articles.feeds f1 ON f1.feed_url = s1.feed_url
58- JOIN feeds f2 ON f2.feed_url = s2.feed_url58+ JOIN articles.feeds f2 ON f2.feed_url = s2.feed_url
59 GROUP BY s1.feed_url, s2.feed_url59 GROUP BY s1.feed_url, s2.feed_url
60 `)60 `)
61 if err != nil {61 if err != nil {
@@ -82,7 +82,7 @@ func (e *Engine) computeDescriptionSimilarity(ctx context.Context, tx *sql.Tx) e
82 INSERT INTO _feed_words (feed_url, word)82 INSERT INTO _feed_words (feed_url, word)
83 WITH feed_tokens AS (83 WITH feed_tokens AS (
84 SELECT feed_url, LOWER(TRIM(value)) AS word84 SELECT feed_url, LOWER(TRIM(value)) AS word
85- FROM feeds,85+ FROM articles.feeds,
86 json_each('["' || REPLACE(LOWER(COALESCE(description, '')), ' ', '","') || '"]')86 json_each('["' || REPLACE(LOWER(COALESCE(description, '')), ' ', '","') || '"]')
87 WHERE description IS NOT NULL AND description != ''87 WHERE description IS NOT NULL AND description != ''
88 )88 )
@@ -140,7 +140,7 @@ func (e *Engine) computeDescriptionSimilarity(ctx context.Context, tx *sql.Tx) e
140 }140 }
141 141
142 descInsert := `142 descInsert := `
143- INSERT OR IGNORE INTO feed_similarity (feed_a, feed_b, jaccard)143+ INSERT OR IGNORE INTO recs.feed_similarity (feed_a, feed_b, jaccard)
144 SELECT feed_a, feed_b, 0 FROM _word_overlap144 SELECT feed_a, feed_b, 0 FROM _word_overlap
145 `145 `
146 if _, err := tx.ExecContext(ctx, descInsert); err != nil {146 if _, err := tx.ExecContext(ctx, descInsert); err != nil {
@@ -148,16 +148,16 @@ func (e *Engine) computeDescriptionSimilarity(ctx context.Context, tx *sql.Tx) e
148 }148 }
149 149
150 descUpdate := fmt.Sprintf(`150 descUpdate := fmt.Sprintf(`
151- UPDATE feed_similarity SET151+ UPDATE recs.feed_similarity SET
152 jaccard = jaccard + %g * CAST(_word_overlap.common AS REAL) / NULLIF(152 jaccard = jaccard + %g * CAST(_word_overlap.common AS REAL) / NULLIF(
153- (SELECT cnt FROM _feed_word_counts WHERE feed_url = feed_similarity.feed_a) +153+ (SELECT cnt FROM _feed_word_counts WHERE feed_url = recs.feed_similarity.feed_a) +
154- (SELECT cnt FROM _feed_word_counts WHERE feed_url = feed_similarity.feed_b) -154+ (SELECT cnt FROM _feed_word_counts WHERE feed_url = recs.feed_similarity.feed_b) -
155 CAST(_word_overlap.common AS REAL),155 CAST(_word_overlap.common AS REAL),
156 0156 0
157 )157 )
158 FROM _word_overlap158 FROM _word_overlap
159- WHERE feed_similarity.feed_a = _word_overlap.feed_a159+ WHERE recs.feed_similarity.feed_a = _word_overlap.feed_a
160- AND feed_similarity.feed_b = _word_overlap.feed_b160+ AND recs.feed_similarity.feed_b = _word_overlap.feed_b
161 `, e.config.DescriptionWeight)161 `, e.config.DescriptionWeight)
162 162
163 if _, err := tx.ExecContext(ctx, descUpdate); err != nil {163 if _, err := tx.ExecContext(ctx, descUpdate); err != nil {
@@ -174,23 +174,23 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
174 }174 }
175 defer func() { _ = tx.Rollback() }()175 defer func() { _ = tx.Rollback() }()
176 176
177- if _, err := tx.ExecContext(ctx, `DELETE FROM user_similarity`); err != nil {177+ if _, err := tx.ExecContext(ctx, `DELETE FROM recs.user_similarity`); err != nil {
178 return err178 return err
179 }179 }
180 180
181 _, err = tx.ExecContext(ctx, `181 _, err = tx.ExecContext(ctx, `
182- INSERT INTO user_similarity (user_a, user_b, jaccard, common_feeds)182+ INSERT INTO recs.user_similarity (user_a, user_b, jaccard, common_feeds)
183 SELECT183 SELECT
184 s1.user_did,184 s1.user_did,
185 s2.user_did,185 s2.user_did,
186 CAST(COUNT(*) AS REAL) / (186 CAST(COUNT(*) AS REAL) / (
187- (SELECT COUNT(*) FROM subscriptions WHERE user_did = s1.user_did) +187+ (SELECT COUNT(*) FROM articles.subscriptions WHERE user_did = s1.user_did) +
188- (SELECT COUNT(*) FROM subscriptions WHERE user_did = s2.user_did) -188+ (SELECT COUNT(*) FROM articles.subscriptions WHERE user_did = s2.user_did) -
189 CAST(COUNT(*) AS REAL)189 CAST(COUNT(*) AS REAL)
190 ),190 ),
191 COUNT(*)191 COUNT(*)
192- FROM subscriptions s1192+ FROM articles.subscriptions s1
193- JOIN subscriptions s2 ON s1.feed_url = s2.feed_url AND s1.user_did < s2.user_did193+ JOIN articles.subscriptions s2 ON s1.feed_url = s2.feed_url AND s1.user_did < s2.user_did
194 GROUP BY s1.user_did, s2.user_did194 GROUP BY s1.user_did, s2.user_did
195 `)195 `)
196 if err != nil {196 if err != nil {
@@ -207,7 +207,7 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
207 }207 }
208 if _, err := tx.ExecContext(ctx, `208 if _, err := tx.ExecContext(ctx, `
209 INSERT INTO _likes_count (author_did, cnt)209 INSERT INTO _likes_count (author_did, cnt)
210- SELECT author_did, COUNT(*) FROM likes GROUP BY author_did210+ SELECT author_did, COUNT(*) FROM articles.likes GROUP BY author_did
211 `); err != nil {211 `); err != nil {
212 return err212 return err
213 }213 }
@@ -227,8 +227,8 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
227 EXP(-0.023 * CAST(julianday('now') - julianday(l1.created_at) AS REAL))227 EXP(-0.023 * CAST(julianday('now') - julianday(l1.created_at) AS REAL))
228 * EXP(-0.023 * CAST(julianday('now') - julianday(l2.created_at) AS REAL))228 * EXP(-0.023 * CAST(julianday('now') - julianday(l2.created_at) AS REAL))
229 ) AS INTEGER)229 ) AS INTEGER)
230- FROM likes l1230+ FROM articles.likes l1
231- JOIN likes l2 ON l1.feed_url = l2.feed_url AND l1.article_url = l2.article_url231+ JOIN articles.likes l2 ON l1.feed_url = l2.feed_url AND l1.article_url = l2.article_url
232 AND l1.author_did < l2.author_did232 AND l1.author_did < l2.author_did
233 WHERE l1.created_at IS NOT NULL AND l2.created_at IS NOT NULL233 WHERE l1.created_at IS NOT NULL AND l2.created_at IS NOT NULL
234 GROUP BY l1.author_did, l2.author_did234 GROUP BY l1.author_did, l2.author_did
@@ -237,17 +237,17 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
237 }237 }
238 238
239 likesUpdate := fmt.Sprintf(`239 likesUpdate := fmt.Sprintf(`
240- UPDATE user_similarity SET240+ UPDATE recs.user_similarity SET
241 jaccard = jaccard + %g * CAST(_likes_overlap.common AS REAL) / NULLIF(241 jaccard = jaccard + %g * CAST(_likes_overlap.common AS REAL) / NULLIF(
242- (SELECT cnt FROM _likes_count WHERE author_did = user_similarity.user_a) +242+ (SELECT cnt FROM _likes_count WHERE author_did = recs.user_similarity.user_a) +
243- (SELECT cnt FROM _likes_count WHERE author_did = user_similarity.user_b) -243+ (SELECT cnt FROM _likes_count WHERE author_did = recs.user_similarity.user_b) -
244 CAST(_likes_overlap.common AS REAL),244 CAST(_likes_overlap.common AS REAL),
245 0245 0
246 ),246 ),
247 common_likes = _likes_overlap.common247 common_likes = _likes_overlap.common
248 FROM _likes_overlap248 FROM _likes_overlap
249- WHERE user_similarity.user_a = _likes_overlap.user_a249+ WHERE recs.user_similarity.user_a = _likes_overlap.user_a
250- AND user_similarity.user_b = _likes_overlap.user_b250+ AND recs.user_similarity.user_b = _likes_overlap.user_b
251 `, e.config.LikesWeight)251 `, e.config.LikesWeight)
252 252
253 if _, err := tx.ExecContext(ctx, likesUpdate); err != nil {253 if _, err := tx.ExecContext(ctx, likesUpdate); err != nil {
@@ -255,7 +255,7 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
255 }255 }
256 256
257 likesInsert := fmt.Sprintf(`257 likesInsert := fmt.Sprintf(`
258- INSERT INTO user_similarity (user_a, user_b, jaccard, common_feeds, common_likes)258+ INSERT INTO recs.user_similarity (user_a, user_b, jaccard, common_feeds, common_likes)
259 SELECT sub.user_a, sub.user_b, sub.jaccard, 0, sub.common259 SELECT sub.user_a, sub.user_b, sub.jaccard, 0, sub.common
260 FROM (260 FROM (
261 SELECT261 SELECT
@@ -289,7 +289,7 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
289 _, err = tx.ExecContext(ctx, `289 _, err = tx.ExecContext(ctx, `
290 INSERT INTO _tag_overlap (user_a, user_b, common)290 INSERT INTO _tag_overlap (user_a, user_b, common)
291 WITH user_tags AS (291 WITH user_tags AS (
292- SELECT author_did, TRIM(value) AS tag FROM annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]')292+ SELECT author_did, TRIM(value) AS tag FROM articles.annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]')
293 WHERE tags IS NOT NULL AND tags != ''293 WHERE tags IS NOT NULL AND tags != ''
294 )294 )
295 SELECT t1.author_did, t2.author_did, COUNT(DISTINCT t1.tag)295 SELECT t1.author_did, t2.author_did, COUNT(DISTINCT t1.tag)
@@ -312,7 +312,7 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
312 if _, err := tx.ExecContext(ctx, `312 if _, err := tx.ExecContext(ctx, `
313 INSERT INTO _tag_count (author_did, cnt)313 INSERT INTO _tag_count (author_did, cnt)
314 WITH user_tags AS (314 WITH user_tags AS (
315- SELECT author_did, TRIM(value) AS tag FROM annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]')315+ SELECT author_did, TRIM(value) AS tag FROM articles.annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]')
316 WHERE tags IS NOT NULL AND tags != ''316 WHERE tags IS NOT NULL AND tags != ''
317 )317 )
318 SELECT author_did, COUNT(DISTINCT tag) FROM user_tags GROUP BY author_did318 SELECT author_did, COUNT(DISTINCT tag) FROM user_tags GROUP BY author_did
@@ -321,7 +321,7 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
321 }321 }
322 322
323 _, err = tx.ExecContext(ctx, `323 _, err = tx.ExecContext(ctx, `
324- INSERT OR IGNORE INTO user_similarity (user_a, user_b, jaccard, common_feeds, common_tags)324+ INSERT OR IGNORE INTO recs.user_similarity (user_a, user_b, jaccard, common_feeds, common_tags)
325 SELECT user_a, user_b, 0, 0, 0 FROM _tag_overlap325 SELECT user_a, user_b, 0, 0, 0 FROM _tag_overlap
326 `)326 `)
327 if err != nil {327 if err != nil {
@@ -329,17 +329,17 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
329 }329 }
330 330
331 tagsUpdate := fmt.Sprintf(`331 tagsUpdate := fmt.Sprintf(`
332- UPDATE user_similarity SET332+ UPDATE recs.user_similarity SET
333 jaccard = jaccard + %g * CAST(_tag_overlap.common AS REAL) / NULLIF(333 jaccard = jaccard + %g * CAST(_tag_overlap.common AS REAL) / NULLIF(
334- (SELECT cnt FROM _tag_count WHERE author_did = user_similarity.user_a) +334+ (SELECT cnt FROM _tag_count WHERE author_did = recs.user_similarity.user_a) +
335- (SELECT cnt FROM _tag_count WHERE author_did = user_similarity.user_b) -335+ (SELECT cnt FROM _tag_count WHERE author_did = recs.user_similarity.user_b) -
336 CAST(_tag_overlap.common AS REAL),336 CAST(_tag_overlap.common AS REAL),
337 0337 0
338 ),338 ),
339 common_tags = _tag_overlap.common339 common_tags = _tag_overlap.common
340 FROM _tag_overlap340 FROM _tag_overlap
341- WHERE user_similarity.user_a = _tag_overlap.user_a341+ WHERE recs.user_similarity.user_a = _tag_overlap.user_a
342- AND user_similarity.user_b = _tag_overlap.user_b342+ AND recs.user_similarity.user_b = _tag_overlap.user_b
343 `, e.config.TagsWeight)343 `, e.config.TagsWeight)
344 344
345 if _, err := tx.ExecContext(ctx, tagsUpdate); err != nil {345 if _, err := tx.ExecContext(ctx, tagsUpdate); err != nil {
@@ -347,13 +347,13 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
347 }347 }
348 348
349 followQuery := fmt.Sprintf(`349 followQuery := fmt.Sprintf(`
350- INSERT INTO user_similarity (user_a, user_b, jaccard, common_feeds, common_likes, common_tags)350+ INSERT INTO recs.user_similarity (user_a, user_b, jaccard, common_feeds, common_likes, common_tags)
351 SELECT351 SELECT
352 MIN(f.user_did, f.target_did),352 MIN(f.user_did, f.target_did),
353 MAX(f.user_did, f.target_did),353 MAX(f.user_did, f.target_did),
354 %g,354 %g,
355 0, 0, 0355 0, 0, 0
356- FROM follows f356+ FROM main.follows f
357 WHERE f.user_did != f.target_did357 WHERE f.user_did != f.target_did
358 GROUP BY MIN(f.user_did, f.target_did), MAX(f.user_did, f.target_did)358 GROUP BY MIN(f.user_did, f.target_did), MAX(f.user_did, f.target_did)
359 ON CONFLICT(user_a, user_b) DO UPDATE SET359 ON CONFLICT(user_a, user_b) DO UPDATE SET
@@ -366,4 +366,4 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
366 366
367 e.logger.Info("user similarity computed")367 e.logger.Info("user similarity computed")
368 return tx.Commit()368 return tx.Commit()
369-}369+}
\ No newline at end of file\ No newline at end of file
modified internal/cluster/scoring.go +49 -50
@@ -41,7 +41,7 @@ type ArticleRecommendation struct {
4141
4242 func (e *Engine) GetFeedRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) {
4343 subCount := 0
44- _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM subscriptions WHERE user_did = ?`, userDID).Scan(&subCount)
44+ _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM articles.subscriptions WHERE user_did = ?`, userDID).Scan(&subCount)
4545
4646 if subCount < 5 {
4747 recs, err := e.ColdStartRecommendations(ctx, userDID, limit*2)
@@ -91,7 +91,7 @@ func (e *Engine) GetWeights(ctx context.Context, userDID string) SignalWeights {
9191 var dbW SignalWeights
9292 err := e.db.QueryRowContext(ctx, `
9393 SELECT w_sub, w_like, w_tag, w_social, w_pop, w_category
94- FROM user_signal_weights WHERE user_did = ?
94+ FROM recs.user_signal_weights WHERE user_did = ?
9595 `, userDID).Scan(&dbW.WSub, &dbW.WLike, &dbW.WTag, &dbW.WSocial, &dbW.WPop, &dbW.WCategory)
9696 if err == nil {
9797 return dbW
@@ -104,46 +104,46 @@ func (e *Engine) ComputeFeedRecommendationsOnDemand(ctx context.Context, userDID
104104
105105 rows, err := e.db.QueryContext(ctx, `
106106 WITH similar_users AS (
107- SELECT user_b AS peer, jaccard FROM user_similarity WHERE user_a = ? AND jaccard > 0.15
107+ SELECT user_b AS peer, jaccard FROM recs.user_similarity WHERE user_a = ? AND jaccard > 0.15
108108 UNION ALL
109- SELECT user_a AS peer, jaccard FROM user_similarity WHERE user_b = ? AND jaccard > 0.15
109+ SELECT user_a AS peer, jaccard FROM recs.user_similarity WHERE user_b = ? AND jaccard > 0.15
110110 ),
111111 candidate_feeds AS (
112112 SELECT s.feed_url,
113113 SUM(su.jaccard) AS sub_signal
114114 FROM similar_users su
115- JOIN subscriptions s ON s.user_did = su.peer
116- WHERE s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
117- AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
115+ JOIN articles.subscriptions s ON s.user_did = su.peer
116+ WHERE s.feed_url NOT IN (SELECT feed_url FROM articles.subscriptions WHERE user_did = ?)
117+ AND s.feed_url NOT IN (SELECT target_id FROM recs.dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
118118 GROUP BY s.feed_url
119119 ),
120120 like_signals AS (
121121 SELECT s.feed_url,
122122 SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal
123123 FROM similar_users su
124- JOIN likes l ON l.author_did = su.peer
125- JOIN subscriptions s ON s.feed_url = l.feed_url
126- WHERE s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
127- AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
124+ JOIN articles.likes l ON l.author_did = su.peer
125+ JOIN articles.subscriptions s ON s.feed_url = l.feed_url
126+ WHERE s.feed_url NOT IN (SELECT feed_url FROM articles.subscriptions WHERE user_did = ?)
127+ AND s.feed_url NOT IN (SELECT target_id FROM recs.dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
128128 GROUP BY s.feed_url
129129 ),
130130 social_boost AS (
131131 SELECT s.feed_url,
132132 SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social
133- FROM follow_distances fd
134- JOIN subscriptions s ON s.user_did = fd.user_b
133+ FROM recs.follow_distances fd
134+ JOIN articles.subscriptions s ON s.user_did = fd.user_b
135135 WHERE fd.user_a = ?
136- AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
137- AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
136+ AND s.feed_url NOT IN (SELECT feed_url FROM articles.subscriptions WHERE user_did = ?)
137+ AND s.feed_url NOT IN (SELECT target_id FROM recs.dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
138138 GROUP BY s.feed_url
139139 ),
140140 category_counts AS (
141141 SELECT category, COUNT(*) AS cnt
142- FROM subscriptions WHERE user_did = ? AND category IS NOT NULL AND category != ''
142+ FROM articles.subscriptions WHERE user_did = ? AND category IS NOT NULL AND category != ''
143143 GROUP BY category
144144 ),
145145 max_subs AS (
146- SELECT CAST(COALESCE(MAX(subscriber_count), 1) AS REAL) AS m FROM feeds
146+ SELECT CAST(COALESCE(MAX(subscriber_count), 1) AS REAL) AS m FROM articles.feeds
147147 )
148148 SELECT cf.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''),
149149 COALESCE(f.description, ''), f.subscriber_count, COALESCE(f.favicon_url, ''),
@@ -157,7 +157,7 @@ func (e *Engine) ComputeFeedRecommendationsOnDemand(ctx context.Context, userDID
157157 ) THEN ? ELSE 0 END
158158 AS score
159159 FROM candidate_feeds cf
160- JOIN feeds f ON f.feed_url = cf.feed_url
160+ JOIN articles.feeds f ON f.feed_url = cf.feed_url
161161 LEFT JOIN like_signals ls ON ls.feed_url = cf.feed_url
162162 LEFT JOIN social_boost sb ON sb.feed_url = cf.feed_url
163163 CROSS JOIN max_subs ms
@@ -187,31 +187,31 @@ func (e *Engine) ComputeArticleRecommendationsOnDemand(ctx context.Context, user
187187
188188 rows, err := e.db.QueryContext(ctx, `
189189 WITH similar_users AS (
190- SELECT user_b AS peer, jaccard FROM user_similarity WHERE user_a = ? AND jaccard > 0.15
190+ SELECT user_b AS peer, jaccard FROM recs.user_similarity WHERE user_a = ? AND jaccard > 0.15
191191 UNION ALL
192- SELECT user_a AS peer, jaccard FROM user_similarity WHERE user_b = ? AND jaccard > 0.15
192+ SELECT user_a AS peer, jaccard FROM recs.user_similarity WHERE user_b = ? AND jaccard > 0.15
193193 ),
194194 liked_articles AS (
195195 SELECT l.feed_url, l.article_url,
196196 SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal
197197 FROM similar_users su
198- JOIN likes l ON l.author_did = su.peer
198+ JOIN articles.likes l ON l.author_did = su.peer
199199 WHERE NOT EXISTS (
200- SELECT 1 FROM likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url
200+ SELECT 1 FROM articles.likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url
201201 )
202202 AND NOT EXISTS (
203- SELECT 1 FROM dismissed_recommendations d WHERE d.user_did = ? AND d.target_type = 'article' AND d.target_id = l.article_url
203+ SELECT 1 FROM recs.dismissed_recommendations d WHERE d.user_did = ? AND d.target_type = 'article' AND d.target_id = l.article_url
204204 )
205205 GROUP BY l.feed_url, l.article_url
206206 ),
207207 social_likes AS (
208208 SELECT l.feed_url, l.article_url,
209209 SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social
210- FROM follow_distances fd
211- JOIN likes l ON l.author_did = fd.user_b
210+ FROM recs.follow_distances fd
211+ JOIN articles.likes l ON l.author_did = fd.user_b
212212 WHERE fd.user_a = ?
213213 AND NOT EXISTS (
214- SELECT 1 FROM likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url
214+ SELECT 1 FROM articles.likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url
215215 )
216216 GROUP BY l.feed_url, l.article_url
217217 )
@@ -223,10 +223,9 @@ func (e *Engine) ComputeArticleRecommendationsOnDemand(ctx context.Context, user
223223 + EXP(-0.023 * CAST(julianday('now') - julianday(a.published) AS REAL)) * 0.2
224224 AS score
225225 FROM liked_articles la
226- JOIN articles a ON a.feed_url = la.feed_url AND a.url = la.article_url
227- LEFT JOIN feeds f ON f.feed_url = la.feed_url
226+ JOIN articles.articles a ON a.feed_url = la.feed_url AND a.url = la.article_url
227+ LEFT JOIN articles.feeds f ON f.feed_url = la.feed_url
228228 LEFT JOIN social_likes sl ON sl.feed_url = la.feed_url AND sl.article_url = la.article_url
229- -- Future-published articles (e.g., scheduled) sort last
230229 ORDER BY score DESC, (CASE WHEN a.published > 'now' THEN 1 ELSE 0 END), a.published DESC
231230 LIMIT ?
232231 `, userDID, userDID, userDID, userDID, userDID, userDID, w.WLike, w.WSocial, limit)
@@ -252,13 +251,13 @@ func (e *Engine) ComputePeopleRecommendationsOnDemand(ctx context.Context, userD
252251 SELECT u.did, u.handle, COALESCE(u.display_name, ''), COALESCE(u.avatar_url, ''),
253252 sim.jaccard, sim.common_feeds, COALESCE(sim.common_likes, 0), COALESCE(sim.common_tags, 0)
254253 FROM (
255- SELECT user_b AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_a = ?
254+ SELECT user_b AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM recs.user_similarity WHERE user_a = ?
256255 UNION ALL
257- SELECT user_a AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_b = ?
256+ SELECT user_a AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM recs.user_similarity WHERE user_b = ?
258257 ) sim
259- JOIN users u ON u.did = sim.peer_did
258+ JOIN main.users u ON u.did = sim.peer_did
260259 WHERE u.handle IS NOT NULL AND u.handle != ''
261- AND EXISTS (SELECT 1 FROM subscriptions s JOIN feeds f ON s.feed_url = f.feed_url WHERE s.user_did = u.did AND f.subscriber_count > 0)
260+ AND EXISTS (SELECT 1 FROM articles.subscriptions s JOIN articles.feeds f ON s.feed_url = f.feed_url WHERE s.user_did = u.did AND f.subscriber_count > 0)
262261 ORDER BY sim.jaccard DESC
263262 LIMIT ?
264263 `, userDID, userDID, limit)
@@ -286,7 +285,7 @@ func (e *Engine) ComputeSignalProfiles(ctx context.Context) error {
286285 }
287286 defer func() { _ = tx.Rollback() }()
288287
289- if _, err := tx.ExecContext(ctx, `DELETE FROM user_signal_profiles`); err != nil {
288+ if _, err := tx.ExecContext(ctx, `DELETE FROM recs.user_signal_profiles`); err != nil {
290289 return err
291290 }
292291
@@ -299,7 +298,7 @@ func (e *Engine) ComputeSignalProfiles(ctx context.Context) error {
299298 return err
300299 }
301300 if _, err := tx.ExecContext(ctx, `
302- INSERT INTO _user_like_counts SELECT author_did, COUNT(*) FROM likes GROUP BY author_did
301+ INSERT INTO _user_like_counts SELECT author_did, COUNT(*) FROM articles.likes GROUP BY author_did
303302 `); err != nil {
304303 return err
305304 }
@@ -316,7 +315,7 @@ func (e *Engine) ComputeSignalProfiles(ctx context.Context) error {
316315 INSERT INTO _user_tag_counts
317316 WITH user_tags AS (
318317 SELECT author_did, TRIM(value) AS tag
319- FROM annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]')
318+ FROM articles.annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]')
320319 WHERE tags IS NOT NULL AND tags != ''
321320 )
322321 SELECT author_did, COUNT(DISTINCT tag) FROM user_tags GROUP BY author_did
@@ -337,7 +336,7 @@ func (e *Engine) ComputeSignalProfiles(ctx context.Context) error {
337336 SELECT user_did, '[' || GROUP_CONCAT('{"c":"' || category || '","n":"' || CAST(cnt AS TEXT) || '}') || ']'
338337 FROM (
339338 SELECT user_did, category, COUNT(*) AS cnt
340- FROM subscriptions
339+ FROM articles.subscriptions
341340 WHERE category IS NOT NULL AND category != ''
342341 GROUP BY user_did, category
343342 ORDER BY COUNT(*) DESC
@@ -349,13 +348,13 @@ func (e *Engine) ComputeSignalProfiles(ctx context.Context) error {
349348 }
350349
351350 _, err = tx.ExecContext(ctx, `
352- INSERT INTO user_signal_profiles (user_did, total_likes, total_tags, top_categories)
351+ INSERT INTO recs.user_signal_profiles (user_did, total_likes, total_tags, top_categories)
353352 SELECT
354353 u.did,
355354 COALESCE(lc.cnt, 0),
356355 COALESCE(tc.cnt, 0),
357356 COALESCE(cc.categories, '[]')
358- FROM users u
357+ FROM main.users u
359358 LEFT JOIN _user_like_counts lc ON lc.user_did = u.did
360359 LEFT JOIN _user_tag_counts tc ON tc.user_did = u.did
361360 LEFT JOIN _user_top_categories cc ON cc.user_did = u.did
@@ -370,7 +369,7 @@ func (e *Engine) ComputeSignalProfiles(ctx context.Context) error {
370369
371370 func (e *Engine) ColdStartRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) {
372371 subCount := 0
373- _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM subscriptions WHERE user_did = ?`, userDID).Scan(&subCount)
372+ _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM articles.subscriptions WHERE user_did = ?`, userDID).Scan(&subCount)
374373 if subCount >= 5 {
375374 return nil, nil
376375 }
@@ -378,19 +377,19 @@ func (e *Engine) ColdStartRecommendations(ctx context.Context, userDID string, l
378377 rows, err := e.db.QueryContext(ctx, `
379378 WITH followed_feeds AS (
380379 SELECT s.feed_url, 1.0 AS weight
381- FROM follow_distances fd
382- JOIN subscriptions s ON s.user_did = fd.user_b
380+ FROM recs.follow_distances fd
381+ JOIN articles.subscriptions s ON s.user_did = fd.user_b
383382 WHERE fd.user_a = ? AND fd.distance = 1
384- AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
385- AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
383+ AND s.feed_url NOT IN (SELECT feed_url FROM articles.subscriptions WHERE user_did = ?)
384+ AND s.feed_url NOT IN (SELECT target_id FROM recs.dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
386385 ),
387386 popular_feeds AS (
388387 SELECT feed_url, subscriber_count,
389- LOG(1 + CAST(subscriber_count AS REAL)) / LOG(1 + CAST((SELECT COALESCE(MAX(subscriber_count), 1) FROM feeds) AS REAL)) AS pop_score
390- FROM feeds
388+ LOG(1 + CAST(subscriber_count AS REAL)) / LOG(1 + CAST((SELECT COALESCE(MAX(subscriber_count), 1) FROM articles.feeds) AS REAL)) AS pop_score
389+ FROM articles.feeds
391390 WHERE subscriber_count > 0
392- AND feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
393- AND feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
391+ AND feed_url NOT IN (SELECT feed_url FROM articles.subscriptions WHERE user_did = ?)
392+ AND feed_url NOT IN (SELECT target_id FROM recs.dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
394393 ORDER BY subscriber_count DESC
395394 LIMIT 50
396395 ),
@@ -410,7 +409,7 @@ func (e *Engine) ColdStartRecommendations(ctx context.Context, userDID string, l
410409 COALESCE(f.favicon_url, ''),
411410 ac.weight AS score
412411 FROM all_candidates ac
413- JOIN feeds f ON f.feed_url = ac.feed_url
412+ JOIN articles.feeds f ON f.feed_url = ac.feed_url
414413 ORDER BY score DESC
415414 LIMIT ?
416415 `, userDID, userDID, userDID, userDID, userDID, limit)
@@ -429,4 +428,4 @@ func (e *Engine) ColdStartRecommendations(ctx context.Context, userDID string, l
429428 results = append(results, rec)
430429 }
431430 return results, rows.Err()
432-}
431+}
\ No newline at end of file
@@ -41,7 +41,7 @@ type ArticleRecommendation struct {
41 41
42 func (e *Engine) GetFeedRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) {42 func (e *Engine) GetFeedRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) {
43 subCount := 043 subCount := 0
44- _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM subscriptions WHERE user_did = ?`, userDID).Scan(&subCount)44+ _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM articles.subscriptions WHERE user_did = ?`, userDID).Scan(&subCount)
45 45
46 if subCount < 5 {46 if subCount < 5 {
47 recs, err := e.ColdStartRecommendations(ctx, userDID, limit*2)47 recs, err := e.ColdStartRecommendations(ctx, userDID, limit*2)
@@ -91,7 +91,7 @@ func (e *Engine) GetWeights(ctx context.Context, userDID string) SignalWeights {
91 var dbW SignalWeights91 var dbW SignalWeights
92 err := e.db.QueryRowContext(ctx, `92 err := e.db.QueryRowContext(ctx, `
93 SELECT w_sub, w_like, w_tag, w_social, w_pop, w_category93 SELECT w_sub, w_like, w_tag, w_social, w_pop, w_category
94- FROM user_signal_weights WHERE user_did = ?94+ FROM recs.user_signal_weights WHERE user_did = ?
95 `, userDID).Scan(&dbW.WSub, &dbW.WLike, &dbW.WTag, &dbW.WSocial, &dbW.WPop, &dbW.WCategory)95 `, userDID).Scan(&dbW.WSub, &dbW.WLike, &dbW.WTag, &dbW.WSocial, &dbW.WPop, &dbW.WCategory)
96 if err == nil {96 if err == nil {
97 return dbW97 return dbW
@@ -104,46 +104,46 @@ func (e *Engine) ComputeFeedRecommendationsOnDemand(ctx context.Context, userDID
104 104
105 rows, err := e.db.QueryContext(ctx, `105 rows, err := e.db.QueryContext(ctx, `
106 WITH similar_users AS (106 WITH similar_users AS (
107- SELECT user_b AS peer, jaccard FROM user_similarity WHERE user_a = ? AND jaccard > 0.15107+ SELECT user_b AS peer, jaccard FROM recs.user_similarity WHERE user_a = ? AND jaccard > 0.15
108 UNION ALL108 UNION ALL
109- SELECT user_a AS peer, jaccard FROM user_similarity WHERE user_b = ? AND jaccard > 0.15109+ SELECT user_a AS peer, jaccard FROM recs.user_similarity WHERE user_b = ? AND jaccard > 0.15
110 ),110 ),
111 candidate_feeds AS (111 candidate_feeds AS (
112 SELECT s.feed_url,112 SELECT s.feed_url,
113 SUM(su.jaccard) AS sub_signal113 SUM(su.jaccard) AS sub_signal
114 FROM similar_users su114 FROM similar_users su
115- JOIN subscriptions s ON s.user_did = su.peer115+ JOIN articles.subscriptions s ON s.user_did = su.peer
116- WHERE s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)116+ WHERE s.feed_url NOT IN (SELECT feed_url FROM articles.subscriptions WHERE user_did = ?)
117- AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')117+ AND s.feed_url NOT IN (SELECT target_id FROM recs.dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
118 GROUP BY s.feed_url118 GROUP BY s.feed_url
119 ),119 ),
120 like_signals AS (120 like_signals AS (
121 SELECT s.feed_url,121 SELECT s.feed_url,
122 SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal122 SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal
123 FROM similar_users su123 FROM similar_users su
124- JOIN likes l ON l.author_did = su.peer124+ JOIN articles.likes l ON l.author_did = su.peer
125- JOIN subscriptions s ON s.feed_url = l.feed_url125+ JOIN articles.subscriptions s ON s.feed_url = l.feed_url
126- WHERE s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)126+ WHERE s.feed_url NOT IN (SELECT feed_url FROM articles.subscriptions WHERE user_did = ?)
127- AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')127+ AND s.feed_url NOT IN (SELECT target_id FROM recs.dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
128 GROUP BY s.feed_url128 GROUP BY s.feed_url
129 ),129 ),
130 social_boost AS (130 social_boost AS (
131 SELECT s.feed_url,131 SELECT s.feed_url,
132 SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social132 SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social
133- FROM follow_distances fd133+ FROM recs.follow_distances fd
134- JOIN subscriptions s ON s.user_did = fd.user_b134+ JOIN articles.subscriptions s ON s.user_did = fd.user_b
135 WHERE fd.user_a = ?135 WHERE fd.user_a = ?
136- AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)136+ AND s.feed_url NOT IN (SELECT feed_url FROM articles.subscriptions WHERE user_did = ?)
137- AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')137+ AND s.feed_url NOT IN (SELECT target_id FROM recs.dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
138 GROUP BY s.feed_url138 GROUP BY s.feed_url
139 ),139 ),
140 category_counts AS (140 category_counts AS (
141 SELECT category, COUNT(*) AS cnt141 SELECT category, COUNT(*) AS cnt
142- FROM subscriptions WHERE user_did = ? AND category IS NOT NULL AND category != ''142+ FROM articles.subscriptions WHERE user_did = ? AND category IS NOT NULL AND category != ''
143 GROUP BY category143 GROUP BY category
144 ),144 ),
145 max_subs AS (145 max_subs AS (
146- SELECT CAST(COALESCE(MAX(subscriber_count), 1) AS REAL) AS m FROM feeds146+ SELECT CAST(COALESCE(MAX(subscriber_count), 1) AS REAL) AS m FROM articles.feeds
147 )147 )
148 SELECT cf.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''),148 SELECT cf.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''),
149 COALESCE(f.description, ''), f.subscriber_count, COALESCE(f.favicon_url, ''),149 COALESCE(f.description, ''), f.subscriber_count, COALESCE(f.favicon_url, ''),
@@ -157,7 +157,7 @@ func (e *Engine) ComputeFeedRecommendationsOnDemand(ctx context.Context, userDID
157 ) THEN ? ELSE 0 END157 ) THEN ? ELSE 0 END
158 AS score158 AS score
159 FROM candidate_feeds cf159 FROM candidate_feeds cf
160- JOIN feeds f ON f.feed_url = cf.feed_url160+ JOIN articles.feeds f ON f.feed_url = cf.feed_url
161 LEFT JOIN like_signals ls ON ls.feed_url = cf.feed_url161 LEFT JOIN like_signals ls ON ls.feed_url = cf.feed_url
162 LEFT JOIN social_boost sb ON sb.feed_url = cf.feed_url162 LEFT JOIN social_boost sb ON sb.feed_url = cf.feed_url
163 CROSS JOIN max_subs ms163 CROSS JOIN max_subs ms
@@ -187,31 +187,31 @@ func (e *Engine) ComputeArticleRecommendationsOnDemand(ctx context.Context, user
187 187
188 rows, err := e.db.QueryContext(ctx, `188 rows, err := e.db.QueryContext(ctx, `
189 WITH similar_users AS (189 WITH similar_users AS (
190- SELECT user_b AS peer, jaccard FROM user_similarity WHERE user_a = ? AND jaccard > 0.15190+ SELECT user_b AS peer, jaccard FROM recs.user_similarity WHERE user_a = ? AND jaccard > 0.15
191 UNION ALL191 UNION ALL
192- SELECT user_a AS peer, jaccard FROM user_similarity WHERE user_b = ? AND jaccard > 0.15192+ SELECT user_a AS peer, jaccard FROM recs.user_similarity WHERE user_b = ? AND jaccard > 0.15
193 ),193 ),
194 liked_articles AS (194 liked_articles AS (
195 SELECT l.feed_url, l.article_url,195 SELECT l.feed_url, l.article_url,
196 SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal196 SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal
197 FROM similar_users su197 FROM similar_users su
198- JOIN likes l ON l.author_did = su.peer198+ JOIN articles.likes l ON l.author_did = su.peer
199 WHERE NOT EXISTS (199 WHERE NOT EXISTS (
200- SELECT 1 FROM likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url200+ SELECT 1 FROM articles.likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url
201 )201 )
202 AND NOT EXISTS (202 AND NOT EXISTS (
203- SELECT 1 FROM dismissed_recommendations d WHERE d.user_did = ? AND d.target_type = 'article' AND d.target_id = l.article_url203+ SELECT 1 FROM recs.dismissed_recommendations d WHERE d.user_did = ? AND d.target_type = 'article' AND d.target_id = l.article_url
204 )204 )
205 GROUP BY l.feed_url, l.article_url205 GROUP BY l.feed_url, l.article_url
206 ),206 ),
207 social_likes AS (207 social_likes AS (
208 SELECT l.feed_url, l.article_url,208 SELECT l.feed_url, l.article_url,
209 SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social209 SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social
210- FROM follow_distances fd210+ FROM recs.follow_distances fd
211- JOIN likes l ON l.author_did = fd.user_b211+ JOIN articles.likes l ON l.author_did = fd.user_b
212 WHERE fd.user_a = ?212 WHERE fd.user_a = ?
213 AND NOT EXISTS (213 AND NOT EXISTS (
214- SELECT 1 FROM likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url214+ SELECT 1 FROM articles.likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url
215 )215 )
216 GROUP BY l.feed_url, l.article_url216 GROUP BY l.feed_url, l.article_url
217 )217 )
@@ -223,10 +223,9 @@ func (e *Engine) ComputeArticleRecommendationsOnDemand(ctx context.Context, user
223 + EXP(-0.023 * CAST(julianday('now') - julianday(a.published) AS REAL)) * 0.2223 + EXP(-0.023 * CAST(julianday('now') - julianday(a.published) AS REAL)) * 0.2
224 AS score224 AS score
225 FROM liked_articles la225 FROM liked_articles la
226- JOIN articles a ON a.feed_url = la.feed_url AND a.url = la.article_url226+ JOIN articles.articles a ON a.feed_url = la.feed_url AND a.url = la.article_url
227- LEFT JOIN feeds f ON f.feed_url = la.feed_url227+ LEFT JOIN articles.feeds f ON f.feed_url = la.feed_url
228 LEFT JOIN social_likes sl ON sl.feed_url = la.feed_url AND sl.article_url = la.article_url228 LEFT JOIN social_likes sl ON sl.feed_url = la.feed_url AND sl.article_url = la.article_url
229- -- Future-published articles (e.g., scheduled) sort last
230 ORDER BY score DESC, (CASE WHEN a.published > 'now' THEN 1 ELSE 0 END), a.published DESC229 ORDER BY score DESC, (CASE WHEN a.published > 'now' THEN 1 ELSE 0 END), a.published DESC
231 LIMIT ?230 LIMIT ?
232 `, userDID, userDID, userDID, userDID, userDID, userDID, w.WLike, w.WSocial, limit)231 `, userDID, userDID, userDID, userDID, userDID, userDID, w.WLike, w.WSocial, limit)
@@ -252,13 +251,13 @@ func (e *Engine) ComputePeopleRecommendationsOnDemand(ctx context.Context, userD
252 SELECT u.did, u.handle, COALESCE(u.display_name, ''), COALESCE(u.avatar_url, ''),251 SELECT u.did, u.handle, COALESCE(u.display_name, ''), COALESCE(u.avatar_url, ''),
253 sim.jaccard, sim.common_feeds, COALESCE(sim.common_likes, 0), COALESCE(sim.common_tags, 0)252 sim.jaccard, sim.common_feeds, COALESCE(sim.common_likes, 0), COALESCE(sim.common_tags, 0)
254 FROM (253 FROM (
255- SELECT user_b AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_a = ?254+ SELECT user_b AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM recs.user_similarity WHERE user_a = ?
256 UNION ALL255 UNION ALL
257- SELECT user_a AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_b = ?256+ SELECT user_a AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM recs.user_similarity WHERE user_b = ?
258 ) sim257 ) sim
259- JOIN users u ON u.did = sim.peer_did258+ JOIN main.users u ON u.did = sim.peer_did
260 WHERE u.handle IS NOT NULL AND u.handle != ''259 WHERE u.handle IS NOT NULL AND u.handle != ''
261- AND EXISTS (SELECT 1 FROM subscriptions s JOIN feeds f ON s.feed_url = f.feed_url WHERE s.user_did = u.did AND f.subscriber_count > 0)260+ AND EXISTS (SELECT 1 FROM articles.subscriptions s JOIN articles.feeds f ON s.feed_url = f.feed_url WHERE s.user_did = u.did AND f.subscriber_count > 0)
262 ORDER BY sim.jaccard DESC261 ORDER BY sim.jaccard DESC
263 LIMIT ?262 LIMIT ?
264 `, userDID, userDID, limit)263 `, userDID, userDID, limit)
@@ -286,7 +285,7 @@ func (e *Engine) ComputeSignalProfiles(ctx context.Context) error {
286 }285 }
287 defer func() { _ = tx.Rollback() }()286 defer func() { _ = tx.Rollback() }()
288 287
289- if _, err := tx.ExecContext(ctx, `DELETE FROM user_signal_profiles`); err != nil {288+ if _, err := tx.ExecContext(ctx, `DELETE FROM recs.user_signal_profiles`); err != nil {
290 return err289 return err
291 }290 }
292 291
@@ -299,7 +298,7 @@ func (e *Engine) ComputeSignalProfiles(ctx context.Context) error {
299 return err298 return err
300 }299 }
301 if _, err := tx.ExecContext(ctx, `300 if _, err := tx.ExecContext(ctx, `
302- INSERT INTO _user_like_counts SELECT author_did, COUNT(*) FROM likes GROUP BY author_did301+ INSERT INTO _user_like_counts SELECT author_did, COUNT(*) FROM articles.likes GROUP BY author_did
303 `); err != nil {302 `); err != nil {
304 return err303 return err
305 }304 }
@@ -316,7 +315,7 @@ func (e *Engine) ComputeSignalProfiles(ctx context.Context) error {
316 INSERT INTO _user_tag_counts315 INSERT INTO _user_tag_counts
317 WITH user_tags AS (316 WITH user_tags AS (
318 SELECT author_did, TRIM(value) AS tag317 SELECT author_did, TRIM(value) AS tag
319- FROM annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]')318+ FROM articles.annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]')
320 WHERE tags IS NOT NULL AND tags != ''319 WHERE tags IS NOT NULL AND tags != ''
321 )320 )
322 SELECT author_did, COUNT(DISTINCT tag) FROM user_tags GROUP BY author_did321 SELECT author_did, COUNT(DISTINCT tag) FROM user_tags GROUP BY author_did
@@ -337,7 +336,7 @@ func (e *Engine) ComputeSignalProfiles(ctx context.Context) error {
337 SELECT user_did, '[' || GROUP_CONCAT('{"c":"' || category || '","n":"' || CAST(cnt AS TEXT) || '}') || ']'336 SELECT user_did, '[' || GROUP_CONCAT('{"c":"' || category || '","n":"' || CAST(cnt AS TEXT) || '}') || ']'
338 FROM (337 FROM (
339 SELECT user_did, category, COUNT(*) AS cnt338 SELECT user_did, category, COUNT(*) AS cnt
340- FROM subscriptions339+ FROM articles.subscriptions
341 WHERE category IS NOT NULL AND category != ''340 WHERE category IS NOT NULL AND category != ''
342 GROUP BY user_did, category341 GROUP BY user_did, category
343 ORDER BY COUNT(*) DESC342 ORDER BY COUNT(*) DESC
@@ -349,13 +348,13 @@ func (e *Engine) ComputeSignalProfiles(ctx context.Context) error {
349 }348 }
350 349
351 _, err = tx.ExecContext(ctx, `350 _, err = tx.ExecContext(ctx, `
352- INSERT INTO user_signal_profiles (user_did, total_likes, total_tags, top_categories)351+ INSERT INTO recs.user_signal_profiles (user_did, total_likes, total_tags, top_categories)
353 SELECT352 SELECT
354 u.did,353 u.did,
355 COALESCE(lc.cnt, 0),354 COALESCE(lc.cnt, 0),
356 COALESCE(tc.cnt, 0),355 COALESCE(tc.cnt, 0),
357 COALESCE(cc.categories, '[]')356 COALESCE(cc.categories, '[]')
358- FROM users u357+ FROM main.users u
359 LEFT JOIN _user_like_counts lc ON lc.user_did = u.did358 LEFT JOIN _user_like_counts lc ON lc.user_did = u.did
360 LEFT JOIN _user_tag_counts tc ON tc.user_did = u.did359 LEFT JOIN _user_tag_counts tc ON tc.user_did = u.did
361 LEFT JOIN _user_top_categories cc ON cc.user_did = u.did360 LEFT JOIN _user_top_categories cc ON cc.user_did = u.did
@@ -370,7 +369,7 @@ func (e *Engine) ComputeSignalProfiles(ctx context.Context) error {
370 369
371 func (e *Engine) ColdStartRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) {370 func (e *Engine) ColdStartRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) {
372 subCount := 0371 subCount := 0
373- _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM subscriptions WHERE user_did = ?`, userDID).Scan(&subCount)372+ _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM articles.subscriptions WHERE user_did = ?`, userDID).Scan(&subCount)
374 if subCount >= 5 {373 if subCount >= 5 {
375 return nil, nil374 return nil, nil
376 }375 }
@@ -378,19 +377,19 @@ func (e *Engine) ColdStartRecommendations(ctx context.Context, userDID string, l
378 rows, err := e.db.QueryContext(ctx, `377 rows, err := e.db.QueryContext(ctx, `
379 WITH followed_feeds AS (378 WITH followed_feeds AS (
380 SELECT s.feed_url, 1.0 AS weight379 SELECT s.feed_url, 1.0 AS weight
381- FROM follow_distances fd380+ FROM recs.follow_distances fd
382- JOIN subscriptions s ON s.user_did = fd.user_b381+ JOIN articles.subscriptions s ON s.user_did = fd.user_b
383 WHERE fd.user_a = ? AND fd.distance = 1382 WHERE fd.user_a = ? AND fd.distance = 1
384- AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)383+ AND s.feed_url NOT IN (SELECT feed_url FROM articles.subscriptions WHERE user_did = ?)
385- AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')384+ AND s.feed_url NOT IN (SELECT target_id FROM recs.dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
386 ),385 ),
387 popular_feeds AS (386 popular_feeds AS (
388 SELECT feed_url, subscriber_count,387 SELECT feed_url, subscriber_count,
389- LOG(1 + CAST(subscriber_count AS REAL)) / LOG(1 + CAST((SELECT COALESCE(MAX(subscriber_count), 1) FROM feeds) AS REAL)) AS pop_score388+ LOG(1 + CAST(subscriber_count AS REAL)) / LOG(1 + CAST((SELECT COALESCE(MAX(subscriber_count), 1) FROM articles.feeds) AS REAL)) AS pop_score
390- FROM feeds389+ FROM articles.feeds
391 WHERE subscriber_count > 0390 WHERE subscriber_count > 0
392- AND feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)391+ AND feed_url NOT IN (SELECT feed_url FROM articles.subscriptions WHERE user_did = ?)
393- AND feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')392+ AND feed_url NOT IN (SELECT target_id FROM recs.dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
394 ORDER BY subscriber_count DESC393 ORDER BY subscriber_count DESC
395 LIMIT 50394 LIMIT 50
396 ),395 ),
@@ -410,7 +409,7 @@ func (e *Engine) ColdStartRecommendations(ctx context.Context, userDID string, l
410 COALESCE(f.favicon_url, ''),409 COALESCE(f.favicon_url, ''),
411 ac.weight AS score410 ac.weight AS score
412 FROM all_candidates ac411 FROM all_candidates ac
413- JOIN feeds f ON f.feed_url = ac.feed_url412+ JOIN articles.feeds f ON f.feed_url = ac.feed_url
414 ORDER BY score DESC413 ORDER BY score DESC
415 LIMIT ?414 LIMIT ?
416 `, userDID, userDID, userDID, userDID, userDID, limit)415 `, userDID, userDID, userDID, userDID, userDID, limit)
@@ -429,4 +428,4 @@ func (e *Engine) ColdStartRecommendations(ctx context.Context, userDID string, l
429 results = append(results, rec)428 results = append(results, rec)
430 }429 }
431 return results, rows.Err()430 return results, rows.Err()
432-}431+}
\ No newline at end of file\ No newline at end of file
modified internal/cluster/social.go +8 -8
@@ -11,18 +11,18 @@ func (e *Engine) ComputeFollowDistances(ctx context.Context) error {
1111 }
1212 defer func() { _ = tx.Rollback() }()
1313
14- if _, err := tx.ExecContext(ctx, `DELETE FROM follow_distances`); err != nil {
14+ if _, err := tx.ExecContext(ctx, `DELETE FROM recs.follow_distances`); err != nil {
1515 return err
1616 }
1717
1818 _, err = tx.ExecContext(ctx, `
19- INSERT INTO follow_distances (user_a, user_b, distance)
19+ INSERT INTO recs.follow_distances (user_a, user_b, distance)
2020 SELECT user_a, user_b, MIN(distance) FROM (
21- SELECT user_did AS user_a, target_did AS user_b, 1 AS distance FROM follows WHERE user_did != target_did
21+ SELECT user_did AS user_a, target_did AS user_b, 1 AS distance FROM main.follows WHERE user_did != target_did
2222 UNION ALL
2323 SELECT f1.user_did, f2.target_did, 2
24- FROM follows f1
25- JOIN follows f2 ON f1.target_did = f2.user_did
24+ FROM main.follows f1
25+ JOIN main.follows f2 ON f1.target_did = f2.user_did
2626 WHERE f1.user_did != f2.target_did
2727 ) GROUP BY user_a, user_b
2828 `)
@@ -37,7 +37,7 @@ func (e *Engine) ComputeFollowDistances(ctx context.Context) error {
3737 func (e *Engine) ComputeFollowDistancesIncremental(ctx context.Context) error {
3838 var maxFollowed string
3939 err := e.db.QueryRowContext(ctx, `
40- SELECT COALESCE(MAX(followed_at), '1970-01-01') FROM follows
40+ SELECT COALESCE(MAX(followed_at), '1970-01-01') FROM main.follows
4141 `).Scan(&maxFollowed)
4242 if err != nil {
4343 return err
@@ -45,7 +45,7 @@ func (e *Engine) ComputeFollowDistancesIncremental(ctx context.Context) error {
4545
4646 var lastComputed string
4747 err = e.db.QueryRowContext(ctx, `
48- SELECT COALESCE(MAX(computed_at), '1970-01-01') FROM user_similarity
48+ SELECT COALESCE(MAX(computed_at), '1970-01-01') FROM recs.user_similarity
4949 `).Scan(&lastComputed)
5050 if err != nil {
5151 return err
@@ -56,4 +56,4 @@ func (e *Engine) ComputeFollowDistancesIncremental(ctx context.Context) error {
5656 }
5757
5858 return e.ComputeFollowDistances(ctx)
59-}
59+}
\ No newline at end of file
@@ -11,18 +11,18 @@ func (e *Engine) ComputeFollowDistances(ctx context.Context) error {
11 }11 }
12 defer func() { _ = tx.Rollback() }()12 defer func() { _ = tx.Rollback() }()
13 13
14- if _, err := tx.ExecContext(ctx, `DELETE FROM follow_distances`); err != nil {14+ if _, err := tx.ExecContext(ctx, `DELETE FROM recs.follow_distances`); err != nil {
15 return err15 return err
16 }16 }
17 17
18 _, err = tx.ExecContext(ctx, `18 _, err = tx.ExecContext(ctx, `
19- INSERT INTO follow_distances (user_a, user_b, distance)19+ INSERT INTO recs.follow_distances (user_a, user_b, distance)
20 SELECT user_a, user_b, MIN(distance) FROM (20 SELECT user_a, user_b, MIN(distance) FROM (
21- SELECT user_did AS user_a, target_did AS user_b, 1 AS distance FROM follows WHERE user_did != target_did21+ SELECT user_did AS user_a, target_did AS user_b, 1 AS distance FROM main.follows WHERE user_did != target_did
22 UNION ALL22 UNION ALL
23 SELECT f1.user_did, f2.target_did, 223 SELECT f1.user_did, f2.target_did, 2
24- FROM follows f124+ FROM main.follows f1
25- JOIN follows f2 ON f1.target_did = f2.user_did25+ JOIN main.follows f2 ON f1.target_did = f2.user_did
26 WHERE f1.user_did != f2.target_did26 WHERE f1.user_did != f2.target_did
27 ) GROUP BY user_a, user_b27 ) GROUP BY user_a, user_b
28 `)28 `)
@@ -37,7 +37,7 @@ func (e *Engine) ComputeFollowDistances(ctx context.Context) error {
37 func (e *Engine) ComputeFollowDistancesIncremental(ctx context.Context) error {37 func (e *Engine) ComputeFollowDistancesIncremental(ctx context.Context) error {
38 var maxFollowed string38 var maxFollowed string
39 err := e.db.QueryRowContext(ctx, `39 err := e.db.QueryRowContext(ctx, `
40- SELECT COALESCE(MAX(followed_at), '1970-01-01') FROM follows40+ SELECT COALESCE(MAX(followed_at), '1970-01-01') FROM main.follows
41 `).Scan(&maxFollowed)41 `).Scan(&maxFollowed)
42 if err != nil {42 if err != nil {
43 return err43 return err
@@ -45,7 +45,7 @@ func (e *Engine) ComputeFollowDistancesIncremental(ctx context.Context) error {
45 45
46 var lastComputed string46 var lastComputed string
47 err = e.db.QueryRowContext(ctx, `47 err = e.db.QueryRowContext(ctx, `
48- SELECT COALESCE(MAX(computed_at), '1970-01-01') FROM user_similarity48+ SELECT COALESCE(MAX(computed_at), '1970-01-01') FROM recs.user_similarity
49 `).Scan(&lastComputed)49 `).Scan(&lastComputed)
50 if err != nil {50 if err != nil {
51 return err51 return err
@@ -56,4 +56,4 @@ func (e *Engine) ComputeFollowDistancesIncremental(ctx context.Context) error {
56 }56 }
57 57
58 return e.ComputeFollowDistances(ctx)58 return e.ComputeFollowDistances(ctx)
59-}59+}
\ No newline at end of file\ No newline at end of file
modified internal/cluster/weights.go +5 -5
@@ -22,18 +22,18 @@ func (e *Engine) PenalizeSignal(ctx context.Context, userDID string, signal stri
2222 func (e *Engine) adjustWeight(ctx context.Context, userDID string, signal string, delta float64) {
2323 var actedCount int
2424 _ = e.db.QueryRowContext(ctx, `
25- SELECT COUNT(*) FROM recommendation_impressions WHERE user_did = ? AND acted = 1
25+ SELECT COUNT(*) FROM recs.recommendation_impressions WHERE user_did = ? AND acted = 1
2626 `, userDID).Scan(&actedCount)
2727 if actedCount < minActionsTune {
2828 return
2929 }
3030
3131 var exists int
32- _ = e.db.QueryRowContext(ctx, `SELECT 1 FROM user_signal_weights WHERE user_did = ?`, userDID).Scan(&exists)
32+ _ = e.db.QueryRowContext(ctx, `SELECT 1 FROM recs.user_signal_weights WHERE user_did = ?`, userDID).Scan(&exists)
3333
3434 if exists == 0 {
3535 _, _ = e.db.ExecContext(ctx, `
36- INSERT INTO user_signal_weights (user_did, w_sub, w_like, w_tag, w_social, w_pop, w_category)
36+ INSERT INTO recs.user_signal_weights (user_did, w_sub, w_like, w_tag, w_social, w_pop, w_category)
3737 VALUES (?, 1.0, 0.5, 0.3, 0.7, 0.2, 0.4)
3838 `, userDID)
3939 }
@@ -45,7 +45,7 @@ func (e *Engine) adjustWeight(ctx context.Context, userDID string, signal string
4545
4646 adj := learningRate * delta
4747 _, _ = e.db.ExecContext(ctx, `
48- UPDATE user_signal_weights SET
48+ UPDATE recs.user_signal_weights SET
4949 `+column+` = MAX(?, MIN(?, `+column+` * (1 + ?))),
5050 updated_at = CURRENT_TIMESTAMP
5151 WHERE user_did = ?
@@ -90,4 +90,4 @@ func (e *Engine) GetDominantSignal(w SignalWeights) string {
9090 }
9191 }
9292 return best
93-}
93+}
\ No newline at end of file
@@ -22,18 +22,18 @@ func (e *Engine) PenalizeSignal(ctx context.Context, userDID string, signal stri
22 func (e *Engine) adjustWeight(ctx context.Context, userDID string, signal string, delta float64) {22 func (e *Engine) adjustWeight(ctx context.Context, userDID string, signal string, delta float64) {
23 var actedCount int23 var actedCount int
24 _ = e.db.QueryRowContext(ctx, `24 _ = e.db.QueryRowContext(ctx, `
25- SELECT COUNT(*) FROM recommendation_impressions WHERE user_did = ? AND acted = 125+ SELECT COUNT(*) FROM recs.recommendation_impressions WHERE user_did = ? AND acted = 1
26 `, userDID).Scan(&actedCount)26 `, userDID).Scan(&actedCount)
27 if actedCount < minActionsTune {27 if actedCount < minActionsTune {
28 return28 return
29 }29 }
30 30
31 var exists int31 var exists int
32- _ = e.db.QueryRowContext(ctx, `SELECT 1 FROM user_signal_weights WHERE user_did = ?`, userDID).Scan(&exists)32+ _ = e.db.QueryRowContext(ctx, `SELECT 1 FROM recs.user_signal_weights WHERE user_did = ?`, userDID).Scan(&exists)
33 33
34 if exists == 0 {34 if exists == 0 {
35 _, _ = e.db.ExecContext(ctx, `35 _, _ = e.db.ExecContext(ctx, `
36- INSERT INTO user_signal_weights (user_did, w_sub, w_like, w_tag, w_social, w_pop, w_category)36+ INSERT INTO recs.user_signal_weights (user_did, w_sub, w_like, w_tag, w_social, w_pop, w_category)
37 VALUES (?, 1.0, 0.5, 0.3, 0.7, 0.2, 0.4)37 VALUES (?, 1.0, 0.5, 0.3, 0.7, 0.2, 0.4)
38 `, userDID)38 `, userDID)
39 }39 }
@@ -45,7 +45,7 @@ func (e *Engine) adjustWeight(ctx context.Context, userDID string, signal string
45 45
46 adj := learningRate * delta46 adj := learningRate * delta
47 _, _ = e.db.ExecContext(ctx, `47 _, _ = e.db.ExecContext(ctx, `
48- UPDATE user_signal_weights SET48+ UPDATE recs.user_signal_weights SET
49 `+column+` = MAX(?, MIN(?, `+column+` * (1 + ?))),49 `+column+` = MAX(?, MIN(?, `+column+` * (1 + ?))),
50 updated_at = CURRENT_TIMESTAMP50 updated_at = CURRENT_TIMESTAMP
51 WHERE user_did = ?51 WHERE user_did = ?
@@ -90,4 +90,4 @@ func (e *Engine) GetDominantSignal(w SignalWeights) string {
90 }90 }
91 }91 }
92 return best92 return best
93-}93+}
\ No newline at end of file\ No newline at end of file
modified internal/db/db.go +43 -46
@@ -9,6 +9,8 @@ import (
99 "github.com/mattn/go-sqlite3"
1010 )
1111
12+const DSN = "_journal_mode=WAL&_busy_timeout=5000&_synchronous=NORMAL&_cache=shared"
13+
1214 func init() {
1315 sql.Register("sqlite3_glean", &sqlite3.SQLiteDriver{
1416 ConnectHook: func(conn *sqlite3.SQLiteConn) error {
@@ -61,7 +63,7 @@ func (d *DB) Close() error {
6163 }
6264
6365 func Open(path string) (*DB, error) {
64- db, err := sql.Open("sqlite3_glean", path+"?_journal_mode=WAL&_busy_timeout=30000&_synchronous=NORMAL&_cache_size=-64000&_stmt_cache_size=64&_mutex=no")
66+ db, err := sql.Open("sqlite3_glean", path+"?cache=shared&"+DSN)
6567 if err != nil {
6668 return nil, err
6769 }
@@ -70,15 +72,16 @@ func Open(path string) (*DB, error) {
7072 db.SetMaxIdleConns(5)
7173 db.SetConnMaxLifetime(30 * time.Minute)
7274
73- if err := initSchema(db); err != nil {
75+ wrapped := &DB{db}
76+ if err := initSchema(wrapped); err != nil {
7477 db.Close()
7578 return nil, err
7679 }
7780
78- return &DB{db}, nil
81+ return wrapped, nil
7982 }
8083
81-func initSchema(db *sql.DB) error {
84+func initSchema(db *DB) error {
8285 for _, s := range schema {
8386 if _, err := db.Exec(s); err != nil {
8487 return err
@@ -115,8 +118,8 @@ var schema = []string{
115118 )`,
116119 `CREATE TABLE IF NOT EXISTS subscriptions (
117120 id INTEGER PRIMARY KEY AUTOINCREMENT,
118- user_did TEXT NOT NULL REFERENCES users(did),
119- feed_url TEXT NOT NULL REFERENCES feeds(feed_url),
121+ user_did TEXT NOT NULL,
122+ feed_url TEXT NOT NULL,
120123 title TEXT,
121124 category TEXT,
122125 added_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
@@ -126,7 +129,7 @@ var schema = []string{
126129 )`,
127130 `CREATE TABLE IF NOT EXISTS articles (
128131 id INTEGER PRIMARY KEY AUTOINCREMENT,
129- feed_url TEXT NOT NULL REFERENCES feeds(feed_url),
132+ feed_url TEXT NOT NULL,
130133 guid TEXT NOT NULL,
131134 title TEXT NOT NULL DEFAULT '',
132135 url TEXT,
@@ -140,8 +143,8 @@ var schema = []string{
140143 UNIQUE(feed_url, guid)
141144 )`,
142145 `CREATE TABLE IF NOT EXISTS read_state (
143- user_did TEXT NOT NULL REFERENCES users(did),
144- article_id INTEGER NOT NULL REFERENCES articles(id),
146+ user_did TEXT NOT NULL,
147+ article_id INTEGER NOT NULL,
145148 is_read BOOLEAN NOT NULL DEFAULT 0,
146149 read_at DATETIME,
147150 PRIMARY KEY (user_did, article_id)
@@ -149,7 +152,7 @@ var schema = []string{
149152 `CREATE TABLE IF NOT EXISTS annotations (
150153 id INTEGER PRIMARY KEY AUTOINCREMENT,
151154 uri TEXT NOT NULL UNIQUE,
152- author_did TEXT NOT NULL REFERENCES users(did),
155+ author_did TEXT NOT NULL,
153156 feed_url TEXT NOT NULL,
154157 article_url TEXT NOT NULL,
155158 quote TEXT,
@@ -162,7 +165,7 @@ var schema = []string{
162165 `CREATE TABLE IF NOT EXISTS likes (
163166 id INTEGER PRIMARY KEY AUTOINCREMENT,
164167 uri TEXT NOT NULL UNIQUE,
165- author_did TEXT NOT NULL REFERENCES users(did),
168+ author_did TEXT NOT NULL,
166169 feed_url TEXT NOT NULL,
167170 article_url TEXT NOT NULL,
168171 created_at DATETIME NOT NULL,
@@ -170,16 +173,16 @@ var schema = []string{
170173 UNIQUE(author_did, feed_url, article_url)
171174 )`,
172175 `CREATE TABLE IF NOT EXISTS feed_similarity (
173- feed_a TEXT NOT NULL REFERENCES feeds(feed_url),
174- feed_b TEXT NOT NULL REFERENCES feeds(feed_url),
176+ feed_a TEXT NOT NULL,
177+ feed_b TEXT NOT NULL,
175178 jaccard REAL NOT NULL,
176179 computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
177180 PRIMARY KEY (feed_a, feed_b),
178181 CHECK(feed_a < feed_b)
179182 )`,
180183 `CREATE TABLE IF NOT EXISTS user_similarity (
181- user_a TEXT NOT NULL REFERENCES users(did),
182- user_b TEXT NOT NULL REFERENCES users(did),
184+ user_a TEXT NOT NULL,
185+ user_b TEXT NOT NULL,
183186 jaccard REAL NOT NULL,
184187 common_feeds INTEGER NOT NULL,
185188 common_likes INTEGER NOT NULL DEFAULT 0,
@@ -189,7 +192,7 @@ var schema = []string{
189192 CHECK(user_a < user_b)
190193 )`,
191194 `CREATE TABLE IF NOT EXISTS follows (
192- user_did TEXT NOT NULL REFERENCES users(did),
195+ user_did TEXT NOT NULL,
193196 target_did TEXT NOT NULL,
194197 uri TEXT,
195198 cid TEXT,
@@ -206,38 +209,16 @@ var schema = []string{
206209 data TEXT NOT NULL,
207210 PRIMARY KEY (account_did, session_id)
208211 )`,
209- `CREATE INDEX IF NOT EXISTS idx_subscriptions_feed ON subscriptions(feed_url)`,
210- `CREATE INDEX IF NOT EXISTS idx_subscriptions_feed_user ON subscriptions(feed_url, user_did)`,
211- `CREATE INDEX IF NOT EXISTS idx_subscriptions_user ON subscriptions(user_did)`,
212- `CREATE INDEX IF NOT EXISTS idx_subscriptions_uri ON subscriptions(uri)`,
213- `CREATE INDEX IF NOT EXISTS idx_likes_author_feed ON likes(author_did, feed_url, created_at)`,
214- `CREATE INDEX IF NOT EXISTS idx_articles_feed ON articles(feed_url)`,
215- `CREATE INDEX IF NOT EXISTS idx_articles_published ON articles(published DESC)`,
216- `CREATE INDEX IF NOT EXISTS idx_articles_url ON articles(url)`,
217- `CREATE INDEX IF NOT EXISTS idx_read_state_unread ON read_state(user_did, is_read) WHERE is_read = 0`,
218- `CREATE INDEX IF NOT EXISTS idx_annotations_article ON annotations(article_url)`,
219- `CREATE INDEX IF NOT EXISTS idx_annotations_author ON annotations(author_did)`,
220- `CREATE INDEX IF NOT EXISTS idx_annotations_created_at ON annotations(created_at DESC)`,
221- `CREATE INDEX IF NOT EXISTS idx_likes_article ON likes(feed_url, article_url)`,
222- `CREATE INDEX IF NOT EXISTS idx_likes_author ON likes(author_did)`,
223- `CREATE INDEX IF NOT EXISTS idx_likes_created_at ON likes(created_at DESC)`,
224- `CREATE INDEX IF NOT EXISTS idx_follows_user ON follows(user_did)`,
225- `CREATE INDEX IF NOT EXISTS idx_follows_target ON follows(target_did)`,
226- `CREATE INDEX IF NOT EXISTS idx_follows_uri ON follows(uri)`,
227- `CREATE INDEX IF NOT EXISTS idx_user_similarity_b ON user_similarity(user_b)`,
228- `CREATE INDEX IF NOT EXISTS idx_user_similarity_a ON user_similarity(user_a)`,
229-
230212 `CREATE TABLE IF NOT EXISTS dismissed_recommendations (
231- user_did TEXT NOT NULL REFERENCES users(did),
213+ user_did TEXT NOT NULL,
232214 target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
233215 target_id TEXT NOT NULL,
234216 reason TEXT,
235217 dismissed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
236218 PRIMARY KEY (user_did, target_type, target_id)
237219 )`,
238-
239220 `CREATE TABLE IF NOT EXISTS recommendation_impressions (
240- user_did TEXT NOT NULL REFERENCES users(did),
221+ user_did TEXT NOT NULL,
241222 target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
242223 target_id TEXT NOT NULL,
243224 first_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
@@ -246,16 +227,14 @@ var schema = []string{
246227 acted BOOLEAN NOT NULL DEFAULT 0,
247228 PRIMARY KEY (user_did, target_type, target_id)
248229 )`,
249-
250230 `CREATE TABLE IF NOT EXISTS follow_distances (
251231 user_a TEXT NOT NULL,
252232 user_b TEXT NOT NULL,
253233 distance INTEGER NOT NULL CHECK(distance IN (1, 2)),
254234 PRIMARY KEY (user_a, user_b)
255235 )`,
256-
257236 `CREATE TABLE IF NOT EXISTS user_signal_weights (
258- user_did TEXT PRIMARY KEY REFERENCES users(did),
237+ user_did TEXT PRIMARY KEY,
259238 w_sub REAL NOT NULL DEFAULT 1.0,
260239 w_like REAL NOT NULL DEFAULT 0.5,
261240 w_tag REAL NOT NULL DEFAULT 0.3,
@@ -264,15 +243,33 @@ var schema = []string{
264243 w_category REAL NOT NULL DEFAULT 0.4,
265244 updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
266245 )`,
267-
268246 `CREATE TABLE IF NOT EXISTS user_signal_profiles (
269- user_did TEXT PRIMARY KEY REFERENCES users(did),
247+ user_did TEXT PRIMARY KEY,
270248 total_likes INTEGER NOT NULL DEFAULT 0,
271249 total_tags INTEGER NOT NULL DEFAULT 0,
272250 top_categories TEXT,
273251 updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
274252 )`,
275-
253+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_feed ON subscriptions(feed_url)`,
254+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_feed_user ON subscriptions(feed_url, user_did)`,
255+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_user ON subscriptions(user_did)`,
256+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_uri ON subscriptions(uri)`,
257+ `CREATE INDEX IF NOT EXISTS idx_likes_author_feed ON likes(author_did, feed_url, created_at)`,
258+ `CREATE INDEX IF NOT EXISTS idx_articles_feed ON articles(feed_url)`,
259+ `CREATE INDEX IF NOT EXISTS idx_articles_published ON articles(published DESC)`,
260+ `CREATE INDEX IF NOT EXISTS idx_articles_url ON articles(url)`,
261+ `CREATE INDEX IF NOT EXISTS idx_read_state_unread ON read_state(user_did, is_read) WHERE is_read = 0`,
262+ `CREATE INDEX IF NOT EXISTS idx_annotations_article ON annotations(article_url)`,
263+ `CREATE INDEX IF NOT EXISTS idx_annotations_author ON annotations(author_did)`,
264+ `CREATE INDEX IF NOT EXISTS idx_annotations_created_at ON annotations(created_at DESC)`,
265+ `CREATE INDEX IF NOT EXISTS idx_likes_article ON likes(feed_url, article_url)`,
266+ `CREATE INDEX IF NOT EXISTS idx_likes_author ON likes(author_did)`,
267+ `CREATE INDEX IF NOT EXISTS idx_likes_created_at ON likes(created_at DESC)`,
268+ `CREATE INDEX IF NOT EXISTS idx_follows_user ON follows(user_did)`,
269+ `CREATE INDEX IF NOT EXISTS idx_follows_target ON follows(target_did)`,
270+ `CREATE INDEX IF NOT EXISTS idx_follows_uri ON follows(uri)`,
271+ `CREATE INDEX IF NOT EXISTS idx_user_similarity_b ON user_similarity(user_b)`,
272+ `CREATE INDEX IF NOT EXISTS idx_user_similarity_a ON user_similarity(user_a)`,
276273 `CREATE INDEX IF NOT EXISTS idx_dismissed_user_type ON dismissed_recommendations(user_did, target_type)`,
277274 `CREATE INDEX IF NOT EXISTS idx_impressions_user_unacted ON recommendation_impressions(user_did, acted, shown_count)`,
278275 `CREATE INDEX IF NOT EXISTS idx_impressions_last_shown ON recommendation_impressions(last_shown_at)`,
@@ -9,6 +9,8 @@ import (
9 "github.com/mattn/go-sqlite3"9 "github.com/mattn/go-sqlite3"
10 )10 )
11 11
12+const DSN = "_journal_mode=WAL&_busy_timeout=5000&_synchronous=NORMAL&_cache=shared"
13+
12 func init() {14 func init() {
13 sql.Register("sqlite3_glean", &sqlite3.SQLiteDriver{15 sql.Register("sqlite3_glean", &sqlite3.SQLiteDriver{
14 ConnectHook: func(conn *sqlite3.SQLiteConn) error {16 ConnectHook: func(conn *sqlite3.SQLiteConn) error {
@@ -61,7 +63,7 @@ func (d *DB) Close() error {
61 }63 }
62 64
63 func Open(path string) (*DB, error) {65 func Open(path string) (*DB, error) {
64- db, err := sql.Open("sqlite3_glean", path+"?_journal_mode=WAL&_busy_timeout=30000&_synchronous=NORMAL&_cache_size=-64000&_stmt_cache_size=64&_mutex=no")66+ db, err := sql.Open("sqlite3_glean", path+"?cache=shared&"+DSN)
65 if err != nil {67 if err != nil {
66 return nil, err68 return nil, err
67 }69 }
@@ -70,15 +72,16 @@ func Open(path string) (*DB, error) {
70 db.SetMaxIdleConns(5)72 db.SetMaxIdleConns(5)
71 db.SetConnMaxLifetime(30 * time.Minute)73 db.SetConnMaxLifetime(30 * time.Minute)
72 74
73- if err := initSchema(db); err != nil {75+ wrapped := &DB{db}
76+ if err := initSchema(wrapped); err != nil {
74 db.Close()77 db.Close()
75 return nil, err78 return nil, err
76 }79 }
77 80
78- return &DB{db}, nil81+ return wrapped, nil
79 }82 }
80 83
81-func initSchema(db *sql.DB) error {84+func initSchema(db *DB) error {
82 for _, s := range schema {85 for _, s := range schema {
83 if _, err := db.Exec(s); err != nil {86 if _, err := db.Exec(s); err != nil {
84 return err87 return err
@@ -115,8 +118,8 @@ var schema = []string{
115 )`,118 )`,
116 `CREATE TABLE IF NOT EXISTS subscriptions (119 `CREATE TABLE IF NOT EXISTS subscriptions (
117 id INTEGER PRIMARY KEY AUTOINCREMENT,120 id INTEGER PRIMARY KEY AUTOINCREMENT,
118- user_did TEXT NOT NULL REFERENCES users(did),121+ user_did TEXT NOT NULL,
119- feed_url TEXT NOT NULL REFERENCES feeds(feed_url),122+ feed_url TEXT NOT NULL,
120 title TEXT,123 title TEXT,
121 category TEXT,124 category TEXT,
122 added_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,125 added_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
@@ -126,7 +129,7 @@ var schema = []string{
126 )`,129 )`,
127 `CREATE TABLE IF NOT EXISTS articles (130 `CREATE TABLE IF NOT EXISTS articles (
128 id INTEGER PRIMARY KEY AUTOINCREMENT,131 id INTEGER PRIMARY KEY AUTOINCREMENT,
129- feed_url TEXT NOT NULL REFERENCES feeds(feed_url),132+ feed_url TEXT NOT NULL,
130 guid TEXT NOT NULL,133 guid TEXT NOT NULL,
131 title TEXT NOT NULL DEFAULT '',134 title TEXT NOT NULL DEFAULT '',
132 url TEXT,135 url TEXT,
@@ -140,8 +143,8 @@ var schema = []string{
140 UNIQUE(feed_url, guid)143 UNIQUE(feed_url, guid)
141 )`,144 )`,
142 `CREATE TABLE IF NOT EXISTS read_state (145 `CREATE TABLE IF NOT EXISTS read_state (
143- user_did TEXT NOT NULL REFERENCES users(did),146+ user_did TEXT NOT NULL,
144- article_id INTEGER NOT NULL REFERENCES articles(id),147+ article_id INTEGER NOT NULL,
145 is_read BOOLEAN NOT NULL DEFAULT 0,148 is_read BOOLEAN NOT NULL DEFAULT 0,
146 read_at DATETIME,149 read_at DATETIME,
147 PRIMARY KEY (user_did, article_id)150 PRIMARY KEY (user_did, article_id)
@@ -149,7 +152,7 @@ var schema = []string{
149 `CREATE TABLE IF NOT EXISTS annotations (152 `CREATE TABLE IF NOT EXISTS annotations (
150 id INTEGER PRIMARY KEY AUTOINCREMENT,153 id INTEGER PRIMARY KEY AUTOINCREMENT,
151 uri TEXT NOT NULL UNIQUE,154 uri TEXT NOT NULL UNIQUE,
152- author_did TEXT NOT NULL REFERENCES users(did),155+ author_did TEXT NOT NULL,
153 feed_url TEXT NOT NULL,156 feed_url TEXT NOT NULL,
154 article_url TEXT NOT NULL,157 article_url TEXT NOT NULL,
155 quote TEXT,158 quote TEXT,
@@ -162,7 +165,7 @@ var schema = []string{
162 `CREATE TABLE IF NOT EXISTS likes (165 `CREATE TABLE IF NOT EXISTS likes (
163 id INTEGER PRIMARY KEY AUTOINCREMENT,166 id INTEGER PRIMARY KEY AUTOINCREMENT,
164 uri TEXT NOT NULL UNIQUE,167 uri TEXT NOT NULL UNIQUE,
165- author_did TEXT NOT NULL REFERENCES users(did),168+ author_did TEXT NOT NULL,
166 feed_url TEXT NOT NULL,169 feed_url TEXT NOT NULL,
167 article_url TEXT NOT NULL,170 article_url TEXT NOT NULL,
168 created_at DATETIME NOT NULL,171 created_at DATETIME NOT NULL,
@@ -170,16 +173,16 @@ var schema = []string{
170 UNIQUE(author_did, feed_url, article_url)173 UNIQUE(author_did, feed_url, article_url)
171 )`,174 )`,
172 `CREATE TABLE IF NOT EXISTS feed_similarity (175 `CREATE TABLE IF NOT EXISTS feed_similarity (
173- feed_a TEXT NOT NULL REFERENCES feeds(feed_url),176+ feed_a TEXT NOT NULL,
174- feed_b TEXT NOT NULL REFERENCES feeds(feed_url),177+ feed_b TEXT NOT NULL,
175 jaccard REAL NOT NULL,178 jaccard REAL NOT NULL,
176 computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,179 computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
177 PRIMARY KEY (feed_a, feed_b),180 PRIMARY KEY (feed_a, feed_b),
178 CHECK(feed_a < feed_b)181 CHECK(feed_a < feed_b)
179 )`,182 )`,
180 `CREATE TABLE IF NOT EXISTS user_similarity (183 `CREATE TABLE IF NOT EXISTS user_similarity (
181- user_a TEXT NOT NULL REFERENCES users(did),184+ user_a TEXT NOT NULL,
182- user_b TEXT NOT NULL REFERENCES users(did),185+ user_b TEXT NOT NULL,
183 jaccard REAL NOT NULL,186 jaccard REAL NOT NULL,
184 common_feeds INTEGER NOT NULL,187 common_feeds INTEGER NOT NULL,
185 common_likes INTEGER NOT NULL DEFAULT 0,188 common_likes INTEGER NOT NULL DEFAULT 0,
@@ -189,7 +192,7 @@ var schema = []string{
189 CHECK(user_a < user_b)192 CHECK(user_a < user_b)
190 )`,193 )`,
191 `CREATE TABLE IF NOT EXISTS follows (194 `CREATE TABLE IF NOT EXISTS follows (
192- user_did TEXT NOT NULL REFERENCES users(did),195+ user_did TEXT NOT NULL,
193 target_did TEXT NOT NULL,196 target_did TEXT NOT NULL,
194 uri TEXT,197 uri TEXT,
195 cid TEXT,198 cid TEXT,
@@ -206,38 +209,16 @@ var schema = []string{
206 data TEXT NOT NULL,209 data TEXT NOT NULL,
207 PRIMARY KEY (account_did, session_id)210 PRIMARY KEY (account_did, session_id)
208 )`,211 )`,
209- `CREATE INDEX IF NOT EXISTS idx_subscriptions_feed ON subscriptions(feed_url)`,
210- `CREATE INDEX IF NOT EXISTS idx_subscriptions_feed_user ON subscriptions(feed_url, user_did)`,
211- `CREATE INDEX IF NOT EXISTS idx_subscriptions_user ON subscriptions(user_did)`,
212- `CREATE INDEX IF NOT EXISTS idx_subscriptions_uri ON subscriptions(uri)`,
213- `CREATE INDEX IF NOT EXISTS idx_likes_author_feed ON likes(author_did, feed_url, created_at)`,
214- `CREATE INDEX IF NOT EXISTS idx_articles_feed ON articles(feed_url)`,
215- `CREATE INDEX IF NOT EXISTS idx_articles_published ON articles(published DESC)`,
216- `CREATE INDEX IF NOT EXISTS idx_articles_url ON articles(url)`,
217- `CREATE INDEX IF NOT EXISTS idx_read_state_unread ON read_state(user_did, is_read) WHERE is_read = 0`,
218- `CREATE INDEX IF NOT EXISTS idx_annotations_article ON annotations(article_url)`,
219- `CREATE INDEX IF NOT EXISTS idx_annotations_author ON annotations(author_did)`,
220- `CREATE INDEX IF NOT EXISTS idx_annotations_created_at ON annotations(created_at DESC)`,
221- `CREATE INDEX IF NOT EXISTS idx_likes_article ON likes(feed_url, article_url)`,
222- `CREATE INDEX IF NOT EXISTS idx_likes_author ON likes(author_did)`,
223- `CREATE INDEX IF NOT EXISTS idx_likes_created_at ON likes(created_at DESC)`,
224- `CREATE INDEX IF NOT EXISTS idx_follows_user ON follows(user_did)`,
225- `CREATE INDEX IF NOT EXISTS idx_follows_target ON follows(target_did)`,
226- `CREATE INDEX IF NOT EXISTS idx_follows_uri ON follows(uri)`,
227- `CREATE INDEX IF NOT EXISTS idx_user_similarity_b ON user_similarity(user_b)`,
228- `CREATE INDEX IF NOT EXISTS idx_user_similarity_a ON user_similarity(user_a)`,
229-
230 `CREATE TABLE IF NOT EXISTS dismissed_recommendations (212 `CREATE TABLE IF NOT EXISTS dismissed_recommendations (
231- user_did TEXT NOT NULL REFERENCES users(did),213+ user_did TEXT NOT NULL,
232 target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),214 target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
233 target_id TEXT NOT NULL,215 target_id TEXT NOT NULL,
234 reason TEXT,216 reason TEXT,
235 dismissed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,217 dismissed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
236 PRIMARY KEY (user_did, target_type, target_id)218 PRIMARY KEY (user_did, target_type, target_id)
237 )`,219 )`,
238-
239 `CREATE TABLE IF NOT EXISTS recommendation_impressions (220 `CREATE TABLE IF NOT EXISTS recommendation_impressions (
240- user_did TEXT NOT NULL REFERENCES users(did),221+ user_did TEXT NOT NULL,
241 target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),222 target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
242 target_id TEXT NOT NULL,223 target_id TEXT NOT NULL,
243 first_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,224 first_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
@@ -246,16 +227,14 @@ var schema = []string{
246 acted BOOLEAN NOT NULL DEFAULT 0,227 acted BOOLEAN NOT NULL DEFAULT 0,
247 PRIMARY KEY (user_did, target_type, target_id)228 PRIMARY KEY (user_did, target_type, target_id)
248 )`,229 )`,
249-
250 `CREATE TABLE IF NOT EXISTS follow_distances (230 `CREATE TABLE IF NOT EXISTS follow_distances (
251 user_a TEXT NOT NULL,231 user_a TEXT NOT NULL,
252 user_b TEXT NOT NULL,232 user_b TEXT NOT NULL,
253 distance INTEGER NOT NULL CHECK(distance IN (1, 2)),233 distance INTEGER NOT NULL CHECK(distance IN (1, 2)),
254 PRIMARY KEY (user_a, user_b)234 PRIMARY KEY (user_a, user_b)
255 )`,235 )`,
256-
257 `CREATE TABLE IF NOT EXISTS user_signal_weights (236 `CREATE TABLE IF NOT EXISTS user_signal_weights (
258- user_did TEXT PRIMARY KEY REFERENCES users(did),237+ user_did TEXT PRIMARY KEY,
259 w_sub REAL NOT NULL DEFAULT 1.0,238 w_sub REAL NOT NULL DEFAULT 1.0,
260 w_like REAL NOT NULL DEFAULT 0.5,239 w_like REAL NOT NULL DEFAULT 0.5,
261 w_tag REAL NOT NULL DEFAULT 0.3,240 w_tag REAL NOT NULL DEFAULT 0.3,
@@ -264,15 +243,33 @@ var schema = []string{
264 w_category REAL NOT NULL DEFAULT 0.4,243 w_category REAL NOT NULL DEFAULT 0.4,
265 updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP244 updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
266 )`,245 )`,
267-
268 `CREATE TABLE IF NOT EXISTS user_signal_profiles (246 `CREATE TABLE IF NOT EXISTS user_signal_profiles (
269- user_did TEXT PRIMARY KEY REFERENCES users(did),247+ user_did TEXT PRIMARY KEY,
270 total_likes INTEGER NOT NULL DEFAULT 0,248 total_likes INTEGER NOT NULL DEFAULT 0,
271 total_tags INTEGER NOT NULL DEFAULT 0,249 total_tags INTEGER NOT NULL DEFAULT 0,
272 top_categories TEXT,250 top_categories TEXT,
273 updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP251 updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
274 )`,252 )`,
275-253+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_feed ON subscriptions(feed_url)`,
254+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_feed_user ON subscriptions(feed_url, user_did)`,
255+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_user ON subscriptions(user_did)`,
256+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_uri ON subscriptions(uri)`,
257+ `CREATE INDEX IF NOT EXISTS idx_likes_author_feed ON likes(author_did, feed_url, created_at)`,
258+ `CREATE INDEX IF NOT EXISTS idx_articles_feed ON articles(feed_url)`,
259+ `CREATE INDEX IF NOT EXISTS idx_articles_published ON articles(published DESC)`,
260+ `CREATE INDEX IF NOT EXISTS idx_articles_url ON articles(url)`,
261+ `CREATE INDEX IF NOT EXISTS idx_read_state_unread ON read_state(user_did, is_read) WHERE is_read = 0`,
262+ `CREATE INDEX IF NOT EXISTS idx_annotations_article ON annotations(article_url)`,
263+ `CREATE INDEX IF NOT EXISTS idx_annotations_author ON annotations(author_did)`,
264+ `CREATE INDEX IF NOT EXISTS idx_annotations_created_at ON annotations(created_at DESC)`,
265+ `CREATE INDEX IF NOT EXISTS idx_likes_article ON likes(feed_url, article_url)`,
266+ `CREATE INDEX IF NOT EXISTS idx_likes_author ON likes(author_did)`,
267+ `CREATE INDEX IF NOT EXISTS idx_likes_created_at ON likes(created_at DESC)`,
268+ `CREATE INDEX IF NOT EXISTS idx_follows_user ON follows(user_did)`,
269+ `CREATE INDEX IF NOT EXISTS idx_follows_target ON follows(target_did)`,
270+ `CREATE INDEX IF NOT EXISTS idx_follows_uri ON follows(uri)`,
271+ `CREATE INDEX IF NOT EXISTS idx_user_similarity_b ON user_similarity(user_b)`,
272+ `CREATE INDEX IF NOT EXISTS idx_user_similarity_a ON user_similarity(user_a)`,
276 `CREATE INDEX IF NOT EXISTS idx_dismissed_user_type ON dismissed_recommendations(user_did, target_type)`,273 `CREATE INDEX IF NOT EXISTS idx_dismissed_user_type ON dismissed_recommendations(user_did, target_type)`,
277 `CREATE INDEX IF NOT EXISTS idx_impressions_user_unacted ON recommendation_impressions(user_did, acted, shown_count)`,274 `CREATE INDEX IF NOT EXISTS idx_impressions_user_unacted ON recommendation_impressions(user_did, acted, shown_count)`,
278 `CREATE INDEX IF NOT EXISTS idx_impressions_last_shown ON recommendation_impressions(last_shown_at)`,275 `CREATE INDEX IF NOT EXISTS idx_impressions_last_shown ON recommendation_impressions(last_shown_at)`,
added internal/db/multi.go +367 -0
new file mode 100644
@@ -0,0 +1,367 @@
1+package db
2+
3+import (
4+ "database/sql"
5+ "fmt"
6+ "math"
7+ "sync/atomic"
8+ "time"
9+
10+ "github.com/mattn/go-sqlite3"
11+)
12+
13+type Databases struct {
14+ Users *DB
15+ Articles *DB
16+ Recs *DB
17+}
18+
19+var multiDriverSeq int64
20+
21+func OpenAll(basePath string) (*Databases, error) {
22+ articlesPath := basePath + "_articles"
23+ recsPath := basePath + "_recs"
24+
25+ seq := atomic.AddInt64(&multiDriverSeq, 1)
26+ driverName := fmt.Sprintf("sqlite3_glean_multi_%d", seq)
27+
28+ sql.Register(driverName, &sqlite3.SQLiteDriver{
29+ ConnectHook: func(conn *sqlite3.SQLiteConn) error {
30+ if err := conn.RegisterFunc("exp", func(x float64) float64 { return math.Exp(x) }, true); err != nil {
31+ return err
32+ }
33+ if err := conn.RegisterFunc("log", func(x float64) float64 { return math.Log(x) }, true); err != nil {
34+ return err
35+ }
36+ for _, p := range []string{
37+ `PRAGMA wal_autocheckpoint = 1000`,
38+ `PRAGMA temp_store = MEMORY`,
39+ `PRAGMA mmap_size = 268435456`,
40+ } {
41+ if _, err := conn.Exec(p, nil); err != nil {
42+ return err
43+ }
44+ }
45+ if _, err := conn.Exec(fmt.Sprintf("ATTACH DATABASE '%s' AS articles", articlesPath), nil); err != nil {
46+ return err
47+ }
48+ if _, err := conn.Exec(fmt.Sprintf("ATTACH DATABASE '%s' AS recs", recsPath), nil); err != nil {
49+ return err
50+ }
51+ return nil
52+ },
53+ })
54+
55+ usersDB, err := sql.Open(driverName, basePath+"_users?cache=shared&"+DSN)
56+ if err != nil {
57+ return nil, err
58+ }
59+ usersDB.SetMaxOpenConns(10)
60+ usersDB.SetMaxIdleConns(5)
61+ usersDB.SetConnMaxLifetime(30 * time.Minute)
62+ users := &DB{usersDB}
63+
64+ articles, err := Open(articlesPath)
65+ if err != nil {
66+ users.Close()
67+ return nil, err
68+ }
69+
70+ recs, err := Open(recsPath)
71+ if err != nil {
72+ users.Close()
73+ articles.Close()
74+ return nil, err
75+ }
76+
77+ if err := initUsersSchema(users); err != nil {
78+ users.Close()
79+ articles.Close()
80+ recs.Close()
81+ return nil, err
82+ }
83+
84+ if err := initArticlesSchema(articles); err != nil {
85+ users.Close()
86+ articles.Close()
87+ recs.Close()
88+ return nil, err
89+ }
90+
91+ if err := initRecsSchema(recs); err != nil {
92+ users.Close()
93+ articles.Close()
94+ recs.Close()
95+ return nil, err
96+ }
97+
98+ return &Databases{
99+ Users: users,
100+ Articles: articles,
101+ Recs: recs,
102+ }, nil
103+}
104+
105+func (d *Databases) Close() error {
106+ if d.Users != nil {
107+ _ = d.Users.Close()
108+ }
109+ if d.Articles != nil {
110+ _ = d.Articles.Close()
111+ }
112+ if d.Recs != nil {
113+ _ = d.Recs.Close()
114+ }
115+ return nil
116+}
117+
118+func initUsersSchema(db *DB) error {
119+ for _, s := range usersSchema {
120+ if _, err := db.Exec(s); err != nil {
121+ return err
122+ }
123+ }
124+ return nil
125+}
126+
127+func initArticlesSchema(db *DB) error {
128+ for _, s := range articlesSchema {
129+ if _, err := db.Exec(s); err != nil {
130+ return err
131+ }
132+ }
133+ return nil
134+}
135+
136+func initRecsSchema(db *DB) error {
137+ for _, s := range recsSchema {
138+ if _, err := db.Exec(s); err != nil {
139+ return err
140+ }
141+ }
142+ return nil
143+}
144+
145+var usersSchema = []string{
146+ `CREATE TABLE IF NOT EXISTS users (
147+ did TEXT PRIMARY KEY,
148+ handle TEXT NOT NULL,
149+ display_name TEXT,
150+ avatar_url TEXT,
151+ indexed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
152+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
153+ )`,
154+
155+ `CREATE TABLE IF NOT EXISTS follows (
156+ user_did TEXT NOT NULL,
157+ target_did TEXT NOT NULL,
158+ uri TEXT,
159+ cid TEXT,
160+ followed_at DATETIME,
161+ PRIMARY KEY (user_did, target_did)
162+ )`,
163+
164+ `CREATE TABLE IF NOT EXISTS oauth_auth_requests (
165+ state TEXT PRIMARY KEY,
166+ data TEXT NOT NULL
167+ )`,
168+
169+ `CREATE TABLE IF NOT EXISTS oauth_sessions (
170+ account_did TEXT NOT NULL,
171+ session_id TEXT NOT NULL,
172+ data TEXT NOT NULL,
173+ PRIMARY KEY (account_did, session_id)
174+ )`,
175+
176+ `CREATE INDEX IF NOT EXISTS idx_follows_user ON follows(user_did)`,
177+ `CREATE INDEX IF NOT EXISTS idx_follows_target ON follows(target_did)`,
178+ `CREATE INDEX IF NOT EXISTS idx_follows_uri ON follows(uri)`,
179+ `CREATE INDEX IF NOT EXISTS idx_follows_followed_at ON follows(followed_at)`,
180+ `CREATE INDEX IF NOT EXISTS idx_users_handle ON users(handle)`,
181+}
182+
183+var articlesSchema = []string{
184+ `CREATE TABLE IF NOT EXISTS feeds (
185+ feed_url TEXT PRIMARY KEY,
186+ title TEXT,
187+ site_url TEXT,
188+ description TEXT,
189+ feed_type TEXT CHECK(feed_type IN ('rss', 'atom', 'json')),
190+ last_fetched_at DATETIME,
191+ last_error TEXT,
192+ subscriber_count INTEGER NOT NULL DEFAULT 0,
193+ etag TEXT,
194+ last_modified TEXT,
195+ fetch_interval_minutes INTEGER NOT NULL DEFAULT 30,
196+ next_fetch_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
197+ consecutive_empty_fetches INTEGER NOT NULL DEFAULT 0,
198+ error_count INTEGER NOT NULL DEFAULT 0,
199+ favicon_url TEXT
200+ )`,
201+
202+ `CREATE TABLE IF NOT EXISTS subscriptions (
203+ id INTEGER PRIMARY KEY AUTOINCREMENT,
204+ user_did TEXT NOT NULL,
205+ feed_url TEXT NOT NULL,
206+ title TEXT,
207+ category TEXT,
208+ added_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
209+ uri TEXT,
210+ cid TEXT,
211+ UNIQUE(user_did, feed_url)
212+ )`,
213+
214+ `CREATE TABLE IF NOT EXISTS articles (
215+ id INTEGER PRIMARY KEY AUTOINCREMENT,
216+ feed_url TEXT NOT NULL,
217+ guid TEXT NOT NULL,
218+ title TEXT NOT NULL DEFAULT '',
219+ url TEXT,
220+ author TEXT,
221+ summary TEXT,
222+ content TEXT,
223+ full_content TEXT,
224+ published DATETIME,
225+ updated DATETIME,
226+ fetched_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
227+ UNIQUE(feed_url, guid)
228+ )`,
229+
230+ `CREATE TABLE IF NOT EXISTS read_state (
231+ user_did TEXT NOT NULL,
232+ article_id INTEGER NOT NULL,
233+ is_read BOOLEAN NOT NULL DEFAULT 0,
234+ read_at DATETIME,
235+ PRIMARY KEY (user_did, article_id)
236+ )`,
237+
238+ `CREATE TABLE IF NOT EXISTS annotations (
239+ id INTEGER PRIMARY KEY AUTOINCREMENT,
240+ uri TEXT NOT NULL UNIQUE,
241+ author_did TEXT NOT NULL,
242+ feed_url TEXT NOT NULL,
243+ article_url TEXT NOT NULL,
244+ quote TEXT,
245+ note TEXT,
246+ tags TEXT,
247+ rating INTEGER,
248+ created_at DATETIME NOT NULL,
249+ cid TEXT
250+ )`,
251+
252+ `CREATE TABLE IF NOT EXISTS likes (
253+ id INTEGER PRIMARY KEY AUTOINCREMENT,
254+ uri TEXT NOT NULL UNIQUE,
255+ author_did TEXT NOT NULL,
256+ feed_url TEXT NOT NULL,
257+ article_url TEXT NOT NULL,
258+ created_at DATETIME NOT NULL,
259+ cid TEXT,
260+ UNIQUE(author_did, feed_url, article_url)
261+ )`,
262+
263+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_feed ON subscriptions(feed_url)`,
264+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_feed_user ON subscriptions(feed_url, user_did)`,
265+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_user ON subscriptions(user_did)`,
266+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_uri ON subscriptions(uri)`,
267+ `CREATE INDEX IF NOT EXISTS idx_likes_author_feed ON likes(author_did, feed_url, created_at)`,
268+ `CREATE INDEX IF NOT EXISTS idx_articles_feed ON articles(feed_url)`,
269+ `CREATE INDEX IF NOT EXISTS idx_articles_published ON articles(published DESC)`,
270+ `CREATE INDEX IF NOT EXISTS idx_articles_url ON articles(url)`,
271+ `CREATE INDEX IF NOT EXISTS idx_read_state_unread ON read_state(user_did, is_read) WHERE is_read = 0`,
272+ `CREATE INDEX IF NOT EXISTS idx_annotations_article ON annotations(article_url)`,
273+ `CREATE INDEX IF NOT EXISTS idx_annotations_author ON annotations(author_did)`,
274+ `CREATE INDEX IF NOT EXISTS idx_annotations_created_at ON annotations(created_at DESC)`,
275+ `CREATE INDEX IF NOT EXISTS idx_likes_article ON likes(feed_url, article_url)`,
276+ `CREATE INDEX IF NOT EXISTS idx_likes_author ON likes(author_did)`,
277+ `CREATE INDEX IF NOT EXISTS idx_likes_created_at ON likes(created_at DESC)`,
278+
279+ `CREATE VIRTUAL TABLE IF NOT EXISTS articles_fts USING fts5(title, summary, content, author, content=articles, content_rowid=id)`,
280+ `CREATE TRIGGER IF NOT EXISTS articles_ai AFTER INSERT ON articles BEGIN
281+ INSERT INTO articles_fts(rowid, title, summary, content, author) VALUES (new.id, new.title, new.summary, new.content, new.author);
282+ END`,
283+ `CREATE TRIGGER IF NOT EXISTS articles_ad AFTER DELETE ON articles BEGIN
284+ INSERT INTO articles_fts(articles_fts, rowid, title, summary, content, author) VALUES('delete', old.id, old.title, old.summary, old.content, old.author);
285+ END`,
286+ `CREATE TRIGGER IF NOT EXISTS articles_au AFTER UPDATE ON articles BEGIN
287+ INSERT INTO articles_fts(articles_fts, rowid, title, summary, content, author) VALUES('delete', old.id, old.title, old.summary, old.content, old.author);
288+ INSERT INTO articles_fts(rowid, title, summary, content, author) VALUES (new.id, new.title, new.summary, new.content, new.author);
289+ END`,
290+}
291+
292+var recsSchema = []string{
293+ `CREATE TABLE IF NOT EXISTS feed_similarity (
294+ feed_a TEXT NOT NULL,
295+ feed_b TEXT NOT NULL,
296+ jaccard REAL NOT NULL,
297+ computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
298+ PRIMARY KEY (feed_a, feed_b),
299+ CHECK(feed_a < feed_b)
300+ )`,
301+
302+ `CREATE TABLE IF NOT EXISTS user_similarity (
303+ user_a TEXT NOT NULL,
304+ user_b TEXT NOT NULL,
305+ jaccard REAL NOT NULL,
306+ common_feeds INTEGER NOT NULL,
307+ common_likes INTEGER NOT NULL DEFAULT 0,
308+ common_tags INTEGER NOT NULL DEFAULT 0,
309+ computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
310+ PRIMARY KEY (user_a, user_b),
311+ CHECK(user_a < user_b)
312+ )`,
313+
314+ `CREATE TABLE IF NOT EXISTS dismissed_recommendations (
315+ user_did TEXT NOT NULL,
316+ target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
317+ target_id TEXT NOT NULL,
318+ reason TEXT,
319+ dismissed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
320+ PRIMARY KEY (user_did, target_type, target_id)
321+ )`,
322+
323+ `CREATE TABLE IF NOT EXISTS recommendation_impressions (
324+ user_did TEXT NOT NULL,
325+ target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
326+ target_id TEXT NOT NULL,
327+ first_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
328+ last_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
329+ shown_count INTEGER NOT NULL DEFAULT 1,
330+ acted BOOLEAN NOT NULL DEFAULT 0,
331+ PRIMARY KEY (user_did, target_type, target_id)
332+ )`,
333+
334+ `CREATE TABLE IF NOT EXISTS follow_distances (
335+ user_a TEXT NOT NULL,
336+ user_b TEXT NOT NULL,
337+ distance INTEGER NOT NULL CHECK(distance IN (1, 2)),
338+ PRIMARY KEY (user_a, user_b)
339+ )`,
340+
341+ `CREATE TABLE IF NOT EXISTS user_signal_weights (
342+ user_did TEXT PRIMARY KEY,
343+ w_sub REAL NOT NULL DEFAULT 1.0,
344+ w_like REAL NOT NULL DEFAULT 0.5,
345+ w_tag REAL NOT NULL DEFAULT 0.3,
346+ w_social REAL NOT NULL DEFAULT 0.7,
347+ w_pop REAL NOT NULL DEFAULT 0.2,
348+ w_category REAL NOT NULL DEFAULT 0.4,
349+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
350+ )`,
351+
352+ `CREATE TABLE IF NOT EXISTS user_signal_profiles (
353+ user_did TEXT PRIMARY KEY,
354+ total_likes INTEGER NOT NULL DEFAULT 0,
355+ total_tags INTEGER NOT NULL DEFAULT 0,
356+ top_categories TEXT,
357+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
358+ )`,
359+
360+ `CREATE INDEX IF NOT EXISTS idx_dismissed_user_type ON dismissed_recommendations(user_did, target_type)`,
361+ `CREATE INDEX IF NOT EXISTS idx_impressions_user_unacted ON recommendation_impressions(user_did, acted, shown_count)`,
362+ `CREATE INDEX IF NOT EXISTS idx_impressions_last_shown ON recommendation_impressions(last_shown_at)`,
363+ `CREATE INDEX IF NOT EXISTS idx_follow_distances_b ON follow_distances(user_b)`,
364+ `CREATE INDEX IF NOT EXISTS idx_follow_distances_a_dist ON follow_distances(user_a, distance)`,
365+ `CREATE INDEX IF NOT EXISTS idx_user_similarity_b ON user_similarity(user_b)`,
366+ `CREATE INDEX IF NOT EXISTS idx_user_similarity_a ON user_similarity(user_a)`,
367+}
new file mode 100644
@@ -0,0 +1,367 @@
1+package db
2+
3+import (
4+ "database/sql"
5+ "fmt"
6+ "math"
7+ "sync/atomic"
8+ "time"
9+
10+ "github.com/mattn/go-sqlite3"
11+)
12+
13+type Databases struct {
14+ Users *DB
15+ Articles *DB
16+ Recs *DB
17+}
18+
19+var multiDriverSeq int64
20+
21+func OpenAll(basePath string) (*Databases, error) {
22+ articlesPath := basePath + "_articles"
23+ recsPath := basePath + "_recs"
24+
25+ seq := atomic.AddInt64(&multiDriverSeq, 1)
26+ driverName := fmt.Sprintf("sqlite3_glean_multi_%d", seq)
27+
28+ sql.Register(driverName, &sqlite3.SQLiteDriver{
29+ ConnectHook: func(conn *sqlite3.SQLiteConn) error {
30+ if err := conn.RegisterFunc("exp", func(x float64) float64 { return math.Exp(x) }, true); err != nil {
31+ return err
32+ }
33+ if err := conn.RegisterFunc("log", func(x float64) float64 { return math.Log(x) }, true); err != nil {
34+ return err
35+ }
36+ for _, p := range []string{
37+ `PRAGMA wal_autocheckpoint = 1000`,
38+ `PRAGMA temp_store = MEMORY`,
39+ `PRAGMA mmap_size = 268435456`,
40+ } {
41+ if _, err := conn.Exec(p, nil); err != nil {
42+ return err
43+ }
44+ }
45+ if _, err := conn.Exec(fmt.Sprintf("ATTACH DATABASE '%s' AS articles", articlesPath), nil); err != nil {
46+ return err
47+ }
48+ if _, err := conn.Exec(fmt.Sprintf("ATTACH DATABASE '%s' AS recs", recsPath), nil); err != nil {
49+ return err
50+ }
51+ return nil
52+ },
53+ })
54+
55+ usersDB, err := sql.Open(driverName, basePath+"_users?cache=shared&"+DSN)
56+ if err != nil {
57+ return nil, err
58+ }
59+ usersDB.SetMaxOpenConns(10)
60+ usersDB.SetMaxIdleConns(5)
61+ usersDB.SetConnMaxLifetime(30 * time.Minute)
62+ users := &DB{usersDB}
63+
64+ articles, err := Open(articlesPath)
65+ if err != nil {
66+ users.Close()
67+ return nil, err
68+ }
69+
70+ recs, err := Open(recsPath)
71+ if err != nil {
72+ users.Close()
73+ articles.Close()
74+ return nil, err
75+ }
76+
77+ if err := initUsersSchema(users); err != nil {
78+ users.Close()
79+ articles.Close()
80+ recs.Close()
81+ return nil, err
82+ }
83+
84+ if err := initArticlesSchema(articles); err != nil {
85+ users.Close()
86+ articles.Close()
87+ recs.Close()
88+ return nil, err
89+ }
90+
91+ if err := initRecsSchema(recs); err != nil {
92+ users.Close()
93+ articles.Close()
94+ recs.Close()
95+ return nil, err
96+ }
97+
98+ return &Databases{
99+ Users: users,
100+ Articles: articles,
101+ Recs: recs,
102+ }, nil
103+}
104+
105+func (d *Databases) Close() error {
106+ if d.Users != nil {
107+ _ = d.Users.Close()
108+ }
109+ if d.Articles != nil {
110+ _ = d.Articles.Close()
111+ }
112+ if d.Recs != nil {
113+ _ = d.Recs.Close()
114+ }
115+ return nil
116+}
117+
118+func initUsersSchema(db *DB) error {
119+ for _, s := range usersSchema {
120+ if _, err := db.Exec(s); err != nil {
121+ return err
122+ }
123+ }
124+ return nil
125+}
126+
127+func initArticlesSchema(db *DB) error {
128+ for _, s := range articlesSchema {
129+ if _, err := db.Exec(s); err != nil {
130+ return err
131+ }
132+ }
133+ return nil
134+}
135+
136+func initRecsSchema(db *DB) error {
137+ for _, s := range recsSchema {
138+ if _, err := db.Exec(s); err != nil {
139+ return err
140+ }
141+ }
142+ return nil
143+}
144+
145+var usersSchema = []string{
146+ `CREATE TABLE IF NOT EXISTS users (
147+ did TEXT PRIMARY KEY,
148+ handle TEXT NOT NULL,
149+ display_name TEXT,
150+ avatar_url TEXT,
151+ indexed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
152+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
153+ )`,
154+
155+ `CREATE TABLE IF NOT EXISTS follows (
156+ user_did TEXT NOT NULL,
157+ target_did TEXT NOT NULL,
158+ uri TEXT,
159+ cid TEXT,
160+ followed_at DATETIME,
161+ PRIMARY KEY (user_did, target_did)
162+ )`,
163+
164+ `CREATE TABLE IF NOT EXISTS oauth_auth_requests (
165+ state TEXT PRIMARY KEY,
166+ data TEXT NOT NULL
167+ )`,
168+
169+ `CREATE TABLE IF NOT EXISTS oauth_sessions (
170+ account_did TEXT NOT NULL,
171+ session_id TEXT NOT NULL,
172+ data TEXT NOT NULL,
173+ PRIMARY KEY (account_did, session_id)
174+ )`,
175+
176+ `CREATE INDEX IF NOT EXISTS idx_follows_user ON follows(user_did)`,
177+ `CREATE INDEX IF NOT EXISTS idx_follows_target ON follows(target_did)`,
178+ `CREATE INDEX IF NOT EXISTS idx_follows_uri ON follows(uri)`,
179+ `CREATE INDEX IF NOT EXISTS idx_follows_followed_at ON follows(followed_at)`,
180+ `CREATE INDEX IF NOT EXISTS idx_users_handle ON users(handle)`,
181+}
182+
183+var articlesSchema = []string{
184+ `CREATE TABLE IF NOT EXISTS feeds (
185+ feed_url TEXT PRIMARY KEY,
186+ title TEXT,
187+ site_url TEXT,
188+ description TEXT,
189+ feed_type TEXT CHECK(feed_type IN ('rss', 'atom', 'json')),
190+ last_fetched_at DATETIME,
191+ last_error TEXT,
192+ subscriber_count INTEGER NOT NULL DEFAULT 0,
193+ etag TEXT,
194+ last_modified TEXT,
195+ fetch_interval_minutes INTEGER NOT NULL DEFAULT 30,
196+ next_fetch_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
197+ consecutive_empty_fetches INTEGER NOT NULL DEFAULT 0,
198+ error_count INTEGER NOT NULL DEFAULT 0,
199+ favicon_url TEXT
200+ )`,
201+
202+ `CREATE TABLE IF NOT EXISTS subscriptions (
203+ id INTEGER PRIMARY KEY AUTOINCREMENT,
204+ user_did TEXT NOT NULL,
205+ feed_url TEXT NOT NULL,
206+ title TEXT,
207+ category TEXT,
208+ added_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
209+ uri TEXT,
210+ cid TEXT,
211+ UNIQUE(user_did, feed_url)
212+ )`,
213+
214+ `CREATE TABLE IF NOT EXISTS articles (
215+ id INTEGER PRIMARY KEY AUTOINCREMENT,
216+ feed_url TEXT NOT NULL,
217+ guid TEXT NOT NULL,
218+ title TEXT NOT NULL DEFAULT '',
219+ url TEXT,
220+ author TEXT,
221+ summary TEXT,
222+ content TEXT,
223+ full_content TEXT,
224+ published DATETIME,
225+ updated DATETIME,
226+ fetched_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
227+ UNIQUE(feed_url, guid)
228+ )`,
229+
230+ `CREATE TABLE IF NOT EXISTS read_state (
231+ user_did TEXT NOT NULL,
232+ article_id INTEGER NOT NULL,
233+ is_read BOOLEAN NOT NULL DEFAULT 0,
234+ read_at DATETIME,
235+ PRIMARY KEY (user_did, article_id)
236+ )`,
237+
238+ `CREATE TABLE IF NOT EXISTS annotations (
239+ id INTEGER PRIMARY KEY AUTOINCREMENT,
240+ uri TEXT NOT NULL UNIQUE,
241+ author_did TEXT NOT NULL,
242+ feed_url TEXT NOT NULL,
243+ article_url TEXT NOT NULL,
244+ quote TEXT,
245+ note TEXT,
246+ tags TEXT,
247+ rating INTEGER,
248+ created_at DATETIME NOT NULL,
249+ cid TEXT
250+ )`,
251+
252+ `CREATE TABLE IF NOT EXISTS likes (
253+ id INTEGER PRIMARY KEY AUTOINCREMENT,
254+ uri TEXT NOT NULL UNIQUE,
255+ author_did TEXT NOT NULL,
256+ feed_url TEXT NOT NULL,
257+ article_url TEXT NOT NULL,
258+ created_at DATETIME NOT NULL,
259+ cid TEXT,
260+ UNIQUE(author_did, feed_url, article_url)
261+ )`,
262+
263+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_feed ON subscriptions(feed_url)`,
264+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_feed_user ON subscriptions(feed_url, user_did)`,
265+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_user ON subscriptions(user_did)`,
266+ `CREATE INDEX IF NOT EXISTS idx_subscriptions_uri ON subscriptions(uri)`,
267+ `CREATE INDEX IF NOT EXISTS idx_likes_author_feed ON likes(author_did, feed_url, created_at)`,
268+ `CREATE INDEX IF NOT EXISTS idx_articles_feed ON articles(feed_url)`,
269+ `CREATE INDEX IF NOT EXISTS idx_articles_published ON articles(published DESC)`,
270+ `CREATE INDEX IF NOT EXISTS idx_articles_url ON articles(url)`,
271+ `CREATE INDEX IF NOT EXISTS idx_read_state_unread ON read_state(user_did, is_read) WHERE is_read = 0`,
272+ `CREATE INDEX IF NOT EXISTS idx_annotations_article ON annotations(article_url)`,
273+ `CREATE INDEX IF NOT EXISTS idx_annotations_author ON annotations(author_did)`,
274+ `CREATE INDEX IF NOT EXISTS idx_annotations_created_at ON annotations(created_at DESC)`,
275+ `CREATE INDEX IF NOT EXISTS idx_likes_article ON likes(feed_url, article_url)`,
276+ `CREATE INDEX IF NOT EXISTS idx_likes_author ON likes(author_did)`,
277+ `CREATE INDEX IF NOT EXISTS idx_likes_created_at ON likes(created_at DESC)`,
278+
279+ `CREATE VIRTUAL TABLE IF NOT EXISTS articles_fts USING fts5(title, summary, content, author, content=articles, content_rowid=id)`,
280+ `CREATE TRIGGER IF NOT EXISTS articles_ai AFTER INSERT ON articles BEGIN
281+ INSERT INTO articles_fts(rowid, title, summary, content, author) VALUES (new.id, new.title, new.summary, new.content, new.author);
282+ END`,
283+ `CREATE TRIGGER IF NOT EXISTS articles_ad AFTER DELETE ON articles BEGIN
284+ INSERT INTO articles_fts(articles_fts, rowid, title, summary, content, author) VALUES('delete', old.id, old.title, old.summary, old.content, old.author);
285+ END`,
286+ `CREATE TRIGGER IF NOT EXISTS articles_au AFTER UPDATE ON articles BEGIN
287+ INSERT INTO articles_fts(articles_fts, rowid, title, summary, content, author) VALUES('delete', old.id, old.title, old.summary, old.content, old.author);
288+ INSERT INTO articles_fts(rowid, title, summary, content, author) VALUES (new.id, new.title, new.summary, new.content, new.author);
289+ END`,
290+}
291+
292+var recsSchema = []string{
293+ `CREATE TABLE IF NOT EXISTS feed_similarity (
294+ feed_a TEXT NOT NULL,
295+ feed_b TEXT NOT NULL,
296+ jaccard REAL NOT NULL,
297+ computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
298+ PRIMARY KEY (feed_a, feed_b),
299+ CHECK(feed_a < feed_b)
300+ )`,
301+
302+ `CREATE TABLE IF NOT EXISTS user_similarity (
303+ user_a TEXT NOT NULL,
304+ user_b TEXT NOT NULL,
305+ jaccard REAL NOT NULL,
306+ common_feeds INTEGER NOT NULL,
307+ common_likes INTEGER NOT NULL DEFAULT 0,
308+ common_tags INTEGER NOT NULL DEFAULT 0,
309+ computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
310+ PRIMARY KEY (user_a, user_b),
311+ CHECK(user_a < user_b)
312+ )`,
313+
314+ `CREATE TABLE IF NOT EXISTS dismissed_recommendations (
315+ user_did TEXT NOT NULL,
316+ target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
317+ target_id TEXT NOT NULL,
318+ reason TEXT,
319+ dismissed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
320+ PRIMARY KEY (user_did, target_type, target_id)
321+ )`,
322+
323+ `CREATE TABLE IF NOT EXISTS recommendation_impressions (
324+ user_did TEXT NOT NULL,
325+ target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
326+ target_id TEXT NOT NULL,
327+ first_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
328+ last_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
329+ shown_count INTEGER NOT NULL DEFAULT 1,
330+ acted BOOLEAN NOT NULL DEFAULT 0,
331+ PRIMARY KEY (user_did, target_type, target_id)
332+ )`,
333+
334+ `CREATE TABLE IF NOT EXISTS follow_distances (
335+ user_a TEXT NOT NULL,
336+ user_b TEXT NOT NULL,
337+ distance INTEGER NOT NULL CHECK(distance IN (1, 2)),
338+ PRIMARY KEY (user_a, user_b)
339+ )`,
340+
341+ `CREATE TABLE IF NOT EXISTS user_signal_weights (
342+ user_did TEXT PRIMARY KEY,
343+ w_sub REAL NOT NULL DEFAULT 1.0,
344+ w_like REAL NOT NULL DEFAULT 0.5,
345+ w_tag REAL NOT NULL DEFAULT 0.3,
346+ w_social REAL NOT NULL DEFAULT 0.7,
347+ w_pop REAL NOT NULL DEFAULT 0.2,
348+ w_category REAL NOT NULL DEFAULT 0.4,
349+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
350+ )`,
351+
352+ `CREATE TABLE IF NOT EXISTS user_signal_profiles (
353+ user_did TEXT PRIMARY KEY,
354+ total_likes INTEGER NOT NULL DEFAULT 0,
355+ total_tags INTEGER NOT NULL DEFAULT 0,
356+ top_categories TEXT,
357+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
358+ )`,
359+
360+ `CREATE INDEX IF NOT EXISTS idx_dismissed_user_type ON dismissed_recommendations(user_did, target_type)`,
361+ `CREATE INDEX IF NOT EXISTS idx_impressions_user_unacted ON recommendation_impressions(user_did, acted, shown_count)`,
362+ `CREATE INDEX IF NOT EXISTS idx_impressions_last_shown ON recommendation_impressions(last_shown_at)`,
363+ `CREATE INDEX IF NOT EXISTS idx_follow_distances_b ON follow_distances(user_b)`,
364+ `CREATE INDEX IF NOT EXISTS idx_follow_distances_a_dist ON follow_distances(user_a, distance)`,
365+ `CREATE INDEX IF NOT EXISTS idx_user_similarity_b ON user_similarity(user_b)`,
366+ `CREATE INDEX IF NOT EXISTS idx_user_similarity_a ON user_similarity(user_a)`,
367+}
modified internal/feed/fetcher.go +3 -0
@@ -101,6 +101,9 @@ func (s *Scheduler) Run(ctx context.Context) error {
101101 ticker := time.NewTicker(s.tickInterval)
102102 defer ticker.Stop()
103103
104+ // fetch all at startup
105+ s.fetchAll(ctx)
106+
104107 for {
105108 select {
106109 case <-ctx.Done():
@@ -101,6 +101,9 @@ func (s *Scheduler) Run(ctx context.Context) error {
101 ticker := time.NewTicker(s.tickInterval)101 ticker := time.NewTicker(s.tickInterval)
102 defer ticker.Stop()102 defer ticker.Stop()
103 103
104+ // fetch all at startup
105+ s.fetchAll(ctx)
106+
104 for {107 for {
105 select {108 select {
106 case <-ctx.Done():109 case <-ctx.Done():
modified internal/server/annotations_handler.go +20 -11
@@ -16,22 +16,26 @@ import (
1616
1717 func (s *Server) handleLibrary(w http.ResponseWriter, r *http.Request) {
1818 user := currentUser(r)
19+ ctx := r.Context()
1920
2021 limit := 20
2122
22- likedPageNum, _ := strconv.Atoi(r.URL.Query().Get("liked_page"))
23- if likedPageNum < 1 {
23+ likedPageNum, err := strconv.Atoi(r.URL.Query().Get("liked_page"))
24+ if err != nil || likedPageNum < 1 {
2425 likedPageNum = 1
2526 }
26- annotPageNum, _ := strconv.Atoi(r.URL.Query().Get("annot_page"))
27- if annotPageNum < 1 {
27+ annotPageNum, err := strconv.Atoi(r.URL.Query().Get("annot_page"))
28+ if err != nil || annotPageNum < 1 {
2829 annotPageNum = 1
2930 }
3031
3132 likedPage := Pagination{Page: likedPageNum, PageSize: limit}
3233 annotPage := Pagination{Page: annotPageNum, PageSize: limit}
3334
34- articles, _ := s.db.ListLikedArticles(r.Context(), user.DID, limit+1, likedPage.Offset())
35+ articles, err := s.dbs.Articles.ListLikedArticles(ctx, user.DID, limit+1, likedPage.Offset())
36+ if err != nil {
37+ s.logger.Warn("failed to list liked articles", "error", err, "did", user.DID)
38+ }
3539 likedHasMore := len(articles) > limit
3640 if likedHasMore {
3741 articles = articles[:limit]
@@ -42,7 +46,10 @@ func (s *Server) handleLibrary(w http.ResponseWriter, r *http.Request) {
4246 likedPage.NextPage = likedPage.Page + 1
4347 }
4448
45- annotations, _ := s.db.ListAnnotations(r.Context(), "", "", user.DID, limit+1, annotPage.Offset())
49+ annotations, err := s.dbs.Articles.ListAnnotations(ctx, "", "", user.DID, limit+1, annotPage.Offset())
50+ if err != nil {
51+ s.logger.Warn("failed to list annotations", "error", err, "did", user.DID)
52+ }
4653 annotHasMore := len(annotations) > limit
4754 if annotHasMore {
4855 annotations = annotations[:limit]
@@ -65,6 +72,7 @@ func (s *Server) handleLibrary(w http.ResponseWriter, r *http.Request) {
6572
6673 func (s *Server) handleCreateAnnotation(w http.ResponseWriter, r *http.Request) {
6774 user := currentUser(r)
75+ ctx := r.Context()
6876 a := &db.Annotation{
6977 AuthorDID: user.DID,
7078 FeedURL: r.FormValue("feed_url"),
@@ -95,7 +103,7 @@ func (s *Server) handleCreateAnnotation(w http.ResponseWriter, r *http.Request)
95103 Tags: tags,
96104 Rating: int(a.Rating.Int64),
97105 }
98- uri, cid, err := client.CreateRecord(r.Context(), user.DID, atproto.CollectionAnnotation, record)
106+ uri, cid, err := client.CreateRecord(ctx, user.DID, atproto.CollectionAnnotation, record)
99107 if err != nil {
100108 s.logger.Error("failed to write annotation to PDS", "error", err)
101109 http.Error(w, "failed to write annotation to PDS: "+err.Error(), http.StatusBadGateway)
@@ -107,7 +115,7 @@ func (s *Server) handleCreateAnnotation(w http.ResponseWriter, r *http.Request)
107115 a.URI = fmt.Sprintf("glean:annotation:%d", time.Now().UnixNano())
108116 }
109117
110- if err := s.db.CreateAnnotation(r.Context(), a); err != nil {
118+ if err := s.dbs.Articles.CreateAnnotation(ctx, a); err != nil {
111119 http.Error(w, err.Error(), http.StatusInternalServerError)
112120 return
113121 }
@@ -121,13 +129,14 @@ func (s *Server) handleCreateAnnotation(w http.ResponseWriter, r *http.Request)
121129
122130 func (s *Server) handleDeleteAnnotation(w http.ResponseWriter, r *http.Request) {
123131 user := currentUser(r)
132+ ctx := r.Context()
124133 id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
125134 if err != nil {
126135 http.Error(w, "invalid id", http.StatusBadRequest)
127136 return
128137 }
129138
130- annotation, err := s.db.GetAnnotation(r.Context(), id)
139+ annotation, err := s.dbs.Articles.GetAnnotation(ctx, id)
131140 if err != nil {
132141 http.Error(w, "annotation not found", http.StatusNotFound)
133142 return
@@ -142,14 +151,14 @@ func (s *Server) handleDeleteAnnotation(w http.ResponseWriter, r *http.Request)
142151 if client := s.pdsClientForUser(r); client != nil {
143152 parsed, ok := atproto.ParseRecordURI(annotation.URI)
144153 if ok {
145- if delErr := client.DeleteRecord(r.Context(), user.DID, parsed.Collection, parsed.RKey); delErr != nil {
154+ if delErr := client.DeleteRecord(ctx, user.DID, parsed.Collection, parsed.RKey); delErr != nil {
146155 s.logger.Error("failed to delete annotation from PDS", "error", delErr)
147156 }
148157 }
149158 }
150159 }
151160
152- if err := s.db.DeleteAnnotation(r.Context(), annotation.URI); err != nil {
161+ if err := s.dbs.Articles.DeleteAnnotation(ctx, annotation.URI); err != nil {
153162 http.Error(w, err.Error(), http.StatusInternalServerError)
154163 return
155164 }
@@ -16,22 +16,26 @@ import (
16 16
17 func (s *Server) handleLibrary(w http.ResponseWriter, r *http.Request) {17 func (s *Server) handleLibrary(w http.ResponseWriter, r *http.Request) {
18 user := currentUser(r)18 user := currentUser(r)
19+ ctx := r.Context()
19 20
20 limit := 2021 limit := 20
21 22
22- likedPageNum, _ := strconv.Atoi(r.URL.Query().Get("liked_page"))23+ likedPageNum, err := strconv.Atoi(r.URL.Query().Get("liked_page"))
23- if likedPageNum < 1 {24+ if err != nil || likedPageNum < 1 {
24 likedPageNum = 125 likedPageNum = 1
25 }26 }
26- annotPageNum, _ := strconv.Atoi(r.URL.Query().Get("annot_page"))27+ annotPageNum, err := strconv.Atoi(r.URL.Query().Get("annot_page"))
27- if annotPageNum < 1 {28+ if err != nil || annotPageNum < 1 {
28 annotPageNum = 129 annotPageNum = 1
29 }30 }
30 31
31 likedPage := Pagination{Page: likedPageNum, PageSize: limit}32 likedPage := Pagination{Page: likedPageNum, PageSize: limit}
32 annotPage := Pagination{Page: annotPageNum, PageSize: limit}33 annotPage := Pagination{Page: annotPageNum, PageSize: limit}
33 34
34- articles, _ := s.db.ListLikedArticles(r.Context(), user.DID, limit+1, likedPage.Offset())35+ articles, err := s.dbs.Articles.ListLikedArticles(ctx, user.DID, limit+1, likedPage.Offset())
36+ if err != nil {
37+ s.logger.Warn("failed to list liked articles", "error", err, "did", user.DID)
38+ }
35 likedHasMore := len(articles) > limit39 likedHasMore := len(articles) > limit
36 if likedHasMore {40 if likedHasMore {
37 articles = articles[:limit]41 articles = articles[:limit]
@@ -42,7 +46,10 @@ func (s *Server) handleLibrary(w http.ResponseWriter, r *http.Request) {
42 likedPage.NextPage = likedPage.Page + 146 likedPage.NextPage = likedPage.Page + 1
43 }47 }
44 48
45- annotations, _ := s.db.ListAnnotations(r.Context(), "", "", user.DID, limit+1, annotPage.Offset())49+ annotations, err := s.dbs.Articles.ListAnnotations(ctx, "", "", user.DID, limit+1, annotPage.Offset())
50+ if err != nil {
51+ s.logger.Warn("failed to list annotations", "error", err, "did", user.DID)
52+ }
46 annotHasMore := len(annotations) > limit53 annotHasMore := len(annotations) > limit
47 if annotHasMore {54 if annotHasMore {
48 annotations = annotations[:limit]55 annotations = annotations[:limit]
@@ -65,6 +72,7 @@ func (s *Server) handleLibrary(w http.ResponseWriter, r *http.Request) {
65 72
66 func (s *Server) handleCreateAnnotation(w http.ResponseWriter, r *http.Request) {73 func (s *Server) handleCreateAnnotation(w http.ResponseWriter, r *http.Request) {
67 user := currentUser(r)74 user := currentUser(r)
75+ ctx := r.Context()
68 a := &db.Annotation{76 a := &db.Annotation{
69 AuthorDID: user.DID,77 AuthorDID: user.DID,
70 FeedURL: r.FormValue("feed_url"),78 FeedURL: r.FormValue("feed_url"),
@@ -95,7 +103,7 @@ func (s *Server) handleCreateAnnotation(w http.ResponseWriter, r *http.Request)
95 Tags: tags,103 Tags: tags,
96 Rating: int(a.Rating.Int64),104 Rating: int(a.Rating.Int64),
97 }105 }
98- uri, cid, err := client.CreateRecord(r.Context(), user.DID, atproto.CollectionAnnotation, record)106+ uri, cid, err := client.CreateRecord(ctx, user.DID, atproto.CollectionAnnotation, record)
99 if err != nil {107 if err != nil {
100 s.logger.Error("failed to write annotation to PDS", "error", err)108 s.logger.Error("failed to write annotation to PDS", "error", err)
101 http.Error(w, "failed to write annotation to PDS: "+err.Error(), http.StatusBadGateway)109 http.Error(w, "failed to write annotation to PDS: "+err.Error(), http.StatusBadGateway)
@@ -107,7 +115,7 @@ func (s *Server) handleCreateAnnotation(w http.ResponseWriter, r *http.Request)
107 a.URI = fmt.Sprintf("glean:annotation:%d", time.Now().UnixNano())115 a.URI = fmt.Sprintf("glean:annotation:%d", time.Now().UnixNano())
108 }116 }
109 117
110- if err := s.db.CreateAnnotation(r.Context(), a); err != nil {118+ if err := s.dbs.Articles.CreateAnnotation(ctx, a); err != nil {
111 http.Error(w, err.Error(), http.StatusInternalServerError)119 http.Error(w, err.Error(), http.StatusInternalServerError)
112 return120 return
113 }121 }
@@ -121,13 +129,14 @@ func (s *Server) handleCreateAnnotation(w http.ResponseWriter, r *http.Request)
121 129
122 func (s *Server) handleDeleteAnnotation(w http.ResponseWriter, r *http.Request) {130 func (s *Server) handleDeleteAnnotation(w http.ResponseWriter, r *http.Request) {
123 user := currentUser(r)131 user := currentUser(r)
132+ ctx := r.Context()
124 id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)133 id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
125 if err != nil {134 if err != nil {
126 http.Error(w, "invalid id", http.StatusBadRequest)135 http.Error(w, "invalid id", http.StatusBadRequest)
127 return136 return
128 }137 }
129 138
130- annotation, err := s.db.GetAnnotation(r.Context(), id)139+ annotation, err := s.dbs.Articles.GetAnnotation(ctx, id)
131 if err != nil {140 if err != nil {
132 http.Error(w, "annotation not found", http.StatusNotFound)141 http.Error(w, "annotation not found", http.StatusNotFound)
133 return142 return
@@ -142,14 +151,14 @@ func (s *Server) handleDeleteAnnotation(w http.ResponseWriter, r *http.Request)
142 if client := s.pdsClientForUser(r); client != nil {151 if client := s.pdsClientForUser(r); client != nil {
143 parsed, ok := atproto.ParseRecordURI(annotation.URI)152 parsed, ok := atproto.ParseRecordURI(annotation.URI)
144 if ok {153 if ok {
145- if delErr := client.DeleteRecord(r.Context(), user.DID, parsed.Collection, parsed.RKey); delErr != nil {154+ if delErr := client.DeleteRecord(ctx, user.DID, parsed.Collection, parsed.RKey); delErr != nil {
146 s.logger.Error("failed to delete annotation from PDS", "error", delErr)155 s.logger.Error("failed to delete annotation from PDS", "error", delErr)
147 }156 }
148 }157 }
149 }158 }
150 }159 }
151 160
152- if err := s.db.DeleteAnnotation(r.Context(), annotation.URI); err != nil {161+ if err := s.dbs.Articles.DeleteAnnotation(ctx, annotation.URI); err != nil {
153 http.Error(w, err.Error(), http.StatusInternalServerError)162 http.Error(w, err.Error(), http.StatusInternalServerError)
154 return163 return
155 }164 }
modified internal/server/articles_handler.go +82 -37
@@ -32,6 +32,7 @@ func writeLikeButton(w http.ResponseWriter, articleID int64, liked bool, count i
3232
3333 func (s *Server) handleArticles(w http.ResponseWriter, r *http.Request) {
3434 user := currentUser(r)
35+ ctx := r.Context()
3536 feedURL := r.URL.Query().Get("feed")
3637 status := r.URL.Query().Get("status")
3738 searchQuery := r.URL.Query().Get("q")
@@ -39,7 +40,10 @@ func (s *Server) handleArticles(w http.ResponseWriter, r *http.Request) {
3940 page := pageFromRequest(r, 50)
4041
4142 if status == "" && searchQuery == "" {
42- unreadCount, _ := s.db.GetUnreadCount(r.Context(), user.DID, feedURL)
43+ unreadCount, err := s.dbs.Articles.GetUnreadCount(ctx, user.DID, feedURL)
44+ if err != nil {
45+ s.logger.Warn("failed to get unread count", "error", err, "did", user.DID)
46+ }
4347 if unreadCount > 0 {
4448 status = "unread"
4549 } else {
@@ -51,15 +55,15 @@ func (s *Server) handleArticles(w http.ResponseWriter, r *http.Request) {
5155 var err error
5256
5357 if searchQuery != "" {
54- articles, err = s.db.SearchArticles(r.Context(), user.DID, searchQuery, page.Limit()+1, page.Offset())
58+ articles, err = s.dbs.Articles.SearchArticles(ctx, user.DID, searchQuery, page.Limit()+1, page.Offset())
5559 } else {
5660 switch status {
5761 case "unread":
58- articles, err = s.db.ListUnreadArticles(r.Context(), user.DID, feedURL, page.Limit()+1, page.Offset())
62+ articles, err = s.dbs.Articles.ListUnreadArticles(ctx, user.DID, feedURL, page.Limit()+1, page.Offset())
5963 case "read":
60- articles, err = s.db.ListReadArticles(r.Context(), user.DID, feedURL, page.Limit()+1, page.Offset())
64+ articles, err = s.dbs.Articles.ListReadArticles(ctx, user.DID, feedURL, page.Limit()+1, page.Offset())
6165 default:
62- articles, err = s.db.ListArticles(r.Context(), user.DID, feedURL, page.Limit()+1, page.Offset())
66+ articles, err = s.dbs.Articles.ListArticles(ctx, user.DID, feedURL, page.Limit()+1, page.Offset())
6367 }
6468 }
6569
@@ -88,10 +92,12 @@ func (s *Server) handleArticles(w http.ResponseWriter, r *http.Request) {
8892 }
8993
9094 if feedURL != "" {
91- if feed, err := s.db.GetFeed(r.Context(), feedURL); err == nil {
95+ if feed, err := s.dbs.Articles.GetFeed(ctx, feedURL); err == nil {
9296 data["Feed"] = feed
97+ } else {
98+ s.logger.Warn("failed to get feed", "error", err, "feed", feedURL)
9399 }
94- if _, err := s.db.GetSubscription(r.Context(), user.DID, feedURL); err == nil {
100+ if _, err := s.dbs.Articles.GetSubscription(ctx, user.DID, feedURL); err == nil {
95101 data["IsSubscribed"] = true
96102 } else {
97103 data["IsSubscribed"] = false
@@ -108,6 +114,7 @@ func (s *Server) handleArticles(w http.ResponseWriter, r *http.Request) {
108114
109115 func (s *Server) handleNewArticleCount(w http.ResponseWriter, r *http.Request) {
110116 user := currentUser(r)
117+ ctx := r.Context()
111118 sinceUnix, err := strconv.ParseInt(r.URL.Query().Get("since"), 10, 64)
112119 if err != nil {
113120 w.WriteHeader(http.StatusBadRequest)
@@ -115,8 +122,9 @@ func (s *Server) handleNewArticleCount(w http.ResponseWriter, r *http.Request) {
115122 }
116123 since := time.Unix(sinceUnix, 0)
117124
118- count, err := s.db.CountNewArticles(r.Context(), user.DID, since)
125+ count, err := s.dbs.Articles.CountNewArticles(ctx, user.DID, since)
119126 if err != nil {
127+ s.logger.Error("failed to count new articles", "error", err, "did", user.DID)
120128 w.WriteHeader(http.StatusInternalServerError)
121129 return
122130 }
@@ -138,29 +146,53 @@ func pluralS(n int) string {
138146
139147 func (s *Server) handleArticleDetail(w http.ResponseWriter, r *http.Request) {
140148 user := currentUser(r)
149+ ctx := r.Context()
141150 id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
142151 if err != nil {
143152 http.Error(w, "invalid id", http.StatusBadRequest)
144153 return
145154 }
146155
147- article, err := s.db.GetArticle(r.Context(), id)
156+ article, err := s.dbs.Articles.GetArticle(ctx, id)
148157 if err != nil {
149158 http.Error(w, "article not found", http.StatusNotFound)
150159 return
151160 }
152161
153- _ = s.db.MarkArticleRead(r.Context(), user.DID, id)
162+ if err := s.dbs.Articles.MarkArticleRead(ctx, user.DID, id); err != nil {
163+ s.logger.Warn("failed to mark article read", "error", err, "id", id)
164+ }
154165
155- readState, _ := s.db.GetReadState(r.Context(), user.DID, id)
166+ readState, err := s.dbs.Articles.GetReadState(ctx, user.DID, id)
167+ if err != nil {
168+ s.logger.Warn("failed to get read state", "error", err, "id", id)
169+ }
170+
171+ var likeCount int
172+ if article.URL.Valid {
173+ likeCount, err = s.dbs.Articles.GetLikeCount(ctx, article.FeedURL, article.URL.String)
174+ if err != nil {
175+ s.logger.Warn("failed to get like count", "error", err, "feed", article.FeedURL)
176+ }
177+ }
156178
157- likeCount, _ := s.db.GetLikeCount(r.Context(), article.FeedURL, article.URL.String)
158179 liked := false
159180 if article.URL.Valid {
160- liked, _ = s.db.HasLiked(r.Context(), user.DID, article.FeedURL, article.URL.String)
181+ liked, err = s.dbs.Articles.HasLiked(ctx, user.DID, article.FeedURL, article.URL.String)
182+ if err != nil {
183+ s.logger.Warn("failed to check if liked", "error", err)
184+ }
185+ }
186+
187+ annotations, err := s.dbs.Articles.ListAnnotations(ctx, "", article.URL.String, "", 20, 0)
188+ if err != nil {
189+ s.logger.Warn("failed to list annotations", "error", err)
190+ }
191+
192+ feed, err := s.dbs.Articles.GetFeed(ctx, article.FeedURL)
193+ if err != nil {
194+ s.logger.Warn("failed to get feed", "error", err, "feed", article.FeedURL)
161195 }
162- annotations, _ := s.db.ListAnnotations(r.Context(), "", article.URL.String, "", 20, 0)
163- feed, _ := s.db.GetFeed(r.Context(), article.FeedURL)
164196
165197 s.render(w, r, "article_detail.html", map[string]any{
166198 "User": user,
@@ -181,7 +213,7 @@ func (s *Server) handleMarkRead(w http.ResponseWriter, r *http.Request) {
181213 http.Error(w, "invalid id", http.StatusBadRequest)
182214 return
183215 }
184- if err := s.db.MarkArticleRead(r.Context(), user.DID, id); err != nil {
216+ if err := s.dbs.Articles.MarkArticleRead(r.Context(), user.DID, id); err != nil {
185217 http.Error(w, err.Error(), http.StatusInternalServerError)
186218 return
187219 }
@@ -196,7 +228,7 @@ func (s *Server) handleMarkUnread(w http.ResponseWriter, r *http.Request) {
196228 http.Error(w, "invalid id", http.StatusBadRequest)
197229 return
198230 }
199- if err := s.db.MarkArticleUnread(r.Context(), user.DID, id); err != nil {
231+ if err := s.dbs.Articles.MarkArticleUnread(r.Context(), user.DID, id); err != nil {
200232 http.Error(w, err.Error(), http.StatusInternalServerError)
201233 return
202234 }
@@ -206,26 +238,27 @@ func (s *Server) handleMarkUnread(w http.ResponseWriter, r *http.Request) {
206238
207239 func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
208240 user := currentUser(r)
241+ ctx := r.Context()
209242 id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
210243 if err != nil {
211244 http.Error(w, "invalid id", http.StatusBadRequest)
212245 return
213246 }
214247
215- article, err := s.db.GetArticle(r.Context(), id)
248+ article, err := s.dbs.Articles.GetArticle(ctx, id)
216249 if err != nil {
217250 http.Error(w, err.Error(), http.StatusNotFound)
218251 return
219252 }
220253
221- liked, err := s.db.HasLiked(r.Context(), user.DID, article.FeedURL, article.URL.String)
254+ liked, err := s.dbs.Articles.HasLiked(ctx, user.DID, article.FeedURL, article.URL.String)
222255 if err != nil {
223256 http.Error(w, err.Error(), http.StatusInternalServerError)
224257 return
225258 }
226259
227260 if liked {
228- existingLike, getErr := s.db.GetLike(r.Context(), user.DID, article.FeedURL, article.URL.String)
261+ existingLike, getErr := s.dbs.Articles.GetLike(ctx, user.DID, article.FeedURL, article.URL.String)
229262 if getErr != nil {
230263 http.Error(w, getErr.Error(), http.StatusInternalServerError)
231264 return
@@ -234,7 +267,7 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
234267 if client := s.pdsClientForUser(r); client != nil {
235268 parsed, ok := atproto.ParseRecordURI(existingLike.URI)
236269 if ok {
237- if delErr := client.DeleteRecord(r.Context(), user.DID, parsed.Collection, parsed.RKey); delErr != nil {
270+ if delErr := client.DeleteRecord(ctx, user.DID, parsed.Collection, parsed.RKey); delErr != nil {
238271 s.logger.Error("failed to delete like from PDS", "error", delErr)
239272 http.Error(w, "failed to delete like from PDS: "+delErr.Error(), http.StatusBadGateway)
240273 return
@@ -242,7 +275,7 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
242275 }
243276 }
244277 }
245- if err := s.db.DeleteLikeByUserArticle(r.Context(), user.DID, article.FeedURL, article.URL.String); err != nil {
278+ if err := s.dbs.Articles.DeleteLikeByUserArticle(ctx, user.DID, article.FeedURL, article.URL.String); err != nil {
246279 http.Error(w, err.Error(), http.StatusInternalServerError)
247280 return
248281 }
@@ -254,7 +287,7 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
254287 }
255288
256289 if client := s.pdsClientForUser(r); client != nil {
257- uri, _, err := client.CreateRecord(r.Context(), user.DID, atproto.CollectionLike, likeRecord)
290+ uri, _, err := client.CreateRecord(ctx, user.DID, atproto.CollectionLike, likeRecord)
258291 if err != nil {
259292 s.logger.Error("failed to write like to PDS", "error", err)
260293 http.Error(w, "failed to write like to PDS: "+err.Error(), http.StatusBadGateway)
@@ -268,13 +301,15 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
268301 ArticleURL: article.URL.String,
269302 CreatedAt: sql.NullTime{Time: time.Now(), Valid: true},
270303 }
271- if err := s.db.CreateLike(r.Context(), like); err != nil && !errors.Is(err, db.ErrDuplicateLike) {
304+ if err := s.dbs.Articles.CreateLike(ctx, like); err != nil && !errors.Is(err, db.ErrDuplicateLike) {
272305 http.Error(w, err.Error(), http.StatusInternalServerError)
273306 return
274307 }
275- _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "article", article.URL.String)
276- sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID))
277- s.engine.RewardSignal(r.Context(), user.DID, sig)
308+ if err := s.engine.MarkImpressionActed(ctx, user.DID, "article", article.URL.String); err != nil {
309+ s.logger.Warn("failed to mark impression acted", "error", err)
310+ }
311+ sig := s.engine.GetDominantSignal(s.engine.GetWeights(ctx, user.DID))
312+ s.engine.RewardSignal(ctx, user.DID, sig)
278313 } else {
279314 like := &db.Like{
280315 URI: fmt.Sprintf("glean:like:%d", time.Now().UnixNano()),
@@ -283,29 +318,38 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
283318 ArticleURL: article.URL.String,
284319 CreatedAt: sql.NullTime{Time: time.Now(), Valid: true},
285320 }
286- if err := s.db.CreateLike(r.Context(), like); err != nil && !errors.Is(err, db.ErrDuplicateLike) {
321+ if err := s.dbs.Articles.CreateLike(ctx, like); err != nil && !errors.Is(err, db.ErrDuplicateLike) {
287322 http.Error(w, err.Error(), http.StatusInternalServerError)
288323 return
289324 }
290- _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "article", article.URL.String)
291- sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID))
292- s.engine.RewardSignal(r.Context(), user.DID, sig)
325+ if err := s.engine.MarkImpressionActed(ctx, user.DID, "article", article.URL.String); err != nil {
326+ s.logger.Warn("failed to mark impression acted", "error", err)
327+ }
328+ sig := s.engine.GetDominantSignal(s.engine.GetWeights(ctx, user.DID))
329+ s.engine.RewardSignal(ctx, user.DID, sig)
293330 }
294331 }
295332
296- likeCount, _ := s.db.GetLikeCount(r.Context(), article.FeedURL, article.URL.String)
333+ likeCount := 0
334+ if article.URL.Valid {
335+ likeCount, err = s.dbs.Articles.GetLikeCount(ctx, article.FeedURL, article.URL.String)
336+ if err != nil {
337+ s.logger.Warn("failed to get like count", "error", err)
338+ }
339+ }
297340 bordered := r.URL.Query().Get("bordered") == "true"
298341 writeLikeButton(w, id, !liked, likeCount, bordered)
299342 }
300343
301344 func (s *Server) handleMarkAllRead(w http.ResponseWriter, r *http.Request) {
302345 user := currentUser(r)
346+ ctx := r.Context()
303347 feedURL := r.FormValue("feed")
304348 var err error
305349 if feedURL != "" {
306- err = s.db.MarkAllRead(r.Context(), user.DID, feedURL)
350+ err = s.dbs.Articles.MarkAllRead(ctx, user.DID, feedURL)
307351 } else {
308- err = s.db.MarkAllSubscribedRead(r.Context(), user.DID)
352+ err = s.dbs.Articles.MarkAllSubscribedRead(ctx, user.DID)
309353 }
310354 if err != nil {
311355 http.Error(w, err.Error(), http.StatusInternalServerError)
@@ -316,13 +360,14 @@ func (s *Server) handleMarkAllRead(w http.ResponseWriter, r *http.Request) {
316360 }
317361
318362 func (s *Server) handleFetchContent(w http.ResponseWriter, r *http.Request) {
363+ ctx := r.Context()
319364 id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
320365 if err != nil {
321366 http.Error(w, "invalid id", http.StatusBadRequest)
322367 return
323368 }
324369
325- article, err := s.db.GetArticle(r.Context(), id)
370+ article, err := s.dbs.Articles.GetArticle(ctx, id)
326371 if err != nil {
327372 http.Error(w, "article not found", http.StatusNotFound)
328373 return
@@ -334,7 +379,7 @@ func (s *Server) handleFetchContent(w http.ResponseWriter, r *http.Request) {
334379 return
335380 }
336381
337- content, err := s.scraper.Scrape(r.Context(), article.URL.String)
382+ content, err := s.scraper.Scrape(ctx, article.URL.String)
338383 if err != nil {
339384 s.logger.Error("failed to scrape article", "error", err, "url", article.URL.String)
340385 w.Header().Set("Content-Type", "text/html")
@@ -350,7 +395,7 @@ func (s *Server) handleFetchContent(w http.ResponseWriter, r *http.Request) {
350395
351396 cleaned := sanitize.HTML(content)
352397
353- if err := s.db.UpdateArticleFullContent(r.Context(), id, cleaned); err != nil {
398+ if err := s.dbs.Articles.UpdateArticleFullContent(ctx, id, cleaned); err != nil {
354399 s.logger.Error("failed to save full content", "error", err, "id", id)
355400 }
356401
@@ -32,6 +32,7 @@ func writeLikeButton(w http.ResponseWriter, articleID int64, liked bool, count i
32 32
33 func (s *Server) handleArticles(w http.ResponseWriter, r *http.Request) {33 func (s *Server) handleArticles(w http.ResponseWriter, r *http.Request) {
34 user := currentUser(r)34 user := currentUser(r)
35+ ctx := r.Context()
35 feedURL := r.URL.Query().Get("feed")36 feedURL := r.URL.Query().Get("feed")
36 status := r.URL.Query().Get("status")37 status := r.URL.Query().Get("status")
37 searchQuery := r.URL.Query().Get("q")38 searchQuery := r.URL.Query().Get("q")
@@ -39,7 +40,10 @@ func (s *Server) handleArticles(w http.ResponseWriter, r *http.Request) {
39 page := pageFromRequest(r, 50)40 page := pageFromRequest(r, 50)
40 41
41 if status == "" && searchQuery == "" {42 if status == "" && searchQuery == "" {
42- unreadCount, _ := s.db.GetUnreadCount(r.Context(), user.DID, feedURL)43+ unreadCount, err := s.dbs.Articles.GetUnreadCount(ctx, user.DID, feedURL)
44+ if err != nil {
45+ s.logger.Warn("failed to get unread count", "error", err, "did", user.DID)
46+ }
43 if unreadCount > 0 {47 if unreadCount > 0 {
44 status = "unread"48 status = "unread"
45 } else {49 } else {
@@ -51,15 +55,15 @@ func (s *Server) handleArticles(w http.ResponseWriter, r *http.Request) {
51 var err error55 var err error
52 56
53 if searchQuery != "" {57 if searchQuery != "" {
54- articles, err = s.db.SearchArticles(r.Context(), user.DID, searchQuery, page.Limit()+1, page.Offset())58+ articles, err = s.dbs.Articles.SearchArticles(ctx, user.DID, searchQuery, page.Limit()+1, page.Offset())
55 } else {59 } else {
56 switch status {60 switch status {
57 case "unread":61 case "unread":
58- articles, err = s.db.ListUnreadArticles(r.Context(), user.DID, feedURL, page.Limit()+1, page.Offset())62+ articles, err = s.dbs.Articles.ListUnreadArticles(ctx, user.DID, feedURL, page.Limit()+1, page.Offset())
59 case "read":63 case "read":
60- articles, err = s.db.ListReadArticles(r.Context(), user.DID, feedURL, page.Limit()+1, page.Offset())64+ articles, err = s.dbs.Articles.ListReadArticles(ctx, user.DID, feedURL, page.Limit()+1, page.Offset())
61 default:65 default:
62- articles, err = s.db.ListArticles(r.Context(), user.DID, feedURL, page.Limit()+1, page.Offset())66+ articles, err = s.dbs.Articles.ListArticles(ctx, user.DID, feedURL, page.Limit()+1, page.Offset())
63 }67 }
64 }68 }
65 69
@@ -88,10 +92,12 @@ func (s *Server) handleArticles(w http.ResponseWriter, r *http.Request) {
88 }92 }
89 93
90 if feedURL != "" {94 if feedURL != "" {
91- if feed, err := s.db.GetFeed(r.Context(), feedURL); err == nil {95+ if feed, err := s.dbs.Articles.GetFeed(ctx, feedURL); err == nil {
92 data["Feed"] = feed96 data["Feed"] = feed
97+ } else {
98+ s.logger.Warn("failed to get feed", "error", err, "feed", feedURL)
93 }99 }
94- if _, err := s.db.GetSubscription(r.Context(), user.DID, feedURL); err == nil {100+ if _, err := s.dbs.Articles.GetSubscription(ctx, user.DID, feedURL); err == nil {
95 data["IsSubscribed"] = true101 data["IsSubscribed"] = true
96 } else {102 } else {
97 data["IsSubscribed"] = false103 data["IsSubscribed"] = false
@@ -108,6 +114,7 @@ func (s *Server) handleArticles(w http.ResponseWriter, r *http.Request) {
108 114
109 func (s *Server) handleNewArticleCount(w http.ResponseWriter, r *http.Request) {115 func (s *Server) handleNewArticleCount(w http.ResponseWriter, r *http.Request) {
110 user := currentUser(r)116 user := currentUser(r)
117+ ctx := r.Context()
111 sinceUnix, err := strconv.ParseInt(r.URL.Query().Get("since"), 10, 64)118 sinceUnix, err := strconv.ParseInt(r.URL.Query().Get("since"), 10, 64)
112 if err != nil {119 if err != nil {
113 w.WriteHeader(http.StatusBadRequest)120 w.WriteHeader(http.StatusBadRequest)
@@ -115,8 +122,9 @@ func (s *Server) handleNewArticleCount(w http.ResponseWriter, r *http.Request) {
115 }122 }
116 since := time.Unix(sinceUnix, 0)123 since := time.Unix(sinceUnix, 0)
117 124
118- count, err := s.db.CountNewArticles(r.Context(), user.DID, since)125+ count, err := s.dbs.Articles.CountNewArticles(ctx, user.DID, since)
119 if err != nil {126 if err != nil {
127+ s.logger.Error("failed to count new articles", "error", err, "did", user.DID)
120 w.WriteHeader(http.StatusInternalServerError)128 w.WriteHeader(http.StatusInternalServerError)
121 return129 return
122 }130 }
@@ -138,29 +146,53 @@ func pluralS(n int) string {
138 146
139 func (s *Server) handleArticleDetail(w http.ResponseWriter, r *http.Request) {147 func (s *Server) handleArticleDetail(w http.ResponseWriter, r *http.Request) {
140 user := currentUser(r)148 user := currentUser(r)
149+ ctx := r.Context()
141 id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)150 id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
142 if err != nil {151 if err != nil {
143 http.Error(w, "invalid id", http.StatusBadRequest)152 http.Error(w, "invalid id", http.StatusBadRequest)
144 return153 return
145 }154 }
146 155
147- article, err := s.db.GetArticle(r.Context(), id)156+ article, err := s.dbs.Articles.GetArticle(ctx, id)
148 if err != nil {157 if err != nil {
149 http.Error(w, "article not found", http.StatusNotFound)158 http.Error(w, "article not found", http.StatusNotFound)
150 return159 return
151 }160 }
152 161
153- _ = s.db.MarkArticleRead(r.Context(), user.DID, id)162+ if err := s.dbs.Articles.MarkArticleRead(ctx, user.DID, id); err != nil {
163+ s.logger.Warn("failed to mark article read", "error", err, "id", id)
164+ }
154 165
155- readState, _ := s.db.GetReadState(r.Context(), user.DID, id)166+ readState, err := s.dbs.Articles.GetReadState(ctx, user.DID, id)
167+ if err != nil {
168+ s.logger.Warn("failed to get read state", "error", err, "id", id)
169+ }
170+
171+ var likeCount int
172+ if article.URL.Valid {
173+ likeCount, err = s.dbs.Articles.GetLikeCount(ctx, article.FeedURL, article.URL.String)
174+ if err != nil {
175+ s.logger.Warn("failed to get like count", "error", err, "feed", article.FeedURL)
176+ }
177+ }
156 178
157- likeCount, _ := s.db.GetLikeCount(r.Context(), article.FeedURL, article.URL.String)
158 liked := false179 liked := false
159 if article.URL.Valid {180 if article.URL.Valid {
160- liked, _ = s.db.HasLiked(r.Context(), user.DID, article.FeedURL, article.URL.String)181+ liked, err = s.dbs.Articles.HasLiked(ctx, user.DID, article.FeedURL, article.URL.String)
182+ if err != nil {
183+ s.logger.Warn("failed to check if liked", "error", err)
184+ }
185+ }
186+
187+ annotations, err := s.dbs.Articles.ListAnnotations(ctx, "", article.URL.String, "", 20, 0)
188+ if err != nil {
189+ s.logger.Warn("failed to list annotations", "error", err)
190+ }
191+
192+ feed, err := s.dbs.Articles.GetFeed(ctx, article.FeedURL)
193+ if err != nil {
194+ s.logger.Warn("failed to get feed", "error", err, "feed", article.FeedURL)
161 }195 }
162- annotations, _ := s.db.ListAnnotations(r.Context(), "", article.URL.String, "", 20, 0)
163- feed, _ := s.db.GetFeed(r.Context(), article.FeedURL)
164 196
165 s.render(w, r, "article_detail.html", map[string]any{197 s.render(w, r, "article_detail.html", map[string]any{
166 "User": user,198 "User": user,
@@ -181,7 +213,7 @@ func (s *Server) handleMarkRead(w http.ResponseWriter, r *http.Request) {
181 http.Error(w, "invalid id", http.StatusBadRequest)213 http.Error(w, "invalid id", http.StatusBadRequest)
182 return214 return
183 }215 }
184- if err := s.db.MarkArticleRead(r.Context(), user.DID, id); err != nil {216+ if err := s.dbs.Articles.MarkArticleRead(r.Context(), user.DID, id); err != nil {
185 http.Error(w, err.Error(), http.StatusInternalServerError)217 http.Error(w, err.Error(), http.StatusInternalServerError)
186 return218 return
187 }219 }
@@ -196,7 +228,7 @@ func (s *Server) handleMarkUnread(w http.ResponseWriter, r *http.Request) {
196 http.Error(w, "invalid id", http.StatusBadRequest)228 http.Error(w, "invalid id", http.StatusBadRequest)
197 return229 return
198 }230 }
199- if err := s.db.MarkArticleUnread(r.Context(), user.DID, id); err != nil {231+ if err := s.dbs.Articles.MarkArticleUnread(r.Context(), user.DID, id); err != nil {
200 http.Error(w, err.Error(), http.StatusInternalServerError)232 http.Error(w, err.Error(), http.StatusInternalServerError)
201 return233 return
202 }234 }
@@ -206,26 +238,27 @@ func (s *Server) handleMarkUnread(w http.ResponseWriter, r *http.Request) {
206 238
207 func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {239 func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
208 user := currentUser(r)240 user := currentUser(r)
241+ ctx := r.Context()
209 id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)242 id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
210 if err != nil {243 if err != nil {
211 http.Error(w, "invalid id", http.StatusBadRequest)244 http.Error(w, "invalid id", http.StatusBadRequest)
212 return245 return
213 }246 }
214 247
215- article, err := s.db.GetArticle(r.Context(), id)248+ article, err := s.dbs.Articles.GetArticle(ctx, id)
216 if err != nil {249 if err != nil {
217 http.Error(w, err.Error(), http.StatusNotFound)250 http.Error(w, err.Error(), http.StatusNotFound)
218 return251 return
219 }252 }
220 253
221- liked, err := s.db.HasLiked(r.Context(), user.DID, article.FeedURL, article.URL.String)254+ liked, err := s.dbs.Articles.HasLiked(ctx, user.DID, article.FeedURL, article.URL.String)
222 if err != nil {255 if err != nil {
223 http.Error(w, err.Error(), http.StatusInternalServerError)256 http.Error(w, err.Error(), http.StatusInternalServerError)
224 return257 return
225 }258 }
226 259
227 if liked {260 if liked {
228- existingLike, getErr := s.db.GetLike(r.Context(), user.DID, article.FeedURL, article.URL.String)261+ existingLike, getErr := s.dbs.Articles.GetLike(ctx, user.DID, article.FeedURL, article.URL.String)
229 if getErr != nil {262 if getErr != nil {
230 http.Error(w, getErr.Error(), http.StatusInternalServerError)263 http.Error(w, getErr.Error(), http.StatusInternalServerError)
231 return264 return
@@ -234,7 +267,7 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
234 if client := s.pdsClientForUser(r); client != nil {267 if client := s.pdsClientForUser(r); client != nil {
235 parsed, ok := atproto.ParseRecordURI(existingLike.URI)268 parsed, ok := atproto.ParseRecordURI(existingLike.URI)
236 if ok {269 if ok {
237- if delErr := client.DeleteRecord(r.Context(), user.DID, parsed.Collection, parsed.RKey); delErr != nil {270+ if delErr := client.DeleteRecord(ctx, user.DID, parsed.Collection, parsed.RKey); delErr != nil {
238 s.logger.Error("failed to delete like from PDS", "error", delErr)271 s.logger.Error("failed to delete like from PDS", "error", delErr)
239 http.Error(w, "failed to delete like from PDS: "+delErr.Error(), http.StatusBadGateway)272 http.Error(w, "failed to delete like from PDS: "+delErr.Error(), http.StatusBadGateway)
240 return273 return
@@ -242,7 +275,7 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
242 }275 }
243 }276 }
244 }277 }
245- if err := s.db.DeleteLikeByUserArticle(r.Context(), user.DID, article.FeedURL, article.URL.String); err != nil {278+ if err := s.dbs.Articles.DeleteLikeByUserArticle(ctx, user.DID, article.FeedURL, article.URL.String); err != nil {
246 http.Error(w, err.Error(), http.StatusInternalServerError)279 http.Error(w, err.Error(), http.StatusInternalServerError)
247 return280 return
248 }281 }
@@ -254,7 +287,7 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
254 }287 }
255 288
256 if client := s.pdsClientForUser(r); client != nil {289 if client := s.pdsClientForUser(r); client != nil {
257- uri, _, err := client.CreateRecord(r.Context(), user.DID, atproto.CollectionLike, likeRecord)290+ uri, _, err := client.CreateRecord(ctx, user.DID, atproto.CollectionLike, likeRecord)
258 if err != nil {291 if err != nil {
259 s.logger.Error("failed to write like to PDS", "error", err)292 s.logger.Error("failed to write like to PDS", "error", err)
260 http.Error(w, "failed to write like to PDS: "+err.Error(), http.StatusBadGateway)293 http.Error(w, "failed to write like to PDS: "+err.Error(), http.StatusBadGateway)
@@ -268,13 +301,15 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
268 ArticleURL: article.URL.String,301 ArticleURL: article.URL.String,
269 CreatedAt: sql.NullTime{Time: time.Now(), Valid: true},302 CreatedAt: sql.NullTime{Time: time.Now(), Valid: true},
270 }303 }
271- if err := s.db.CreateLike(r.Context(), like); err != nil && !errors.Is(err, db.ErrDuplicateLike) {304+ if err := s.dbs.Articles.CreateLike(ctx, like); err != nil && !errors.Is(err, db.ErrDuplicateLike) {
272 http.Error(w, err.Error(), http.StatusInternalServerError)305 http.Error(w, err.Error(), http.StatusInternalServerError)
273 return306 return
274 }307 }
275- _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "article", article.URL.String)308+ if err := s.engine.MarkImpressionActed(ctx, user.DID, "article", article.URL.String); err != nil {
276- sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID))309+ s.logger.Warn("failed to mark impression acted", "error", err)
277- s.engine.RewardSignal(r.Context(), user.DID, sig)310+ }
311+ sig := s.engine.GetDominantSignal(s.engine.GetWeights(ctx, user.DID))
312+ s.engine.RewardSignal(ctx, user.DID, sig)
278 } else {313 } else {
279 like := &db.Like{314 like := &db.Like{
280 URI: fmt.Sprintf("glean:like:%d", time.Now().UnixNano()),315 URI: fmt.Sprintf("glean:like:%d", time.Now().UnixNano()),
@@ -283,29 +318,38 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
283 ArticleURL: article.URL.String,318 ArticleURL: article.URL.String,
284 CreatedAt: sql.NullTime{Time: time.Now(), Valid: true},319 CreatedAt: sql.NullTime{Time: time.Now(), Valid: true},
285 }320 }
286- if err := s.db.CreateLike(r.Context(), like); err != nil && !errors.Is(err, db.ErrDuplicateLike) {321+ if err := s.dbs.Articles.CreateLike(ctx, like); err != nil && !errors.Is(err, db.ErrDuplicateLike) {
287 http.Error(w, err.Error(), http.StatusInternalServerError)322 http.Error(w, err.Error(), http.StatusInternalServerError)
288 return323 return
289 }324 }
290- _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "article", article.URL.String)325+ if err := s.engine.MarkImpressionActed(ctx, user.DID, "article", article.URL.String); err != nil {
291- sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID))326+ s.logger.Warn("failed to mark impression acted", "error", err)
292- s.engine.RewardSignal(r.Context(), user.DID, sig)327+ }
328+ sig := s.engine.GetDominantSignal(s.engine.GetWeights(ctx, user.DID))
329+ s.engine.RewardSignal(ctx, user.DID, sig)
293 }330 }
294 }331 }
295 332
296- likeCount, _ := s.db.GetLikeCount(r.Context(), article.FeedURL, article.URL.String)333+ likeCount := 0
334+ if article.URL.Valid {
335+ likeCount, err = s.dbs.Articles.GetLikeCount(ctx, article.FeedURL, article.URL.String)
336+ if err != nil {
337+ s.logger.Warn("failed to get like count", "error", err)
338+ }
339+ }
297 bordered := r.URL.Query().Get("bordered") == "true"340 bordered := r.URL.Query().Get("bordered") == "true"
298 writeLikeButton(w, id, !liked, likeCount, bordered)341 writeLikeButton(w, id, !liked, likeCount, bordered)
299 }342 }
300 343
301 func (s *Server) handleMarkAllRead(w http.ResponseWriter, r *http.Request) {344 func (s *Server) handleMarkAllRead(w http.ResponseWriter, r *http.Request) {
302 user := currentUser(r)345 user := currentUser(r)
346+ ctx := r.Context()
303 feedURL := r.FormValue("feed")347 feedURL := r.FormValue("feed")
304 var err error348 var err error
305 if feedURL != "" {349 if feedURL != "" {
306- err = s.db.MarkAllRead(r.Context(), user.DID, feedURL)350+ err = s.dbs.Articles.MarkAllRead(ctx, user.DID, feedURL)
307 } else {351 } else {
308- err = s.db.MarkAllSubscribedRead(r.Context(), user.DID)352+ err = s.dbs.Articles.MarkAllSubscribedRead(ctx, user.DID)
309 }353 }
310 if err != nil {354 if err != nil {
311 http.Error(w, err.Error(), http.StatusInternalServerError)355 http.Error(w, err.Error(), http.StatusInternalServerError)
@@ -316,13 +360,14 @@ func (s *Server) handleMarkAllRead(w http.ResponseWriter, r *http.Request) {
316 }360 }
317 361
318 func (s *Server) handleFetchContent(w http.ResponseWriter, r *http.Request) {362 func (s *Server) handleFetchContent(w http.ResponseWriter, r *http.Request) {
363+ ctx := r.Context()
319 id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)364 id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
320 if err != nil {365 if err != nil {
321 http.Error(w, "invalid id", http.StatusBadRequest)366 http.Error(w, "invalid id", http.StatusBadRequest)
322 return367 return
323 }368 }
324 369
325- article, err := s.db.GetArticle(r.Context(), id)370+ article, err := s.dbs.Articles.GetArticle(ctx, id)
326 if err != nil {371 if err != nil {
327 http.Error(w, "article not found", http.StatusNotFound)372 http.Error(w, "article not found", http.StatusNotFound)
328 return373 return
@@ -334,7 +379,7 @@ func (s *Server) handleFetchContent(w http.ResponseWriter, r *http.Request) {
334 return379 return
335 }380 }
336 381
337- content, err := s.scraper.Scrape(r.Context(), article.URL.String)382+ content, err := s.scraper.Scrape(ctx, article.URL.String)
338 if err != nil {383 if err != nil {
339 s.logger.Error("failed to scrape article", "error", err, "url", article.URL.String)384 s.logger.Error("failed to scrape article", "error", err, "url", article.URL.String)
340 w.Header().Set("Content-Type", "text/html")385 w.Header().Set("Content-Type", "text/html")
@@ -350,7 +395,7 @@ func (s *Server) handleFetchContent(w http.ResponseWriter, r *http.Request) {
350 395
351 cleaned := sanitize.HTML(content)396 cleaned := sanitize.HTML(content)
352 397
353- if err := s.db.UpdateArticleFullContent(r.Context(), id, cleaned); err != nil {398+ if err := s.dbs.Articles.UpdateArticleFullContent(ctx, id, cleaned); err != nil {
354 s.logger.Error("failed to save full content", "error", err, "id", id)399 s.logger.Error("failed to save full content", "error", err, "id", id)
355 }400 }
356 401
modified internal/server/auth_handler.go +3 -3
@@ -33,7 +33,7 @@ func (s *Server) handleAuthStart(w http.ResponseWriter, r *http.Request) {
3333 http.Error(w, "could not resolve handle", http.StatusInternalServerError)
3434 return
3535 }
36- user, createErr := s.db.CreateUser(r.Context(), did, handle, "", "")
36+ user, createErr := s.dbs.Users.CreateUser(r.Context(), did, handle, "", "")
3737 if createErr != nil {
3838 http.Error(w, createErr.Error(), http.StatusInternalServerError)
3939 return
@@ -67,7 +67,7 @@ func (s *Server) handleAuthCallback(w http.ResponseWriter, r *http.Request) {
6767 return
6868 }
6969
70- user, err := s.db.CreateUser(r.Context(), did, handle, "", "")
70+ user, err := s.dbs.Users.CreateUser(r.Context(), did, handle, "", "")
7171 if err != nil {
7272 s.logger.Error("failed to create user", "error", err)
7373 http.Error(w, err.Error(), http.StatusInternalServerError)
@@ -102,7 +102,7 @@ func (s *Server) handleOAuthCallback(w http.ResponseWriter, r *http.Request) {
102102 }
103103 }
104104
105- user, err := s.db.CreateUser(r.Context(), did, handle, displayName, avatarURL)
105+ user, err := s.dbs.Users.CreateUser(r.Context(), did, handle, displayName, avatarURL)
106106 if err != nil {
107107 s.logger.Error("failed to create user", "error", err)
108108 http.Error(w, err.Error(), http.StatusInternalServerError)
@@ -33,7 +33,7 @@ func (s *Server) handleAuthStart(w http.ResponseWriter, r *http.Request) {
33 http.Error(w, "could not resolve handle", http.StatusInternalServerError)33 http.Error(w, "could not resolve handle", http.StatusInternalServerError)
34 return34 return
35 }35 }
36- user, createErr := s.db.CreateUser(r.Context(), did, handle, "", "")36+ user, createErr := s.dbs.Users.CreateUser(r.Context(), did, handle, "", "")
37 if createErr != nil {37 if createErr != nil {
38 http.Error(w, createErr.Error(), http.StatusInternalServerError)38 http.Error(w, createErr.Error(), http.StatusInternalServerError)
39 return39 return
@@ -67,7 +67,7 @@ func (s *Server) handleAuthCallback(w http.ResponseWriter, r *http.Request) {
67 return67 return
68 }68 }
69 69
70- user, err := s.db.CreateUser(r.Context(), did, handle, "", "")70+ user, err := s.dbs.Users.CreateUser(r.Context(), did, handle, "", "")
71 if err != nil {71 if err != nil {
72 s.logger.Error("failed to create user", "error", err)72 s.logger.Error("failed to create user", "error", err)
73 http.Error(w, err.Error(), http.StatusInternalServerError)73 http.Error(w, err.Error(), http.StatusInternalServerError)
@@ -102,7 +102,7 @@ func (s *Server) handleOAuthCallback(w http.ResponseWriter, r *http.Request) {
102 }102 }
103 }103 }
104 104
105- user, err := s.db.CreateUser(r.Context(), did, handle, displayName, avatarURL)105+ user, err := s.dbs.Users.CreateUser(r.Context(), did, handle, displayName, avatarURL)
106 if err != nil {106 if err != nil {
107 s.logger.Error("failed to create user", "error", err)107 s.logger.Error("failed to create user", "error", err)
108 http.Error(w, err.Error(), http.StatusInternalServerError)108 http.Error(w, err.Error(), http.StatusInternalServerError)
modified internal/server/dashboard_handler.go +41 -9
@@ -9,21 +9,43 @@ import (
99
1010 func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) {
1111 user := currentUser(r)
12+ ctx := r.Context()
1213
13- unreadCount, _ := s.db.GetUnreadCount(r.Context(), user.DID, "")
14- subCount, _ := s.db.GetSubscriptionCount(r.Context(), user.DID)
14+ unreadCount, err := s.dbs.Articles.GetUnreadCount(ctx, user.DID, "")
15+ if err != nil {
16+ s.logger.Warn("failed to get unread count", "error", err, "did", user.DID)
17+ }
18+
19+ subCount, err := s.dbs.Articles.GetSubscriptionCount(ctx, user.DID)
20+ if err != nil {
21+ s.logger.Warn("failed to get subscription count", "error", err, "did", user.DID)
22+ }
1523
1624 page := pageFromRequest(r, 25)
17- articles, _ := s.db.ListUnreadArticles(r.Context(), user.DID, "", page.Limit()+1, page.Offset())
25+ articles, err := s.dbs.Articles.ListUnreadArticles(ctx, user.DID, "", page.Limit()+1, page.Offset())
26+ if err != nil {
27+ s.logger.Warn("failed to list unread articles", "error", err, "did", user.DID)
28+ }
1829 totalFetched := len(articles)
1930 page = page.Paginate(totalFetched)
2031 if page.HasNext {
2132 articles = articles[:page.PageSize]
2233 }
2334
24- articleRecs, _ := s.engine.GetArticleRecommendations(r.Context(), user.DID, 5)
25- peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5)
26- feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 5)
35+ articleRecs, err := s.engine.GetArticleRecommendations(ctx, user.DID, 5)
36+ if err != nil {
37+ s.logger.Warn("failed to get article recommendations", "error", err, "did", user.DID)
38+ }
39+
40+ peopleRecs, err := s.engine.GetPeopleRecommendations(ctx, user.DID, 5)
41+ if err != nil {
42+ s.logger.Warn("failed to get people recommendations", "error", err, "did", user.DID)
43+ }
44+
45+ feedRecs, err := s.engine.GetFeedRecommendations(ctx, user.DID, 5)
46+ if err != nil {
47+ s.logger.Warn("failed to get feed recommendations", "error", err, "did", user.DID)
48+ }
2749
2850 var impressions []cluster.Impression
2951 for _, rec := range feedRecs {
@@ -33,12 +55,22 @@ func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) {
3355 impressions = append(impressions, cluster.Impression{TargetType: "article", TargetID: rec.URL})
3456 }
3557 if len(impressions) > 0 {
36- _ = s.engine.RecordImpressions(r.Context(), user.DID, impressions)
58+ if err := s.engine.RecordImpressions(ctx, user.DID, impressions); err != nil {
59+ s.logger.Warn("failed to record impressions", "error", err)
60+ }
3761 }
3862
3963 since := time.Now().AddDate(0, 0, -7).Format(time.RFC3339)
40- personalTrending, _ := s.db.ListTrendingArticlesForUser(r.Context(), user.DID, since, 5, 0)
41- globalTrending, _ := s.db.ListTrendingArticles(r.Context(), user.DID, since, 10, 0)
64+
65+ personalTrending, err := s.dbs.Articles.ListTrendingArticlesForUser(ctx, user.DID, since, 5, 0)
66+ if err != nil {
67+ s.logger.Warn("failed to list personal trending", "error", err, "did", user.DID)
68+ }
69+
70+ globalTrending, err := s.dbs.Articles.ListTrendingArticles(ctx, user.DID, since, 10, 0)
71+ if err != nil {
72+ s.logger.Warn("failed to list global trending", "error", err, "did", user.DID)
73+ }
4274
4375 s.render(w, r, "dashboard.html", map[string]any{
4476 "User": user,
@@ -9,21 +9,43 @@ import (
9 9
10 func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) {10 func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) {
11 user := currentUser(r)11 user := currentUser(r)
12+ ctx := r.Context()
12 13
13- unreadCount, _ := s.db.GetUnreadCount(r.Context(), user.DID, "")14+ unreadCount, err := s.dbs.Articles.GetUnreadCount(ctx, user.DID, "")
14- subCount, _ := s.db.GetSubscriptionCount(r.Context(), user.DID)15+ if err != nil {
16+ s.logger.Warn("failed to get unread count", "error", err, "did", user.DID)
17+ }
18+
19+ subCount, err := s.dbs.Articles.GetSubscriptionCount(ctx, user.DID)
20+ if err != nil {
21+ s.logger.Warn("failed to get subscription count", "error", err, "did", user.DID)
22+ }
15 23
16 page := pageFromRequest(r, 25)24 page := pageFromRequest(r, 25)
17- articles, _ := s.db.ListUnreadArticles(r.Context(), user.DID, "", page.Limit()+1, page.Offset())25+ articles, err := s.dbs.Articles.ListUnreadArticles(ctx, user.DID, "", page.Limit()+1, page.Offset())
26+ if err != nil {
27+ s.logger.Warn("failed to list unread articles", "error", err, "did", user.DID)
28+ }
18 totalFetched := len(articles)29 totalFetched := len(articles)
19 page = page.Paginate(totalFetched)30 page = page.Paginate(totalFetched)
20 if page.HasNext {31 if page.HasNext {
21 articles = articles[:page.PageSize]32 articles = articles[:page.PageSize]
22 }33 }
23 34
24- articleRecs, _ := s.engine.GetArticleRecommendations(r.Context(), user.DID, 5)35+ articleRecs, err := s.engine.GetArticleRecommendations(ctx, user.DID, 5)
25- peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5)36+ if err != nil {
26- feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 5)37+ s.logger.Warn("failed to get article recommendations", "error", err, "did", user.DID)
38+ }
39+
40+ peopleRecs, err := s.engine.GetPeopleRecommendations(ctx, user.DID, 5)
41+ if err != nil {
42+ s.logger.Warn("failed to get people recommendations", "error", err, "did", user.DID)
43+ }
44+
45+ feedRecs, err := s.engine.GetFeedRecommendations(ctx, user.DID, 5)
46+ if err != nil {
47+ s.logger.Warn("failed to get feed recommendations", "error", err, "did", user.DID)
48+ }
27 49
28 var impressions []cluster.Impression50 var impressions []cluster.Impression
29 for _, rec := range feedRecs {51 for _, rec := range feedRecs {
@@ -33,12 +55,22 @@ func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) {
33 impressions = append(impressions, cluster.Impression{TargetType: "article", TargetID: rec.URL})55 impressions = append(impressions, cluster.Impression{TargetType: "article", TargetID: rec.URL})
34 }56 }
35 if len(impressions) > 0 {57 if len(impressions) > 0 {
36- _ = s.engine.RecordImpressions(r.Context(), user.DID, impressions)58+ if err := s.engine.RecordImpressions(ctx, user.DID, impressions); err != nil {
59+ s.logger.Warn("failed to record impressions", "error", err)
60+ }
37 }61 }
38 62
39 since := time.Now().AddDate(0, 0, -7).Format(time.RFC3339)63 since := time.Now().AddDate(0, 0, -7).Format(time.RFC3339)
40- personalTrending, _ := s.db.ListTrendingArticlesForUser(r.Context(), user.DID, since, 5, 0)64+
41- globalTrending, _ := s.db.ListTrendingArticles(r.Context(), user.DID, since, 10, 0)65+ personalTrending, err := s.dbs.Articles.ListTrendingArticlesForUser(ctx, user.DID, since, 5, 0)
66+ if err != nil {
67+ s.logger.Warn("failed to list personal trending", "error", err, "did", user.DID)
68+ }
69+
70+ globalTrending, err := s.dbs.Articles.ListTrendingArticles(ctx, user.DID, since, 10, 0)
71+ if err != nil {
72+ s.logger.Warn("failed to list global trending", "error", err, "did", user.DID)
73+ }
42 74
43 s.render(w, r, "dashboard.html", map[string]any{75 s.render(w, r, "dashboard.html", map[string]any{
44 "User": user,76 "User": user,
modified internal/server/feeds_handler.go +75 -27
@@ -16,29 +16,53 @@ import (
1616 func (s *Server) handleFeeds(w http.ResponseWriter, r *http.Request) {
1717 user := currentUser(r)
1818 category := r.URL.Query().Get("category")
19+ ctx := r.Context()
1920
2021 page := pageFromRequest(r, 50)
21- subs, _ := s.db.ListSubscriptions(r.Context(), user.DID, category, page.Limit()+1, page.Offset())
22+ subs, err := s.dbs.Articles.ListSubscriptions(ctx, user.DID, category, page.Limit()+1, page.Offset())
23+ if err != nil {
24+ s.logger.Warn("failed to list subscriptions", "error", err, "did", user.DID)
25+ }
2226 totalFetched := len(subs)
2327 page = page.Paginate(totalFetched)
2428 if page.HasNext {
2529 subs = subs[:page.PageSize]
2630 }
2731
28- allSubs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 1000, 0)
29- feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 6)
30- peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5)
32+ allSubs, err := s.dbs.Articles.ListSubscriptions(ctx, user.DID, "", 1000, 0)
33+ if err != nil {
34+ s.logger.Warn("failed to list all subscriptions", "error", err, "did", user.DID)
35+ }
36+
37+ feedRecs, err := s.engine.GetFeedRecommendations(ctx, user.DID, 6)
38+ if err != nil {
39+ s.logger.Warn("failed to get feed recommendations", "error", err, "did", user.DID)
40+ }
41+
42+ peopleRecs, err := s.engine.GetPeopleRecommendations(ctx, user.DID, 5)
43+ if err != nil {
44+ s.logger.Warn("failed to get people recommendations", "error", err, "did", user.DID)
45+ }
3146
3247 if len(feedRecs) > 0 {
3348 impressions := make([]cluster.Impression, len(feedRecs))
3449 for i, rec := range feedRecs {
3550 impressions[i] = cluster.Impression{TargetType: "feed", TargetID: rec.FeedURL}
3651 }
37- _ = s.engine.RecordImpressions(r.Context(), user.DID, impressions)
52+ if err := s.engine.RecordImpressions(ctx, user.DID, impressions); err != nil {
53+ s.logger.Warn("failed to record impressions", "error", err)
54+ }
55+ }
56+
57+ deadFeeds, err := s.dbs.Articles.ListDeadFeeds(ctx, user.DID, 7)
58+ if err != nil {
59+ s.logger.Warn("failed to list dead feeds", "error", err, "did", user.DID)
3860 }
39- deadFeeds, _ := s.db.ListDeadFeeds(r.Context(), user.DID, 7)
4061
41- categories, _ := s.db.GetCategories(r.Context(), user.DID)
62+ categories, err := s.dbs.Articles.GetCategories(ctx, user.DID)
63+ if err != nil {
64+ s.logger.Warn("failed to get categories", "error", err, "did", user.DID)
65+ }
4266
4367 s.render(w, r, "feeds.html", map[string]any{
4468 "User": user,
@@ -80,7 +104,7 @@ func (s *Server) handleAddFeed(w http.ResponseWriter, r *http.Request) {
80104 if faviconURL == "" {
81105 go func() {
82106 if f := feed.ResolveFavicon(context.Background(), feedURL, result.Feed.SiteURL); f != "" {
83- _ = s.db.UpdateFeedFavicon(context.Background(), feedURL, f)
107+ _ = s.dbs.Articles.UpdateFeedFavicon(context.Background(), feedURL, f)
84108 }
85109 }()
86110 }
@@ -94,7 +118,7 @@ func (s *Server) handleAddFeed(w http.ResponseWriter, r *http.Request) {
94118 FeedType: nullString(result.Feed.Type),
95119 FaviconURL: nullString(faviconURL),
96120 }
97- if err := s.db.UpsertFeed(r.Context(), f); err != nil {
121+ if err := s.dbs.Articles.UpsertFeed(r.Context(), f); err != nil {
98122 s.logger.Error("failed to upsert feed", "error", err)
99123 http.Error(w, err.Error(), http.StatusInternalServerError)
100124 return
@@ -118,7 +142,7 @@ func (s *Server) handleAddFeed(w http.ResponseWriter, r *http.Request) {
118142 subCID = cid
119143 }
120144
121- if err := s.db.CreateSubscription(r.Context(), user.DID, feedURL, feedTitle, category, subURI, subCID); err != nil {
145+ if err := s.dbs.Articles.CreateSubscription(r.Context(), user.DID, feedURL, feedTitle, category, subURI, subCID); err != nil {
122146 if errors.Is(err, db.ErrDuplicateSubscription) {
123147 http.Error(w, "Already subscribed to this feed.", http.StatusConflict)
124148 return
@@ -128,12 +152,15 @@ func (s *Server) handleAddFeed(w http.ResponseWriter, r *http.Request) {
128152 return
129153 }
130154
131- _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "feed", feedURL)
155+ if err := s.engine.MarkImpressionActed(r.Context(), user.DID, "feed", feedURL); err != nil {
156+ s.logger.Warn("failed to mark impression acted", "error", err)
157+ }
132158 sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID))
133159 s.engine.RewardSignal(r.Context(), user.DID, sig)
134160
135- sub, _ := s.db.GetSubscription(r.Context(), user.DID, feedURL)
136- if sub == nil {
161+ sub, err := s.dbs.Articles.GetSubscription(r.Context(), user.DID, feedURL)
162+ if err != nil {
163+ s.logger.Warn("failed to get subscription", "error", err)
137164 w.WriteHeader(http.StatusNoContent)
138165 return
139166 }
@@ -156,7 +183,7 @@ func (s *Server) handleRemoveFeed(w http.ResponseWriter, r *http.Request) {
156183 return
157184 }
158185
159- sub, err := s.db.GetSubscription(r.Context(), user.DID, feedURL)
186+ sub, err := s.dbs.Articles.GetSubscription(r.Context(), user.DID, feedURL)
160187 if err == nil && sub.URI.Valid {
161188 if client := s.pdsClientForUser(r); client != nil {
162189 parsed, ok := atproto.ParseRecordURI(sub.URI.String)
@@ -170,7 +197,7 @@ func (s *Server) handleRemoveFeed(w http.ResponseWriter, r *http.Request) {
170197 }
171198 }
172199
173- if err := s.db.DeleteSubscription(r.Context(), user.DID, feedURL); err != nil {
200+ if err := s.dbs.Articles.DeleteSubscription(r.Context(), user.DID, feedURL); err != nil {
174201 s.logger.Error("failed to delete subscription", "error", err)
175202 http.Error(w, err.Error(), http.StatusInternalServerError)
176203 return
@@ -182,7 +209,10 @@ func (s *Server) handleRemoveFeed(w http.ResponseWriter, r *http.Request) {
182209 func (s *Server) handleClearAllSubscriptions(w http.ResponseWriter, r *http.Request) {
183210 user := currentUser(r)
184211
185- subs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 1000, 0)
212+ subs, err := s.dbs.Articles.ListSubscriptions(r.Context(), user.DID, "", 1000, 0)
213+ if err != nil {
214+ s.logger.Warn("failed to list subscriptions", "error", err, "did", user.DID)
215+ }
186216 if client := s.pdsClientForUser(r); client != nil {
187217 for _, sub := range subs {
188218 if sub.URI.Valid {
@@ -196,7 +226,7 @@ func (s *Server) handleClearAllSubscriptions(w http.ResponseWriter, r *http.Requ
196226 }
197227 }
198228
199- if err := s.db.DeleteAllSubscriptions(r.Context(), user.DID); err != nil {
229+ if err := s.dbs.Articles.DeleteAllSubscriptions(r.Context(), user.DID); err != nil {
200230 s.logger.Error("failed to clear subscriptions", "error", err)
201231 http.Error(w, err.Error(), http.StatusInternalServerError)
202232 return
@@ -231,14 +261,16 @@ func (s *Server) handleOPMLUpload(w http.ResponseWriter, r *http.Request) {
231261 SiteURL: nullString(fu.SiteURL),
232262 Description: nullString(fu.Description),
233263 }
234- if upsertErr := s.db.UpsertFeed(r.Context(), f); upsertErr != nil {
264+ if upsertErr := s.dbs.Articles.UpsertFeed(r.Context(), f); upsertErr != nil {
235265 s.logger.Error("failed to upsert feed", "error", upsertErr)
236266 continue
237267 }
238268
239269 go func(feedURL, siteURL string) {
240270 if fav := feed.ResolveFavicon(context.Background(), feedURL, siteURL); fav != "" {
241- _ = s.db.UpdateFeedFavicon(context.Background(), feedURL, fav)
271+ if err := s.dbs.Articles.UpdateFeedFavicon(context.Background(), feedURL, fav); err != nil {
272+ s.logger.Warn("failed to update favicon", "error", err, "feed", feedURL)
273+ }
242274 }
243275 }(fu.URL, fu.SiteURL)
244276
@@ -259,7 +291,7 @@ func (s *Server) handleOPMLUpload(w http.ResponseWriter, r *http.Request) {
259291 subCID = cid
260292 }
261293
262- if subErr := s.db.CreateSubscription(r.Context(), user.DID, fu.URL, fu.Title, fu.Category, subURI, subCID); subErr != nil {
294+ if subErr := s.dbs.Articles.CreateSubscription(r.Context(), user.DID, fu.URL, fu.Title, fu.Category, subURI, subCID); subErr != nil {
263295 if !errors.Is(subErr, db.ErrDuplicateSubscription) {
264296 s.logger.Error("failed to create subscription", "error", subErr)
265297 }
@@ -274,7 +306,10 @@ func (s *Server) handleOPMLUpload(w http.ResponseWriter, r *http.Request) {
274306
275307 func (s *Server) handleOPMLDownload(w http.ResponseWriter, r *http.Request) {
276308 user := currentUser(r)
277- subs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 1000, 0)
309+ subs, err := s.dbs.Articles.ListSubscriptions(r.Context(), user.DID, "", 1000, 0)
310+ if err != nil {
311+ s.logger.Warn("failed to list subscriptions", "error", err, "did", user.DID)
312+ }
278313
279314 var feedURLs []feed.FeedURL
280315 for _, sub := range subs {
@@ -298,7 +333,10 @@ func (s *Server) handleOPMLDownload(w http.ResponseWriter, r *http.Request) {
298333
299334 func (s *Server) handleFeedList(w http.ResponseWriter, r *http.Request) {
300335 user := currentUser(r)
301- subs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 100, 0)
336+ subs, err := s.dbs.Articles.ListSubscriptions(r.Context(), user.DID, "", 100, 0)
337+ if err != nil {
338+ s.logger.Warn("failed to list subscriptions", "error", err, "did", user.DID)
339+ }
302340 s.render(w, r, "feeds.html", map[string]any{
303341 "User": user,
304342 "Subscriptions": subs,
@@ -308,7 +346,10 @@ func (s *Server) handleFeedList(w http.ResponseWriter, r *http.Request) {
308346 func (s *Server) handleRefreshFeeds(w http.ResponseWriter, r *http.Request) {
309347 user := currentUser(r)
310348
311- subs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 100, 0)
349+ subs, err := s.dbs.Articles.ListSubscriptions(r.Context(), user.DID, "", 100, 0)
350+ if err != nil {
351+ s.logger.Warn("failed to list subscriptions", "error", err, "did", user.DID)
352+ }
312353 seen := make(map[string]bool)
313354 for _, sub := range subs {
314355 if seen[sub.FeedURL] {
@@ -316,8 +357,9 @@ func (s *Server) handleRefreshFeeds(w http.ResponseWriter, r *http.Request) {
316357 }
317358 seen[sub.FeedURL] = true
318359
319- f, err := s.db.GetFeed(r.Context(), sub.FeedURL)
360+ f, err := s.dbs.Articles.GetFeed(r.Context(), sub.FeedURL)
320361 if err != nil {
362+ s.logger.Warn("failed to get feed", "error", err, "feed", sub.FeedURL)
321363 continue
322364 }
323365 ff := &feed.Feed{
@@ -332,7 +374,10 @@ func (s *Server) handleRefreshFeeds(w http.ResponseWriter, r *http.Request) {
332374 s.scheduler.FetchFeed(r.Context(), ff)
333375 }
334376
335- subs, _ = s.db.ListSubscriptions(r.Context(), user.DID, "", 100, 0)
377+ subs, err = s.dbs.Articles.ListSubscriptions(r.Context(), user.DID, "", 100, 0)
378+ if err != nil {
379+ s.logger.Warn("failed to list subscriptions", "error", err, "did", user.DID)
380+ }
336381 s.render(w, r, "feed-list.html", map[string]any{
337382 "User": user,
338383 "Subscriptions": subs,
@@ -346,7 +391,7 @@ func (s *Server) handleRetryFeed(w http.ResponseWriter, r *http.Request) {
346391 return
347392 }
348393
349- f, err := s.db.GetFeed(r.Context(), feedURL)
394+ f, err := s.dbs.Articles.GetFeed(r.Context(), feedURL)
350395 if err != nil {
351396 http.Error(w, "feed not found", http.StatusNotFound)
352397 return
@@ -364,7 +409,10 @@ func (s *Server) handleRetryFeed(w http.ResponseWriter, r *http.Request) {
364409 s.scheduler.FetchFeed(r.Context(), ff)
365410
366411 user := currentUser(r)
367- deadFeeds, _ := s.db.ListDeadFeeds(r.Context(), user.DID, 7)
412+ deadFeeds, err := s.dbs.Articles.ListDeadFeeds(r.Context(), user.DID, 7)
413+ if err != nil {
414+ s.logger.Warn("failed to list dead feeds", "error", err, "did", user.DID)
415+ }
368416 if len(deadFeeds) == 0 {
369417 w.Header().Set("Content-Type", "text/html")
370418 w.Write([]byte(""))
@@ -16,29 +16,53 @@ import (
16 func (s *Server) handleFeeds(w http.ResponseWriter, r *http.Request) {16 func (s *Server) handleFeeds(w http.ResponseWriter, r *http.Request) {
17 user := currentUser(r)17 user := currentUser(r)
18 category := r.URL.Query().Get("category")18 category := r.URL.Query().Get("category")
19+ ctx := r.Context()
19 20
20 page := pageFromRequest(r, 50)21 page := pageFromRequest(r, 50)
21- subs, _ := s.db.ListSubscriptions(r.Context(), user.DID, category, page.Limit()+1, page.Offset())22+ subs, err := s.dbs.Articles.ListSubscriptions(ctx, user.DID, category, page.Limit()+1, page.Offset())
23+ if err != nil {
24+ s.logger.Warn("failed to list subscriptions", "error", err, "did", user.DID)
25+ }
22 totalFetched := len(subs)26 totalFetched := len(subs)
23 page = page.Paginate(totalFetched)27 page = page.Paginate(totalFetched)
24 if page.HasNext {28 if page.HasNext {
25 subs = subs[:page.PageSize]29 subs = subs[:page.PageSize]
26 }30 }
27 31
28- allSubs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 1000, 0)32+ allSubs, err := s.dbs.Articles.ListSubscriptions(ctx, user.DID, "", 1000, 0)
29- feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 6)33+ if err != nil {
30- peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5)34+ s.logger.Warn("failed to list all subscriptions", "error", err, "did", user.DID)
35+ }
36+
37+ feedRecs, err := s.engine.GetFeedRecommendations(ctx, user.DID, 6)
38+ if err != nil {
39+ s.logger.Warn("failed to get feed recommendations", "error", err, "did", user.DID)
40+ }
41+
42+ peopleRecs, err := s.engine.GetPeopleRecommendations(ctx, user.DID, 5)
43+ if err != nil {
44+ s.logger.Warn("failed to get people recommendations", "error", err, "did", user.DID)
45+ }
31 46
32 if len(feedRecs) > 0 {47 if len(feedRecs) > 0 {
33 impressions := make([]cluster.Impression, len(feedRecs))48 impressions := make([]cluster.Impression, len(feedRecs))
34 for i, rec := range feedRecs {49 for i, rec := range feedRecs {
35 impressions[i] = cluster.Impression{TargetType: "feed", TargetID: rec.FeedURL}50 impressions[i] = cluster.Impression{TargetType: "feed", TargetID: rec.FeedURL}
36 }51 }
37- _ = s.engine.RecordImpressions(r.Context(), user.DID, impressions)52+ if err := s.engine.RecordImpressions(ctx, user.DID, impressions); err != nil {
53+ s.logger.Warn("failed to record impressions", "error", err)
54+ }
55+ }
56+
57+ deadFeeds, err := s.dbs.Articles.ListDeadFeeds(ctx, user.DID, 7)
58+ if err != nil {
59+ s.logger.Warn("failed to list dead feeds", "error", err, "did", user.DID)
38 }60 }
39- deadFeeds, _ := s.db.ListDeadFeeds(r.Context(), user.DID, 7)
40 61
41- categories, _ := s.db.GetCategories(r.Context(), user.DID)62+ categories, err := s.dbs.Articles.GetCategories(ctx, user.DID)
63+ if err != nil {
64+ s.logger.Warn("failed to get categories", "error", err, "did", user.DID)
65+ }
42 66
43 s.render(w, r, "feeds.html", map[string]any{67 s.render(w, r, "feeds.html", map[string]any{
44 "User": user,68 "User": user,
@@ -80,7 +104,7 @@ func (s *Server) handleAddFeed(w http.ResponseWriter, r *http.Request) {
80 if faviconURL == "" {104 if faviconURL == "" {
81 go func() {105 go func() {
82 if f := feed.ResolveFavicon(context.Background(), feedURL, result.Feed.SiteURL); f != "" {106 if f := feed.ResolveFavicon(context.Background(), feedURL, result.Feed.SiteURL); f != "" {
83- _ = s.db.UpdateFeedFavicon(context.Background(), feedURL, f)107+ _ = s.dbs.Articles.UpdateFeedFavicon(context.Background(), feedURL, f)
84 }108 }
85 }()109 }()
86 }110 }
@@ -94,7 +118,7 @@ func (s *Server) handleAddFeed(w http.ResponseWriter, r *http.Request) {
94 FeedType: nullString(result.Feed.Type),118 FeedType: nullString(result.Feed.Type),
95 FaviconURL: nullString(faviconURL),119 FaviconURL: nullString(faviconURL),
96 }120 }
97- if err := s.db.UpsertFeed(r.Context(), f); err != nil {121+ if err := s.dbs.Articles.UpsertFeed(r.Context(), f); err != nil {
98 s.logger.Error("failed to upsert feed", "error", err)122 s.logger.Error("failed to upsert feed", "error", err)
99 http.Error(w, err.Error(), http.StatusInternalServerError)123 http.Error(w, err.Error(), http.StatusInternalServerError)
100 return124 return
@@ -118,7 +142,7 @@ func (s *Server) handleAddFeed(w http.ResponseWriter, r *http.Request) {
118 subCID = cid142 subCID = cid
119 }143 }
120 144
121- if err := s.db.CreateSubscription(r.Context(), user.DID, feedURL, feedTitle, category, subURI, subCID); err != nil {145+ if err := s.dbs.Articles.CreateSubscription(r.Context(), user.DID, feedURL, feedTitle, category, subURI, subCID); err != nil {
122 if errors.Is(err, db.ErrDuplicateSubscription) {146 if errors.Is(err, db.ErrDuplicateSubscription) {
123 http.Error(w, "Already subscribed to this feed.", http.StatusConflict)147 http.Error(w, "Already subscribed to this feed.", http.StatusConflict)
124 return148 return
@@ -128,12 +152,15 @@ func (s *Server) handleAddFeed(w http.ResponseWriter, r *http.Request) {
128 return152 return
129 }153 }
130 154
131- _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "feed", feedURL)155+ if err := s.engine.MarkImpressionActed(r.Context(), user.DID, "feed", feedURL); err != nil {
156+ s.logger.Warn("failed to mark impression acted", "error", err)
157+ }
132 sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID))158 sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID))
133 s.engine.RewardSignal(r.Context(), user.DID, sig)159 s.engine.RewardSignal(r.Context(), user.DID, sig)
134 160
135- sub, _ := s.db.GetSubscription(r.Context(), user.DID, feedURL)161+ sub, err := s.dbs.Articles.GetSubscription(r.Context(), user.DID, feedURL)
136- if sub == nil {162+ if err != nil {
163+ s.logger.Warn("failed to get subscription", "error", err)
137 w.WriteHeader(http.StatusNoContent)164 w.WriteHeader(http.StatusNoContent)
138 return165 return
139 }166 }
@@ -156,7 +183,7 @@ func (s *Server) handleRemoveFeed(w http.ResponseWriter, r *http.Request) {
156 return183 return
157 }184 }
158 185
159- sub, err := s.db.GetSubscription(r.Context(), user.DID, feedURL)186+ sub, err := s.dbs.Articles.GetSubscription(r.Context(), user.DID, feedURL)
160 if err == nil && sub.URI.Valid {187 if err == nil && sub.URI.Valid {
161 if client := s.pdsClientForUser(r); client != nil {188 if client := s.pdsClientForUser(r); client != nil {
162 parsed, ok := atproto.ParseRecordURI(sub.URI.String)189 parsed, ok := atproto.ParseRecordURI(sub.URI.String)
@@ -170,7 +197,7 @@ func (s *Server) handleRemoveFeed(w http.ResponseWriter, r *http.Request) {
170 }197 }
171 }198 }
172 199
173- if err := s.db.DeleteSubscription(r.Context(), user.DID, feedURL); err != nil {200+ if err := s.dbs.Articles.DeleteSubscription(r.Context(), user.DID, feedURL); err != nil {
174 s.logger.Error("failed to delete subscription", "error", err)201 s.logger.Error("failed to delete subscription", "error", err)
175 http.Error(w, err.Error(), http.StatusInternalServerError)202 http.Error(w, err.Error(), http.StatusInternalServerError)
176 return203 return
@@ -182,7 +209,10 @@ func (s *Server) handleRemoveFeed(w http.ResponseWriter, r *http.Request) {
182 func (s *Server) handleClearAllSubscriptions(w http.ResponseWriter, r *http.Request) {209 func (s *Server) handleClearAllSubscriptions(w http.ResponseWriter, r *http.Request) {
183 user := currentUser(r)210 user := currentUser(r)
184 211
185- subs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 1000, 0)212+ subs, err := s.dbs.Articles.ListSubscriptions(r.Context(), user.DID, "", 1000, 0)
213+ if err != nil {
214+ s.logger.Warn("failed to list subscriptions", "error", err, "did", user.DID)
215+ }
186 if client := s.pdsClientForUser(r); client != nil {216 if client := s.pdsClientForUser(r); client != nil {
187 for _, sub := range subs {217 for _, sub := range subs {
188 if sub.URI.Valid {218 if sub.URI.Valid {
@@ -196,7 +226,7 @@ func (s *Server) handleClearAllSubscriptions(w http.ResponseWriter, r *http.Requ
196 }226 }
197 }227 }
198 228
199- if err := s.db.DeleteAllSubscriptions(r.Context(), user.DID); err != nil {229+ if err := s.dbs.Articles.DeleteAllSubscriptions(r.Context(), user.DID); err != nil {
200 s.logger.Error("failed to clear subscriptions", "error", err)230 s.logger.Error("failed to clear subscriptions", "error", err)
201 http.Error(w, err.Error(), http.StatusInternalServerError)231 http.Error(w, err.Error(), http.StatusInternalServerError)
202 return232 return
@@ -231,14 +261,16 @@ func (s *Server) handleOPMLUpload(w http.ResponseWriter, r *http.Request) {
231 SiteURL: nullString(fu.SiteURL),261 SiteURL: nullString(fu.SiteURL),
232 Description: nullString(fu.Description),262 Description: nullString(fu.Description),
233 }263 }
234- if upsertErr := s.db.UpsertFeed(r.Context(), f); upsertErr != nil {264+ if upsertErr := s.dbs.Articles.UpsertFeed(r.Context(), f); upsertErr != nil {
235 s.logger.Error("failed to upsert feed", "error", upsertErr)265 s.logger.Error("failed to upsert feed", "error", upsertErr)
236 continue266 continue
237 }267 }
238 268
239 go func(feedURL, siteURL string) {269 go func(feedURL, siteURL string) {
240 if fav := feed.ResolveFavicon(context.Background(), feedURL, siteURL); fav != "" {270 if fav := feed.ResolveFavicon(context.Background(), feedURL, siteURL); fav != "" {
241- _ = s.db.UpdateFeedFavicon(context.Background(), feedURL, fav)271+ if err := s.dbs.Articles.UpdateFeedFavicon(context.Background(), feedURL, fav); err != nil {
272+ s.logger.Warn("failed to update favicon", "error", err, "feed", feedURL)
273+ }
242 }274 }
243 }(fu.URL, fu.SiteURL)275 }(fu.URL, fu.SiteURL)
244 276
@@ -259,7 +291,7 @@ func (s *Server) handleOPMLUpload(w http.ResponseWriter, r *http.Request) {
259 subCID = cid291 subCID = cid
260 }292 }
261 293
262- if subErr := s.db.CreateSubscription(r.Context(), user.DID, fu.URL, fu.Title, fu.Category, subURI, subCID); subErr != nil {294+ if subErr := s.dbs.Articles.CreateSubscription(r.Context(), user.DID, fu.URL, fu.Title, fu.Category, subURI, subCID); subErr != nil {
263 if !errors.Is(subErr, db.ErrDuplicateSubscription) {295 if !errors.Is(subErr, db.ErrDuplicateSubscription) {
264 s.logger.Error("failed to create subscription", "error", subErr)296 s.logger.Error("failed to create subscription", "error", subErr)
265 }297 }
@@ -274,7 +306,10 @@ func (s *Server) handleOPMLUpload(w http.ResponseWriter, r *http.Request) {
274 306
275 func (s *Server) handleOPMLDownload(w http.ResponseWriter, r *http.Request) {307 func (s *Server) handleOPMLDownload(w http.ResponseWriter, r *http.Request) {
276 user := currentUser(r)308 user := currentUser(r)
277- subs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 1000, 0)309+ subs, err := s.dbs.Articles.ListSubscriptions(r.Context(), user.DID, "", 1000, 0)
310+ if err != nil {
311+ s.logger.Warn("failed to list subscriptions", "error", err, "did", user.DID)
312+ }
278 313
279 var feedURLs []feed.FeedURL314 var feedURLs []feed.FeedURL
280 for _, sub := range subs {315 for _, sub := range subs {
@@ -298,7 +333,10 @@ func (s *Server) handleOPMLDownload(w http.ResponseWriter, r *http.Request) {
298 333
299 func (s *Server) handleFeedList(w http.ResponseWriter, r *http.Request) {334 func (s *Server) handleFeedList(w http.ResponseWriter, r *http.Request) {
300 user := currentUser(r)335 user := currentUser(r)
301- subs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 100, 0)336+ subs, err := s.dbs.Articles.ListSubscriptions(r.Context(), user.DID, "", 100, 0)
337+ if err != nil {
338+ s.logger.Warn("failed to list subscriptions", "error", err, "did", user.DID)
339+ }
302 s.render(w, r, "feeds.html", map[string]any{340 s.render(w, r, "feeds.html", map[string]any{
303 "User": user,341 "User": user,
304 "Subscriptions": subs,342 "Subscriptions": subs,
@@ -308,7 +346,10 @@ func (s *Server) handleFeedList(w http.ResponseWriter, r *http.Request) {
308 func (s *Server) handleRefreshFeeds(w http.ResponseWriter, r *http.Request) {346 func (s *Server) handleRefreshFeeds(w http.ResponseWriter, r *http.Request) {
309 user := currentUser(r)347 user := currentUser(r)
310 348
311- subs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 100, 0)349+ subs, err := s.dbs.Articles.ListSubscriptions(r.Context(), user.DID, "", 100, 0)
350+ if err != nil {
351+ s.logger.Warn("failed to list subscriptions", "error", err, "did", user.DID)
352+ }
312 seen := make(map[string]bool)353 seen := make(map[string]bool)
313 for _, sub := range subs {354 for _, sub := range subs {
314 if seen[sub.FeedURL] {355 if seen[sub.FeedURL] {
@@ -316,8 +357,9 @@ func (s *Server) handleRefreshFeeds(w http.ResponseWriter, r *http.Request) {
316 }357 }
317 seen[sub.FeedURL] = true358 seen[sub.FeedURL] = true
318 359
319- f, err := s.db.GetFeed(r.Context(), sub.FeedURL)360+ f, err := s.dbs.Articles.GetFeed(r.Context(), sub.FeedURL)
320 if err != nil {361 if err != nil {
362+ s.logger.Warn("failed to get feed", "error", err, "feed", sub.FeedURL)
321 continue363 continue
322 }364 }
323 ff := &feed.Feed{365 ff := &feed.Feed{
@@ -332,7 +374,10 @@ func (s *Server) handleRefreshFeeds(w http.ResponseWriter, r *http.Request) {
332 s.scheduler.FetchFeed(r.Context(), ff)374 s.scheduler.FetchFeed(r.Context(), ff)
333 }375 }
334 376
335- subs, _ = s.db.ListSubscriptions(r.Context(), user.DID, "", 100, 0)377+ subs, err = s.dbs.Articles.ListSubscriptions(r.Context(), user.DID, "", 100, 0)
378+ if err != nil {
379+ s.logger.Warn("failed to list subscriptions", "error", err, "did", user.DID)
380+ }
336 s.render(w, r, "feed-list.html", map[string]any{381 s.render(w, r, "feed-list.html", map[string]any{
337 "User": user,382 "User": user,
338 "Subscriptions": subs,383 "Subscriptions": subs,
@@ -346,7 +391,7 @@ func (s *Server) handleRetryFeed(w http.ResponseWriter, r *http.Request) {
346 return391 return
347 }392 }
348 393
349- f, err := s.db.GetFeed(r.Context(), feedURL)394+ f, err := s.dbs.Articles.GetFeed(r.Context(), feedURL)
350 if err != nil {395 if err != nil {
351 http.Error(w, "feed not found", http.StatusNotFound)396 http.Error(w, "feed not found", http.StatusNotFound)
352 return397 return
@@ -364,7 +409,10 @@ func (s *Server) handleRetryFeed(w http.ResponseWriter, r *http.Request) {
364 s.scheduler.FetchFeed(r.Context(), ff)409 s.scheduler.FetchFeed(r.Context(), ff)
365 410
366 user := currentUser(r)411 user := currentUser(r)
367- deadFeeds, _ := s.db.ListDeadFeeds(r.Context(), user.DID, 7)412+ deadFeeds, err := s.dbs.Articles.ListDeadFeeds(r.Context(), user.DID, 7)
413+ if err != nil {
414+ s.logger.Warn("failed to list dead feeds", "error", err, "did", user.DID)
415+ }
368 if len(deadFeeds) == 0 {416 if len(deadFeeds) == 0 {
369 w.Header().Set("Content-Type", "text/html")417 w.Header().Set("Content-Type", "text/html")
370 w.Write([]byte(""))418 w.Write([]byte(""))
modified internal/server/profile_handler.go +25 -9
@@ -10,18 +10,20 @@ import (
1010 )
1111
1212 func (s *Server) handleProfile(w http.ResponseWriter, r *http.Request) {
13+ ctx := r.Context()
1314 param := chi.URLParam(r, "did")
1415
1516 var did string
1617 if strings.HasPrefix(param, "did:") {
1718 did = param
1819 } else {
19- profileUser, err := s.db.GetUserByHandle(r.Context(), param)
20+ profileUser, err := s.dbs.Articles.GetUserByHandle(ctx, param)
2021 if err == nil {
2122 did = profileUser.DID
2223 } else {
23- resolved, err := atproto.ResolveHandle(r.Context(), param)
24+ resolved, err := atproto.ResolveHandle(ctx, param)
2425 if err != nil {
26+ s.logger.Warn("failed to resolve handle", "error", err, "handle", param)
2527 http.Error(w, "handle not found", http.StatusNotFound)
2628 return
2729 }
@@ -29,24 +31,38 @@ func (s *Server) handleProfile(w http.ResponseWriter, r *http.Request) {
2931 }
3032 }
3133
32- profileUser, err := s.db.GetUser(r.Context(), did)
34+ profileUser, err := s.dbs.Articles.GetUser(ctx, did)
3335 if err != nil {
34- http.Error(w, err.Error(), http.StatusNotFound)
36+ s.logger.Warn("failed to get user", "error", err, "did", did)
37+ http.Error(w, "user not found", http.StatusNotFound)
3538 return
3639 }
3740
3841 if !profileUser.AvatarURL.Valid || profileUser.AvatarURL.String == "" {
39- _, displayName, avatarURL, err := atproto.FetchProfile(r.Context(), did)
42+ _, displayName, avatarURL, err := atproto.FetchProfile(ctx, did)
4043 if err == nil && avatarURL != "" {
41- _ = s.db.UpdateUserProfile(r.Context(), did, displayName, avatarURL)
44+ if err := s.dbs.Articles.UpdateUserProfile(ctx, did, displayName, avatarURL); err != nil {
45+ s.logger.Warn("failed to update user profile", "error", err, "did", did)
46+ }
4247 profileUser.DisplayName = nullString(displayName)
4348 profileUser.AvatarURL = nullString(avatarURL)
4449 }
4550 }
4651
47- subs, _ := s.db.ListSubscriptions(r.Context(), did, "", 50, 0)
48- annotations, _ := s.db.ListAnnotations(r.Context(), "", "", did, 50, 0)
49- subCount, _ := s.db.GetSubscriptionCount(r.Context(), did)
52+ subs, err := s.dbs.Articles.ListSubscriptions(ctx, did, "", 50, 0)
53+ if err != nil {
54+ s.logger.Warn("failed to list subscriptions", "error", err, "did", did)
55+ }
56+
57+ annotations, err := s.dbs.Articles.ListAnnotations(ctx, "", "", did, 50, 0)
58+ if err != nil {
59+ s.logger.Warn("failed to list annotations", "error", err, "did", did)
60+ }
61+
62+ subCount, err := s.dbs.Articles.GetSubscriptionCount(ctx, did)
63+ if err != nil {
64+ s.logger.Warn("failed to get subscription count", "error", err, "did", did)
65+ }
5066
5167 user := currentUser(r)
5268
@@ -10,18 +10,20 @@ import (
10 )10 )
11 11
12 func (s *Server) handleProfile(w http.ResponseWriter, r *http.Request) {12 func (s *Server) handleProfile(w http.ResponseWriter, r *http.Request) {
13+ ctx := r.Context()
13 param := chi.URLParam(r, "did")14 param := chi.URLParam(r, "did")
14 15
15 var did string16 var did string
16 if strings.HasPrefix(param, "did:") {17 if strings.HasPrefix(param, "did:") {
17 did = param18 did = param
18 } else {19 } else {
19- profileUser, err := s.db.GetUserByHandle(r.Context(), param)20+ profileUser, err := s.dbs.Articles.GetUserByHandle(ctx, param)
20 if err == nil {21 if err == nil {
21 did = profileUser.DID22 did = profileUser.DID
22 } else {23 } else {
23- resolved, err := atproto.ResolveHandle(r.Context(), param)24+ resolved, err := atproto.ResolveHandle(ctx, param)
24 if err != nil {25 if err != nil {
26+ s.logger.Warn("failed to resolve handle", "error", err, "handle", param)
25 http.Error(w, "handle not found", http.StatusNotFound)27 http.Error(w, "handle not found", http.StatusNotFound)
26 return28 return
27 }29 }
@@ -29,24 +31,38 @@ func (s *Server) handleProfile(w http.ResponseWriter, r *http.Request) {
29 }31 }
30 }32 }
31 33
32- profileUser, err := s.db.GetUser(r.Context(), did)34+ profileUser, err := s.dbs.Articles.GetUser(ctx, did)
33 if err != nil {35 if err != nil {
34- http.Error(w, err.Error(), http.StatusNotFound)36+ s.logger.Warn("failed to get user", "error", err, "did", did)
37+ http.Error(w, "user not found", http.StatusNotFound)
35 return38 return
36 }39 }
37 40
38 if !profileUser.AvatarURL.Valid || profileUser.AvatarURL.String == "" {41 if !profileUser.AvatarURL.Valid || profileUser.AvatarURL.String == "" {
39- _, displayName, avatarURL, err := atproto.FetchProfile(r.Context(), did)42+ _, displayName, avatarURL, err := atproto.FetchProfile(ctx, did)
40 if err == nil && avatarURL != "" {43 if err == nil && avatarURL != "" {
41- _ = s.db.UpdateUserProfile(r.Context(), did, displayName, avatarURL)44+ if err := s.dbs.Articles.UpdateUserProfile(ctx, did, displayName, avatarURL); err != nil {
45+ s.logger.Warn("failed to update user profile", "error", err, "did", did)
46+ }
42 profileUser.DisplayName = nullString(displayName)47 profileUser.DisplayName = nullString(displayName)
43 profileUser.AvatarURL = nullString(avatarURL)48 profileUser.AvatarURL = nullString(avatarURL)
44 }49 }
45 }50 }
46 51
47- subs, _ := s.db.ListSubscriptions(r.Context(), did, "", 50, 0)52+ subs, err := s.dbs.Articles.ListSubscriptions(ctx, did, "", 50, 0)
48- annotations, _ := s.db.ListAnnotations(r.Context(), "", "", did, 50, 0)53+ if err != nil {
49- subCount, _ := s.db.GetSubscriptionCount(r.Context(), did)54+ s.logger.Warn("failed to list subscriptions", "error", err, "did", did)
55+ }
56+
57+ annotations, err := s.dbs.Articles.ListAnnotations(ctx, "", "", did, 50, 0)
58+ if err != nil {
59+ s.logger.Warn("failed to list annotations", "error", err, "did", did)
60+ }
61+
62+ subCount, err := s.dbs.Articles.GetSubscriptionCount(ctx, did)
63+ if err != nil {
64+ s.logger.Warn("failed to get subscription count", "error", err, "did", did)
65+ }
50 66
51 user := currentUser(r)67 user := currentUser(r)
52 68
modified internal/server/server.go +15 -15
@@ -56,7 +56,7 @@ func splitString(s, sep string) []string {
5656 }
5757
5858 type Server struct {
59- db *db.DB
59+ dbs *db.Databases
6060 router *chi.Mux
6161 templates *template.Template
6262 logger *slog.Logger
@@ -70,8 +70,8 @@ type Server struct {
7070 callbackURL string
7171 }
7272
73-func New(database *db.DB, clientID, callbackURL, addr string, scheduler *feed.Scheduler, engine *cluster.Engine, logger *slog.Logger) *Server {
74- oauthStore := db.NewOAuthStore(database)
73+func New(dbs *db.Databases, clientID, callbackURL, addr string, scheduler *feed.Scheduler, engine *cluster.Engine, logger *slog.Logger) *Server {
74+ oauthStore := db.NewOAuthStore(dbs.Users)
7575
7676 var config oauth.ClientConfig
7777 if clientID == "" {
@@ -87,7 +87,7 @@ func New(database *db.DB, clientID, callbackURL, addr string, scheduler *feed.Sc
8787 oauthClient := oauth.NewClientApp(&config, oauthStore)
8888
8989 s := &Server{
90- db: database,
90+ dbs: dbs,
9191 router: chi.NewRouter(),
9292 logger: logger,
9393 oauth: oauthClient,
@@ -200,7 +200,7 @@ func (s *Server) setupRoutes() {
200200 s.router.Post("/auth/logout", s.handleAuthLogout)
201201 s.router.Get("/oauth/client-metadata", s.handleOAuthClientMetadata)
202202
203- xrpc := atproto.NewXRPCHandler(s.db.DB, s.engine)
203+ xrpc := atproto.NewXRPCHandler(s.dbs.Articles.DB, s.engine)
204204 s.router.Get("/xrpc/at.glean.listSubscriptions", xrpc.ListSubscriptions)
205205 s.router.Get("/xrpc/at.glean.listAnnotations", xrpc.ListAnnotations)
206206 s.router.Get("/xrpc/at.glean.listLikes", xrpc.ListLikes)
@@ -382,11 +382,11 @@ func (s *Server) syncUserInBackground(userDID string, client *atproto.Client) {
382382 defer cancel()
383383
384384 isNewUser := false
385- if count, err := s.db.GetSubscriptionCount(ctx, userDID); err == nil && count == 0 {
385+ if count, err := s.dbs.Articles.GetSubscriptionCount(ctx, userDID); err == nil && count == 0 {
386386 isNewUser = true
387387 }
388388
389- sync := atproto.NewSync(s.db, client, s.logger)
389+ sync := atproto.NewSync(s.dbs.Articles, s.dbs.Users, client, s.logger)
390390 if err := sync.Run(ctx, userDID); err != nil {
391391 s.logger.Error("background sync failed", "error", err, "did", userDID)
392392 }
@@ -398,7 +398,7 @@ func (s *Server) syncUserInBackground(userDID string, client *atproto.Client) {
398398 }
399399
400400 func (s *Server) refreshUserFeeds(ctx context.Context, userDID string) {
401- subs, err := s.db.ListSubscriptions(ctx, userDID, "", 1000, 0)
401+ subs, err := s.dbs.Articles.ListSubscriptions(ctx, userDID, "", 1000, 0)
402402 if err != nil {
403403 s.logger.Error("failed to list subscriptions for initial fetch", "error", err, "did", userDID)
404404 return
@@ -415,7 +415,7 @@ func (s *Server) refreshUserFeeds(ctx context.Context, userDID string) {
415415 }
416416 seen[sub.FeedURL] = true
417417
418- f, err := s.db.GetFeed(ctx, sub.FeedURL)
418+ f, err := s.dbs.Articles.GetFeed(ctx, sub.FeedURL)
419419 if err != nil {
420420 continue
421421 }
@@ -458,7 +458,7 @@ func (s *Server) BackfillFromCollectionDir(ctx context.Context, collectionDirURL
458458 return
459459 }
460460
461- existing, err := s.db.ListUserDIDs(ctx)
461+ existing, err := s.dbs.Users.ListUserDIDs(ctx)
462462 if err != nil {
463463 s.logger.Error("failed to list existing users", "error", err)
464464 return
@@ -499,7 +499,7 @@ func (s *Server) BackfillFromCollectionDir(ctx context.Context, collectionDirURL
499499 avatarURL = avatar
500500 }
501501
502- if _, err := s.db.CreateUser(ctx, did, handle, displayName, avatarURL); err != nil {
502+ if _, err := s.dbs.Users.CreateUser(ctx, did, handle, displayName, avatarURL); err != nil {
503503 s.logger.Error("failed to create user during backfill", "error", err, "did", did)
504504 return
505505 }
@@ -511,7 +511,7 @@ func (s *Server) BackfillFromCollectionDir(ctx context.Context, collectionDirURL
511511 }
512512
513513 client := atproto.NewUnauthenticatedClient(pdsURL)
514- sync := atproto.NewSync(s.db, client, s.logger)
514+ sync := atproto.NewSync(s.dbs.Articles, s.dbs.Users, client, s.logger)
515515 if err := sync.Run(ctx, did); err != nil {
516516 s.logger.Error("backfill sync failed", "error", err, "did", did)
517517 }
@@ -525,7 +525,7 @@ func (s *Server) BackfillFromCollectionDir(ctx context.Context, collectionDirURL
525525 }
526526
527527 func (s *Server) runSyncAll(ctx context.Context) {
528- users, err := s.db.ListUsers(ctx)
528+ users, err := s.dbs.Users.ListUsers(ctx)
529529 if err != nil {
530530 s.logger.Error("failed to list users for sync", "error", err)
531531 return
@@ -549,14 +549,14 @@ func (s *Server) runSyncAll(ctx context.Context) {
549549 }
550550
551551 client := atproto.NewClient(sess.APIClient())
552- sync := atproto.NewSync(s.db, client, s.logger)
552+ sync := atproto.NewSync(s.dbs.Articles, s.dbs.Users, client, s.logger)
553553 if err := sync.Run(ctx, u.DID); err != nil {
554554 metrics.SyncErrors.Inc()
555555 s.logger.Error("periodic sync failed", "error", err, "did", u.DID)
556556 }
557557
558558 if dn, avatar, err := client.GetProfile(ctx, u.DID); err == nil {
559- _ = s.db.UpdateUserProfile(ctx, u.DID, dn, avatar)
559+ _ = s.dbs.Users.UpdateUserProfile(ctx, u.DID, dn, avatar)
560560 }
561561
562562 metrics.SyncRuns.Inc()
@@ -56,7 +56,7 @@ func splitString(s, sep string) []string {
56 }56 }
57 57
58 type Server struct {58 type Server struct {
59- db *db.DB59+ dbs *db.Databases
60 router *chi.Mux60 router *chi.Mux
61 templates *template.Template61 templates *template.Template
62 logger *slog.Logger62 logger *slog.Logger
@@ -70,8 +70,8 @@ type Server struct {
70 callbackURL string70 callbackURL string
71 }71 }
72 72
73-func New(database *db.DB, clientID, callbackURL, addr string, scheduler *feed.Scheduler, engine *cluster.Engine, logger *slog.Logger) *Server {73+func New(dbs *db.Databases, clientID, callbackURL, addr string, scheduler *feed.Scheduler, engine *cluster.Engine, logger *slog.Logger) *Server {
74- oauthStore := db.NewOAuthStore(database)74+ oauthStore := db.NewOAuthStore(dbs.Users)
75 75
76 var config oauth.ClientConfig76 var config oauth.ClientConfig
77 if clientID == "" {77 if clientID == "" {
@@ -87,7 +87,7 @@ func New(database *db.DB, clientID, callbackURL, addr string, scheduler *feed.Sc
87 oauthClient := oauth.NewClientApp(&config, oauthStore)87 oauthClient := oauth.NewClientApp(&config, oauthStore)
88 88
89 s := &Server{89 s := &Server{
90- db: database,90+ dbs: dbs,
91 router: chi.NewRouter(),91 router: chi.NewRouter(),
92 logger: logger,92 logger: logger,
93 oauth: oauthClient,93 oauth: oauthClient,
@@ -200,7 +200,7 @@ func (s *Server) setupRoutes() {
200 s.router.Post("/auth/logout", s.handleAuthLogout)200 s.router.Post("/auth/logout", s.handleAuthLogout)
201 s.router.Get("/oauth/client-metadata", s.handleOAuthClientMetadata)201 s.router.Get("/oauth/client-metadata", s.handleOAuthClientMetadata)
202 202
203- xrpc := atproto.NewXRPCHandler(s.db.DB, s.engine)203+ xrpc := atproto.NewXRPCHandler(s.dbs.Articles.DB, s.engine)
204 s.router.Get("/xrpc/at.glean.listSubscriptions", xrpc.ListSubscriptions)204 s.router.Get("/xrpc/at.glean.listSubscriptions", xrpc.ListSubscriptions)
205 s.router.Get("/xrpc/at.glean.listAnnotations", xrpc.ListAnnotations)205 s.router.Get("/xrpc/at.glean.listAnnotations", xrpc.ListAnnotations)
206 s.router.Get("/xrpc/at.glean.listLikes", xrpc.ListLikes)206 s.router.Get("/xrpc/at.glean.listLikes", xrpc.ListLikes)
@@ -382,11 +382,11 @@ func (s *Server) syncUserInBackground(userDID string, client *atproto.Client) {
382 defer cancel()382 defer cancel()
383 383
384 isNewUser := false384 isNewUser := false
385- if count, err := s.db.GetSubscriptionCount(ctx, userDID); err == nil && count == 0 {385+ if count, err := s.dbs.Articles.GetSubscriptionCount(ctx, userDID); err == nil && count == 0 {
386 isNewUser = true386 isNewUser = true
387 }387 }
388 388
389- sync := atproto.NewSync(s.db, client, s.logger)389+ sync := atproto.NewSync(s.dbs.Articles, s.dbs.Users, client, s.logger)
390 if err := sync.Run(ctx, userDID); err != nil {390 if err := sync.Run(ctx, userDID); err != nil {
391 s.logger.Error("background sync failed", "error", err, "did", userDID)391 s.logger.Error("background sync failed", "error", err, "did", userDID)
392 }392 }
@@ -398,7 +398,7 @@ func (s *Server) syncUserInBackground(userDID string, client *atproto.Client) {
398 }398 }
399 399
400 func (s *Server) refreshUserFeeds(ctx context.Context, userDID string) {400 func (s *Server) refreshUserFeeds(ctx context.Context, userDID string) {
401- subs, err := s.db.ListSubscriptions(ctx, userDID, "", 1000, 0)401+ subs, err := s.dbs.Articles.ListSubscriptions(ctx, userDID, "", 1000, 0)
402 if err != nil {402 if err != nil {
403 s.logger.Error("failed to list subscriptions for initial fetch", "error", err, "did", userDID)403 s.logger.Error("failed to list subscriptions for initial fetch", "error", err, "did", userDID)
404 return404 return
@@ -415,7 +415,7 @@ func (s *Server) refreshUserFeeds(ctx context.Context, userDID string) {
415 }415 }
416 seen[sub.FeedURL] = true416 seen[sub.FeedURL] = true
417 417
418- f, err := s.db.GetFeed(ctx, sub.FeedURL)418+ f, err := s.dbs.Articles.GetFeed(ctx, sub.FeedURL)
419 if err != nil {419 if err != nil {
420 continue420 continue
421 }421 }
@@ -458,7 +458,7 @@ func (s *Server) BackfillFromCollectionDir(ctx context.Context, collectionDirURL
458 return458 return
459 }459 }
460 460
461- existing, err := s.db.ListUserDIDs(ctx)461+ existing, err := s.dbs.Users.ListUserDIDs(ctx)
462 if err != nil {462 if err != nil {
463 s.logger.Error("failed to list existing users", "error", err)463 s.logger.Error("failed to list existing users", "error", err)
464 return464 return
@@ -499,7 +499,7 @@ func (s *Server) BackfillFromCollectionDir(ctx context.Context, collectionDirURL
499 avatarURL = avatar499 avatarURL = avatar
500 }500 }
501 501
502- if _, err := s.db.CreateUser(ctx, did, handle, displayName, avatarURL); err != nil {502+ if _, err := s.dbs.Users.CreateUser(ctx, did, handle, displayName, avatarURL); err != nil {
503 s.logger.Error("failed to create user during backfill", "error", err, "did", did)503 s.logger.Error("failed to create user during backfill", "error", err, "did", did)
504 return504 return
505 }505 }
@@ -511,7 +511,7 @@ func (s *Server) BackfillFromCollectionDir(ctx context.Context, collectionDirURL
511 }511 }
512 512
513 client := atproto.NewUnauthenticatedClient(pdsURL)513 client := atproto.NewUnauthenticatedClient(pdsURL)
514- sync := atproto.NewSync(s.db, client, s.logger)514+ sync := atproto.NewSync(s.dbs.Articles, s.dbs.Users, client, s.logger)
515 if err := sync.Run(ctx, did); err != nil {515 if err := sync.Run(ctx, did); err != nil {
516 s.logger.Error("backfill sync failed", "error", err, "did", did)516 s.logger.Error("backfill sync failed", "error", err, "did", did)
517 }517 }
@@ -525,7 +525,7 @@ func (s *Server) BackfillFromCollectionDir(ctx context.Context, collectionDirURL
525 }525 }
526 526
527 func (s *Server) runSyncAll(ctx context.Context) {527 func (s *Server) runSyncAll(ctx context.Context) {
528- users, err := s.db.ListUsers(ctx)528+ users, err := s.dbs.Users.ListUsers(ctx)
529 if err != nil {529 if err != nil {
530 s.logger.Error("failed to list users for sync", "error", err)530 s.logger.Error("failed to list users for sync", "error", err)
531 return531 return
@@ -549,14 +549,14 @@ func (s *Server) runSyncAll(ctx context.Context) {
549 }549 }
550 550
551 client := atproto.NewClient(sess.APIClient())551 client := atproto.NewClient(sess.APIClient())
552- sync := atproto.NewSync(s.db, client, s.logger)552+ sync := atproto.NewSync(s.dbs.Articles, s.dbs.Users, client, s.logger)
553 if err := sync.Run(ctx, u.DID); err != nil {553 if err := sync.Run(ctx, u.DID); err != nil {
554 metrics.SyncErrors.Inc()554 metrics.SyncErrors.Inc()
555 s.logger.Error("periodic sync failed", "error", err, "did", u.DID)555 s.logger.Error("periodic sync failed", "error", err, "did", u.DID)
556 }556 }
557 557
558 if dn, avatar, err := client.GetProfile(ctx, u.DID); err == nil {558 if dn, avatar, err := client.GetProfile(ctx, u.DID); err == nil {
559- _ = s.db.UpdateUserProfile(ctx, u.DID, dn, avatar)559+ _ = s.dbs.Users.UpdateUserProfile(ctx, u.DID, dn, avatar)
560 }560 }
561 561
562 metrics.SyncRuns.Inc()562 metrics.SyncRuns.Inc()
modified internal/server/session.go +1 -1
@@ -34,7 +34,7 @@ func (s *Server) getUserFromSession(r *http.Request) *db.User {
3434 return nil
3535 }
3636
37- user, err := s.db.GetUser(r.Context(), data.DID)
37+ user, err := s.dbs.Users.GetUser(r.Context(), data.DID)
3838 if err != nil {
3939 return nil
4040 }
@@ -34,7 +34,7 @@ func (s *Server) getUserFromSession(r *http.Request) *db.User {
34 return nil34 return nil
35 }35 }
36 36
37- user, err := s.db.GetUser(r.Context(), data.DID)37+ user, err := s.dbs.Users.GetUser(r.Context(), data.DID)
38 if err != nil {38 if err != nil {
39 return nil39 return nil
40 }40 }
modified main.go +7 -7
@@ -34,26 +34,26 @@ func main() {
3434
3535 logger := slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{Level: slog.LevelInfo}))
3636
37- database, err := db.Open(*dbPath)
37+ dbs, err := db.OpenAll(*dbPath)
3838 if err != nil {
39- logger.Error("failed to open database", "error", err)
39+ logger.Error("failed to open databases", "error", err)
4040 os.Exit(1)
4141 }
42- defer database.Close()
42+ defer dbs.Close()
4343
4444 clientID := envOr("GLEAN_OAUTH_CLIENT_ID", "")
4545 callbackURL := envOr("GLEAN_OAUTH_REDIRECT_URL", "")
4646
47- storeAdapter := db.NewFeedStoreAdapter(database)
47+ storeAdapter := db.NewFeedStoreAdapter(dbs.Articles)
4848 scheduler := feed.NewScheduler(storeAdapter, logger, *fetchInterval, 30*time.Minute)
4949
50- engine := cluster.NewEngine(database.DB, logger)
50+ engine := cluster.NewEngine(dbs.Users.DB, logger)
5151
52- srv := server.New(database, clientID, callbackURL, *addr, scheduler, engine, logger)
52+ srv := server.New(dbs, clientID, callbackURL, *addr, scheduler, engine, logger)
5353
5454 cron := cluster.NewCron(engine, *clusterInterval, logger)
5555
56- handler := atproto.NewStreamDBHandler(database, logger)
56+ handler := atproto.NewStreamDBHandler(dbs.Articles, dbs.Users, logger)
5757 jetstream := atproto.NewJetstreamConsumer(*jetstreamURL, handler.Handle, logger)
5858
5959 ctx, cancel := context.WithCancel(context.Background())
@@ -34,26 +34,26 @@ func main() {
34 34
35 logger := slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{Level: slog.LevelInfo}))35 logger := slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{Level: slog.LevelInfo}))
36 36
37- database, err := db.Open(*dbPath)37+ dbs, err := db.OpenAll(*dbPath)
38 if err != nil {38 if err != nil {
39- logger.Error("failed to open database", "error", err)39+ logger.Error("failed to open databases", "error", err)
40 os.Exit(1)40 os.Exit(1)
41 }41 }
42- defer database.Close()42+ defer dbs.Close()
43 43
44 clientID := envOr("GLEAN_OAUTH_CLIENT_ID", "")44 clientID := envOr("GLEAN_OAUTH_CLIENT_ID", "")
45 callbackURL := envOr("GLEAN_OAUTH_REDIRECT_URL", "")45 callbackURL := envOr("GLEAN_OAUTH_REDIRECT_URL", "")
46 46
47- storeAdapter := db.NewFeedStoreAdapter(database)47+ storeAdapter := db.NewFeedStoreAdapter(dbs.Articles)
48 scheduler := feed.NewScheduler(storeAdapter, logger, *fetchInterval, 30*time.Minute)48 scheduler := feed.NewScheduler(storeAdapter, logger, *fetchInterval, 30*time.Minute)
49 49
50- engine := cluster.NewEngine(database.DB, logger)50+ engine := cluster.NewEngine(dbs.Users.DB, logger)
51 51
52- srv := server.New(database, clientID, callbackURL, *addr, scheduler, engine, logger)52+ srv := server.New(dbs, clientID, callbackURL, *addr, scheduler, engine, logger)
53 53
54 cron := cluster.NewCron(engine, *clusterInterval, logger)54 cron := cluster.NewCron(engine, *clusterInterval, logger)
55 55
56- handler := atproto.NewStreamDBHandler(database, logger)56+ handler := atproto.NewStreamDBHandler(dbs.Articles, dbs.Users, logger)
57 jetstream := atproto.NewJetstreamConsumer(*jetstreamURL, handler.Handle, logger)57 jetstream := atproto.NewJetstreamConsumer(*jetstreamURL, handler.Handle, logger)
58 58
59 ctx, cancel := context.WithCancel(context.Background())59 ctx, cancel := context.WithCancel(context.Background())
modified readme.md +1 -1
@@ -40,7 +40,7 @@ Then open `http://localhost:8080`.
4040 | Variable | Default | What it does |
4141 | -------------------------- | -------------------------- | --------------------------------------------------------- |
4242 | `GLEAN_ADDR` | `:8080` | Listen address |
43-| `GLEAN_DB` | `glean.db` | SQLite database path |
43+| `GLEAN_DB` | `glean.db` | SQLite base path (`_users`, `_articles`, `_recs` suffixes) |
4444 | `GLEAN_JETSTREAM` | `wss://jetstream.glean.at` | Jetstream WebSocket URL |
4545 | `GLEAN_SYNC_INTERVAL` | `1h` | PDS sync interval (Go duration: `30m`, `2h30m`, etc.) |
4646 | `GLEAN_CLUSTER_INTERVAL` | `10m` | Cluster recomputation interval (Go duration) |
@@ -40,7 +40,7 @@ Then open `http://localhost:8080`.
40 | Variable | Default | What it does |40 | Variable | Default | What it does |
41 | -------------------------- | -------------------------- | --------------------------------------------------------- |41 | -------------------------- | -------------------------- | --------------------------------------------------------- |
42 | `GLEAN_ADDR` | `:8080` | Listen address |42 | `GLEAN_ADDR` | `:8080` | Listen address |
43-| `GLEAN_DB` | `glean.db` | SQLite database path |43+| `GLEAN_DB` | `glean.db` | SQLite base path (`_users`, `_articles`, `_recs` suffixes) |
44 | `GLEAN_JETSTREAM` | `wss://jetstream.glean.at` | Jetstream WebSocket URL |44 | `GLEAN_JETSTREAM` | `wss://jetstream.glean.at` | Jetstream WebSocket URL |
45 | `GLEAN_SYNC_INTERVAL` | `1h` | PDS sync interval (Go duration: `30m`, `2h30m`, etc.) |45 | `GLEAN_SYNC_INTERVAL` | `1h` | PDS sync interval (Go duration: `30m`, `2h30m`, etc.) |
46 | `GLEAN_CLUSTER_INTERVAL` | `10m` | Cluster recomputation interval (Go duration) |46 | `GLEAN_CLUSTER_INTERVAL` | `10m` | Cluster recomputation interval (Go duration) |