nandi/gleanpublic⑂ Fork 0
⑂ b8b7aa8
Commits
⬇ Clone ▾
git clone https://git.rickub.com/nandi/glean.git
git clone ssh://git@rickub.com/nandi/glean.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

Refactor recommendation engine and add dismissal logicUnverified

Julien Robert committed 2026-04-22T02:04:40+02:00 Browse files
b8b7aa8 parent: c5ad568
modified docs/specs.md +144 -67
@@ -143,13 +143,13 @@ Glean also indexes records from the Skyreader lexicon (`app.skyreader.feed.subsc
143143
144144 The mapping from Skyreader subscription to Glean subscription:
145145
146-| Skyreader field | Glean field | Notes |
147-| --------------- | ------------- | ------------------------------------ |
148-| `feedUrl` | `feed_url` | Direct mapping |
149-| `title` | `title` | Direct mapping |
150-| `siteUrl` | `site_url` | Stored on the feed record |
151-| `createdAt` | `added_at` | Direct mapping |
152-| _(none)_ | `category` | Empty (Skyreader has no categories) |
146+| Skyreader field | Glean field | Notes |
147+| --------------- | ----------- | ----------------------------------- |
148+| `feedUrl` | `feed_url` | Direct mapping |
149+| `title` | `title` | Direct mapping |
150+| `siteUrl` | `site_url` | Stored on the feed record |
151+| `createdAt` | `added_at` | Direct mapping |
152+| _(none)_ | `category` | Empty (Skyreader has no categories) |
153153
154154 If a Glean subscription already exists for the same `feed_url`, the existing one is kept. If the existing subscription has no URI (was created locally without PDS sync), the Skyreader URI/CID is backfilled.
155155
@@ -661,12 +661,20 @@ CREATE TABLE oauth_sessions (
661661 );
662662 ```
663663
664-Glean has two complementary recommendation signals:
664+Glean uses a multi-signal recommendation system that combines subscription overlap, like patterns, social graph distance, and user behavior feedback.
665665
666-- **Subscriptions** (Jaccard similarity): "Who reads the same feeds?" → feed and people discovery
667-- **Likes** (co-occurrence): "Who likes the same articles?" → article and feed discovery
666+### 7.1 Signals
668667
669-### 7.1 Feed Co-occurrence (Jaccard Similarity)
668+| Signal | Source | Weight (default) | Description |
669+|--------|--------|-------------------|-------------|
670+| Subscription | `subscriptions` | 1.0 | Jaccard over subscriber sets between similar users |
671+| Like | `likes` | 0.5 | Time-decayed like co-occurrence (30-day half-life) |
672+| Tag | `annotations.tags` | 0.3 | Jaccard over annotation tag sets |
673+| Social | `follow_distances` | 0.7 | Follow distance: 1-hop=1.0, 2-hop=0.3 |
674+| Popularity | `feeds.subscriber_count` | 0.2 | `log(1 + subscribers) / log(1 + max)` |
675+| Category | `subscriptions.category` | 0.4 | Boost feeds matching user's existing categories |
676+
677+### 7.2 Feed Co-occurrence (Jaccard Similarity)
670678
671679 For any two feeds, the similarity is the Jaccard index of their subscriber sets:
672680
@@ -674,89 +682,153 @@ For any two feeds, the similarity is the Jaccard index of their subscriber sets:
674682 J(A, B) = |subscribers(A) ∩ subscribers(B)| / |subscribers(A) ∪ subscribers(B)|
675683 ```
676684
677-This is recomputed periodically (cron job) or incrementally when subscriptions change.
685+Feed description text similarity is also computed (word overlap after stopword removal) and added as a boost.
678686
679-### 7.2 User Similarity
687+### 7.3 User Similarity
680688
681-For any two users, compute Jaccard over their subscription sets:
689+For any two users, compute Jaccard over their subscription sets, plus like co-occurrence (time-decayed) and tag overlap:
682690
683691 ```
684-J(U1, U2) = |feeds(U1) ∩ feeds(U2)| / |feeds(U1) ∪ feeds(U2)|
692+J(U1, U2) = jaccard_subscriptions + 0.3 * jaccard_likes + 0.2 * jaccard_tags + follow_boost
685693 ```
686694
687-### 7.3 Recommendation Algorithms
695+Like overlap uses exponential time decay: `EXP(-0.023 * age_days)` (30-day half-life).
696+
697+### 7.4 On-Demand Scoring
688698
689-**Feed recommendations (on glean.at):**
699+Recommendations are computed **on-demand** at query time, not pre-materialized. This avoids write amplification on every cron run.
690700
691-1. Find users with Jaccard > 0.2 (similar readers)
692-2. Collect feeds those users subscribe to that the target user does not
693-3. Rank by frequency (how many similar users subscribe) and average similarity
694-4. Return top N feeds as recommendations
701+**Feed recommendation score** (computed in SQL):
695702
696703 ```
697-score(feed) = Σ J(target, U) for each user U subscribed to feed
704+score = sub_signal * w_sub
705+ + like_signal * w_like
706+ + social_signal * w_social
707+ + pop_signal * w_pop
708+ + category_signal * w_category
698709 ```
699710
700-**Article recommendations (on glean.at, from likes):**
711+Where:
712+- `sub_signal = SUM(jaccard(target, U))` for similar users U subscribed to feed
713+- `like_signal = SUM(jaccard(target, U) * time_decay)` for likes in that feed by similar users
714+- `social_signal = SUM(distance_weight)` from follow_distances
715+- `pop_signal = log(1 + subscriber_count) / log(1 + max_subscribers)`
716+- `category_signal = 1` if feed description matches user's top categories
701717
702-1. Find users who liked articles that the target user also liked
703-2. Collect articles those users liked that the target has not
704-3. Rank by frequency and recency
705-4. Return top N articles as recommendations
718+**Article recommendation score**:
706719
707720 ```
708-score(article) = Σ J(target, U) for each similar user U who liked the article
721+score = like_signal * w_like
722+ + social_signal * w_social
723+ + recency_signal * 0.2
709724 ```
710725
711-**People recommendations (to follow on Bluesky):**
726+### 7.5 User Feedback (Dismiss)
712727
713-1. Compute user similarity for all pairs
714-2. Return users with highest Jaccard, linking to their Bluesky profile for follow
728+Users can dismiss recommendations they don't want to see again:
715729
716-### 7.4 Implementation
730+- `POST /feeds/dismiss` — dismiss a feed recommendation
731+- `POST /articles/dismiss` — dismiss an article recommendation
732+- Dismissals are stored locally in `dismissed_recommendations` (not on PDS)
733+- Dismissed items are excluded from all future recommendation queries
734+- Auto-dismiss: items shown >15 times over >30 days without action are auto-dismissed
717735
718-For the initial version, brute-force Jaccard with SQLite is sufficient (scale: ~10k users, ~100k subscriptions). The query is:
736+Impression tracking (`recommendation_impressions`) records how many times each recommendation was shown and whether the user acted on it.
719737
720-```sql
721-SELECT s2.feed_url, COUNT(*) as overlap_count
722-FROM subscriptions s1
723-JOIN subscriptions s2 ON s1.feed_url = s2.feed_url
724-WHERE s1.user_did = ? AND s2.user_did != ?
725-AND s2.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
726-GROUP BY s2.feed_url
727-ORDER BY overlap_count DESC
728-LIMIT 20;
738+### 7.6 Auto-Tuned Signal Weights
739+
740+Each user has a row in `user_signal_weights` with per-signal weights. When a user acts on a recommendation (subscribes, likes), the dominant signal that produced that recommendation is rewarded:
741+
742+```
743+new_weight = MAX(0.1, MIN(3.0, old_weight * (1 + learning_rate * delta)))
729744 ```
730745
731-For larger scale, move to MinHash + LSH (banded hashing) to approximate Jaccard in sub-linear time.
746+- `learning_rate = 0.1`, `delta = +1` for reward, `-1` for penalty
747+- Only activates after `minActionsTune = 5` positive actions
748+- Defaults are used when no row exists for a user
749+
750+### 7.7 Social Graph
751+
752+Follow distances (1-hop and 2-hop) are pre-computed in `follow_distances` during the cron job:
753+
754+- 1-hop: direct follows (weight 1.0)
755+- 2-hop: friends-of-friends (weight 0.3)
756+- 3-hop is excluded due to noise and computational cost
757+
758+### 7.8 Diversity & Freshness
759+
760+After scoring, diversity filtering is applied in Go (not SQL):
761+
762+- **Domain diversity**: max 2 feeds from the same domain in results
763+- **Category diversity**: max 3 feeds from the same category in results
764+- This prevents recommendation clustering on a single source
765+
766+### 7.9 Cold Start
767+
768+New users with <5 subscriptions get a fallback strategy:
769+
770+1. Feeds from 1-hop followed users (70% weight)
771+2. Globally popular feeds by subscriber count (30% weight)
732772
733-### 7.5 Clustering Engine (Cron)
773+### 7.10 Clustering Engine (Cron)
734774
735775 A background goroutine runs on a configurable schedule (`GLEAN_CLUSTER_INTERVAL`, default 10m):
736776
737-1. **Compute feed similarity**: Batch-update the `feed_similarity` table (Jaccard over subscriber sets)
738-2. **Compute user similarity**: Batch-update the `user_similarity` table (Jaccard over subscription sets, boosted by follow relationships)
739-3. **Generate feed recommendations**: Materialize top feed recommendations per user into `user_feed_recommendations`
740-4. **Generate article recommendations**: Materialize top article recommendations per user into `user_article_recommendations`
777+1. **Compute feed similarity**: Batch-update `feed_similarity` table (Jaccard over subscriber sets + description similarity)
778+2. **Compute user similarity**: Batch-update `user_similarity` table (subscription Jaccard + time-decayed likes + tags + follow boost)
779+3. **Compute follow distances**: 1-hop and 2-hop from `follows` table
780+4. **Compute signal profiles**: Per-user category/tag/like summaries
781+5. **Auto-dismiss stale**: Dismiss items shown >15 times over >30 days without action
741782
742783 Jetstream ingestion and record indexing happen in a separate persistent goroutine (the Jetstream consumer), not in the cron.
743784
785+### 7.11 New Database Tables
786+
744787 ```sql
745-CREATE TABLE user_feed_recommendations (
746- user_did TEXT NOT NULL REFERENCES users(did),
747- feed_url TEXT NOT NULL REFERENCES feeds(feed_url),
748- score REAL NOT NULL,
749- computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
750- PRIMARY KEY (user_did, feed_url)
788+CREATE TABLE dismissed_recommendations (
789+ user_did TEXT NOT NULL REFERENCES users(did),
790+ target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
791+ target_id TEXT NOT NULL,
792+ reason TEXT,
793+ dismissed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
794+ PRIMARY KEY (user_did, target_type, target_id)
751795 );
752796
753-CREATE TABLE user_article_recommendations (
754- user_did TEXT NOT NULL REFERENCES users(did),
755- feed_url TEXT NOT NULL,
756- article_url TEXT NOT NULL,
757- score REAL NOT NULL,
758- computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
759- PRIMARY KEY (user_did, feed_url, article_url)
797+CREATE TABLE recommendation_impressions (
798+ user_did TEXT NOT NULL REFERENCES users(did),
799+ target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
800+ target_id TEXT NOT NULL,
801+ first_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
802+ last_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
803+ shown_count INTEGER NOT NULL DEFAULT 1,
804+ acted BOOLEAN NOT NULL DEFAULT 0,
805+ PRIMARY KEY (user_did, target_type, target_id)
806+);
807+
808+CREATE TABLE follow_distances (
809+ user_a TEXT NOT NULL,
810+ user_b TEXT NOT NULL,
811+ distance INTEGER NOT NULL CHECK(distance IN (1, 2)),
812+ PRIMARY KEY (user_a, user_b)
813+);
814+
815+CREATE TABLE user_signal_weights (
816+ user_did TEXT PRIMARY KEY REFERENCES users(did),
817+ w_sub REAL NOT NULL DEFAULT 1.0,
818+ w_like REAL NOT NULL DEFAULT 0.5,
819+ w_tag REAL NOT NULL DEFAULT 0.3,
820+ w_social REAL NOT NULL DEFAULT 0.7,
821+ w_pop REAL NOT NULL DEFAULT 0.2,
822+ w_category REAL NOT NULL DEFAULT 0.4,
823+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
824+);
825+
826+CREATE TABLE user_signal_profiles (
827+ user_did TEXT PRIMARY KEY REFERENCES users(did),
828+ total_likes INTEGER NOT NULL DEFAULT 0,
829+ total_tags INTEGER NOT NULL DEFAULT 0,
830+ top_categories TEXT,
831+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
760832 );
761833 ```
762834
@@ -778,15 +850,17 @@ The server renders HTML fragments that htmx swaps into the page. No JSON API nee
778850 | `/feeds/add` | POST | Add a single feed URL |
779851 | `/feeds/remove` | DELETE | Remove a feed |
780852 | `/feeds/refresh` | POST | Refresh all subscribed feeds |
781-| `/feeds/clear` | POST | Clear all subscriptions |
853+| `/feeds/clear` | POST | Clear all subscriptions |
854+| `/feeds/dismiss` | POST | Dismiss a feed recommendation |
782855 | `/articles` | GET | Read articles (paginated, filterable by feed) |
783856 | `/articles/{id}` | GET | Article detail view |
784857 | `/articles/{id}/read` | POST | Mark article as read |
785858 | `/articles/{id}/unread` | POST | Mark article as unread |
786859 | `/articles/{id}/like` | POST | Like an article |
787860 | `/articles/{id}/fetch-content` | POST | Fetch full article content from original URL |
788-| `/articles/mark-all-read` | POST | Mark all articles as read |
789-| `/trending` | GET | Community feed: articles ranked by likes |
861+| `/articles/mark-all-read` | POST | Mark all articles as read |
862+| `/articles/dismiss` | POST | Dismiss an article recommendation |
863+| `/trending` | GET | Community feed: articles ranked by likes |
790864 | `/library` | GET | Liked articles and annotations |
791865 | `/library/create` | POST | Create annotation on an article |
792866 | `/library/{id}/delete` | POST | Delete an annotation |
@@ -846,7 +920,12 @@ glean/
846920 │ │ └── metrics.go # Prometheus metrics definitions
847921 │ ├── cluster/
848922 │ │ ├── jaccard.go # Jaccard similarity computation
849-│ │ ├── recommender.go # Feed + people recommendation queries
923+│ │ ├── recommender.go # Feed + people recommendation queries (on-demand)
924+│ │ ├── scoring.go # Multi-signal composite scoring queries
925+│ │ ├── social.go # Follow-distance computation (1-2 hop)
926+│ │ ├── dismiss.go # Dismiss + impression tracking
927+│ │ ├── weights.go # Bandit-style signal weight auto-tuning
928+│ │ ├── diversity.go # Post-query domain/category diversity filtering
850929 │ │ └── cron.go # Background recomputation scheduler
851930 │ ├── server/
852931 │ │ ├── server.go # HTTP server, router setup
@@ -993,6 +1072,4 @@ All PDS records are public. There is no notion of private data on the AT Protoco
9931072
9941073 ## 13. Future Considerations
9951074
996-- **MinHash/LSH**: Replace brute-force Jaccard when user count exceeds ~50k
997-- **Full-text search**: Add FTS5 virtual table on articles for search
9981075 - **Email digest**: Periodic email with top articles from subscribed feeds
@@ -143,13 +143,13 @@ Glean also indexes records from the Skyreader lexicon (`app.skyreader.feed.subsc
143 143
144 The mapping from Skyreader subscription to Glean subscription:144 The mapping from Skyreader subscription to Glean subscription:
145 145
146-| Skyreader field | Glean field | Notes |146+| Skyreader field | Glean field | Notes |
147-| --------------- | ------------- | ------------------------------------ |147+| --------------- | ----------- | ----------------------------------- |
148-| `feedUrl` | `feed_url` | Direct mapping |148+| `feedUrl` | `feed_url` | Direct mapping |
149-| `title` | `title` | Direct mapping |149+| `title` | `title` | Direct mapping |
150-| `siteUrl` | `site_url` | Stored on the feed record |150+| `siteUrl` | `site_url` | Stored on the feed record |
151-| `createdAt` | `added_at` | Direct mapping |151+| `createdAt` | `added_at` | Direct mapping |
152-| _(none)_ | `category` | Empty (Skyreader has no categories) |152+| _(none)_ | `category` | Empty (Skyreader has no categories) |
153 153
154 If a Glean subscription already exists for the same `feed_url`, the existing one is kept. If the existing subscription has no URI (was created locally without PDS sync), the Skyreader URI/CID is backfilled.154 If a Glean subscription already exists for the same `feed_url`, the existing one is kept. If the existing subscription has no URI (was created locally without PDS sync), the Skyreader URI/CID is backfilled.
155 155
@@ -661,12 +661,20 @@ CREATE TABLE oauth_sessions (
661 );661 );
662 ```662 ```
663 663
664-Glean has two complementary recommendation signals:664+Glean uses a multi-signal recommendation system that combines subscription overlap, like patterns, social graph distance, and user behavior feedback.
665 665
666-- **Subscriptions** (Jaccard similarity): "Who reads the same feeds?" → feed and people discovery666+### 7.1 Signals
667-- **Likes** (co-occurrence): "Who likes the same articles?" → article and feed discovery
668 667
669-### 7.1 Feed Co-occurrence (Jaccard Similarity)668+| Signal | Source | Weight (default) | Description |
669+|--------|--------|-------------------|-------------|
670+| Subscription | `subscriptions` | 1.0 | Jaccard over subscriber sets between similar users |
671+| Like | `likes` | 0.5 | Time-decayed like co-occurrence (30-day half-life) |
672+| Tag | `annotations.tags` | 0.3 | Jaccard over annotation tag sets |
673+| Social | `follow_distances` | 0.7 | Follow distance: 1-hop=1.0, 2-hop=0.3 |
674+| Popularity | `feeds.subscriber_count` | 0.2 | `log(1 + subscribers) / log(1 + max)` |
675+| Category | `subscriptions.category` | 0.4 | Boost feeds matching user's existing categories |
676+
677+### 7.2 Feed Co-occurrence (Jaccard Similarity)
670 678
671 For any two feeds, the similarity is the Jaccard index of their subscriber sets:679 For any two feeds, the similarity is the Jaccard index of their subscriber sets:
672 680
@@ -674,89 +682,153 @@ For any two feeds, the similarity is the Jaccard index of their subscriber sets:
674 J(A, B) = |subscribers(A) ∩ subscribers(B)| / |subscribers(A) ∪ subscribers(B)|682 J(A, B) = |subscribers(A) ∩ subscribers(B)| / |subscribers(A) ∪ subscribers(B)|
675 ```683 ```
676 684
677-This is recomputed periodically (cron job) or incrementally when subscriptions change.685+Feed description text similarity is also computed (word overlap after stopword removal) and added as a boost.
678 686
679-### 7.2 User Similarity687+### 7.3 User Similarity
680 688
681-For any two users, compute Jaccard over their subscription sets:689+For any two users, compute Jaccard over their subscription sets, plus like co-occurrence (time-decayed) and tag overlap:
682 690
683 ```691 ```
684-J(U1, U2) = |feeds(U1) ∩ feeds(U2)| / |feeds(U1) ∪ feeds(U2)|692+J(U1, U2) = jaccard_subscriptions + 0.3 * jaccard_likes + 0.2 * jaccard_tags + follow_boost
685 ```693 ```
686 694
687-### 7.3 Recommendation Algorithms695+Like overlap uses exponential time decay: `EXP(-0.023 * age_days)` (30-day half-life).
696+
697+### 7.4 On-Demand Scoring
688 698
689-**Feed recommendations (on glean.at):**699+Recommendations are computed **on-demand** at query time, not pre-materialized. This avoids write amplification on every cron run.
690 700
691-1. Find users with Jaccard > 0.2 (similar readers)701+**Feed recommendation score** (computed in SQL):
692-2. Collect feeds those users subscribe to that the target user does not
693-3. Rank by frequency (how many similar users subscribe) and average similarity
694-4. Return top N feeds as recommendations
695 702
696 ```703 ```
697-score(feed) = Σ J(target, U) for each user U subscribed to feed704+score = sub_signal * w_sub
705+ + like_signal * w_like
706+ + social_signal * w_social
707+ + pop_signal * w_pop
708+ + category_signal * w_category
698 ```709 ```
699 710
700-**Article recommendations (on glean.at, from likes):**711+Where:
712+- `sub_signal = SUM(jaccard(target, U))` for similar users U subscribed to feed
713+- `like_signal = SUM(jaccard(target, U) * time_decay)` for likes in that feed by similar users
714+- `social_signal = SUM(distance_weight)` from follow_distances
715+- `pop_signal = log(1 + subscriber_count) / log(1 + max_subscribers)`
716+- `category_signal = 1` if feed description matches user's top categories
701 717
702-1. Find users who liked articles that the target user also liked718+**Article recommendation score**:
703-2. Collect articles those users liked that the target has not
704-3. Rank by frequency and recency
705-4. Return top N articles as recommendations
706 719
707 ```720 ```
708-score(article) = Σ J(target, U) for each similar user U who liked the article721+score = like_signal * w_like
722+ + social_signal * w_social
723+ + recency_signal * 0.2
709 ```724 ```
710 725
711-**People recommendations (to follow on Bluesky):**726+### 7.5 User Feedback (Dismiss)
712 727
713-1. Compute user similarity for all pairs728+Users can dismiss recommendations they don't want to see again:
714-2. Return users with highest Jaccard, linking to their Bluesky profile for follow
715 729
716-### 7.4 Implementation730+- `POST /feeds/dismiss` — dismiss a feed recommendation
731+- `POST /articles/dismiss` — dismiss an article recommendation
732+- Dismissals are stored locally in `dismissed_recommendations` (not on PDS)
733+- Dismissed items are excluded from all future recommendation queries
734+- Auto-dismiss: items shown >15 times over >30 days without action are auto-dismissed
717 735
718-For the initial version, brute-force Jaccard with SQLite is sufficient (scale: ~10k users, ~100k subscriptions). The query is:736+Impression tracking (`recommendation_impressions`) records how many times each recommendation was shown and whether the user acted on it.
719 737
720-```sql738+### 7.6 Auto-Tuned Signal Weights
721-SELECT s2.feed_url, COUNT(*) as overlap_count739+
722-FROM subscriptions s1740+Each user has a row in `user_signal_weights` with per-signal weights. When a user acts on a recommendation (subscribes, likes), the dominant signal that produced that recommendation is rewarded:
723-JOIN subscriptions s2 ON s1.feed_url = s2.feed_url741+
724-WHERE s1.user_did = ? AND s2.user_did != ?742+```
725-AND s2.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)743+new_weight = MAX(0.1, MIN(3.0, old_weight * (1 + learning_rate * delta)))
726-GROUP BY s2.feed_url
727-ORDER BY overlap_count DESC
728-LIMIT 20;
729 ```744 ```
730 745
731-For larger scale, move to MinHash + LSH (banded hashing) to approximate Jaccard in sub-linear time.746+- `learning_rate = 0.1`, `delta = +1` for reward, `-1` for penalty
747+- Only activates after `minActionsTune = 5` positive actions
748+- Defaults are used when no row exists for a user
749+
750+### 7.7 Social Graph
751+
752+Follow distances (1-hop and 2-hop) are pre-computed in `follow_distances` during the cron job:
753+
754+- 1-hop: direct follows (weight 1.0)
755+- 2-hop: friends-of-friends (weight 0.3)
756+- 3-hop is excluded due to noise and computational cost
757+
758+### 7.8 Diversity & Freshness
759+
760+After scoring, diversity filtering is applied in Go (not SQL):
761+
762+- **Domain diversity**: max 2 feeds from the same domain in results
763+- **Category diversity**: max 3 feeds from the same category in results
764+- This prevents recommendation clustering on a single source
765+
766+### 7.9 Cold Start
767+
768+New users with <5 subscriptions get a fallback strategy:
769+
770+1. Feeds from 1-hop followed users (70% weight)
771+2. Globally popular feeds by subscriber count (30% weight)
732 772
733-### 7.5 Clustering Engine (Cron)773+### 7.10 Clustering Engine (Cron)
734 774
735 A background goroutine runs on a configurable schedule (`GLEAN_CLUSTER_INTERVAL`, default 10m):775 A background goroutine runs on a configurable schedule (`GLEAN_CLUSTER_INTERVAL`, default 10m):
736 776
737-1. **Compute feed similarity**: Batch-update the `feed_similarity` table (Jaccard over subscriber sets)777+1. **Compute feed similarity**: Batch-update `feed_similarity` table (Jaccard over subscriber sets + description similarity)
738-2. **Compute user similarity**: Batch-update the `user_similarity` table (Jaccard over subscription sets, boosted by follow relationships)778+2. **Compute user similarity**: Batch-update `user_similarity` table (subscription Jaccard + time-decayed likes + tags + follow boost)
739-3. **Generate feed recommendations**: Materialize top feed recommendations per user into `user_feed_recommendations`779+3. **Compute follow distances**: 1-hop and 2-hop from `follows` table
740-4. **Generate article recommendations**: Materialize top article recommendations per user into `user_article_recommendations`780+4. **Compute signal profiles**: Per-user category/tag/like summaries
781+5. **Auto-dismiss stale**: Dismiss items shown >15 times over >30 days without action
741 782
742 Jetstream ingestion and record indexing happen in a separate persistent goroutine (the Jetstream consumer), not in the cron.783 Jetstream ingestion and record indexing happen in a separate persistent goroutine (the Jetstream consumer), not in the cron.
743 784
785+### 7.11 New Database Tables
786+
744 ```sql787 ```sql
745-CREATE TABLE user_feed_recommendations (788+CREATE TABLE dismissed_recommendations (
746- user_did TEXT NOT NULL REFERENCES users(did),789+ user_did TEXT NOT NULL REFERENCES users(did),
747- feed_url TEXT NOT NULL REFERENCES feeds(feed_url),790+ target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
748- score REAL NOT NULL,791+ target_id TEXT NOT NULL,
749- computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,792+ reason TEXT,
750- PRIMARY KEY (user_did, feed_url)793+ dismissed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
794+ PRIMARY KEY (user_did, target_type, target_id)
751 );795 );
752 796
753-CREATE TABLE user_article_recommendations (797+CREATE TABLE recommendation_impressions (
754- user_did TEXT NOT NULL REFERENCES users(did),798+ user_did TEXT NOT NULL REFERENCES users(did),
755- feed_url TEXT NOT NULL,799+ target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
756- article_url TEXT NOT NULL,800+ target_id TEXT NOT NULL,
757- score REAL NOT NULL,801+ first_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
758- computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,802+ last_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
759- PRIMARY KEY (user_did, feed_url, article_url)803+ shown_count INTEGER NOT NULL DEFAULT 1,
804+ acted BOOLEAN NOT NULL DEFAULT 0,
805+ PRIMARY KEY (user_did, target_type, target_id)
806+);
807+
808+CREATE TABLE follow_distances (
809+ user_a TEXT NOT NULL,
810+ user_b TEXT NOT NULL,
811+ distance INTEGER NOT NULL CHECK(distance IN (1, 2)),
812+ PRIMARY KEY (user_a, user_b)
813+);
814+
815+CREATE TABLE user_signal_weights (
816+ user_did TEXT PRIMARY KEY REFERENCES users(did),
817+ w_sub REAL NOT NULL DEFAULT 1.0,
818+ w_like REAL NOT NULL DEFAULT 0.5,
819+ w_tag REAL NOT NULL DEFAULT 0.3,
820+ w_social REAL NOT NULL DEFAULT 0.7,
821+ w_pop REAL NOT NULL DEFAULT 0.2,
822+ w_category REAL NOT NULL DEFAULT 0.4,
823+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
824+);
825+
826+CREATE TABLE user_signal_profiles (
827+ user_did TEXT PRIMARY KEY REFERENCES users(did),
828+ total_likes INTEGER NOT NULL DEFAULT 0,
829+ total_tags INTEGER NOT NULL DEFAULT 0,
830+ top_categories TEXT,
831+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
760 );832 );
761 ```833 ```
762 834
@@ -778,15 +850,17 @@ The server renders HTML fragments that htmx swaps into the page. No JSON API nee
778 | `/feeds/add` | POST | Add a single feed URL |850 | `/feeds/add` | POST | Add a single feed URL |
779 | `/feeds/remove` | DELETE | Remove a feed |851 | `/feeds/remove` | DELETE | Remove a feed |
780 | `/feeds/refresh` | POST | Refresh all subscribed feeds |852 | `/feeds/refresh` | POST | Refresh all subscribed feeds |
781-| `/feeds/clear` | POST | Clear all subscriptions |853+| `/feeds/clear` | POST | Clear all subscriptions |
854+| `/feeds/dismiss` | POST | Dismiss a feed recommendation |
782 | `/articles` | GET | Read articles (paginated, filterable by feed) |855 | `/articles` | GET | Read articles (paginated, filterable by feed) |
783 | `/articles/{id}` | GET | Article detail view |856 | `/articles/{id}` | GET | Article detail view |
784 | `/articles/{id}/read` | POST | Mark article as read |857 | `/articles/{id}/read` | POST | Mark article as read |
785 | `/articles/{id}/unread` | POST | Mark article as unread |858 | `/articles/{id}/unread` | POST | Mark article as unread |
786 | `/articles/{id}/like` | POST | Like an article |859 | `/articles/{id}/like` | POST | Like an article |
787 | `/articles/{id}/fetch-content` | POST | Fetch full article content from original URL |860 | `/articles/{id}/fetch-content` | POST | Fetch full article content from original URL |
788-| `/articles/mark-all-read` | POST | Mark all articles as read |861+| `/articles/mark-all-read` | POST | Mark all articles as read |
789-| `/trending` | GET | Community feed: articles ranked by likes |862+| `/articles/dismiss` | POST | Dismiss an article recommendation |
863+| `/trending` | GET | Community feed: articles ranked by likes |
790 | `/library` | GET | Liked articles and annotations |864 | `/library` | GET | Liked articles and annotations |
791 | `/library/create` | POST | Create annotation on an article |865 | `/library/create` | POST | Create annotation on an article |
792 | `/library/{id}/delete` | POST | Delete an annotation |866 | `/library/{id}/delete` | POST | Delete an annotation |
@@ -846,7 +920,12 @@ glean/
846 │ │ └── metrics.go # Prometheus metrics definitions920 │ │ └── metrics.go # Prometheus metrics definitions
847 │ ├── cluster/921 │ ├── cluster/
848 │ │ ├── jaccard.go # Jaccard similarity computation922 │ │ ├── jaccard.go # Jaccard similarity computation
849-│ │ ├── recommender.go # Feed + people recommendation queries923+│ │ ├── recommender.go # Feed + people recommendation queries (on-demand)
924+│ │ ├── scoring.go # Multi-signal composite scoring queries
925+│ │ ├── social.go # Follow-distance computation (1-2 hop)
926+│ │ ├── dismiss.go # Dismiss + impression tracking
927+│ │ ├── weights.go # Bandit-style signal weight auto-tuning
928+│ │ ├── diversity.go # Post-query domain/category diversity filtering
850 │ │ └── cron.go # Background recomputation scheduler929 │ │ └── cron.go # Background recomputation scheduler
851 │ ├── server/930 │ ├── server/
852 │ │ ├── server.go # HTTP server, router setup931 │ │ ├── server.go # HTTP server, router setup
@@ -993,6 +1072,4 @@ All PDS records are public. There is no notion of private data on the AT Protoco
993 1072
994 ## 13. Future Considerations1073 ## 13. Future Considerations
995 1074
996-- **MinHash/LSH**: Replace brute-force Jaccard when user count exceeds ~50k
997-- **Full-text search**: Add FTS5 virtual table on articles for search
998 - **Email digest**: Periodic email with top articles from subscribed feeds1075 - **Email digest**: Periodic email with top articles from subscribed feeds
modified internal/atproto/xrpc.go +26 -54
@@ -8,14 +8,17 @@ import (
88 "strings"
99
1010 "github.com/go-chi/chi/v5"
11+
12+ "pkg.rbrt.fr/glean/internal/cluster"
1113 )
1214
1315 type XRPCHandler struct {
14- db *sql.DB
16+ db *sql.DB
17+ engine *cluster.Engine
1518 }
1619
17-func NewXRPCHandler(db *sql.DB) *XRPCHandler {
18- return &XRPCHandler{db: db}
20+func NewXRPCHandler(db *sql.DB, engine *cluster.Engine) *XRPCHandler {
21+ return &XRPCHandler{db: db, engine: engine}
1922 }
2023
2124 func (h *XRPCHandler) ListSubscriptions(w http.ResponseWriter, r *http.Request) {
@@ -303,72 +306,41 @@ func (h *XRPCHandler) GetRecommendations(w http.ResponseWriter, r *http.Request)
303306 repo := r.URL.Query().Get("repo")
304307 limit := min(parseIntParam(r, "limit", 20), 50)
305308
306- feedRows, err := h.db.QueryContext(r.Context(), `
307- SELECT r.feed_url, f.title, f.site_url, f.description, f.subscriber_count, r.score
308- FROM user_feed_recommendations r
309- JOIN feeds f ON r.feed_url = f.feed_url
310- WHERE r.user_did = ?
311- ORDER BY r.score DESC
312- LIMIT ?
313- `, repo, limit)
309+ ctx := r.Context()
310+
311+ feedRecs, err := h.engine.GetFeedRecommendations(ctx, repo, limit)
314312 if err != nil {
315313 http.Error(w, err.Error(), http.StatusInternalServerError)
316314 return
317315 }
318- defer feedRows.Close()
319316
320- feeds := make([]RecommendedFeed, 0)
321- for feedRows.Next() {
322- var feedURL, title, siteURL, description string
323- var subscriberCount int
324- var score float64
325- if err := feedRows.Scan(&feedURL, &title, &siteURL, &description, &subscriberCount, &score); err != nil {
326- http.Error(w, err.Error(), http.StatusInternalServerError)
327- return
328- }
317+ feeds := make([]RecommendedFeed, 0, len(feedRecs))
318+ for _, rec := range feedRecs {
329319 feeds = append(feeds, RecommendedFeed{
330- FeedURL: feedURL,
331- Title: title,
332- SiteURL: siteURL,
333- Description: description,
334- SubscriberCount: subscriberCount,
335- Score: score,
320+ FeedURL: rec.FeedURL,
321+ Title: rec.Title,
322+ SiteURL: rec.SiteURL,
323+ Description: rec.Description,
324+ SubscriberCount: rec.SubscriberCount,
325+ Score: rec.Score,
336326 })
337327 }
338328
339- peopleRows, err := h.db.QueryContext(r.Context(), `
340- SELECT u.did, u.handle, u.display_name, u.avatar_url, s.jaccard, s.common_feeds
341- FROM user_similarity s
342- JOIN users u ON (
343- CASE WHEN s.user_a = ? THEN s.user_b ELSE s.user_a END
344- ) = u.did
345- WHERE s.user_a = ? OR s.user_b = ?
346- ORDER BY s.jaccard DESC
347- LIMIT ?
348- `, repo, repo, repo, limit)
329+ peopleRecs, err := h.engine.GetPeopleRecommendations(ctx, repo, limit)
349330 if err != nil {
350331 http.Error(w, err.Error(), http.StatusInternalServerError)
351332 return
352333 }
353- defer peopleRows.Close()
354334
355- people := make([]RecommendedPerson, 0)
356- for peopleRows.Next() {
357- var did, handle string
358- var displayName, avatar sql.NullString
359- var jaccard float64
360- var commonFeeds int
361- if err := peopleRows.Scan(&did, &handle, &displayName, &avatar, &jaccard, &commonFeeds); err != nil {
362- http.Error(w, err.Error(), http.StatusInternalServerError)
363- return
364- }
335+ people := make([]RecommendedPerson, 0, len(peopleRecs))
336+ for _, rec := range peopleRecs {
365337 people = append(people, RecommendedPerson{
366- DID: did,
367- Handle: handle,
368- DisplayName: displayName.String,
369- Avatar: avatar.String,
370- Jaccard: jaccard,
371- CommonFeeds: commonFeeds,
338+ DID: rec.DID,
339+ Handle: rec.Handle,
340+ DisplayName: rec.DisplayName,
341+ Avatar: rec.AvatarURL,
342+ Jaccard: rec.Jaccard,
343+ CommonFeeds: rec.CommonFeeds,
372344 })
373345 }
374346
@@ -8,14 +8,17 @@ import (
8 "strings"8 "strings"
9 9
10 "github.com/go-chi/chi/v5"10 "github.com/go-chi/chi/v5"
11+
12+ "pkg.rbrt.fr/glean/internal/cluster"
11 )13 )
12 14
13 type XRPCHandler struct {15 type XRPCHandler struct {
14- db *sql.DB16+ db *sql.DB
17+ engine *cluster.Engine
15 }18 }
16 19
17-func NewXRPCHandler(db *sql.DB) *XRPCHandler {20+func NewXRPCHandler(db *sql.DB, engine *cluster.Engine) *XRPCHandler {
18- return &XRPCHandler{db: db}21+ return &XRPCHandler{db: db, engine: engine}
19 }22 }
20 23
21 func (h *XRPCHandler) ListSubscriptions(w http.ResponseWriter, r *http.Request) {24 func (h *XRPCHandler) ListSubscriptions(w http.ResponseWriter, r *http.Request) {
@@ -303,72 +306,41 @@ func (h *XRPCHandler) GetRecommendations(w http.ResponseWriter, r *http.Request)
303 repo := r.URL.Query().Get("repo")306 repo := r.URL.Query().Get("repo")
304 limit := min(parseIntParam(r, "limit", 20), 50)307 limit := min(parseIntParam(r, "limit", 20), 50)
305 308
306- feedRows, err := h.db.QueryContext(r.Context(), `309+ ctx := r.Context()
307- SELECT r.feed_url, f.title, f.site_url, f.description, f.subscriber_count, r.score310+
308- FROM user_feed_recommendations r311+ feedRecs, err := h.engine.GetFeedRecommendations(ctx, repo, limit)
309- JOIN feeds f ON r.feed_url = f.feed_url
310- WHERE r.user_did = ?
311- ORDER BY r.score DESC
312- LIMIT ?
313- `, repo, limit)
314 if err != nil {312 if err != nil {
315 http.Error(w, err.Error(), http.StatusInternalServerError)313 http.Error(w, err.Error(), http.StatusInternalServerError)
316 return314 return
317 }315 }
318- defer feedRows.Close()
319 316
320- feeds := make([]RecommendedFeed, 0)317+ feeds := make([]RecommendedFeed, 0, len(feedRecs))
321- for feedRows.Next() {318+ for _, rec := range feedRecs {
322- var feedURL, title, siteURL, description string
323- var subscriberCount int
324- var score float64
325- if err := feedRows.Scan(&feedURL, &title, &siteURL, &description, &subscriberCount, &score); err != nil {
326- http.Error(w, err.Error(), http.StatusInternalServerError)
327- return
328- }
329 feeds = append(feeds, RecommendedFeed{319 feeds = append(feeds, RecommendedFeed{
330- FeedURL: feedURL,320+ FeedURL: rec.FeedURL,
331- Title: title,321+ Title: rec.Title,
332- SiteURL: siteURL,322+ SiteURL: rec.SiteURL,
333- Description: description,323+ Description: rec.Description,
334- SubscriberCount: subscriberCount,324+ SubscriberCount: rec.SubscriberCount,
335- Score: score,325+ Score: rec.Score,
336 })326 })
337 }327 }
338 328
339- peopleRows, err := h.db.QueryContext(r.Context(), `329+ peopleRecs, err := h.engine.GetPeopleRecommendations(ctx, repo, limit)
340- SELECT u.did, u.handle, u.display_name, u.avatar_url, s.jaccard, s.common_feeds
341- FROM user_similarity s
342- JOIN users u ON (
343- CASE WHEN s.user_a = ? THEN s.user_b ELSE s.user_a END
344- ) = u.did
345- WHERE s.user_a = ? OR s.user_b = ?
346- ORDER BY s.jaccard DESC
347- LIMIT ?
348- `, repo, repo, repo, limit)
349 if err != nil {330 if err != nil {
350 http.Error(w, err.Error(), http.StatusInternalServerError)331 http.Error(w, err.Error(), http.StatusInternalServerError)
351 return332 return
352 }333 }
353- defer peopleRows.Close()
354 334
355- people := make([]RecommendedPerson, 0)335+ people := make([]RecommendedPerson, 0, len(peopleRecs))
356- for peopleRows.Next() {336+ for _, rec := range peopleRecs {
357- var did, handle string
358- var displayName, avatar sql.NullString
359- var jaccard float64
360- var commonFeeds int
361- if err := peopleRows.Scan(&did, &handle, &displayName, &avatar, &jaccard, &commonFeeds); err != nil {
362- http.Error(w, err.Error(), http.StatusInternalServerError)
363- return
364- }
365 people = append(people, RecommendedPerson{337 people = append(people, RecommendedPerson{
366- DID: did,338+ DID: rec.DID,
367- Handle: handle,339+ Handle: rec.Handle,
368- DisplayName: displayName.String,340+ DisplayName: rec.DisplayName,
369- Avatar: avatar.String,341+ Avatar: rec.AvatarURL,
370- Jaccard: jaccard,342+ Jaccard: rec.Jaccard,
371- CommonFeeds: commonFeeds,343+ CommonFeeds: rec.CommonFeeds,
372 })344 })
373 }345 }
374 346
modified internal/cluster/cron.go +8 -2
@@ -32,8 +32,14 @@ func (c *Cron) Run(ctx context.Context) error {
3232 if err := c.engine.ComputeUserSimilarity(ctx); err != nil {
3333 c.engine.logger.Error("user similarity failed", "error", err)
3434 }
35- if err := c.engine.ComputeRecommendations(ctx); err != nil {
36- c.engine.logger.Error("recommendations failed", "error", err)
35+ if err := c.engine.ComputeFollowDistances(ctx); err != nil {
36+ c.engine.logger.Error("follow distances failed", "error", err)
37+ }
38+ if err := c.engine.ComputeSignalProfiles(ctx); err != nil {
39+ c.engine.logger.Error("signal profiles failed", "error", err)
40+ }
41+ if err := c.engine.AutoDismissStale(ctx, 15, 30); err != nil {
42+ c.engine.logger.Error("auto dismiss failed", "error", err)
3743 }
3844 c.engine.mu.Unlock()
3945 }
@@ -32,8 +32,14 @@ func (c *Cron) Run(ctx context.Context) error {
32 if err := c.engine.ComputeUserSimilarity(ctx); err != nil {32 if err := c.engine.ComputeUserSimilarity(ctx); err != nil {
33 c.engine.logger.Error("user similarity failed", "error", err)33 c.engine.logger.Error("user similarity failed", "error", err)
34 }34 }
35- if err := c.engine.ComputeRecommendations(ctx); err != nil {35+ if err := c.engine.ComputeFollowDistances(ctx); err != nil {
36- c.engine.logger.Error("recommendations failed", "error", err)36+ c.engine.logger.Error("follow distances failed", "error", err)
37+ }
38+ if err := c.engine.ComputeSignalProfiles(ctx); err != nil {
39+ c.engine.logger.Error("signal profiles failed", "error", err)
40+ }
41+ if err := c.engine.AutoDismissStale(ctx, 15, 30); err != nil {
42+ c.engine.logger.Error("auto dismiss failed", "error", err)
37 }43 }
38 c.engine.mu.Unlock()44 c.engine.mu.Unlock()
39 }45 }
added internal/cluster/dismiss.go +82 -0
new file mode 100644
@@ -0,0 +1,82 @@
1+package cluster
2+
3+import (
4+ "context"
5+ "time"
6+)
7+
8+type Impression struct {
9+ TargetType string
10+ TargetID string
11+}
12+
13+func (e *Engine) DismissFeed(ctx context.Context, userDID, feedURL, reason string) error {
14+ _, err := e.db.ExecContext(ctx, `
15+ INSERT INTO dismissed_recommendations (user_did, target_type, target_id, reason)
16+ VALUES (?, 'feed', ?, ?)
17+ ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP
18+ `, userDID, feedURL, reason)
19+ return err
20+}
21+
22+func (e *Engine) DismissArticle(ctx context.Context, userDID, articleURL, reason string) error {
23+ _, err := e.db.ExecContext(ctx, `
24+ INSERT INTO dismissed_recommendations (user_did, target_type, target_id, reason)
25+ VALUES (?, 'article', ?, ?)
26+ ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP
27+ `, userDID, articleURL, reason)
28+ return err
29+}
30+
31+func (e *Engine) RecordImpressions(ctx context.Context, userDID string, impressions []Impression) error {
32+ tx, err := e.db.BeginTx(ctx, nil)
33+ if err != nil {
34+ return err
35+ }
36+ defer func() { _ = tx.Rollback() }()
37+
38+ for _, imp := range impressions {
39+ _, err := tx.ExecContext(ctx, `
40+ INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count)
41+ VALUES (?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, 1)
42+ ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET
43+ last_shown_at = CURRENT_TIMESTAMP,
44+ shown_count = shown_count + 1
45+ `, userDID, imp.TargetType, imp.TargetID)
46+ if err != nil {
47+ return err
48+ }
49+ }
50+ return tx.Commit()
51+}
52+
53+func (e *Engine) MarkImpressionActed(ctx context.Context, userDID, targetType, targetID string) error {
54+ _, err := e.db.ExecContext(ctx, `
55+ UPDATE recommendation_impressions SET acted = 1
56+ WHERE user_did = ? AND target_type = ? AND target_id = ?
57+ `, userDID, targetType, targetID)
58+ return err
59+}
60+
61+func (e *Engine) AutoDismissStale(ctx context.Context, minShownCount int, maxAgeDays int) error {
62+ cutoff := time.Now().AddDate(0, 0, -maxAgeDays).Format(time.RFC3339)
63+
64+ _, err := e.db.ExecContext(ctx, `
65+ INSERT OR IGNORE INTO dismissed_recommendations (user_did, target_type, target_id, reason, dismissed_at)
66+ SELECT user_did, target_type, target_id, 'auto_stale', CURRENT_TIMESTAMP
67+ FROM recommendation_impressions
68+ WHERE acted = 0
69+ AND shown_count >= ?
70+ AND first_shown_at < ?
71+ `, minShownCount, cutoff)
72+ return err
73+}
74+
75+func (e *Engine) IsFeedDismissed(ctx context.Context, userDID, feedURL string) (bool, error) {
76+ var count int
77+ err := e.db.QueryRowContext(ctx, `
78+ SELECT COUNT(1) FROM dismissed_recommendations
79+ WHERE user_did = ? AND target_type = 'feed' AND target_id = ?
80+ `, userDID, feedURL).Scan(&count)
81+ return count > 0, err
82+}
new file mode 100644
@@ -0,0 +1,82 @@
1+package cluster
2+
3+import (
4+ "context"
5+ "time"
6+)
7+
8+type Impression struct {
9+ TargetType string
10+ TargetID string
11+}
12+
13+func (e *Engine) DismissFeed(ctx context.Context, userDID, feedURL, reason string) error {
14+ _, err := e.db.ExecContext(ctx, `
15+ INSERT INTO dismissed_recommendations (user_did, target_type, target_id, reason)
16+ VALUES (?, 'feed', ?, ?)
17+ ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP
18+ `, userDID, feedURL, reason)
19+ return err
20+}
21+
22+func (e *Engine) DismissArticle(ctx context.Context, userDID, articleURL, reason string) error {
23+ _, err := e.db.ExecContext(ctx, `
24+ INSERT INTO dismissed_recommendations (user_did, target_type, target_id, reason)
25+ VALUES (?, 'article', ?, ?)
26+ ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP
27+ `, userDID, articleURL, reason)
28+ return err
29+}
30+
31+func (e *Engine) RecordImpressions(ctx context.Context, userDID string, impressions []Impression) error {
32+ tx, err := e.db.BeginTx(ctx, nil)
33+ if err != nil {
34+ return err
35+ }
36+ defer func() { _ = tx.Rollback() }()
37+
38+ for _, imp := range impressions {
39+ _, err := tx.ExecContext(ctx, `
40+ INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count)
41+ VALUES (?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, 1)
42+ ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET
43+ last_shown_at = CURRENT_TIMESTAMP,
44+ shown_count = shown_count + 1
45+ `, userDID, imp.TargetType, imp.TargetID)
46+ if err != nil {
47+ return err
48+ }
49+ }
50+ return tx.Commit()
51+}
52+
53+func (e *Engine) MarkImpressionActed(ctx context.Context, userDID, targetType, targetID string) error {
54+ _, err := e.db.ExecContext(ctx, `
55+ UPDATE recommendation_impressions SET acted = 1
56+ WHERE user_did = ? AND target_type = ? AND target_id = ?
57+ `, userDID, targetType, targetID)
58+ return err
59+}
60+
61+func (e *Engine) AutoDismissStale(ctx context.Context, minShownCount int, maxAgeDays int) error {
62+ cutoff := time.Now().AddDate(0, 0, -maxAgeDays).Format(time.RFC3339)
63+
64+ _, err := e.db.ExecContext(ctx, `
65+ INSERT OR IGNORE INTO dismissed_recommendations (user_did, target_type, target_id, reason, dismissed_at)
66+ SELECT user_did, target_type, target_id, 'auto_stale', CURRENT_TIMESTAMP
67+ FROM recommendation_impressions
68+ WHERE acted = 0
69+ AND shown_count >= ?
70+ AND first_shown_at < ?
71+ `, minShownCount, cutoff)
72+ return err
73+}
74+
75+func (e *Engine) IsFeedDismissed(ctx context.Context, userDID, feedURL string) (bool, error) {
76+ var count int
77+ err := e.db.QueryRowContext(ctx, `
78+ SELECT COUNT(1) FROM dismissed_recommendations
79+ WHERE user_did = ? AND target_type = 'feed' AND target_id = ?
80+ `, userDID, feedURL).Scan(&count)
81+ return count > 0, err
82+}
added internal/cluster/diversity.go +68 -0
new file mode 100644
@@ -0,0 +1,68 @@
1+package cluster
2+
3+import (
4+ "net/url"
5+ "strings"
6+)
7+
8+const maxPerDomain = 2
9+const maxPerCategory = 3
10+
11+func ApplyDiversity(candidates []*FeedRecommendation, topN int) []*FeedRecommendation {
12+ domainCount := make(map[string]int, len(candidates))
13+ categoryCount := make(map[string]int)
14+ result := make([]*FeedRecommendation, 0, topN)
15+
16+ for _, c := range candidates {
17+ if len(result) >= topN {
18+ break
19+ }
20+
21+ domain := extractDomain(c.SiteURL)
22+ if domain != "" && domainCount[domain] >= maxPerDomain {
23+ continue
24+ }
25+
26+ cat := extractCategory(c.Description)
27+ if cat != "" && categoryCount[cat] >= maxPerCategory {
28+ continue
29+ }
30+
31+ if domain != "" {
32+ domainCount[domain]++
33+ }
34+ if cat != "" {
35+ categoryCount[cat]++
36+ }
37+ result = append(result, c)
38+ }
39+
40+ return result
41+}
42+
43+func extractDomain(siteURL string) string {
44+ if siteURL == "" {
45+ return ""
46+ }
47+ u, err := url.Parse(siteURL)
48+ if err != nil {
49+ return ""
50+ }
51+ host := u.Hostname()
52+ parts := strings.Split(host, ".")
53+ if len(parts) > 2 {
54+ return strings.Join(parts[len(parts)-2:], ".")
55+ }
56+ return host
57+}
58+
59+func extractCategory(description string) string {
60+ if description == "" {
61+ return ""
62+ }
63+ words := strings.Fields(strings.ToLower(description))
64+ if len(words) == 0 {
65+ return ""
66+ }
67+ return words[0]
68+}
new file mode 100644
@@ -0,0 +1,68 @@
1+package cluster
2+
3+import (
4+ "net/url"
5+ "strings"
6+)
7+
8+const maxPerDomain = 2
9+const maxPerCategory = 3
10+
11+func ApplyDiversity(candidates []*FeedRecommendation, topN int) []*FeedRecommendation {
12+ domainCount := make(map[string]int, len(candidates))
13+ categoryCount := make(map[string]int)
14+ result := make([]*FeedRecommendation, 0, topN)
15+
16+ for _, c := range candidates {
17+ if len(result) >= topN {
18+ break
19+ }
20+
21+ domain := extractDomain(c.SiteURL)
22+ if domain != "" && domainCount[domain] >= maxPerDomain {
23+ continue
24+ }
25+
26+ cat := extractCategory(c.Description)
27+ if cat != "" && categoryCount[cat] >= maxPerCategory {
28+ continue
29+ }
30+
31+ if domain != "" {
32+ domainCount[domain]++
33+ }
34+ if cat != "" {
35+ categoryCount[cat]++
36+ }
37+ result = append(result, c)
38+ }
39+
40+ return result
41+}
42+
43+func extractDomain(siteURL string) string {
44+ if siteURL == "" {
45+ return ""
46+ }
47+ u, err := url.Parse(siteURL)
48+ if err != nil {
49+ return ""
50+ }
51+ host := u.Hostname()
52+ parts := strings.Split(host, ".")
53+ if len(parts) > 2 {
54+ return strings.Join(parts[len(parts)-2:], ".")
55+ }
56+ return host
57+}
58+
59+func extractCategory(description string) string {
60+ if description == "" {
61+ return ""
62+ }
63+ words := strings.Fields(strings.ToLower(description))
64+ if len(words) == 0 {
65+ return ""
66+ }
67+ return words[0]
68+}
modified internal/cluster/jaccard.go +14 -99
@@ -9,20 +9,18 @@ import (
99 )
1010
1111 type Config struct {
12- SimilarityThreshold float64
13- FollowBoost float64
14- LikesWeight float64
15- TagsWeight float64
16- DescriptionWeight float64
12+ FollowBoost float64
13+ LikesWeight float64
14+ TagsWeight float64
15+ DescriptionWeight float64
1716 }
1817
1918 func DefaultConfig() Config {
2019 return Config{
21- SimilarityThreshold: 0.2,
22- FollowBoost: 0.5,
23- LikesWeight: 0.3,
24- TagsWeight: 0.2,
25- DescriptionWeight: 0.15,
20+ FollowBoost: 0.5,
21+ LikesWeight: 0.3,
22+ TagsWeight: 0.2,
23+ DescriptionWeight: 0.15,
2624 }
2725 }
2826
@@ -37,48 +35,6 @@ func NewEngine(db *sql.DB, logger *slog.Logger) *Engine {
3735 return &Engine{db: db, logger: logger, config: DefaultConfig()}
3836 }
3937
40-func (e *Engine) ComputeArticleRecommendations(ctx context.Context) error {
41- tx, err := e.db.BeginTx(ctx, nil)
42- if err != nil {
43- return err
44- }
45- defer func() { _ = tx.Rollback() }()
46-
47- if _, err := tx.ExecContext(ctx, `DELETE FROM user_article_recommendations`); err != nil {
48- return err
49- }
50-
51- query := fmt.Sprintf(`
52- INSERT INTO user_article_recommendations (user_did, feed_url, article_url, score)
53- SELECT targets.target, l.feed_url, l.article_url, SUM(targets.jaccard) AS score
54- FROM (
55- SELECT us.user_a AS target, us.user_b AS peer, us.jaccard
56- FROM user_similarity us
57- WHERE us.jaccard > %g
58- UNION ALL
59- SELECT us.user_b AS target, us.user_a AS peer, us.jaccard
60- FROM user_similarity us
61- WHERE us.jaccard > %g
62- ) targets
63- JOIN likes l ON l.author_did = targets.peer
64- WHERE NOT EXISTS (
65- SELECT 1 FROM subscriptions sub WHERE sub.user_did = targets.target AND sub.feed_url = l.feed_url
66- )
67- AND NOT EXISTS (
68- SELECT 1 FROM likes ul WHERE ul.author_did = targets.target AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url
69- )
70- GROUP BY targets.target, l.feed_url, l.article_url
71- ORDER BY score DESC
72- `, e.config.SimilarityThreshold, e.config.SimilarityThreshold)
73-
74- if _, err := tx.ExecContext(ctx, query); err != nil {
75- return err
76- }
77-
78- e.logger.Info("article recommendations computed")
79- return tx.Commit()
80-}
81-
8238 func (e *Engine) ComputeFeedSimilarity(ctx context.Context) error {
8339 tx, err := e.db.BeginTx(ctx, nil)
8440 if err != nil {
@@ -260,10 +216,15 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
260216 }
261217 if _, err := tx.ExecContext(ctx, `
262218 INSERT INTO _likes_overlap (user_a, user_b, common)
263- SELECT l1.author_did, l2.author_did, COUNT(*)
219+ SELECT l1.author_did, l2.author_did,
220+ CAST(SUM(
221+ EXP(-0.023 * CAST(julianday('now') - julianday(l1.created_at) AS REAL))
222+ * EXP(-0.023 * CAST(julianday('now') - julianday(l2.created_at) AS REAL))
223+ ) AS INTEGER)
264224 FROM likes l1
265225 JOIN likes l2 ON l1.feed_url = l2.feed_url AND l1.article_url = l2.article_url
266226 AND l1.author_did < l2.author_did
227+ WHERE l1.created_at IS NOT NULL AND l2.created_at IS NOT NULL
267228 GROUP BY l1.author_did, l2.author_did
268229 `); err != nil {
269230 return err
@@ -400,49 +361,3 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
400361 e.logger.Info("user similarity computed")
401362 return tx.Commit()
402363 }
403-
404-func (e *Engine) ComputeRecommendations(ctx context.Context) error {
405- tx, err := e.db.BeginTx(ctx, nil)
406- if err != nil {
407- return err
408- }
409- defer func() { _ = tx.Rollback() }()
410-
411- if _, err := tx.ExecContext(ctx, `DELETE FROM user_feed_recommendations`); err != nil {
412- return err
413- }
414-
415- recQuery := fmt.Sprintf(`
416- INSERT INTO user_feed_recommendations (user_did, feed_url, score)
417- SELECT target, feed_url, SUM(jaccard) AS score
418- FROM (
419- SELECT us.user_a AS target, s.feed_url, us.jaccard
420- FROM user_similarity us
421- JOIN subscriptions s ON s.user_did = us.user_b
422- WHERE us.jaccard > %g
423- AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = us.user_a)
424-
425- UNION ALL
426-
427- SELECT us.user_b AS target, s.feed_url, us.jaccard
428- FROM user_similarity us
429- JOIN subscriptions s ON s.user_did = us.user_a
430- WHERE us.jaccard > %g
431- AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = us.user_b)
432- )
433- GROUP BY target, feed_url
434- ORDER BY score DESC
435- `, e.config.SimilarityThreshold, e.config.SimilarityThreshold)
436-
437- if _, err := tx.ExecContext(ctx, recQuery); err != nil {
438- return err
439- }
440-
441- e.logger.Info("feed recommendations computed")
442-
443- if err := tx.Commit(); err != nil {
444- return err
445- }
446-
447- return e.ComputeArticleRecommendations(ctx)
448-}
@@ -9,20 +9,18 @@ import (
9 )9 )
10 10
11 type Config struct {11 type Config struct {
12- SimilarityThreshold float6412+ FollowBoost float64
13- FollowBoost float6413+ LikesWeight float64
14- LikesWeight float6414+ TagsWeight float64
15- TagsWeight float6415+ DescriptionWeight float64
16- DescriptionWeight float64
17 }16 }
18 17
19 func DefaultConfig() Config {18 func DefaultConfig() Config {
20 return Config{19 return Config{
21- SimilarityThreshold: 0.2,20+ FollowBoost: 0.5,
22- FollowBoost: 0.5,21+ LikesWeight: 0.3,
23- LikesWeight: 0.3,22+ TagsWeight: 0.2,
24- TagsWeight: 0.2,23+ DescriptionWeight: 0.15,
25- DescriptionWeight: 0.15,
26 }24 }
27 }25 }
28 26
@@ -37,48 +35,6 @@ func NewEngine(db *sql.DB, logger *slog.Logger) *Engine {
37 return &Engine{db: db, logger: logger, config: DefaultConfig()}35 return &Engine{db: db, logger: logger, config: DefaultConfig()}
38 }36 }
39 37
40-func (e *Engine) ComputeArticleRecommendations(ctx context.Context) error {
41- tx, err := e.db.BeginTx(ctx, nil)
42- if err != nil {
43- return err
44- }
45- defer func() { _ = tx.Rollback() }()
46-
47- if _, err := tx.ExecContext(ctx, `DELETE FROM user_article_recommendations`); err != nil {
48- return err
49- }
50-
51- query := fmt.Sprintf(`
52- INSERT INTO user_article_recommendations (user_did, feed_url, article_url, score)
53- SELECT targets.target, l.feed_url, l.article_url, SUM(targets.jaccard) AS score
54- FROM (
55- SELECT us.user_a AS target, us.user_b AS peer, us.jaccard
56- FROM user_similarity us
57- WHERE us.jaccard > %g
58- UNION ALL
59- SELECT us.user_b AS target, us.user_a AS peer, us.jaccard
60- FROM user_similarity us
61- WHERE us.jaccard > %g
62- ) targets
63- JOIN likes l ON l.author_did = targets.peer
64- WHERE NOT EXISTS (
65- SELECT 1 FROM subscriptions sub WHERE sub.user_did = targets.target AND sub.feed_url = l.feed_url
66- )
67- AND NOT EXISTS (
68- SELECT 1 FROM likes ul WHERE ul.author_did = targets.target AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url
69- )
70- GROUP BY targets.target, l.feed_url, l.article_url
71- ORDER BY score DESC
72- `, e.config.SimilarityThreshold, e.config.SimilarityThreshold)
73-
74- if _, err := tx.ExecContext(ctx, query); err != nil {
75- return err
76- }
77-
78- e.logger.Info("article recommendations computed")
79- return tx.Commit()
80-}
81-
82 func (e *Engine) ComputeFeedSimilarity(ctx context.Context) error {38 func (e *Engine) ComputeFeedSimilarity(ctx context.Context) error {
83 tx, err := e.db.BeginTx(ctx, nil)39 tx, err := e.db.BeginTx(ctx, nil)
84 if err != nil {40 if err != nil {
@@ -260,10 +216,15 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
260 }216 }
261 if _, err := tx.ExecContext(ctx, `217 if _, err := tx.ExecContext(ctx, `
262 INSERT INTO _likes_overlap (user_a, user_b, common)218 INSERT INTO _likes_overlap (user_a, user_b, common)
263- SELECT l1.author_did, l2.author_did, COUNT(*)219+ SELECT l1.author_did, l2.author_did,
220+ CAST(SUM(
221+ EXP(-0.023 * CAST(julianday('now') - julianday(l1.created_at) AS REAL))
222+ * EXP(-0.023 * CAST(julianday('now') - julianday(l2.created_at) AS REAL))
223+ ) AS INTEGER)
264 FROM likes l1224 FROM likes l1
265 JOIN likes l2 ON l1.feed_url = l2.feed_url AND l1.article_url = l2.article_url225 JOIN likes l2 ON l1.feed_url = l2.feed_url AND l1.article_url = l2.article_url
266 AND l1.author_did < l2.author_did226 AND l1.author_did < l2.author_did
227+ WHERE l1.created_at IS NOT NULL AND l2.created_at IS NOT NULL
267 GROUP BY l1.author_did, l2.author_did228 GROUP BY l1.author_did, l2.author_did
268 `); err != nil {229 `); err != nil {
269 return err230 return err
@@ -400,49 +361,3 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error {
400 e.logger.Info("user similarity computed")361 e.logger.Info("user similarity computed")
401 return tx.Commit()362 return tx.Commit()
402 }363 }
403-
404-func (e *Engine) ComputeRecommendations(ctx context.Context) error {
405- tx, err := e.db.BeginTx(ctx, nil)
406- if err != nil {
407- return err
408- }
409- defer func() { _ = tx.Rollback() }()
410-
411- if _, err := tx.ExecContext(ctx, `DELETE FROM user_feed_recommendations`); err != nil {
412- return err
413- }
414-
415- recQuery := fmt.Sprintf(`
416- INSERT INTO user_feed_recommendations (user_did, feed_url, score)
417- SELECT target, feed_url, SUM(jaccard) AS score
418- FROM (
419- SELECT us.user_a AS target, s.feed_url, us.jaccard
420- FROM user_similarity us
421- JOIN subscriptions s ON s.user_did = us.user_b
422- WHERE us.jaccard > %g
423- AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = us.user_a)
424-
425- UNION ALL
426-
427- SELECT us.user_b AS target, s.feed_url, us.jaccard
428- FROM user_similarity us
429- JOIN subscriptions s ON s.user_did = us.user_a
430- WHERE us.jaccard > %g
431- AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = us.user_b)
432- )
433- GROUP BY target, feed_url
434- ORDER BY score DESC
435- `, e.config.SimilarityThreshold, e.config.SimilarityThreshold)
436-
437- if _, err := tx.ExecContext(ctx, recQuery); err != nil {
438- return err
439- }
440-
441- e.logger.Info("feed recommendations computed")
442-
443- if err := tx.Commit(); err != nil {
444- return err
445- }
446-
447- return e.ComputeArticleRecommendations(ctx)
448-}
modified internal/cluster/jaccard_test.go +323 -36
@@ -2,6 +2,7 @@ package cluster
22
33 import (
44 "context"
5+ "fmt"
56 "os"
67 "testing"
78
@@ -64,6 +65,18 @@ func seedClusterData(t *testing.T, ctx context.Context, database *db.DB) {
6465 }
6566 }
6667
68+func seedFollowData(t *testing.T, ctx context.Context, database *db.DB) {
69+ t.Helper()
70+ follows := []struct{ user, target string }{
71+ {"did:test:alice", "did:test:bob"},
72+ {"did:test:bob", "did:test:carol"},
73+ }
74+ for _, f := range follows {
75+ _, err := database.ExecContext(ctx, `INSERT OR IGNORE INTO follows (user_did, target_did) VALUES (?, ?)`, f.user, f.target)
76+ assert.NilError(t, err)
77+ }
78+}
79+
6780 func TestComputeFeedSimilarity(t *testing.T) {
6881 ctx := context.Background()
6982 database := setupClusterTestDB(t)
@@ -94,7 +107,7 @@ func TestComputeUserSimilarity(t *testing.T) {
94107 assert.Assert(t, count > 0, "expected user similarity pairs")
95108 }
96109
97-func TestComputeRecommendations_GeneratesFeedRecsForNewUser(t *testing.T) {
110+func TestOnDemandFeedRecommendations(t *testing.T) {
98111 ctx := context.Background()
99112 database := setupClusterTestDB(t)
100113 seedClusterData(t, ctx, database)
@@ -102,7 +115,6 @@ func TestComputeRecommendations_GeneratesFeedRecsForNewUser(t *testing.T) {
102115 engine := NewEngine(database.DB, slog.Default())
103116 assert.NilError(t, engine.ComputeFeedSimilarity(ctx))
104117 assert.NilError(t, engine.ComputeUserSimilarity(ctx))
105- assert.NilError(t, engine.ComputeRecommendations(ctx))
106118
107119 recs, err := engine.GetFeedRecommendations(ctx, "did:test:carol", 10)
108120 assert.NilError(t, err)
@@ -117,7 +129,7 @@ func TestComputeRecommendations_GeneratesFeedRecsForNewUser(t *testing.T) {
117129 assert.Assert(t, found, "carol should be recommended feeds she doesn't subscribe to")
118130 }
119131
120-func TestComputeRecommendations_NoSelfRecommendations(t *testing.T) {
132+func TestNoSelfRecommendations(t *testing.T) {
121133 ctx := context.Background()
122134 database := setupClusterTestDB(t)
123135 seedClusterData(t, ctx, database)
@@ -125,7 +137,6 @@ func TestComputeRecommendations_NoSelfRecommendations(t *testing.T) {
125137 engine := NewEngine(database.DB, slog.Default())
126138 assert.NilError(t, engine.ComputeFeedSimilarity(ctx))
127139 assert.NilError(t, engine.ComputeUserSimilarity(ctx))
128- assert.NilError(t, engine.ComputeRecommendations(ctx))
129140
130141 recs, err := engine.GetFeedRecommendations(ctx, "did:test:alice", 10)
131142 assert.NilError(t, err)
@@ -141,65 +152,341 @@ func TestComputeRecommendations_NoSelfRecommendations(t *testing.T) {
141152 }
142153 }
143154
144-func TestLikesBasedSimilarity(t *testing.T) {
155+func TestDismissedFeedsExcluded(t *testing.T) {
145156 ctx := context.Background()
146157 database := setupClusterTestDB(t)
147158 seedClusterData(t, ctx, database)
148159
149- _, err := database.ExecContext(ctx, `INSERT INTO articles (feed_url, guid, title, url) VALUES (?, ?, ?, ?)`,
150- "https://a.com/feed", "art1", "Article 1", "https://a.com/art1")
160+ engine := NewEngine(database.DB, slog.Default())
161+ assert.NilError(t, engine.ComputeFeedSimilarity(ctx))
162+ assert.NilError(t, engine.ComputeUserSimilarity(ctx))
163+
164+ assert.NilError(t, engine.DismissFeed(ctx, "did:test:carol", "https://a.com/feed", "not_interested"))
165+
166+ recs, err := engine.GetFeedRecommendations(ctx, "did:test:carol", 10)
151167 assert.NilError(t, err)
152- _, err = database.ExecContext(ctx, `INSERT INTO articles (feed_url, guid, title, url) VALUES (?, ?, ?, ?)`,
153- "https://a.com/feed", "art2", "Article 2", "https://a.com/art2")
168+
169+ for _, r := range recs {
170+ assert.Assert(t, r.FeedURL != "https://a.com/feed",
171+ "dismissed feed should not appear in recommendations")
172+ }
173+}
174+
175+func TestIsFeedDismissed(t *testing.T) {
176+ ctx := context.Background()
177+ database := setupClusterTestDB(t)
178+ seedClusterData(t, ctx, database)
179+
180+ engine := NewEngine(database.DB, slog.Default())
181+
182+ dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://a.com/feed")
154183 assert.NilError(t, err)
184+ assert.Assert(t, !dismissed, "feed should not be dismissed initially")
185+
186+ assert.NilError(t, engine.DismissFeed(ctx, "did:test:alice", "https://a.com/feed", "not_interested"))
155187
156- _, err = database.ExecContext(ctx, `INSERT INTO likes (uri, author_did, feed_url, article_url, created_at) VALUES (?, ?, ?, ?, datetime('now'))`,
157- "at://alice/like/1", "did:test:alice", "https://a.com/feed", "https://a.com/art1")
188+ dismissed, err = engine.IsFeedDismissed(ctx, "did:test:alice", "https://a.com/feed")
158189 assert.NilError(t, err)
159- _, err = database.ExecContext(ctx, `INSERT INTO likes (uri, author_did, feed_url, article_url, created_at) VALUES (?, ?, ?, ?, datetime('now'))`,
160- "at://alice/like/2", "did:test:alice", "https://a.com/feed", "https://a.com/art2")
190+ assert.Assert(t, dismissed, "feed should be dismissed after dismiss call")
191+}
192+
193+func TestRecordImpressions(t *testing.T) {
194+ ctx := context.Background()
195+ database := setupClusterTestDB(t)
196+ seedClusterData(t, ctx, database)
197+
198+ engine := NewEngine(database.DB, slog.Default())
199+
200+ impressions := []Impression{
201+ {TargetType: "feed", TargetID: "https://a.com/feed"},
202+ {TargetType: "feed", TargetID: "https://b.com/feed"},
203+ }
204+ assert.NilError(t, engine.RecordImpressions(ctx, "did:test:alice", impressions))
205+
206+ var count int
207+ assert.NilError(t, database.QueryRowContext(ctx,
208+ `SELECT COUNT(*) FROM recommendation_impressions WHERE user_did = 'did:test:alice'`).Scan(&count))
209+ assert.Equal(t, count, 2)
210+
211+ assert.NilError(t, engine.RecordImpressions(ctx, "did:test:alice", impressions))
212+
213+ var shownCount int
214+ assert.NilError(t, database.QueryRowContext(ctx,
215+ `SELECT shown_count FROM recommendation_impressions WHERE user_did = 'did:test:alice' AND target_id = 'https://a.com/feed'`).Scan(&shownCount))
216+ assert.Equal(t, shownCount, 2, "shown_count should increment on repeated impression")
217+}
218+
219+func TestMarkImpressionActed(t *testing.T) {
220+ ctx := context.Background()
221+ database := setupClusterTestDB(t)
222+ seedClusterData(t, ctx, database)
223+
224+ engine := NewEngine(database.DB, slog.Default())
225+
226+ impressions := []Impression{{TargetType: "feed", TargetID: "https://a.com/feed"}}
227+ assert.NilError(t, engine.RecordImpressions(ctx, "did:test:alice", impressions))
228+
229+ assert.NilError(t, engine.MarkImpressionActed(ctx, "did:test:alice", "feed", "https://a.com/feed"))
230+
231+ var acted bool
232+ assert.NilError(t, database.QueryRowContext(ctx,
233+ `SELECT acted FROM recommendation_impressions WHERE user_did = 'did:test:alice' AND target_id = 'https://a.com/feed'`).Scan(&acted))
234+ assert.Assert(t, acted, "impression should be marked as acted")
235+}
236+
237+func TestComputeFollowDistances(t *testing.T) {
238+ ctx := context.Background()
239+ database := setupClusterTestDB(t)
240+ seedClusterData(t, ctx, database)
241+ seedFollowData(t, ctx, database)
242+
243+ engine := NewEngine(database.DB, slog.Default())
244+ assert.NilError(t, engine.ComputeFollowDistances(ctx))
245+
246+ var d1, d2 int
247+ assert.NilError(t, database.QueryRowContext(ctx,
248+ `SELECT COUNT(*) FROM follow_distances WHERE distance = 1`).Scan(&d1))
249+ assert.NilError(t, database.QueryRowContext(ctx,
250+ `SELECT COUNT(*) FROM follow_distances WHERE distance = 2`).Scan(&d2))
251+ assert.Assert(t, d1 >= 2, "expected at least 2 direct follow distances")
252+ assert.Assert(t, d2 >= 1, "expected at least 1 two-hop distance (alice -> bob -> carol)")
253+
254+ var dist int
255+ assert.NilError(t, database.QueryRowContext(ctx,
256+ `SELECT distance FROM follow_distances WHERE user_a = 'did:test:alice' AND user_b = 'did:test:carol'`).Scan(&dist))
257+ assert.Equal(t, dist, 2, "alice should be 2 hops from carol")
258+}
259+
260+func TestAutoDismissStale(t *testing.T) {
261+ ctx := context.Background()
262+ database := setupClusterTestDB(t)
263+ seedClusterData(t, ctx, database)
264+
265+ engine := NewEngine(database.DB, slog.Default())
266+
267+ _, err := database.ExecContext(ctx, `
268+ INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted)
269+ VALUES ('did:test:alice', 'feed', 'https://stale.com/feed', datetime('now', '-31 days'), datetime('now'), 20, 0)
270+ `)
161271 assert.NilError(t, err)
162- _, err = database.ExecContext(ctx, `INSERT INTO likes (uri, author_did, feed_url, article_url, created_at) VALUES (?, ?, ?, ?, datetime('now'))`,
163- "at://carol/like/1", "did:test:carol", "https://a.com/feed", "https://a.com/art1")
272+
273+ assert.NilError(t, engine.AutoDismissStale(ctx, 15, 30))
274+
275+ dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://stale.com/feed")
164276 assert.NilError(t, err)
277+ assert.Assert(t, dismissed, "stale recommendation should be auto-dismissed")
278+}
279+
280+func TestAutoDismissStale_DoesNotDismissRecent(t *testing.T) {
281+ ctx := context.Background()
282+ database := setupClusterTestDB(t)
283+ seedClusterData(t, ctx, database)
165284
166285 engine := NewEngine(database.DB, slog.Default())
167- assert.NilError(t, engine.ComputeUserSimilarity(ctx))
168286
169- var jaccard float64
170- var commonLikes int
171- assert.NilError(t, database.QueryRowContext(ctx,
172- `SELECT jaccard, common_likes FROM user_similarity WHERE user_a = ? AND user_b = ?`,
173- "did:test:alice", "did:test:carol").Scan(&jaccard, &commonLikes))
174- assert.Equal(t, commonLikes, 1, "alice and carol share 1 liked article")
175- assert.Assert(t, jaccard > 0, "likes should contribute to similarity, got %f", jaccard)
287+ _, err := database.ExecContext(ctx, `
288+ INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted)
289+ VALUES ('did:test:alice', 'feed', 'https://recent.com/feed', datetime('now'), datetime('now'), 5, 0)
290+ `)
291+ assert.NilError(t, err)
292+
293+ assert.NilError(t, engine.AutoDismissStale(ctx, 15, 30))
294+
295+ dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://recent.com/feed")
296+ assert.NilError(t, err)
297+ assert.Assert(t, !dismissed, "recent impression should not be auto-dismissed")
298+}
299+
300+func TestAutoDismissStale_DoesNotDismissActed(t *testing.T) {
301+ ctx := context.Background()
302+ database := setupClusterTestDB(t)
303+ seedClusterData(t, ctx, database)
304+
305+ engine := NewEngine(database.DB, slog.Default())
306+
307+ _, err := database.ExecContext(ctx, `
308+ INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted)
309+ VALUES ('did:test:alice', 'feed', 'https://acted.com/feed', datetime('now', '-31 days'), datetime('now'), 20, 1)
310+ `)
311+ assert.NilError(t, err)
312+
313+ assert.NilError(t, engine.AutoDismissStale(ctx, 15, 30))
314+
315+ dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://acted.com/feed")
316+ assert.NilError(t, err)
317+ assert.Assert(t, !dismissed, "acted recommendation should not be auto-dismissed")
318+}
319+
320+func TestDiversityFiltering(t *testing.T) {
321+ candidates := []*FeedRecommendation{
322+ {FeedURL: "https://a.com/1", SiteURL: "https://a.com", Score: 1.0},
323+ {FeedURL: "https://a.com/2", SiteURL: "https://a.com", Score: 0.9},
324+ {FeedURL: "https://a.com/3", SiteURL: "https://a.com", Score: 0.8},
325+ {FeedURL: "https://b.com/1", SiteURL: "https://b.com", Score: 0.7},
326+ {FeedURL: "https://b.com/2", SiteURL: "https://b.com", Score: 0.6},
327+ {FeedURL: "https://c.com/1", SiteURL: "https://c.com", Score: 0.5},
328+ }
329+
330+ result := ApplyDiversity(candidates, 6)
331+
332+ aCount := 0
333+ bCount := 0
334+ cCount := 0
335+ for _, r := range result {
336+ switch extractDomain(r.SiteURL) {
337+ case "a.com":
338+ aCount++
339+ case "b.com":
340+ bCount++
341+ case "c.com":
342+ cCount++
343+ }
344+ }
345+ assert.Assert(t, aCount <= maxPerDomain, "should limit feeds from same domain")
346+ assert.Assert(t, len(result) <= 6, "should respect topN limit")
347+ assert.Assert(t, cCount >= 1, "should include feeds from different domains")
348+}
349+
350+func TestDiversityFiltering_EmptySiteURL(t *testing.T) {
351+ candidates := []*FeedRecommendation{
352+ {FeedURL: "https://a.com/1", SiteURL: "", Score: 1.0},
353+ {FeedURL: "https://b.com/1", SiteURL: "", Score: 0.9},
354+ }
355+ result := ApplyDiversity(candidates, 5)
356+ assert.Equal(t, len(result), 2, "feeds without site_url should not be filtered out")
357+}
358+
359+func TestSignalWeights_Default(t *testing.T) {
360+ ctx := context.Background()
361+ database := setupClusterTestDB(t)
362+ seedClusterData(t, ctx, database)
363+
364+ engine := NewEngine(database.DB, slog.Default())
365+ w := engine.GetWeights(ctx, "did:test:alice")
366+
367+ assert.Equal(t, w.WSub, 1.0)
368+ assert.Equal(t, w.WLike, 0.5)
369+ assert.Equal(t, w.WTag, 0.3)
370+ assert.Equal(t, w.WSocial, 0.7)
371+ assert.Equal(t, w.WPop, 0.2)
372+ assert.Equal(t, w.WCategory, 0.4)
373+}
374+
375+func TestSignalWeights_RewardPenalize(t *testing.T) {
376+ ctx := context.Background()
377+ database := setupClusterTestDB(t)
378+ seedClusterData(t, ctx, database)
379+
380+ engine := NewEngine(database.DB, slog.Default())
381+
382+ _, err := database.ExecContext(ctx, `
383+ INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted)
384+ VALUES ('did:test:alice', 'feed', 'https://a.com/feed', datetime('now'), datetime('now'), 1, 1)
385+ `)
386+ assert.NilError(t, err)
387+ for i := range minActionsTune {
388+ _, err = database.ExecContext(ctx, `
389+ INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted)
390+ VALUES ('did:test:alice', 'feed', ?, datetime('now'), datetime('now'), 1, 1)
391+ `, fmt.Sprintf("https://%d.com/feed", i))
392+ assert.NilError(t, err)
393+ }
394+
395+ engine.RewardSignal(ctx, "did:test:alice", "social")
396+
397+ w := engine.GetWeights(ctx, "did:test:alice")
398+ assert.Assert(t, w.WSocial > 0.7, "rewarding social signal should increase w_social, got %f", w.WSocial)
176399 }
177400
178-func TestTagsBasedSimilarity(t *testing.T) {
401+func TestColdStartRecommendations(t *testing.T) {
179402 ctx := context.Background()
180403 database := setupClusterTestDB(t)
181404 seedClusterData(t, ctx, database)
405+ seedFollowData(t, ctx, database)
406+
407+ engine := NewEngine(database.DB, slog.Default())
408+ assert.NilError(t, engine.ComputeFollowDistances(ctx))
182409
183- _, err := database.ExecContext(ctx, `INSERT INTO annotations (uri, author_did, feed_url, article_url, tags, created_at) VALUES (?, ?, ?, ?, ?, datetime('now'))`,
184- "at://alice/ann/1", "did:test:alice", "https://a.com/feed", "https://a.com/art1", "go,programming")
410+ _, err := database.ExecContext(ctx, `UPDATE feeds SET subscriber_count = 2 WHERE feed_url = 'https://a.com/feed'`)
185411 assert.NilError(t, err)
186- _, err = database.ExecContext(ctx, `INSERT INTO annotations (uri, author_did, feed_url, article_url, tags, created_at) VALUES (?, ?, ?, ?, ?, datetime('now'))`,
187- "at://alice/ann/2", "did:test:alice", "https://a.com/feed", "https://a.com/art2", "rust,programming")
412+ _, err = database.ExecContext(ctx, `UPDATE feeds SET subscriber_count = 2 WHERE feed_url = 'https://b.com/feed'`)
188413 assert.NilError(t, err)
189- _, err = database.ExecContext(ctx, `INSERT INTO annotations (uri, author_did, feed_url, article_url, tags, created_at) VALUES (?, ?, ?, ?, ?, datetime('now'))`,
190- "at://carol/ann/1", "did:test:carol", "https://c.com/feed", "https://c.com/art1", "go,web")
414+
415+ _, err = database.ExecContext(ctx, `INSERT INTO users (did, handle) VALUES (?, ?)`, "did:test:newuser", "newuser")
191416 assert.NilError(t, err)
192417
418+ recs, err := engine.ColdStartRecommendations(ctx, "did:test:newuser", 10)
419+ assert.NilError(t, err)
420+ assert.Assert(t, len(recs) > 0, "new user should get cold start recommendations")
421+}
422+
423+func TestColdStartRecommendations_NotTriggeredForEstablishedUser(t *testing.T) {
424+ ctx := context.Background()
425+ database := setupClusterTestDB(t)
426+ seedClusterData(t, ctx, database)
427+
428+ engine := NewEngine(database.DB, slog.Default())
429+
430+ recs, err := engine.ColdStartRecommendations(ctx, "did:test:alice", 10)
431+ assert.NilError(t, err)
432+ assert.Assert(t, recs == nil, "established user should not get cold start recommendations")
433+}
434+
435+func TestOnDemandPeopleRecommendations(t *testing.T) {
436+ ctx := context.Background()
437+ database := setupClusterTestDB(t)
438+ seedClusterData(t, ctx, database)
439+
193440 engine := NewEngine(database.DB, slog.Default())
194441 assert.NilError(t, engine.ComputeUserSimilarity(ctx))
195442
196- var jaccard float64
197- var commonTags int
443+ recs, err := engine.GetPeopleRecommendations(ctx, "did:test:carol", 10)
444+ assert.NilError(t, err)
445+ assert.Assert(t, len(recs) > 0, "carol should get people recommendations")
446+}
447+
448+func TestDismissArticle(t *testing.T) {
449+ ctx := context.Background()
450+ database := setupClusterTestDB(t)
451+ seedClusterData(t, ctx, database)
452+
453+ engine := NewEngine(database.DB, slog.Default())
454+
455+ assert.NilError(t, engine.DismissArticle(ctx, "did:test:alice", "https://a.com/article1", "not_interested"))
456+
457+ var count int
458+ assert.NilError(t, database.QueryRowContext(ctx,
459+ `SELECT COUNT(*) FROM dismissed_recommendations WHERE user_did = 'did:test:alice' AND target_type = 'article'`).Scan(&count))
460+ assert.Equal(t, count, 1)
461+}
462+
463+func TestComputeSignalProfiles(t *testing.T) {
464+ ctx := context.Background()
465+ database := setupClusterTestDB(t)
466+ seedClusterData(t, ctx, database)
467+
468+ engine := NewEngine(database.DB, slog.Default())
469+ assert.NilError(t, engine.ComputeSignalProfiles(ctx))
470+
471+ var count int
472+ assert.NilError(t, database.QueryRowContext(ctx, `SELECT COUNT(*) FROM user_signal_profiles`).Scan(&count))
473+ assert.Assert(t, count >= 3, "expected signal profiles for all users")
474+}
475+
476+func TestDismissFeed_Idempotent(t *testing.T) {
477+ ctx := context.Background()
478+ database := setupClusterTestDB(t)
479+ seedClusterData(t, ctx, database)
480+
481+ engine := NewEngine(database.DB, slog.Default())
482+
483+ assert.NilError(t, engine.DismissFeed(ctx, "did:test:alice", "https://a.com/feed", "reason1"))
484+ assert.NilError(t, engine.DismissFeed(ctx, "did:test:alice", "https://a.com/feed", "reason2"))
485+
486+ var count int
198487 assert.NilError(t, database.QueryRowContext(ctx,
199- `SELECT jaccard, common_tags FROM user_similarity WHERE user_a = ? AND user_b = ?`,
200- "did:test:alice", "did:test:carol").Scan(&jaccard, &commonTags))
201- assert.Equal(t, commonTags, 1, "alice and carol share 1 tag (go)")
202- assert.Assert(t, jaccard > 0, "tags should contribute to similarity, got %f", jaccard)
488+ `SELECT COUNT(*) FROM dismissed_recommendations WHERE user_did = 'did:test:alice' AND target_type = 'feed'`).Scan(&count))
489+ assert.Equal(t, count, 1, "duplicate dismiss should not create extra rows")
203490 }
204491
205492 func TestDescriptionBasedFeedSimilarity(t *testing.T) {
@@ -2,6 +2,7 @@ package cluster
2 2
3 import (3 import (
4 "context"4 "context"
5+ "fmt"
5 "os"6 "os"
6 "testing"7 "testing"
7 8
@@ -64,6 +65,18 @@ func seedClusterData(t *testing.T, ctx context.Context, database *db.DB) {
64 }65 }
65 }66 }
66 67
68+func seedFollowData(t *testing.T, ctx context.Context, database *db.DB) {
69+ t.Helper()
70+ follows := []struct{ user, target string }{
71+ {"did:test:alice", "did:test:bob"},
72+ {"did:test:bob", "did:test:carol"},
73+ }
74+ for _, f := range follows {
75+ _, err := database.ExecContext(ctx, `INSERT OR IGNORE INTO follows (user_did, target_did) VALUES (?, ?)`, f.user, f.target)
76+ assert.NilError(t, err)
77+ }
78+}
79+
67 func TestComputeFeedSimilarity(t *testing.T) {80 func TestComputeFeedSimilarity(t *testing.T) {
68 ctx := context.Background()81 ctx := context.Background()
69 database := setupClusterTestDB(t)82 database := setupClusterTestDB(t)
@@ -94,7 +107,7 @@ func TestComputeUserSimilarity(t *testing.T) {
94 assert.Assert(t, count > 0, "expected user similarity pairs")107 assert.Assert(t, count > 0, "expected user similarity pairs")
95 }108 }
96 109
97-func TestComputeRecommendations_GeneratesFeedRecsForNewUser(t *testing.T) {110+func TestOnDemandFeedRecommendations(t *testing.T) {
98 ctx := context.Background()111 ctx := context.Background()
99 database := setupClusterTestDB(t)112 database := setupClusterTestDB(t)
100 seedClusterData(t, ctx, database)113 seedClusterData(t, ctx, database)
@@ -102,7 +115,6 @@ func TestComputeRecommendations_GeneratesFeedRecsForNewUser(t *testing.T) {
102 engine := NewEngine(database.DB, slog.Default())115 engine := NewEngine(database.DB, slog.Default())
103 assert.NilError(t, engine.ComputeFeedSimilarity(ctx))116 assert.NilError(t, engine.ComputeFeedSimilarity(ctx))
104 assert.NilError(t, engine.ComputeUserSimilarity(ctx))117 assert.NilError(t, engine.ComputeUserSimilarity(ctx))
105- assert.NilError(t, engine.ComputeRecommendations(ctx))
106 118
107 recs, err := engine.GetFeedRecommendations(ctx, "did:test:carol", 10)119 recs, err := engine.GetFeedRecommendations(ctx, "did:test:carol", 10)
108 assert.NilError(t, err)120 assert.NilError(t, err)
@@ -117,7 +129,7 @@ func TestComputeRecommendations_GeneratesFeedRecsForNewUser(t *testing.T) {
117 assert.Assert(t, found, "carol should be recommended feeds she doesn't subscribe to")129 assert.Assert(t, found, "carol should be recommended feeds she doesn't subscribe to")
118 }130 }
119 131
120-func TestComputeRecommendations_NoSelfRecommendations(t *testing.T) {132+func TestNoSelfRecommendations(t *testing.T) {
121 ctx := context.Background()133 ctx := context.Background()
122 database := setupClusterTestDB(t)134 database := setupClusterTestDB(t)
123 seedClusterData(t, ctx, database)135 seedClusterData(t, ctx, database)
@@ -125,7 +137,6 @@ func TestComputeRecommendations_NoSelfRecommendations(t *testing.T) {
125 engine := NewEngine(database.DB, slog.Default())137 engine := NewEngine(database.DB, slog.Default())
126 assert.NilError(t, engine.ComputeFeedSimilarity(ctx))138 assert.NilError(t, engine.ComputeFeedSimilarity(ctx))
127 assert.NilError(t, engine.ComputeUserSimilarity(ctx))139 assert.NilError(t, engine.ComputeUserSimilarity(ctx))
128- assert.NilError(t, engine.ComputeRecommendations(ctx))
129 140
130 recs, err := engine.GetFeedRecommendations(ctx, "did:test:alice", 10)141 recs, err := engine.GetFeedRecommendations(ctx, "did:test:alice", 10)
131 assert.NilError(t, err)142 assert.NilError(t, err)
@@ -141,65 +152,341 @@ func TestComputeRecommendations_NoSelfRecommendations(t *testing.T) {
141 }152 }
142 }153 }
143 154
144-func TestLikesBasedSimilarity(t *testing.T) {155+func TestDismissedFeedsExcluded(t *testing.T) {
145 ctx := context.Background()156 ctx := context.Background()
146 database := setupClusterTestDB(t)157 database := setupClusterTestDB(t)
147 seedClusterData(t, ctx, database)158 seedClusterData(t, ctx, database)
148 159
149- _, err := database.ExecContext(ctx, `INSERT INTO articles (feed_url, guid, title, url) VALUES (?, ?, ?, ?)`,160+ engine := NewEngine(database.DB, slog.Default())
150- "https://a.com/feed", "art1", "Article 1", "https://a.com/art1")161+ assert.NilError(t, engine.ComputeFeedSimilarity(ctx))
162+ assert.NilError(t, engine.ComputeUserSimilarity(ctx))
163+
164+ assert.NilError(t, engine.DismissFeed(ctx, "did:test:carol", "https://a.com/feed", "not_interested"))
165+
166+ recs, err := engine.GetFeedRecommendations(ctx, "did:test:carol", 10)
151 assert.NilError(t, err)167 assert.NilError(t, err)
152- _, err = database.ExecContext(ctx, `INSERT INTO articles (feed_url, guid, title, url) VALUES (?, ?, ?, ?)`,168+
153- "https://a.com/feed", "art2", "Article 2", "https://a.com/art2")169+ for _, r := range recs {
170+ assert.Assert(t, r.FeedURL != "https://a.com/feed",
171+ "dismissed feed should not appear in recommendations")
172+ }
173+}
174+
175+func TestIsFeedDismissed(t *testing.T) {
176+ ctx := context.Background()
177+ database := setupClusterTestDB(t)
178+ seedClusterData(t, ctx, database)
179+
180+ engine := NewEngine(database.DB, slog.Default())
181+
182+ dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://a.com/feed")
154 assert.NilError(t, err)183 assert.NilError(t, err)
184+ assert.Assert(t, !dismissed, "feed should not be dismissed initially")
185+
186+ assert.NilError(t, engine.DismissFeed(ctx, "did:test:alice", "https://a.com/feed", "not_interested"))
155 187
156- _, err = database.ExecContext(ctx, `INSERT INTO likes (uri, author_did, feed_url, article_url, created_at) VALUES (?, ?, ?, ?, datetime('now'))`,188+ dismissed, err = engine.IsFeedDismissed(ctx, "did:test:alice", "https://a.com/feed")
157- "at://alice/like/1", "did:test:alice", "https://a.com/feed", "https://a.com/art1")
158 assert.NilError(t, err)189 assert.NilError(t, err)
159- _, err = database.ExecContext(ctx, `INSERT INTO likes (uri, author_did, feed_url, article_url, created_at) VALUES (?, ?, ?, ?, datetime('now'))`,190+ assert.Assert(t, dismissed, "feed should be dismissed after dismiss call")
160- "at://alice/like/2", "did:test:alice", "https://a.com/feed", "https://a.com/art2")191+}
192+
193+func TestRecordImpressions(t *testing.T) {
194+ ctx := context.Background()
195+ database := setupClusterTestDB(t)
196+ seedClusterData(t, ctx, database)
197+
198+ engine := NewEngine(database.DB, slog.Default())
199+
200+ impressions := []Impression{
201+ {TargetType: "feed", TargetID: "https://a.com/feed"},
202+ {TargetType: "feed", TargetID: "https://b.com/feed"},
203+ }
204+ assert.NilError(t, engine.RecordImpressions(ctx, "did:test:alice", impressions))
205+
206+ var count int
207+ assert.NilError(t, database.QueryRowContext(ctx,
208+ `SELECT COUNT(*) FROM recommendation_impressions WHERE user_did = 'did:test:alice'`).Scan(&count))
209+ assert.Equal(t, count, 2)
210+
211+ assert.NilError(t, engine.RecordImpressions(ctx, "did:test:alice", impressions))
212+
213+ var shownCount int
214+ assert.NilError(t, database.QueryRowContext(ctx,
215+ `SELECT shown_count FROM recommendation_impressions WHERE user_did = 'did:test:alice' AND target_id = 'https://a.com/feed'`).Scan(&shownCount))
216+ assert.Equal(t, shownCount, 2, "shown_count should increment on repeated impression")
217+}
218+
219+func TestMarkImpressionActed(t *testing.T) {
220+ ctx := context.Background()
221+ database := setupClusterTestDB(t)
222+ seedClusterData(t, ctx, database)
223+
224+ engine := NewEngine(database.DB, slog.Default())
225+
226+ impressions := []Impression{{TargetType: "feed", TargetID: "https://a.com/feed"}}
227+ assert.NilError(t, engine.RecordImpressions(ctx, "did:test:alice", impressions))
228+
229+ assert.NilError(t, engine.MarkImpressionActed(ctx, "did:test:alice", "feed", "https://a.com/feed"))
230+
231+ var acted bool
232+ assert.NilError(t, database.QueryRowContext(ctx,
233+ `SELECT acted FROM recommendation_impressions WHERE user_did = 'did:test:alice' AND target_id = 'https://a.com/feed'`).Scan(&acted))
234+ assert.Assert(t, acted, "impression should be marked as acted")
235+}
236+
237+func TestComputeFollowDistances(t *testing.T) {
238+ ctx := context.Background()
239+ database := setupClusterTestDB(t)
240+ seedClusterData(t, ctx, database)
241+ seedFollowData(t, ctx, database)
242+
243+ engine := NewEngine(database.DB, slog.Default())
244+ assert.NilError(t, engine.ComputeFollowDistances(ctx))
245+
246+ var d1, d2 int
247+ assert.NilError(t, database.QueryRowContext(ctx,
248+ `SELECT COUNT(*) FROM follow_distances WHERE distance = 1`).Scan(&d1))
249+ assert.NilError(t, database.QueryRowContext(ctx,
250+ `SELECT COUNT(*) FROM follow_distances WHERE distance = 2`).Scan(&d2))
251+ assert.Assert(t, d1 >= 2, "expected at least 2 direct follow distances")
252+ assert.Assert(t, d2 >= 1, "expected at least 1 two-hop distance (alice -> bob -> carol)")
253+
254+ var dist int
255+ assert.NilError(t, database.QueryRowContext(ctx,
256+ `SELECT distance FROM follow_distances WHERE user_a = 'did:test:alice' AND user_b = 'did:test:carol'`).Scan(&dist))
257+ assert.Equal(t, dist, 2, "alice should be 2 hops from carol")
258+}
259+
260+func TestAutoDismissStale(t *testing.T) {
261+ ctx := context.Background()
262+ database := setupClusterTestDB(t)
263+ seedClusterData(t, ctx, database)
264+
265+ engine := NewEngine(database.DB, slog.Default())
266+
267+ _, err := database.ExecContext(ctx, `
268+ INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted)
269+ VALUES ('did:test:alice', 'feed', 'https://stale.com/feed', datetime('now', '-31 days'), datetime('now'), 20, 0)
270+ `)
161 assert.NilError(t, err)271 assert.NilError(t, err)
162- _, err = database.ExecContext(ctx, `INSERT INTO likes (uri, author_did, feed_url, article_url, created_at) VALUES (?, ?, ?, ?, datetime('now'))`,272+
163- "at://carol/like/1", "did:test:carol", "https://a.com/feed", "https://a.com/art1")273+ assert.NilError(t, engine.AutoDismissStale(ctx, 15, 30))
274+
275+ dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://stale.com/feed")
164 assert.NilError(t, err)276 assert.NilError(t, err)
277+ assert.Assert(t, dismissed, "stale recommendation should be auto-dismissed")
278+}
279+
280+func TestAutoDismissStale_DoesNotDismissRecent(t *testing.T) {
281+ ctx := context.Background()
282+ database := setupClusterTestDB(t)
283+ seedClusterData(t, ctx, database)
165 284
166 engine := NewEngine(database.DB, slog.Default())285 engine := NewEngine(database.DB, slog.Default())
167- assert.NilError(t, engine.ComputeUserSimilarity(ctx))
168 286
169- var jaccard float64287+ _, err := database.ExecContext(ctx, `
170- var commonLikes int288+ INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted)
171- assert.NilError(t, database.QueryRowContext(ctx,289+ VALUES ('did:test:alice', 'feed', 'https://recent.com/feed', datetime('now'), datetime('now'), 5, 0)
172- `SELECT jaccard, common_likes FROM user_similarity WHERE user_a = ? AND user_b = ?`,290+ `)
173- "did:test:alice", "did:test:carol").Scan(&jaccard, &commonLikes))291+ assert.NilError(t, err)
174- assert.Equal(t, commonLikes, 1, "alice and carol share 1 liked article")292+
175- assert.Assert(t, jaccard > 0, "likes should contribute to similarity, got %f", jaccard)293+ assert.NilError(t, engine.AutoDismissStale(ctx, 15, 30))
294+
295+ dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://recent.com/feed")
296+ assert.NilError(t, err)
297+ assert.Assert(t, !dismissed, "recent impression should not be auto-dismissed")
298+}
299+
300+func TestAutoDismissStale_DoesNotDismissActed(t *testing.T) {
301+ ctx := context.Background()
302+ database := setupClusterTestDB(t)
303+ seedClusterData(t, ctx, database)
304+
305+ engine := NewEngine(database.DB, slog.Default())
306+
307+ _, err := database.ExecContext(ctx, `
308+ INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted)
309+ VALUES ('did:test:alice', 'feed', 'https://acted.com/feed', datetime('now', '-31 days'), datetime('now'), 20, 1)
310+ `)
311+ assert.NilError(t, err)
312+
313+ assert.NilError(t, engine.AutoDismissStale(ctx, 15, 30))
314+
315+ dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://acted.com/feed")
316+ assert.NilError(t, err)
317+ assert.Assert(t, !dismissed, "acted recommendation should not be auto-dismissed")
318+}
319+
320+func TestDiversityFiltering(t *testing.T) {
321+ candidates := []*FeedRecommendation{
322+ {FeedURL: "https://a.com/1", SiteURL: "https://a.com", Score: 1.0},
323+ {FeedURL: "https://a.com/2", SiteURL: "https://a.com", Score: 0.9},
324+ {FeedURL: "https://a.com/3", SiteURL: "https://a.com", Score: 0.8},
325+ {FeedURL: "https://b.com/1", SiteURL: "https://b.com", Score: 0.7},
326+ {FeedURL: "https://b.com/2", SiteURL: "https://b.com", Score: 0.6},
327+ {FeedURL: "https://c.com/1", SiteURL: "https://c.com", Score: 0.5},
328+ }
329+
330+ result := ApplyDiversity(candidates, 6)
331+
332+ aCount := 0
333+ bCount := 0
334+ cCount := 0
335+ for _, r := range result {
336+ switch extractDomain(r.SiteURL) {
337+ case "a.com":
338+ aCount++
339+ case "b.com":
340+ bCount++
341+ case "c.com":
342+ cCount++
343+ }
344+ }
345+ assert.Assert(t, aCount <= maxPerDomain, "should limit feeds from same domain")
346+ assert.Assert(t, len(result) <= 6, "should respect topN limit")
347+ assert.Assert(t, cCount >= 1, "should include feeds from different domains")
348+}
349+
350+func TestDiversityFiltering_EmptySiteURL(t *testing.T) {
351+ candidates := []*FeedRecommendation{
352+ {FeedURL: "https://a.com/1", SiteURL: "", Score: 1.0},
353+ {FeedURL: "https://b.com/1", SiteURL: "", Score: 0.9},
354+ }
355+ result := ApplyDiversity(candidates, 5)
356+ assert.Equal(t, len(result), 2, "feeds without site_url should not be filtered out")
357+}
358+
359+func TestSignalWeights_Default(t *testing.T) {
360+ ctx := context.Background()
361+ database := setupClusterTestDB(t)
362+ seedClusterData(t, ctx, database)
363+
364+ engine := NewEngine(database.DB, slog.Default())
365+ w := engine.GetWeights(ctx, "did:test:alice")
366+
367+ assert.Equal(t, w.WSub, 1.0)
368+ assert.Equal(t, w.WLike, 0.5)
369+ assert.Equal(t, w.WTag, 0.3)
370+ assert.Equal(t, w.WSocial, 0.7)
371+ assert.Equal(t, w.WPop, 0.2)
372+ assert.Equal(t, w.WCategory, 0.4)
373+}
374+
375+func TestSignalWeights_RewardPenalize(t *testing.T) {
376+ ctx := context.Background()
377+ database := setupClusterTestDB(t)
378+ seedClusterData(t, ctx, database)
379+
380+ engine := NewEngine(database.DB, slog.Default())
381+
382+ _, err := database.ExecContext(ctx, `
383+ INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted)
384+ VALUES ('did:test:alice', 'feed', 'https://a.com/feed', datetime('now'), datetime('now'), 1, 1)
385+ `)
386+ assert.NilError(t, err)
387+ for i := range minActionsTune {
388+ _, err = database.ExecContext(ctx, `
389+ INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted)
390+ VALUES ('did:test:alice', 'feed', ?, datetime('now'), datetime('now'), 1, 1)
391+ `, fmt.Sprintf("https://%d.com/feed", i))
392+ assert.NilError(t, err)
393+ }
394+
395+ engine.RewardSignal(ctx, "did:test:alice", "social")
396+
397+ w := engine.GetWeights(ctx, "did:test:alice")
398+ assert.Assert(t, w.WSocial > 0.7, "rewarding social signal should increase w_social, got %f", w.WSocial)
176 }399 }
177 400
178-func TestTagsBasedSimilarity(t *testing.T) {401+func TestColdStartRecommendations(t *testing.T) {
179 ctx := context.Background()402 ctx := context.Background()
180 database := setupClusterTestDB(t)403 database := setupClusterTestDB(t)
181 seedClusterData(t, ctx, database)404 seedClusterData(t, ctx, database)
405+ seedFollowData(t, ctx, database)
406+
407+ engine := NewEngine(database.DB, slog.Default())
408+ assert.NilError(t, engine.ComputeFollowDistances(ctx))
182 409
183- _, err := database.ExecContext(ctx, `INSERT INTO annotations (uri, author_did, feed_url, article_url, tags, created_at) VALUES (?, ?, ?, ?, ?, datetime('now'))`,410+ _, err := database.ExecContext(ctx, `UPDATE feeds SET subscriber_count = 2 WHERE feed_url = 'https://a.com/feed'`)
184- "at://alice/ann/1", "did:test:alice", "https://a.com/feed", "https://a.com/art1", "go,programming")
185 assert.NilError(t, err)411 assert.NilError(t, err)
186- _, err = database.ExecContext(ctx, `INSERT INTO annotations (uri, author_did, feed_url, article_url, tags, created_at) VALUES (?, ?, ?, ?, ?, datetime('now'))`,412+ _, err = database.ExecContext(ctx, `UPDATE feeds SET subscriber_count = 2 WHERE feed_url = 'https://b.com/feed'`)
187- "at://alice/ann/2", "did:test:alice", "https://a.com/feed", "https://a.com/art2", "rust,programming")
188 assert.NilError(t, err)413 assert.NilError(t, err)
189- _, err = database.ExecContext(ctx, `INSERT INTO annotations (uri, author_did, feed_url, article_url, tags, created_at) VALUES (?, ?, ?, ?, ?, datetime('now'))`,414+
190- "at://carol/ann/1", "did:test:carol", "https://c.com/feed", "https://c.com/art1", "go,web")415+ _, err = database.ExecContext(ctx, `INSERT INTO users (did, handle) VALUES (?, ?)`, "did:test:newuser", "newuser")
191 assert.NilError(t, err)416 assert.NilError(t, err)
192 417
418+ recs, err := engine.ColdStartRecommendations(ctx, "did:test:newuser", 10)
419+ assert.NilError(t, err)
420+ assert.Assert(t, len(recs) > 0, "new user should get cold start recommendations")
421+}
422+
423+func TestColdStartRecommendations_NotTriggeredForEstablishedUser(t *testing.T) {
424+ ctx := context.Background()
425+ database := setupClusterTestDB(t)
426+ seedClusterData(t, ctx, database)
427+
428+ engine := NewEngine(database.DB, slog.Default())
429+
430+ recs, err := engine.ColdStartRecommendations(ctx, "did:test:alice", 10)
431+ assert.NilError(t, err)
432+ assert.Assert(t, recs == nil, "established user should not get cold start recommendations")
433+}
434+
435+func TestOnDemandPeopleRecommendations(t *testing.T) {
436+ ctx := context.Background()
437+ database := setupClusterTestDB(t)
438+ seedClusterData(t, ctx, database)
439+
193 engine := NewEngine(database.DB, slog.Default())440 engine := NewEngine(database.DB, slog.Default())
194 assert.NilError(t, engine.ComputeUserSimilarity(ctx))441 assert.NilError(t, engine.ComputeUserSimilarity(ctx))
195 442
196- var jaccard float64443+ recs, err := engine.GetPeopleRecommendations(ctx, "did:test:carol", 10)
197- var commonTags int444+ assert.NilError(t, err)
445+ assert.Assert(t, len(recs) > 0, "carol should get people recommendations")
446+}
447+
448+func TestDismissArticle(t *testing.T) {
449+ ctx := context.Background()
450+ database := setupClusterTestDB(t)
451+ seedClusterData(t, ctx, database)
452+
453+ engine := NewEngine(database.DB, slog.Default())
454+
455+ assert.NilError(t, engine.DismissArticle(ctx, "did:test:alice", "https://a.com/article1", "not_interested"))
456+
457+ var count int
458+ assert.NilError(t, database.QueryRowContext(ctx,
459+ `SELECT COUNT(*) FROM dismissed_recommendations WHERE user_did = 'did:test:alice' AND target_type = 'article'`).Scan(&count))
460+ assert.Equal(t, count, 1)
461+}
462+
463+func TestComputeSignalProfiles(t *testing.T) {
464+ ctx := context.Background()
465+ database := setupClusterTestDB(t)
466+ seedClusterData(t, ctx, database)
467+
468+ engine := NewEngine(database.DB, slog.Default())
469+ assert.NilError(t, engine.ComputeSignalProfiles(ctx))
470+
471+ var count int
472+ assert.NilError(t, database.QueryRowContext(ctx, `SELECT COUNT(*) FROM user_signal_profiles`).Scan(&count))
473+ assert.Assert(t, count >= 3, "expected signal profiles for all users")
474+}
475+
476+func TestDismissFeed_Idempotent(t *testing.T) {
477+ ctx := context.Background()
478+ database := setupClusterTestDB(t)
479+ seedClusterData(t, ctx, database)
480+
481+ engine := NewEngine(database.DB, slog.Default())
482+
483+ assert.NilError(t, engine.DismissFeed(ctx, "did:test:alice", "https://a.com/feed", "reason1"))
484+ assert.NilError(t, engine.DismissFeed(ctx, "did:test:alice", "https://a.com/feed", "reason2"))
485+
486+ var count int
198 assert.NilError(t, database.QueryRowContext(ctx,487 assert.NilError(t, database.QueryRowContext(ctx,
199- `SELECT jaccard, common_tags FROM user_similarity WHERE user_a = ? AND user_b = ?`,488+ `SELECT COUNT(*) FROM dismissed_recommendations WHERE user_did = 'did:test:alice' AND target_type = 'feed'`).Scan(&count))
200- "did:test:alice", "did:test:carol").Scan(&jaccard, &commonTags))489+ assert.Equal(t, count, 1, "duplicate dismiss should not create extra rows")
201- assert.Equal(t, commonTags, 1, "alice and carol share 1 tag (go)")
202- assert.Assert(t, jaccard > 0, "tags should contribute to similarity, got %f", jaccard)
203 }490 }
204 491
205 func TestDescriptionBasedFeedSimilarity(t *testing.T) {492 func TestDescriptionBasedFeedSimilarity(t *testing.T) {
deleted internal/cluster/recommender.go +0 -165
deleted file mode 100644
@@ -1,165 +0,0 @@
1-package cluster
2-
3-import (
4- "context"
5- "database/sql"
6-)
7-
8-type FeedRecommendation struct {
9- FeedURL string
10- Title string
11- SiteURL string
12- Description string
13- SubscriberCount int
14- FaviconURL string
15- Score float64
16-}
17-
18-type PersonRecommendation struct {
19- DID string
20- Handle string
21- DisplayName string
22- AvatarURL string
23- Jaccard float64
24- CommonFeeds int
25- CommonLikes int
26- CommonTags int
27-}
28-
29-type ArticleRecommendation struct {
30- ArticleID int64
31- Title string
32- URL string
33- FeedURL string
34- FeedTitle string
35- Author string
36- Summary string
37- Published sql.NullTime
38- Score float64
39-}
40-
41-type SimilarFeed struct {
42- FeedURL string
43- Title string
44- SiteURL string
45- Description string
46- FeedType string
47- Jaccard float64
48-}
49-
50-func (e *Engine) GetFeedRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) {
51- rows, err := e.db.QueryContext(ctx, `
52- SELECT r.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''),
53- COALESCE(f.description, ''), f.subscriber_count, COALESCE(f.favicon_url, ''), r.score
54- FROM user_feed_recommendations r
55- JOIN feeds f ON f.feed_url = r.feed_url
56- WHERE r.user_did = ?
57- AND r.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
58- ORDER BY r.score DESC
59- LIMIT ?
60- `, userDID, userDID, limit)
61- if err != nil {
62- return nil, err
63- }
64- defer rows.Close()
65-
66- var results []*FeedRecommendation
67- for rows.Next() {
68- rec := &FeedRecommendation{}
69- if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description,
70- &rec.SubscriberCount, &rec.FaviconURL, &rec.Score); err != nil {
71- return nil, err
72- }
73- results = append(results, rec)
74- }
75- return results, rows.Err()
76-}
77-
78-func (e *Engine) GetPeopleRecommendations(ctx context.Context, userDID string, limit int) ([]*PersonRecommendation, error) {
79- rows, err := e.db.QueryContext(ctx, `
80- SELECT u.did, u.handle, COALESCE(u.display_name, ''), COALESCE(u.avatar_url, ''),
81- sim.jaccard, sim.common_feeds, COALESCE(sim.common_likes, 0), COALESCE(sim.common_tags, 0)
82- FROM (
83- SELECT user_b AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_a = ?
84- UNION ALL
85- SELECT user_a AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_b = ?
86- ) sim
87- JOIN users u ON u.did = sim.peer_did
88- WHERE u.handle IS NOT NULL AND u.handle != ''
89- ORDER BY sim.jaccard DESC
90- LIMIT ?
91- `, userDID, userDID, limit)
92- if err != nil {
93- return nil, err
94- }
95- defer rows.Close()
96-
97- var results []*PersonRecommendation
98- for rows.Next() {
99- rec := &PersonRecommendation{}
100- if err := rows.Scan(&rec.DID, &rec.Handle, &rec.DisplayName, &rec.AvatarURL,
101- &rec.Jaccard, &rec.CommonFeeds, &rec.CommonLikes, &rec.CommonTags); err != nil {
102- return nil, err
103- }
104- results = append(results, rec)
105- }
106- return results, rows.Err()
107-}
108-
109-func (e *Engine) GetArticleRecommendations(ctx context.Context, userDID string, limit int) ([]*ArticleRecommendation, error) {
110- rows, err := e.db.QueryContext(ctx, `
111- SELECT a.id, a.title, COALESCE(a.url, ''), r.feed_url, COALESCE(f.title, ''),
112- COALESCE(a.author, ''), COALESCE(a.summary, ''), a.published, r.score
113- FROM user_article_recommendations r
114- JOIN articles a ON a.feed_url = r.feed_url AND a.url = r.article_url
115- LEFT JOIN feeds f ON f.feed_url = r.feed_url
116- WHERE r.user_did = ?
117- ORDER BY r.score DESC
118- LIMIT ?
119- `, userDID, limit)
120- if err != nil {
121- return nil, err
122- }
123- defer rows.Close()
124-
125- var recs []*ArticleRecommendation
126- for rows.Next() {
127- rec := &ArticleRecommendation{}
128- if err := rows.Scan(&rec.ArticleID, &rec.Title, &rec.URL, &rec.FeedURL, &rec.FeedTitle,
129- &rec.Author, &rec.Summary, &rec.Published, &rec.Score); err != nil {
130- return nil, err
131- }
132- recs = append(recs, rec)
133- }
134- return recs, rows.Err()
135-}
136-
137-func (e *Engine) GetSimilarFeeds(ctx context.Context, feedURL string, limit int) ([]*SimilarFeed, error) {
138- rows, err := e.db.QueryContext(ctx, `
139- SELECT f.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''),
140- COALESCE(f.description, ''), COALESCE(f.feed_type, ''), sim.jaccard
141- FROM (
142- SELECT feed_b AS peer_url, jaccard FROM feed_similarity WHERE feed_a = ?
143- UNION ALL
144- SELECT feed_a AS peer_url, jaccard FROM feed_similarity WHERE feed_b = ?
145- ) sim
146- JOIN feeds f ON f.feed_url = sim.peer_url
147- ORDER BY sim.jaccard DESC
148- LIMIT ?
149- `, feedURL, feedURL, limit)
150- if err != nil {
151- return nil, err
152- }
153- defer rows.Close()
154-
155- var results []*SimilarFeed
156- for rows.Next() {
157- rec := &SimilarFeed{}
158- if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description,
159- &rec.FeedType, &rec.Jaccard); err != nil {
160- return nil, err
161- }
162- results = append(results, rec)
163- }
164- return results, rows.Err()
165-}
deleted file mode 100644
@@ -1,165 +0,0 @@
1-package cluster
2-
3-import (
4- "context"
5- "database/sql"
6-)
7-
8-type FeedRecommendation struct {
9- FeedURL string
10- Title string
11- SiteURL string
12- Description string
13- SubscriberCount int
14- FaviconURL string
15- Score float64
16-}
17-
18-type PersonRecommendation struct {
19- DID string
20- Handle string
21- DisplayName string
22- AvatarURL string
23- Jaccard float64
24- CommonFeeds int
25- CommonLikes int
26- CommonTags int
27-}
28-
29-type ArticleRecommendation struct {
30- ArticleID int64
31- Title string
32- URL string
33- FeedURL string
34- FeedTitle string
35- Author string
36- Summary string
37- Published sql.NullTime
38- Score float64
39-}
40-
41-type SimilarFeed struct {
42- FeedURL string
43- Title string
44- SiteURL string
45- Description string
46- FeedType string
47- Jaccard float64
48-}
49-
50-func (e *Engine) GetFeedRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) {
51- rows, err := e.db.QueryContext(ctx, `
52- SELECT r.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''),
53- COALESCE(f.description, ''), f.subscriber_count, COALESCE(f.favicon_url, ''), r.score
54- FROM user_feed_recommendations r
55- JOIN feeds f ON f.feed_url = r.feed_url
56- WHERE r.user_did = ?
57- AND r.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
58- ORDER BY r.score DESC
59- LIMIT ?
60- `, userDID, userDID, limit)
61- if err != nil {
62- return nil, err
63- }
64- defer rows.Close()
65-
66- var results []*FeedRecommendation
67- for rows.Next() {
68- rec := &FeedRecommendation{}
69- if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description,
70- &rec.SubscriberCount, &rec.FaviconURL, &rec.Score); err != nil {
71- return nil, err
72- }
73- results = append(results, rec)
74- }
75- return results, rows.Err()
76-}
77-
78-func (e *Engine) GetPeopleRecommendations(ctx context.Context, userDID string, limit int) ([]*PersonRecommendation, error) {
79- rows, err := e.db.QueryContext(ctx, `
80- SELECT u.did, u.handle, COALESCE(u.display_name, ''), COALESCE(u.avatar_url, ''),
81- sim.jaccard, sim.common_feeds, COALESCE(sim.common_likes, 0), COALESCE(sim.common_tags, 0)
82- FROM (
83- SELECT user_b AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_a = ?
84- UNION ALL
85- SELECT user_a AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_b = ?
86- ) sim
87- JOIN users u ON u.did = sim.peer_did
88- WHERE u.handle IS NOT NULL AND u.handle != ''
89- ORDER BY sim.jaccard DESC
90- LIMIT ?
91- `, userDID, userDID, limit)
92- if err != nil {
93- return nil, err
94- }
95- defer rows.Close()
96-
97- var results []*PersonRecommendation
98- for rows.Next() {
99- rec := &PersonRecommendation{}
100- if err := rows.Scan(&rec.DID, &rec.Handle, &rec.DisplayName, &rec.AvatarURL,
101- &rec.Jaccard, &rec.CommonFeeds, &rec.CommonLikes, &rec.CommonTags); err != nil {
102- return nil, err
103- }
104- results = append(results, rec)
105- }
106- return results, rows.Err()
107-}
108-
109-func (e *Engine) GetArticleRecommendations(ctx context.Context, userDID string, limit int) ([]*ArticleRecommendation, error) {
110- rows, err := e.db.QueryContext(ctx, `
111- SELECT a.id, a.title, COALESCE(a.url, ''), r.feed_url, COALESCE(f.title, ''),
112- COALESCE(a.author, ''), COALESCE(a.summary, ''), a.published, r.score
113- FROM user_article_recommendations r
114- JOIN articles a ON a.feed_url = r.feed_url AND a.url = r.article_url
115- LEFT JOIN feeds f ON f.feed_url = r.feed_url
116- WHERE r.user_did = ?
117- ORDER BY r.score DESC
118- LIMIT ?
119- `, userDID, limit)
120- if err != nil {
121- return nil, err
122- }
123- defer rows.Close()
124-
125- var recs []*ArticleRecommendation
126- for rows.Next() {
127- rec := &ArticleRecommendation{}
128- if err := rows.Scan(&rec.ArticleID, &rec.Title, &rec.URL, &rec.FeedURL, &rec.FeedTitle,
129- &rec.Author, &rec.Summary, &rec.Published, &rec.Score); err != nil {
130- return nil, err
131- }
132- recs = append(recs, rec)
133- }
134- return recs, rows.Err()
135-}
136-
137-func (e *Engine) GetSimilarFeeds(ctx context.Context, feedURL string, limit int) ([]*SimilarFeed, error) {
138- rows, err := e.db.QueryContext(ctx, `
139- SELECT f.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''),
140- COALESCE(f.description, ''), COALESCE(f.feed_type, ''), sim.jaccard
141- FROM (
142- SELECT feed_b AS peer_url, jaccard FROM feed_similarity WHERE feed_a = ?
143- UNION ALL
144- SELECT feed_a AS peer_url, jaccard FROM feed_similarity WHERE feed_b = ?
145- ) sim
146- JOIN feeds f ON f.feed_url = sim.peer_url
147- ORDER BY sim.jaccard DESC
148- LIMIT ?
149- `, feedURL, feedURL, limit)
150- if err != nil {
151- return nil, err
152- }
153- defer rows.Close()
154-
155- var results []*SimilarFeed
156- for rows.Next() {
157- rec := &SimilarFeed{}
158- if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description,
159- &rec.FeedType, &rec.Jaccard); err != nil {
160- return nil, err
161- }
162- results = append(results, rec)
163- }
164- return results, rows.Err()
165-}
added internal/cluster/scoring.go +376 -0
new file mode 100644
@@ -0,0 +1,376 @@
1+package cluster
2+
3+import (
4+ "context"
5+ "database/sql"
6+)
7+
8+type FeedRecommendation struct {
9+ FeedURL string
10+ Title string
11+ SiteURL string
12+ Description string
13+ SubscriberCount int
14+ FaviconURL string
15+ Score float64
16+}
17+
18+type PersonRecommendation struct {
19+ DID string
20+ Handle string
21+ DisplayName string
22+ AvatarURL string
23+ Jaccard float64
24+ CommonFeeds int
25+ CommonLikes int
26+ CommonTags int
27+}
28+
29+type ArticleRecommendation struct {
30+ ArticleID int64
31+ Title string
32+ URL string
33+ FeedURL string
34+ FeedTitle string
35+ Author string
36+ Summary string
37+ Published sql.NullTime
38+ Score float64
39+}
40+
41+func (e *Engine) GetFeedRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) {
42+ subCount := 0
43+ _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM subscriptions WHERE user_did = ?`, userDID).Scan(&subCount)
44+
45+ if subCount < 5 {
46+ recs, err := e.ColdStartRecommendations(ctx, userDID, limit*2)
47+ if err == nil && len(recs) > 0 {
48+ return ApplyDiversity(recs, limit), nil
49+ }
50+ }
51+
52+ recs, err := e.ComputeFeedRecommendationsOnDemand(ctx, userDID, limit*2)
53+ if err != nil {
54+ return nil, err
55+ }
56+
57+ return ApplyDiversity(recs, limit), nil
58+}
59+
60+func (e *Engine) GetPeopleRecommendations(ctx context.Context, userDID string, limit int) ([]*PersonRecommendation, error) {
61+ return e.ComputePeopleRecommendationsOnDemand(ctx, userDID, limit)
62+}
63+
64+func (e *Engine) GetArticleRecommendations(ctx context.Context, userDID string, limit int) ([]*ArticleRecommendation, error) {
65+ return e.ComputeArticleRecommendationsOnDemand(ctx, userDID, limit)
66+}
67+
68+type SignalWeights struct {
69+ WSub float64
70+ WLike float64
71+ WTag float64
72+ WSocial float64
73+ WPop float64
74+ WCategory float64
75+}
76+
77+func defaultWeights() SignalWeights {
78+ return SignalWeights{
79+ WSub: 1.0,
80+ WLike: 0.5,
81+ WTag: 0.3,
82+ WSocial: 0.7,
83+ WPop: 0.2,
84+ WCategory: 0.4,
85+ }
86+}
87+
88+func (e *Engine) GetWeights(ctx context.Context, userDID string) SignalWeights {
89+ w := defaultWeights()
90+ var dbW SignalWeights
91+ err := e.db.QueryRowContext(ctx, `
92+ SELECT w_sub, w_like, w_tag, w_social, w_pop, w_category
93+ FROM user_signal_weights WHERE user_did = ?
94+ `, userDID).Scan(&dbW.WSub, &dbW.WLike, &dbW.WTag, &dbW.WSocial, &dbW.WPop, &dbW.WCategory)
95+ if err == nil {
96+ return dbW
97+ }
98+ return w
99+}
100+
101+func (e *Engine) ComputeFeedRecommendationsOnDemand(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) {
102+ w := e.GetWeights(ctx, userDID)
103+
104+ rows, err := e.db.QueryContext(ctx, `
105+ WITH similar_users AS (
106+ SELECT user_b AS peer, jaccard FROM user_similarity WHERE user_a = ? AND jaccard > 0.15
107+ UNION ALL
108+ SELECT user_a AS peer, jaccard FROM user_similarity WHERE user_b = ? AND jaccard > 0.15
109+ ),
110+ candidate_feeds AS (
111+ SELECT s.feed_url,
112+ SUM(su.jaccard) AS sub_signal
113+ FROM similar_users su
114+ JOIN subscriptions s ON s.user_did = su.peer
115+ WHERE s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
116+ AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
117+ GROUP BY s.feed_url
118+ ),
119+ like_signals AS (
120+ SELECT s.feed_url,
121+ SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal
122+ FROM similar_users su
123+ JOIN likes l ON l.author_did = su.peer
124+ JOIN subscriptions s ON s.feed_url = l.feed_url
125+ WHERE s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
126+ AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
127+ GROUP BY s.feed_url
128+ ),
129+ social_boost AS (
130+ SELECT s.feed_url,
131+ SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social
132+ FROM follow_distances fd
133+ JOIN subscriptions s ON s.user_did = fd.user_b
134+ WHERE fd.user_a = ?
135+ AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
136+ AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
137+ GROUP BY s.feed_url
138+ ),
139+ category_counts AS (
140+ SELECT category, COUNT(*) AS cnt
141+ FROM subscriptions WHERE user_did = ? AND category IS NOT NULL AND category != ''
142+ GROUP BY category
143+ ),
144+ max_subs AS (
145+ SELECT CAST(COALESCE(MAX(subscriber_count), 1) AS REAL) AS m FROM feeds
146+ )
147+ SELECT cf.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''),
148+ COALESCE(f.description, ''), f.subscriber_count, COALESCE(f.favicon_url, ''),
149+ COALESCE(cf.sub_signal, 0) * ?
150+ + COALESCE(ls.like_signal, 0) * ?
151+ + COALESCE(sb.social, 0) * ?
152+ + COALESCE(LOG(1 + CAST(f.subscriber_count AS REAL)) / LOG(1 + ms.m), 0) * ?
153+ + CASE WHEN f.description IS NOT NULL AND EXISTS (
154+ SELECT 1 FROM category_counts cc
155+ WHERE LOWER(f.description) LIKE '%' || LOWER(cc.category) || '%'
156+ ) THEN ? ELSE 0 END
157+ AS score
158+ FROM candidate_feeds cf
159+ JOIN feeds f ON f.feed_url = cf.feed_url
160+ LEFT JOIN like_signals ls ON ls.feed_url = cf.feed_url
161+ LEFT JOIN social_boost sb ON sb.feed_url = cf.feed_url
162+ CROSS JOIN max_subs ms
163+ ORDER BY score DESC
164+ LIMIT ?
165+ `, userDID, userDID, userDID, userDID, userDID, userDID, userDID, userDID, userDID, userDID,
166+ w.WSub, w.WLike, w.WSocial, w.WPop, w.WCategory, limit)
167+ if err != nil {
168+ return nil, err
169+ }
170+ defer rows.Close()
171+
172+ var results []*FeedRecommendation
173+ for rows.Next() {
174+ rec := &FeedRecommendation{}
175+ if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description,
176+ &rec.SubscriberCount, &rec.FaviconURL, &rec.Score); err != nil {
177+ return nil, err
178+ }
179+ results = append(results, rec)
180+ }
181+ return results, rows.Err()
182+}
183+
184+func (e *Engine) ComputeArticleRecommendationsOnDemand(ctx context.Context, userDID string, limit int) ([]*ArticleRecommendation, error) {
185+ w := e.GetWeights(ctx, userDID)
186+
187+ rows, err := e.db.QueryContext(ctx, `
188+ WITH similar_users AS (
189+ SELECT user_b AS peer, jaccard FROM user_similarity WHERE user_a = ? AND jaccard > 0.15
190+ UNION ALL
191+ SELECT user_a AS peer, jaccard FROM user_similarity WHERE user_b = ? AND jaccard > 0.15
192+ ),
193+ liked_articles AS (
194+ SELECT l.feed_url, l.article_url,
195+ SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal
196+ FROM similar_users su
197+ JOIN likes l ON l.author_did = su.peer
198+ WHERE NOT EXISTS (
199+ SELECT 1 FROM likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url
200+ )
201+ AND NOT EXISTS (
202+ SELECT 1 FROM dismissed_recommendations d WHERE d.user_did = ? AND d.target_type = 'article' AND d.target_id = l.article_url
203+ )
204+ GROUP BY l.feed_url, l.article_url
205+ ),
206+ social_likes AS (
207+ SELECT l.feed_url, l.article_url,
208+ SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social
209+ FROM follow_distances fd
210+ JOIN likes l ON l.author_did = fd.user_b
211+ WHERE fd.user_a = ?
212+ AND NOT EXISTS (
213+ SELECT 1 FROM likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url
214+ )
215+ GROUP BY l.feed_url, l.article_url
216+ )
217+ SELECT a.id, a.title, COALESCE(a.url, ''), la.feed_url, COALESCE(f.title, ''),
218+ COALESCE(a.author, ''), COALESCE(a.summary, ''), a.published,
219+ COALESCE(la.like_signal, 0) * ?
220+ + COALESCE(sl.social, 0) * ?
221+ + EXP(-0.023 * CAST(julianday('now') - julianday(a.published) AS REAL)) * 0.2
222+ AS score
223+ FROM liked_articles la
224+ JOIN articles a ON a.feed_url = la.feed_url AND a.url = la.article_url
225+ LEFT JOIN feeds f ON f.feed_url = la.feed_url
226+ LEFT JOIN social_likes sl ON sl.feed_url = la.feed_url AND sl.article_url = la.article_url
227+ ORDER BY score DESC
228+ LIMIT ?
229+ `, userDID, userDID, userDID, userDID, userDID, userDID, w.WLike, w.WSocial, limit)
230+ if err != nil {
231+ return nil, err
232+ }
233+ defer rows.Close()
234+
235+ var recs []*ArticleRecommendation
236+ for rows.Next() {
237+ rec := &ArticleRecommendation{}
238+ if err := rows.Scan(&rec.ArticleID, &rec.Title, &rec.URL, &rec.FeedURL, &rec.FeedTitle,
239+ &rec.Author, &rec.Summary, &rec.Published, &rec.Score); err != nil {
240+ return nil, err
241+ }
242+ recs = append(recs, rec)
243+ }
244+ return recs, rows.Err()
245+}
246+
247+func (e *Engine) ComputePeopleRecommendationsOnDemand(ctx context.Context, userDID string, limit int) ([]*PersonRecommendation, error) {
248+ rows, err := e.db.QueryContext(ctx, `
249+ SELECT u.did, u.handle, COALESCE(u.display_name, ''), COALESCE(u.avatar_url, ''),
250+ sim.jaccard, sim.common_feeds, COALESCE(sim.common_likes, 0), COALESCE(sim.common_tags, 0)
251+ FROM (
252+ SELECT user_b AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_a = ?
253+ UNION ALL
254+ SELECT user_a AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_b = ?
255+ ) sim
256+ JOIN users u ON u.did = sim.peer_did
257+ WHERE u.handle IS NOT NULL AND u.handle != ''
258+ ORDER BY sim.jaccard DESC
259+ LIMIT ?
260+ `, userDID, userDID, limit)
261+ if err != nil {
262+ return nil, err
263+ }
264+ defer rows.Close()
265+
266+ var results []*PersonRecommendation
267+ for rows.Next() {
268+ rec := &PersonRecommendation{}
269+ if err := rows.Scan(&rec.DID, &rec.Handle, &rec.DisplayName, &rec.AvatarURL,
270+ &rec.Jaccard, &rec.CommonFeeds, &rec.CommonLikes, &rec.CommonTags); err != nil {
271+ return nil, err
272+ }
273+ results = append(results, rec)
274+ }
275+ return results, rows.Err()
276+}
277+
278+func (e *Engine) ComputeSignalProfiles(ctx context.Context) error {
279+ tx, err := e.db.BeginTx(ctx, nil)
280+ if err != nil {
281+ return err
282+ }
283+ defer func() { _ = tx.Rollback() }()
284+
285+ if _, err := tx.ExecContext(ctx, `DELETE FROM user_signal_profiles`); err != nil {
286+ return err
287+ }
288+
289+ _, err = tx.ExecContext(ctx, `
290+ INSERT INTO user_signal_profiles (user_did, total_likes, total_tags, top_categories)
291+ SELECT
292+ u.did,
293+ (SELECT COUNT(*) FROM likes WHERE author_did = u.did),
294+ COALESCE((SELECT COUNT(DISTINCT TRIM(value))
295+ FROM annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]')
296+ WHERE author_did = u.did AND tags IS NOT NULL AND tags != ''
297+ ), 0),
298+ (SELECT '[' || GROUP_CONCAT('{"c":"' || category || '","n":"' || CAST(cnt AS TEXT) || '}') || ']'
299+ FROM (
300+ SELECT category, COUNT(*) AS cnt
301+ FROM subscriptions WHERE user_did = u.did AND category IS NOT NULL AND category != ''
302+ GROUP BY category ORDER BY cnt DESC LIMIT 5
303+ )
304+ )
305+ FROM users u
306+ `)
307+ if err != nil {
308+ return err
309+ }
310+
311+ e.logger.Info("signal profiles computed")
312+ return tx.Commit()
313+}
314+
315+func (e *Engine) ColdStartRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) {
316+ subCount := 0
317+ _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM subscriptions WHERE user_did = ?`, userDID).Scan(&subCount)
318+ if subCount >= 5 {
319+ return nil, nil
320+ }
321+
322+ rows, err := e.db.QueryContext(ctx, `
323+ WITH followed_feeds AS (
324+ SELECT s.feed_url, 1.0 AS weight
325+ FROM follow_distances fd
326+ JOIN subscriptions s ON s.user_did = fd.user_b
327+ WHERE fd.user_a = ? AND fd.distance = 1
328+ AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
329+ AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
330+ ),
331+ popular_feeds AS (
332+ SELECT feed_url, subscriber_count,
333+ LOG(1 + CAST(subscriber_count AS REAL)) / LOG(1 + CAST((SELECT COALESCE(MAX(subscriber_count), 1) FROM feeds) AS REAL)) AS pop_score
334+ FROM feeds
335+ WHERE subscriber_count > 0
336+ AND feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
337+ AND feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
338+ ORDER BY subscriber_count DESC
339+ LIMIT 50
340+ ),
341+ all_candidates AS (
342+ SELECT feed_url, MAX(weight) AS weight FROM (
343+ SELECT feed_url, weight FROM followed_feeds
344+ UNION ALL
345+ SELECT feed_url, pop_score AS weight FROM popular_feeds
346+ )
347+ GROUP BY feed_url
348+ )
349+ SELECT ac.feed_url,
350+ COALESCE(f.title, ''),
351+ COALESCE(f.site_url, ''),
352+ COALESCE(f.description, ''),
353+ f.subscriber_count,
354+ COALESCE(f.favicon_url, ''),
355+ ac.weight AS score
356+ FROM all_candidates ac
357+ JOIN feeds f ON f.feed_url = ac.feed_url
358+ ORDER BY score DESC
359+ LIMIT ?
360+ `, userDID, userDID, userDID, userDID, userDID, limit)
361+ if err != nil {
362+ return nil, err
363+ }
364+ defer rows.Close()
365+
366+ var results []*FeedRecommendation
367+ for rows.Next() {
368+ rec := &FeedRecommendation{}
369+ if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description,
370+ &rec.SubscriberCount, &rec.FaviconURL, &rec.Score); err != nil {
371+ return nil, err
372+ }
373+ results = append(results, rec)
374+ }
375+ return results, rows.Err()
376+}
new file mode 100644
@@ -0,0 +1,376 @@
1+package cluster
2+
3+import (
4+ "context"
5+ "database/sql"
6+)
7+
8+type FeedRecommendation struct {
9+ FeedURL string
10+ Title string
11+ SiteURL string
12+ Description string
13+ SubscriberCount int
14+ FaviconURL string
15+ Score float64
16+}
17+
18+type PersonRecommendation struct {
19+ DID string
20+ Handle string
21+ DisplayName string
22+ AvatarURL string
23+ Jaccard float64
24+ CommonFeeds int
25+ CommonLikes int
26+ CommonTags int
27+}
28+
29+type ArticleRecommendation struct {
30+ ArticleID int64
31+ Title string
32+ URL string
33+ FeedURL string
34+ FeedTitle string
35+ Author string
36+ Summary string
37+ Published sql.NullTime
38+ Score float64
39+}
40+
41+func (e *Engine) GetFeedRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) {
42+ subCount := 0
43+ _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM subscriptions WHERE user_did = ?`, userDID).Scan(&subCount)
44+
45+ if subCount < 5 {
46+ recs, err := e.ColdStartRecommendations(ctx, userDID, limit*2)
47+ if err == nil && len(recs) > 0 {
48+ return ApplyDiversity(recs, limit), nil
49+ }
50+ }
51+
52+ recs, err := e.ComputeFeedRecommendationsOnDemand(ctx, userDID, limit*2)
53+ if err != nil {
54+ return nil, err
55+ }
56+
57+ return ApplyDiversity(recs, limit), nil
58+}
59+
60+func (e *Engine) GetPeopleRecommendations(ctx context.Context, userDID string, limit int) ([]*PersonRecommendation, error) {
61+ return e.ComputePeopleRecommendationsOnDemand(ctx, userDID, limit)
62+}
63+
64+func (e *Engine) GetArticleRecommendations(ctx context.Context, userDID string, limit int) ([]*ArticleRecommendation, error) {
65+ return e.ComputeArticleRecommendationsOnDemand(ctx, userDID, limit)
66+}
67+
68+type SignalWeights struct {
69+ WSub float64
70+ WLike float64
71+ WTag float64
72+ WSocial float64
73+ WPop float64
74+ WCategory float64
75+}
76+
77+func defaultWeights() SignalWeights {
78+ return SignalWeights{
79+ WSub: 1.0,
80+ WLike: 0.5,
81+ WTag: 0.3,
82+ WSocial: 0.7,
83+ WPop: 0.2,
84+ WCategory: 0.4,
85+ }
86+}
87+
88+func (e *Engine) GetWeights(ctx context.Context, userDID string) SignalWeights {
89+ w := defaultWeights()
90+ var dbW SignalWeights
91+ err := e.db.QueryRowContext(ctx, `
92+ SELECT w_sub, w_like, w_tag, w_social, w_pop, w_category
93+ FROM user_signal_weights WHERE user_did = ?
94+ `, userDID).Scan(&dbW.WSub, &dbW.WLike, &dbW.WTag, &dbW.WSocial, &dbW.WPop, &dbW.WCategory)
95+ if err == nil {
96+ return dbW
97+ }
98+ return w
99+}
100+
101+func (e *Engine) ComputeFeedRecommendationsOnDemand(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) {
102+ w := e.GetWeights(ctx, userDID)
103+
104+ rows, err := e.db.QueryContext(ctx, `
105+ WITH similar_users AS (
106+ SELECT user_b AS peer, jaccard FROM user_similarity WHERE user_a = ? AND jaccard > 0.15
107+ UNION ALL
108+ SELECT user_a AS peer, jaccard FROM user_similarity WHERE user_b = ? AND jaccard > 0.15
109+ ),
110+ candidate_feeds AS (
111+ SELECT s.feed_url,
112+ SUM(su.jaccard) AS sub_signal
113+ FROM similar_users su
114+ JOIN subscriptions s ON s.user_did = su.peer
115+ WHERE s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
116+ AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
117+ GROUP BY s.feed_url
118+ ),
119+ like_signals AS (
120+ SELECT s.feed_url,
121+ SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal
122+ FROM similar_users su
123+ JOIN likes l ON l.author_did = su.peer
124+ JOIN subscriptions s ON s.feed_url = l.feed_url
125+ WHERE s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
126+ AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
127+ GROUP BY s.feed_url
128+ ),
129+ social_boost AS (
130+ SELECT s.feed_url,
131+ SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social
132+ FROM follow_distances fd
133+ JOIN subscriptions s ON s.user_did = fd.user_b
134+ WHERE fd.user_a = ?
135+ AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
136+ AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
137+ GROUP BY s.feed_url
138+ ),
139+ category_counts AS (
140+ SELECT category, COUNT(*) AS cnt
141+ FROM subscriptions WHERE user_did = ? AND category IS NOT NULL AND category != ''
142+ GROUP BY category
143+ ),
144+ max_subs AS (
145+ SELECT CAST(COALESCE(MAX(subscriber_count), 1) AS REAL) AS m FROM feeds
146+ )
147+ SELECT cf.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''),
148+ COALESCE(f.description, ''), f.subscriber_count, COALESCE(f.favicon_url, ''),
149+ COALESCE(cf.sub_signal, 0) * ?
150+ + COALESCE(ls.like_signal, 0) * ?
151+ + COALESCE(sb.social, 0) * ?
152+ + COALESCE(LOG(1 + CAST(f.subscriber_count AS REAL)) / LOG(1 + ms.m), 0) * ?
153+ + CASE WHEN f.description IS NOT NULL AND EXISTS (
154+ SELECT 1 FROM category_counts cc
155+ WHERE LOWER(f.description) LIKE '%' || LOWER(cc.category) || '%'
156+ ) THEN ? ELSE 0 END
157+ AS score
158+ FROM candidate_feeds cf
159+ JOIN feeds f ON f.feed_url = cf.feed_url
160+ LEFT JOIN like_signals ls ON ls.feed_url = cf.feed_url
161+ LEFT JOIN social_boost sb ON sb.feed_url = cf.feed_url
162+ CROSS JOIN max_subs ms
163+ ORDER BY score DESC
164+ LIMIT ?
165+ `, userDID, userDID, userDID, userDID, userDID, userDID, userDID, userDID, userDID, userDID,
166+ w.WSub, w.WLike, w.WSocial, w.WPop, w.WCategory, limit)
167+ if err != nil {
168+ return nil, err
169+ }
170+ defer rows.Close()
171+
172+ var results []*FeedRecommendation
173+ for rows.Next() {
174+ rec := &FeedRecommendation{}
175+ if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description,
176+ &rec.SubscriberCount, &rec.FaviconURL, &rec.Score); err != nil {
177+ return nil, err
178+ }
179+ results = append(results, rec)
180+ }
181+ return results, rows.Err()
182+}
183+
184+func (e *Engine) ComputeArticleRecommendationsOnDemand(ctx context.Context, userDID string, limit int) ([]*ArticleRecommendation, error) {
185+ w := e.GetWeights(ctx, userDID)
186+
187+ rows, err := e.db.QueryContext(ctx, `
188+ WITH similar_users AS (
189+ SELECT user_b AS peer, jaccard FROM user_similarity WHERE user_a = ? AND jaccard > 0.15
190+ UNION ALL
191+ SELECT user_a AS peer, jaccard FROM user_similarity WHERE user_b = ? AND jaccard > 0.15
192+ ),
193+ liked_articles AS (
194+ SELECT l.feed_url, l.article_url,
195+ SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal
196+ FROM similar_users su
197+ JOIN likes l ON l.author_did = su.peer
198+ WHERE NOT EXISTS (
199+ SELECT 1 FROM likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url
200+ )
201+ AND NOT EXISTS (
202+ SELECT 1 FROM dismissed_recommendations d WHERE d.user_did = ? AND d.target_type = 'article' AND d.target_id = l.article_url
203+ )
204+ GROUP BY l.feed_url, l.article_url
205+ ),
206+ social_likes AS (
207+ SELECT l.feed_url, l.article_url,
208+ SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social
209+ FROM follow_distances fd
210+ JOIN likes l ON l.author_did = fd.user_b
211+ WHERE fd.user_a = ?
212+ AND NOT EXISTS (
213+ SELECT 1 FROM likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url
214+ )
215+ GROUP BY l.feed_url, l.article_url
216+ )
217+ SELECT a.id, a.title, COALESCE(a.url, ''), la.feed_url, COALESCE(f.title, ''),
218+ COALESCE(a.author, ''), COALESCE(a.summary, ''), a.published,
219+ COALESCE(la.like_signal, 0) * ?
220+ + COALESCE(sl.social, 0) * ?
221+ + EXP(-0.023 * CAST(julianday('now') - julianday(a.published) AS REAL)) * 0.2
222+ AS score
223+ FROM liked_articles la
224+ JOIN articles a ON a.feed_url = la.feed_url AND a.url = la.article_url
225+ LEFT JOIN feeds f ON f.feed_url = la.feed_url
226+ LEFT JOIN social_likes sl ON sl.feed_url = la.feed_url AND sl.article_url = la.article_url
227+ ORDER BY score DESC
228+ LIMIT ?
229+ `, userDID, userDID, userDID, userDID, userDID, userDID, w.WLike, w.WSocial, limit)
230+ if err != nil {
231+ return nil, err
232+ }
233+ defer rows.Close()
234+
235+ var recs []*ArticleRecommendation
236+ for rows.Next() {
237+ rec := &ArticleRecommendation{}
238+ if err := rows.Scan(&rec.ArticleID, &rec.Title, &rec.URL, &rec.FeedURL, &rec.FeedTitle,
239+ &rec.Author, &rec.Summary, &rec.Published, &rec.Score); err != nil {
240+ return nil, err
241+ }
242+ recs = append(recs, rec)
243+ }
244+ return recs, rows.Err()
245+}
246+
247+func (e *Engine) ComputePeopleRecommendationsOnDemand(ctx context.Context, userDID string, limit int) ([]*PersonRecommendation, error) {
248+ rows, err := e.db.QueryContext(ctx, `
249+ SELECT u.did, u.handle, COALESCE(u.display_name, ''), COALESCE(u.avatar_url, ''),
250+ sim.jaccard, sim.common_feeds, COALESCE(sim.common_likes, 0), COALESCE(sim.common_tags, 0)
251+ FROM (
252+ SELECT user_b AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_a = ?
253+ UNION ALL
254+ SELECT user_a AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_b = ?
255+ ) sim
256+ JOIN users u ON u.did = sim.peer_did
257+ WHERE u.handle IS NOT NULL AND u.handle != ''
258+ ORDER BY sim.jaccard DESC
259+ LIMIT ?
260+ `, userDID, userDID, limit)
261+ if err != nil {
262+ return nil, err
263+ }
264+ defer rows.Close()
265+
266+ var results []*PersonRecommendation
267+ for rows.Next() {
268+ rec := &PersonRecommendation{}
269+ if err := rows.Scan(&rec.DID, &rec.Handle, &rec.DisplayName, &rec.AvatarURL,
270+ &rec.Jaccard, &rec.CommonFeeds, &rec.CommonLikes, &rec.CommonTags); err != nil {
271+ return nil, err
272+ }
273+ results = append(results, rec)
274+ }
275+ return results, rows.Err()
276+}
277+
278+func (e *Engine) ComputeSignalProfiles(ctx context.Context) error {
279+ tx, err := e.db.BeginTx(ctx, nil)
280+ if err != nil {
281+ return err
282+ }
283+ defer func() { _ = tx.Rollback() }()
284+
285+ if _, err := tx.ExecContext(ctx, `DELETE FROM user_signal_profiles`); err != nil {
286+ return err
287+ }
288+
289+ _, err = tx.ExecContext(ctx, `
290+ INSERT INTO user_signal_profiles (user_did, total_likes, total_tags, top_categories)
291+ SELECT
292+ u.did,
293+ (SELECT COUNT(*) FROM likes WHERE author_did = u.did),
294+ COALESCE((SELECT COUNT(DISTINCT TRIM(value))
295+ FROM annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]')
296+ WHERE author_did = u.did AND tags IS NOT NULL AND tags != ''
297+ ), 0),
298+ (SELECT '[' || GROUP_CONCAT('{"c":"' || category || '","n":"' || CAST(cnt AS TEXT) || '}') || ']'
299+ FROM (
300+ SELECT category, COUNT(*) AS cnt
301+ FROM subscriptions WHERE user_did = u.did AND category IS NOT NULL AND category != ''
302+ GROUP BY category ORDER BY cnt DESC LIMIT 5
303+ )
304+ )
305+ FROM users u
306+ `)
307+ if err != nil {
308+ return err
309+ }
310+
311+ e.logger.Info("signal profiles computed")
312+ return tx.Commit()
313+}
314+
315+func (e *Engine) ColdStartRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) {
316+ subCount := 0
317+ _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM subscriptions WHERE user_did = ?`, userDID).Scan(&subCount)
318+ if subCount >= 5 {
319+ return nil, nil
320+ }
321+
322+ rows, err := e.db.QueryContext(ctx, `
323+ WITH followed_feeds AS (
324+ SELECT s.feed_url, 1.0 AS weight
325+ FROM follow_distances fd
326+ JOIN subscriptions s ON s.user_did = fd.user_b
327+ WHERE fd.user_a = ? AND fd.distance = 1
328+ AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
329+ AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
330+ ),
331+ popular_feeds AS (
332+ SELECT feed_url, subscriber_count,
333+ LOG(1 + CAST(subscriber_count AS REAL)) / LOG(1 + CAST((SELECT COALESCE(MAX(subscriber_count), 1) FROM feeds) AS REAL)) AS pop_score
334+ FROM feeds
335+ WHERE subscriber_count > 0
336+ AND feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?)
337+ AND feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed')
338+ ORDER BY subscriber_count DESC
339+ LIMIT 50
340+ ),
341+ all_candidates AS (
342+ SELECT feed_url, MAX(weight) AS weight FROM (
343+ SELECT feed_url, weight FROM followed_feeds
344+ UNION ALL
345+ SELECT feed_url, pop_score AS weight FROM popular_feeds
346+ )
347+ GROUP BY feed_url
348+ )
349+ SELECT ac.feed_url,
350+ COALESCE(f.title, ''),
351+ COALESCE(f.site_url, ''),
352+ COALESCE(f.description, ''),
353+ f.subscriber_count,
354+ COALESCE(f.favicon_url, ''),
355+ ac.weight AS score
356+ FROM all_candidates ac
357+ JOIN feeds f ON f.feed_url = ac.feed_url
358+ ORDER BY score DESC
359+ LIMIT ?
360+ `, userDID, userDID, userDID, userDID, userDID, limit)
361+ if err != nil {
362+ return nil, err
363+ }
364+ defer rows.Close()
365+
366+ var results []*FeedRecommendation
367+ for rows.Next() {
368+ rec := &FeedRecommendation{}
369+ if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description,
370+ &rec.SubscriberCount, &rec.FaviconURL, &rec.Score); err != nil {
371+ return nil, err
372+ }
373+ results = append(results, rec)
374+ }
375+ return results, rows.Err()
376+}
added internal/cluster/social.go +63 -0
new file mode 100644
@@ -0,0 +1,63 @@
1+package cluster
2+
3+import (
4+ "context"
5+)
6+
7+func (e *Engine) ComputeFollowDistances(ctx context.Context) error {
8+ tx, err := e.db.BeginTx(ctx, nil)
9+ if err != nil {
10+ return err
11+ }
12+ defer func() { _ = tx.Rollback() }()
13+
14+ if _, err := tx.ExecContext(ctx, `DELETE FROM follow_distances`); err != nil {
15+ return err
16+ }
17+
18+ if _, err := tx.ExecContext(ctx, `
19+ INSERT INTO follow_distances (user_a, user_b, distance)
20+ SELECT user_did, target_did, 1
21+ FROM follows
22+ WHERE user_did != target_did
23+ `); err != nil {
24+ return err
25+ }
26+
27+ if _, err := tx.ExecContext(ctx, `
28+ INSERT OR IGNORE INTO follow_distances (user_a, user_b, distance)
29+ SELECT f1.user_did, f2.target_did, 2
30+ FROM follows f1
31+ JOIN follows f2 ON f1.target_did = f2.user_did
32+ WHERE f1.user_did != f2.target_did
33+ `); err != nil {
34+ return err
35+ }
36+
37+ e.logger.Info("follow distances computed")
38+ return tx.Commit()
39+}
40+
41+func (e *Engine) ComputeFollowDistancesIncremental(ctx context.Context) error {
42+ var maxFollowed string
43+ err := e.db.QueryRowContext(ctx, `
44+ SELECT COALESCE(MAX(followed_at), '1970-01-01') FROM follows
45+ `).Scan(&maxFollowed)
46+ if err != nil {
47+ return err
48+ }
49+
50+ var lastComputed string
51+ err = e.db.QueryRowContext(ctx, `
52+ SELECT COALESCE(MAX(updated_at), '1970-01-01') FROM user_similarity
53+ `).Scan(&lastComputed)
54+ if err != nil {
55+ return err
56+ }
57+
58+ if maxFollowed <= lastComputed {
59+ return nil
60+ }
61+
62+ return e.ComputeFollowDistances(ctx)
63+}
new file mode 100644
@@ -0,0 +1,63 @@
1+package cluster
2+
3+import (
4+ "context"
5+)
6+
7+func (e *Engine) ComputeFollowDistances(ctx context.Context) error {
8+ tx, err := e.db.BeginTx(ctx, nil)
9+ if err != nil {
10+ return err
11+ }
12+ defer func() { _ = tx.Rollback() }()
13+
14+ if _, err := tx.ExecContext(ctx, `DELETE FROM follow_distances`); err != nil {
15+ return err
16+ }
17+
18+ if _, err := tx.ExecContext(ctx, `
19+ INSERT INTO follow_distances (user_a, user_b, distance)
20+ SELECT user_did, target_did, 1
21+ FROM follows
22+ WHERE user_did != target_did
23+ `); err != nil {
24+ return err
25+ }
26+
27+ if _, err := tx.ExecContext(ctx, `
28+ INSERT OR IGNORE INTO follow_distances (user_a, user_b, distance)
29+ SELECT f1.user_did, f2.target_did, 2
30+ FROM follows f1
31+ JOIN follows f2 ON f1.target_did = f2.user_did
32+ WHERE f1.user_did != f2.target_did
33+ `); err != nil {
34+ return err
35+ }
36+
37+ e.logger.Info("follow distances computed")
38+ return tx.Commit()
39+}
40+
41+func (e *Engine) ComputeFollowDistancesIncremental(ctx context.Context) error {
42+ var maxFollowed string
43+ err := e.db.QueryRowContext(ctx, `
44+ SELECT COALESCE(MAX(followed_at), '1970-01-01') FROM follows
45+ `).Scan(&maxFollowed)
46+ if err != nil {
47+ return err
48+ }
49+
50+ var lastComputed string
51+ err = e.db.QueryRowContext(ctx, `
52+ SELECT COALESCE(MAX(updated_at), '1970-01-01') FROM user_similarity
53+ `).Scan(&lastComputed)
54+ if err != nil {
55+ return err
56+ }
57+
58+ if maxFollowed <= lastComputed {
59+ return nil
60+ }
61+
62+ return e.ComputeFollowDistances(ctx)
63+}
added internal/cluster/weights.go +93 -0
new file mode 100644
@@ -0,0 +1,93 @@
1+package cluster
2+
3+import (
4+ "context"
5+)
6+
7+const (
8+ learningRate = 0.1
9+ weightMin = 0.1
10+ weightMax = 3.0
11+ minActionsTune = 5
12+)
13+
14+func (e *Engine) RewardSignal(ctx context.Context, userDID string, signal string) {
15+ e.adjustWeight(ctx, userDID, signal, 1.0)
16+}
17+
18+func (e *Engine) PenalizeSignal(ctx context.Context, userDID string, signal string) {
19+ e.adjustWeight(ctx, userDID, signal, -1.0)
20+}
21+
22+func (e *Engine) adjustWeight(ctx context.Context, userDID string, signal string, delta float64) {
23+ var actedCount int
24+ _ = e.db.QueryRowContext(ctx, `
25+ SELECT COUNT(*) FROM recommendation_impressions WHERE user_did = ? AND acted = 1
26+ `, userDID).Scan(&actedCount)
27+ if actedCount < minActionsTune {
28+ return
29+ }
30+
31+ var exists int
32+ _ = e.db.QueryRowContext(ctx, `SELECT 1 FROM user_signal_weights WHERE user_did = ?`, userDID).Scan(&exists)
33+
34+ if exists == 0 {
35+ _, _ = e.db.ExecContext(ctx, `
36+ INSERT INTO user_signal_weights (user_did, w_sub, w_like, w_tag, w_social, w_pop, w_category)
37+ VALUES (?, 1.0, 0.5, 0.3, 0.7, 0.2, 0.4)
38+ `, userDID)
39+ }
40+
41+ column := signalToColumn(signal)
42+ if column == "" {
43+ return
44+ }
45+
46+ adj := learningRate * delta
47+ _, _ = e.db.ExecContext(ctx, `
48+ UPDATE user_signal_weights SET
49+ `+column+` = MAX(?, MIN(?, `+column+` * (1 + ?))),
50+ updated_at = CURRENT_TIMESTAMP
51+ WHERE user_did = ?
52+ `, weightMin, weightMax, adj, userDID)
53+}
54+
55+func signalToColumn(signal string) string {
56+ switch signal {
57+ case "sub":
58+ return "w_sub"
59+ case "like":
60+ return "w_like"
61+ case "tag":
62+ return "w_tag"
63+ case "social":
64+ return "w_social"
65+ case "pop":
66+ return "w_pop"
67+ case "category":
68+ return "w_category"
69+ default:
70+ return ""
71+ }
72+}
73+
74+func (e *Engine) GetDominantSignal(w SignalWeights) string {
75+ signals := map[string]float64{
76+ "sub": w.WSub,
77+ "like": w.WLike,
78+ "tag": w.WTag,
79+ "social": w.WSocial,
80+ "pop": w.WPop,
81+ "category": w.WCategory,
82+ }
83+
84+ var best string
85+ bestVal := -1.0
86+ for s, v := range signals {
87+ if v > bestVal {
88+ bestVal = v
89+ best = s
90+ }
91+ }
92+ return best
93+}
new file mode 100644
@@ -0,0 +1,93 @@
1+package cluster
2+
3+import (
4+ "context"
5+)
6+
7+const (
8+ learningRate = 0.1
9+ weightMin = 0.1
10+ weightMax = 3.0
11+ minActionsTune = 5
12+)
13+
14+func (e *Engine) RewardSignal(ctx context.Context, userDID string, signal string) {
15+ e.adjustWeight(ctx, userDID, signal, 1.0)
16+}
17+
18+func (e *Engine) PenalizeSignal(ctx context.Context, userDID string, signal string) {
19+ e.adjustWeight(ctx, userDID, signal, -1.0)
20+}
21+
22+func (e *Engine) adjustWeight(ctx context.Context, userDID string, signal string, delta float64) {
23+ var actedCount int
24+ _ = e.db.QueryRowContext(ctx, `
25+ SELECT COUNT(*) FROM recommendation_impressions WHERE user_did = ? AND acted = 1
26+ `, userDID).Scan(&actedCount)
27+ if actedCount < minActionsTune {
28+ return
29+ }
30+
31+ var exists int
32+ _ = e.db.QueryRowContext(ctx, `SELECT 1 FROM user_signal_weights WHERE user_did = ?`, userDID).Scan(&exists)
33+
34+ if exists == 0 {
35+ _, _ = e.db.ExecContext(ctx, `
36+ INSERT INTO user_signal_weights (user_did, w_sub, w_like, w_tag, w_social, w_pop, w_category)
37+ VALUES (?, 1.0, 0.5, 0.3, 0.7, 0.2, 0.4)
38+ `, userDID)
39+ }
40+
41+ column := signalToColumn(signal)
42+ if column == "" {
43+ return
44+ }
45+
46+ adj := learningRate * delta
47+ _, _ = e.db.ExecContext(ctx, `
48+ UPDATE user_signal_weights SET
49+ `+column+` = MAX(?, MIN(?, `+column+` * (1 + ?))),
50+ updated_at = CURRENT_TIMESTAMP
51+ WHERE user_did = ?
52+ `, weightMin, weightMax, adj, userDID)
53+}
54+
55+func signalToColumn(signal string) string {
56+ switch signal {
57+ case "sub":
58+ return "w_sub"
59+ case "like":
60+ return "w_like"
61+ case "tag":
62+ return "w_tag"
63+ case "social":
64+ return "w_social"
65+ case "pop":
66+ return "w_pop"
67+ case "category":
68+ return "w_category"
69+ default:
70+ return ""
71+ }
72+}
73+
74+func (e *Engine) GetDominantSignal(w SignalWeights) string {
75+ signals := map[string]float64{
76+ "sub": w.WSub,
77+ "like": w.WLike,
78+ "tag": w.WTag,
79+ "social": w.WSocial,
80+ "pop": w.WPop,
81+ "category": w.WCategory,
82+ }
83+
84+ var best string
85+ bestVal := -1.0
86+ for s, v := range signals {
87+ if v > bestVal {
88+ bestVal = v
89+ best = s
90+ }
91+ }
92+ return best
93+}
modified internal/db/db.go +83 -16
@@ -1,10 +1,13 @@
11 package db
22
33 import (
4+ "context"
45 "database/sql"
5- _ "github.com/mattn/go-sqlite3"
6+ "math"
67 "strings"
78 "time"
9+
10+ "github.com/mattn/go-sqlite3"
811 )
912
1013 func NullStr(s string) sql.NullString {
@@ -40,6 +43,31 @@ func Open(path string) (*DB, error) {
4043 db.SetMaxIdleConns(2)
4144 db.SetConnMaxLifetime(30 * time.Minute)
4245
46+ conn, err := db.Conn(context.Background())
47+ if err != nil {
48+ db.Close()
49+ return nil, err
50+ }
51+
52+ err = conn.Raw(func(driverConn any) error {
53+ sqliteConn, ok := driverConn.(*sqlite3.SQLiteConn)
54+ if !ok {
55+ return nil
56+ }
57+ if err := sqliteConn.RegisterFunc("exp", func(x float64) float64 { return math.Exp(x) }, true); err != nil {
58+ return err
59+ }
60+ if err := sqliteConn.RegisterFunc("log", func(x float64) float64 { return math.Log(x) }, true); err != nil {
61+ return err
62+ }
63+ return nil
64+ })
65+ _ = conn.Close()
66+ if err != nil {
67+ db.Close()
68+ return nil, err
69+ }
70+
4371 if err := initSchema(db); err != nil {
4472 db.Close()
4573 return nil, err
@@ -158,21 +186,6 @@ var schema = []string{
158186 PRIMARY KEY (user_a, user_b),
159187 CHECK(user_a < user_b)
160188 )`,
161- `CREATE TABLE IF NOT EXISTS user_feed_recommendations (
162- user_did TEXT NOT NULL REFERENCES users(did),
163- feed_url TEXT NOT NULL REFERENCES feeds(feed_url),
164- score REAL NOT NULL,
165- computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
166- PRIMARY KEY (user_did, feed_url)
167- )`,
168- `CREATE TABLE IF NOT EXISTS user_article_recommendations (
169- user_did TEXT NOT NULL REFERENCES users(did),
170- feed_url TEXT NOT NULL,
171- article_url TEXT NOT NULL,
172- score REAL NOT NULL,
173- computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
174- PRIMARY KEY (user_did, feed_url, article_url)
175- )`,
176189 `CREATE TABLE IF NOT EXISTS follows (
177190 user_did TEXT NOT NULL REFERENCES users(did),
178191 target_did TEXT NOT NULL,
@@ -208,6 +221,60 @@ var schema = []string{
208221 `CREATE INDEX IF NOT EXISTS idx_follows_target ON follows(target_did)`,
209222 `CREATE INDEX IF NOT EXISTS idx_follows_uri ON follows(uri)`,
210223 `CREATE INDEX IF NOT EXISTS idx_user_similarity_b ON user_similarity(user_b)`,
224+
225+ `CREATE TABLE IF NOT EXISTS dismissed_recommendations (
226+ user_did TEXT NOT NULL REFERENCES users(did),
227+ target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
228+ target_id TEXT NOT NULL,
229+ reason TEXT,
230+ dismissed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
231+ PRIMARY KEY (user_did, target_type, target_id)
232+ )`,
233+
234+ `CREATE TABLE IF NOT EXISTS recommendation_impressions (
235+ user_did TEXT NOT NULL REFERENCES users(did),
236+ target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
237+ target_id TEXT NOT NULL,
238+ first_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
239+ last_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
240+ shown_count INTEGER NOT NULL DEFAULT 1,
241+ acted BOOLEAN NOT NULL DEFAULT 0,
242+ PRIMARY KEY (user_did, target_type, target_id)
243+ )`,
244+
245+ `CREATE TABLE IF NOT EXISTS follow_distances (
246+ user_a TEXT NOT NULL,
247+ user_b TEXT NOT NULL,
248+ distance INTEGER NOT NULL CHECK(distance IN (1, 2)),
249+ PRIMARY KEY (user_a, user_b)
250+ )`,
251+
252+ `CREATE TABLE IF NOT EXISTS user_signal_weights (
253+ user_did TEXT PRIMARY KEY REFERENCES users(did),
254+ w_sub REAL NOT NULL DEFAULT 1.0,
255+ w_like REAL NOT NULL DEFAULT 0.5,
256+ w_tag REAL NOT NULL DEFAULT 0.3,
257+ w_social REAL NOT NULL DEFAULT 0.7,
258+ w_pop REAL NOT NULL DEFAULT 0.2,
259+ w_category REAL NOT NULL DEFAULT 0.4,
260+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
261+ )`,
262+
263+ `CREATE TABLE IF NOT EXISTS user_signal_profiles (
264+ user_did TEXT PRIMARY KEY REFERENCES users(did),
265+ total_likes INTEGER NOT NULL DEFAULT 0,
266+ total_tags INTEGER NOT NULL DEFAULT 0,
267+ top_categories TEXT,
268+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
269+ )`,
270+
271+ `CREATE INDEX IF NOT EXISTS idx_dismissed_user_type ON dismissed_recommendations(user_did, target_type)`,
272+ `CREATE INDEX IF NOT EXISTS idx_impressions_user_unacted ON recommendation_impressions(user_did, acted, shown_count)`,
273+ `CREATE INDEX IF NOT EXISTS idx_impressions_last_shown ON recommendation_impressions(last_shown_at)`,
274+ `CREATE INDEX IF NOT EXISTS idx_follow_distances_b ON follow_distances(user_b)`,
275+ `CREATE INDEX IF NOT EXISTS idx_follow_distances_a_dist ON follow_distances(user_a, distance)`,
276+ `CREATE INDEX IF NOT EXISTS idx_likes_author_feed ON likes(author_did, feed_url, created_at)`,
277+ `CREATE INDEX IF NOT EXISTS idx_follows_followed_at ON follows(followed_at)`,
211278 `CREATE INDEX IF NOT EXISTS idx_users_handle ON users(handle)`,
212279 `CREATE VIRTUAL TABLE IF NOT EXISTS articles_fts USING fts5(title, summary, content, author, content=articles, content_rowid=id)`,
213280 `CREATE TRIGGER IF NOT EXISTS articles_ai AFTER INSERT ON articles BEGIN
@@ -1,10 +1,13 @@
1 package db1 package db
2 2
3 import (3 import (
4+ "context"
4 "database/sql"5 "database/sql"
5- _ "github.com/mattn/go-sqlite3"6+ "math"
6 "strings"7 "strings"
7 "time"8 "time"
9+
10+ "github.com/mattn/go-sqlite3"
8 )11 )
9 12
10 func NullStr(s string) sql.NullString {13 func NullStr(s string) sql.NullString {
@@ -40,6 +43,31 @@ func Open(path string) (*DB, error) {
40 db.SetMaxIdleConns(2)43 db.SetMaxIdleConns(2)
41 db.SetConnMaxLifetime(30 * time.Minute)44 db.SetConnMaxLifetime(30 * time.Minute)
42 45
46+ conn, err := db.Conn(context.Background())
47+ if err != nil {
48+ db.Close()
49+ return nil, err
50+ }
51+
52+ err = conn.Raw(func(driverConn any) error {
53+ sqliteConn, ok := driverConn.(*sqlite3.SQLiteConn)
54+ if !ok {
55+ return nil
56+ }
57+ if err := sqliteConn.RegisterFunc("exp", func(x float64) float64 { return math.Exp(x) }, true); err != nil {
58+ return err
59+ }
60+ if err := sqliteConn.RegisterFunc("log", func(x float64) float64 { return math.Log(x) }, true); err != nil {
61+ return err
62+ }
63+ return nil
64+ })
65+ _ = conn.Close()
66+ if err != nil {
67+ db.Close()
68+ return nil, err
69+ }
70+
43 if err := initSchema(db); err != nil {71 if err := initSchema(db); err != nil {
44 db.Close()72 db.Close()
45 return nil, err73 return nil, err
@@ -158,21 +186,6 @@ var schema = []string{
158 PRIMARY KEY (user_a, user_b),186 PRIMARY KEY (user_a, user_b),
159 CHECK(user_a < user_b)187 CHECK(user_a < user_b)
160 )`,188 )`,
161- `CREATE TABLE IF NOT EXISTS user_feed_recommendations (
162- user_did TEXT NOT NULL REFERENCES users(did),
163- feed_url TEXT NOT NULL REFERENCES feeds(feed_url),
164- score REAL NOT NULL,
165- computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
166- PRIMARY KEY (user_did, feed_url)
167- )`,
168- `CREATE TABLE IF NOT EXISTS user_article_recommendations (
169- user_did TEXT NOT NULL REFERENCES users(did),
170- feed_url TEXT NOT NULL,
171- article_url TEXT NOT NULL,
172- score REAL NOT NULL,
173- computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
174- PRIMARY KEY (user_did, feed_url, article_url)
175- )`,
176 `CREATE TABLE IF NOT EXISTS follows (189 `CREATE TABLE IF NOT EXISTS follows (
177 user_did TEXT NOT NULL REFERENCES users(did),190 user_did TEXT NOT NULL REFERENCES users(did),
178 target_did TEXT NOT NULL,191 target_did TEXT NOT NULL,
@@ -208,6 +221,60 @@ var schema = []string{
208 `CREATE INDEX IF NOT EXISTS idx_follows_target ON follows(target_did)`,221 `CREATE INDEX IF NOT EXISTS idx_follows_target ON follows(target_did)`,
209 `CREATE INDEX IF NOT EXISTS idx_follows_uri ON follows(uri)`,222 `CREATE INDEX IF NOT EXISTS idx_follows_uri ON follows(uri)`,
210 `CREATE INDEX IF NOT EXISTS idx_user_similarity_b ON user_similarity(user_b)`,223 `CREATE INDEX IF NOT EXISTS idx_user_similarity_b ON user_similarity(user_b)`,
224+
225+ `CREATE TABLE IF NOT EXISTS dismissed_recommendations (
226+ user_did TEXT NOT NULL REFERENCES users(did),
227+ target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
228+ target_id TEXT NOT NULL,
229+ reason TEXT,
230+ dismissed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
231+ PRIMARY KEY (user_did, target_type, target_id)
232+ )`,
233+
234+ `CREATE TABLE IF NOT EXISTS recommendation_impressions (
235+ user_did TEXT NOT NULL REFERENCES users(did),
236+ target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')),
237+ target_id TEXT NOT NULL,
238+ first_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
239+ last_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
240+ shown_count INTEGER NOT NULL DEFAULT 1,
241+ acted BOOLEAN NOT NULL DEFAULT 0,
242+ PRIMARY KEY (user_did, target_type, target_id)
243+ )`,
244+
245+ `CREATE TABLE IF NOT EXISTS follow_distances (
246+ user_a TEXT NOT NULL,
247+ user_b TEXT NOT NULL,
248+ distance INTEGER NOT NULL CHECK(distance IN (1, 2)),
249+ PRIMARY KEY (user_a, user_b)
250+ )`,
251+
252+ `CREATE TABLE IF NOT EXISTS user_signal_weights (
253+ user_did TEXT PRIMARY KEY REFERENCES users(did),
254+ w_sub REAL NOT NULL DEFAULT 1.0,
255+ w_like REAL NOT NULL DEFAULT 0.5,
256+ w_tag REAL NOT NULL DEFAULT 0.3,
257+ w_social REAL NOT NULL DEFAULT 0.7,
258+ w_pop REAL NOT NULL DEFAULT 0.2,
259+ w_category REAL NOT NULL DEFAULT 0.4,
260+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
261+ )`,
262+
263+ `CREATE TABLE IF NOT EXISTS user_signal_profiles (
264+ user_did TEXT PRIMARY KEY REFERENCES users(did),
265+ total_likes INTEGER NOT NULL DEFAULT 0,
266+ total_tags INTEGER NOT NULL DEFAULT 0,
267+ top_categories TEXT,
268+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
269+ )`,
270+
271+ `CREATE INDEX IF NOT EXISTS idx_dismissed_user_type ON dismissed_recommendations(user_did, target_type)`,
272+ `CREATE INDEX IF NOT EXISTS idx_impressions_user_unacted ON recommendation_impressions(user_did, acted, shown_count)`,
273+ `CREATE INDEX IF NOT EXISTS idx_impressions_last_shown ON recommendation_impressions(last_shown_at)`,
274+ `CREATE INDEX IF NOT EXISTS idx_follow_distances_b ON follow_distances(user_b)`,
275+ `CREATE INDEX IF NOT EXISTS idx_follow_distances_a_dist ON follow_distances(user_a, distance)`,
276+ `CREATE INDEX IF NOT EXISTS idx_likes_author_feed ON likes(author_did, feed_url, created_at)`,
277+ `CREATE INDEX IF NOT EXISTS idx_follows_followed_at ON follows(followed_at)`,
211 `CREATE INDEX IF NOT EXISTS idx_users_handle ON users(handle)`,278 `CREATE INDEX IF NOT EXISTS idx_users_handle ON users(handle)`,
212 `CREATE VIRTUAL TABLE IF NOT EXISTS articles_fts USING fts5(title, summary, content, author, content=articles, content_rowid=id)`,279 `CREATE VIRTUAL TABLE IF NOT EXISTS articles_fts USING fts5(title, summary, content, author, content=articles, content_rowid=id)`,
213 `CREATE TRIGGER IF NOT EXISTS articles_ai AFTER INSERT ON articles BEGIN280 `CREATE TRIGGER IF NOT EXISTS articles_ai AFTER INSERT ON articles BEGIN
modified internal/server/articles_handler.go +6 -0
@@ -269,6 +269,9 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
269269 http.Error(w, err.Error(), http.StatusInternalServerError)
270270 return
271271 }
272+ _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "article", article.URL.String)
273+ sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID))
274+ s.engine.RewardSignal(r.Context(), user.DID, sig)
272275 } else {
273276 like := &db.Like{
274277 URI: fmt.Sprintf("glean:like:%d", time.Now().UnixNano()),
@@ -281,6 +284,9 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
281284 http.Error(w, err.Error(), http.StatusInternalServerError)
282285 return
283286 }
287+ _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "article", article.URL.String)
288+ sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID))
289+ s.engine.RewardSignal(r.Context(), user.DID, sig)
284290 }
285291 }
286292
@@ -269,6 +269,9 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
269 http.Error(w, err.Error(), http.StatusInternalServerError)269 http.Error(w, err.Error(), http.StatusInternalServerError)
270 return270 return
271 }271 }
272+ _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "article", article.URL.String)
273+ sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID))
274+ s.engine.RewardSignal(r.Context(), user.DID, sig)
272 } else {275 } else {
273 like := &db.Like{276 like := &db.Like{
274 URI: fmt.Sprintf("glean:like:%d", time.Now().UnixNano()),277 URI: fmt.Sprintf("glean:like:%d", time.Now().UnixNano()),
@@ -281,6 +284,9 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) {
281 http.Error(w, err.Error(), http.StatusInternalServerError)284 http.Error(w, err.Error(), http.StatusInternalServerError)
282 return285 return
283 }286 }
287+ _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "article", article.URL.String)
288+ sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID))
289+ s.engine.RewardSignal(r.Context(), user.DID, sig)
284 }290 }
285 }291 }
286 292
modified internal/server/dashboard_handler.go +35 -0
@@ -3,6 +3,8 @@ package server
33 import (
44 "net/http"
55 "time"
6+
7+ "pkg.rbrt.fr/glean/internal/cluster"
68 )
79
810 func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) {
@@ -23,6 +25,17 @@ func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) {
2325 peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5)
2426 feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 5)
2527
28+ var impressions []cluster.Impression
29+ for _, rec := range feedRecs {
30+ impressions = append(impressions, cluster.Impression{TargetType: "feed", TargetID: rec.FeedURL})
31+ }
32+ for _, rec := range articleRecs {
33+ impressions = append(impressions, cluster.Impression{TargetType: "article", TargetID: rec.URL})
34+ }
35+ if len(impressions) > 0 {
36+ _ = s.engine.RecordImpressions(r.Context(), user.DID, impressions)
37+ }
38+
2639 since := time.Now().AddDate(0, 0, -7).Format(time.RFC3339)
2740 personalTrending, _ := s.db.ListTrendingArticlesForUser(r.Context(), user.DID, since, 5, 0)
2841 globalTrending, _ := s.db.ListTrendingArticles(r.Context(), user.DID, since, 10, 0)
@@ -43,3 +56,25 @@ func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) {
4356 "Now": time.Now(),
4457 })
4558 }
59+
60+func (s *Server) handleDismissArticleRecommendation(w http.ResponseWriter, r *http.Request) {
61+ user := currentUser(r)
62+ articleURL := r.FormValue("article_url")
63+ if articleURL == "" {
64+ http.Error(w, "article_url required", http.StatusBadRequest)
65+ return
66+ }
67+
68+ reason := r.FormValue("reason")
69+ if reason == "" {
70+ reason = "not_interested"
71+ }
72+
73+ if err := s.engine.DismissArticle(r.Context(), user.DID, articleURL, reason); err != nil {
74+ s.logger.Error("failed to dismiss article recommendation", "error", err)
75+ http.Error(w, err.Error(), http.StatusInternalServerError)
76+ return
77+ }
78+
79+ w.WriteHeader(http.StatusOK)
80+}
@@ -3,6 +3,8 @@ package server
3 import (3 import (
4 "net/http"4 "net/http"
5 "time"5 "time"
6+
7+ "pkg.rbrt.fr/glean/internal/cluster"
6 )8 )
7 9
8 func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) {10 func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) {
@@ -23,6 +25,17 @@ func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) {
23 peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5)25 peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5)
24 feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 5)26 feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 5)
25 27
28+ var impressions []cluster.Impression
29+ for _, rec := range feedRecs {
30+ impressions = append(impressions, cluster.Impression{TargetType: "feed", TargetID: rec.FeedURL})
31+ }
32+ for _, rec := range articleRecs {
33+ impressions = append(impressions, cluster.Impression{TargetType: "article", TargetID: rec.URL})
34+ }
35+ if len(impressions) > 0 {
36+ _ = s.engine.RecordImpressions(r.Context(), user.DID, impressions)
37+ }
38+
26 since := time.Now().AddDate(0, 0, -7).Format(time.RFC3339)39 since := time.Now().AddDate(0, 0, -7).Format(time.RFC3339)
27 personalTrending, _ := s.db.ListTrendingArticlesForUser(r.Context(), user.DID, since, 5, 0)40 personalTrending, _ := s.db.ListTrendingArticlesForUser(r.Context(), user.DID, since, 5, 0)
28 globalTrending, _ := s.db.ListTrendingArticles(r.Context(), user.DID, since, 10, 0)41 globalTrending, _ := s.db.ListTrendingArticles(r.Context(), user.DID, since, 10, 0)
@@ -43,3 +56,25 @@ func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) {
43 "Now": time.Now(),56 "Now": time.Now(),
44 })57 })
45 }58 }
59+
60+func (s *Server) handleDismissArticleRecommendation(w http.ResponseWriter, r *http.Request) {
61+ user := currentUser(r)
62+ articleURL := r.FormValue("article_url")
63+ if articleURL == "" {
64+ http.Error(w, "article_url required", http.StatusBadRequest)
65+ return
66+ }
67+
68+ reason := r.FormValue("reason")
69+ if reason == "" {
70+ reason = "not_interested"
71+ }
72+
73+ if err := s.engine.DismissArticle(r.Context(), user.DID, articleURL, reason); err != nil {
74+ s.logger.Error("failed to dismiss article recommendation", "error", err)
75+ http.Error(w, err.Error(), http.StatusInternalServerError)
76+ return
77+ }
78+
79+ w.WriteHeader(http.StatusOK)
80+}
modified internal/server/feeds_handler.go +35 -0
@@ -9,6 +9,7 @@ import (
99 "time"
1010
1111 "pkg.rbrt.fr/glean/internal/atproto"
12+ "pkg.rbrt.fr/glean/internal/cluster"
1213 "pkg.rbrt.fr/glean/internal/db"
1314 "pkg.rbrt.fr/glean/internal/feed"
1415 )
@@ -28,6 +29,14 @@ func (s *Server) handleFeeds(w http.ResponseWriter, r *http.Request) {
2829 allSubs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 1000, 0)
2930 feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 10)
3031 peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5)
32+
33+ if len(feedRecs) > 0 {
34+ impressions := make([]cluster.Impression, len(feedRecs))
35+ for i, rec := range feedRecs {
36+ impressions[i] = cluster.Impression{TargetType: "feed", TargetID: rec.FeedURL}
37+ }
38+ _ = s.engine.RecordImpressions(r.Context(), user.DID, impressions)
39+ }
3140 deadFeeds, _ := s.db.ListDeadFeeds(r.Context(), user.DID, 7)
3241
3342 categories, _ := s.db.GetCategories(r.Context(), user.DID)
@@ -122,6 +131,10 @@ func (s *Server) handleAddFeed(w http.ResponseWriter, r *http.Request) {
122131 return
123132 }
124133
134+ _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "feed", feedURL)
135+ sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID))
136+ s.engine.RewardSignal(r.Context(), user.DID, sig)
137+
125138 sub, _ := s.db.GetSubscription(r.Context(), user.DID, feedURL)
126139 if sub == nil {
127140 w.WriteHeader(http.StatusNoContent)
@@ -384,6 +397,28 @@ func (s *Server) handleDiscoverFeedURL(w http.ResponseWriter, r *http.Request) {
384397 })
385398 }
386399
400+func (s *Server) handleDismissFeedRecommendation(w http.ResponseWriter, r *http.Request) {
401+ user := currentUser(r)
402+ feedURL := r.FormValue("feed_url")
403+ if feedURL == "" {
404+ http.Error(w, "feed_url required", http.StatusBadRequest)
405+ return
406+ }
407+
408+ reason := r.FormValue("reason")
409+ if reason == "" {
410+ reason = "not_interested"
411+ }
412+
413+ if err := s.engine.DismissFeed(r.Context(), user.DID, feedURL, reason); err != nil {
414+ s.logger.Error("failed to dismiss feed recommendation", "error", err)
415+ http.Error(w, err.Error(), http.StatusInternalServerError)
416+ return
417+ }
418+
419+ w.WriteHeader(http.StatusOK)
420+}
421+
387422 func nullString(s string) sql.NullString {
388423 return sql.NullString{String: s, Valid: s != ""}
389424 }
@@ -9,6 +9,7 @@ import (
9 "time"9 "time"
10 10
11 "pkg.rbrt.fr/glean/internal/atproto"11 "pkg.rbrt.fr/glean/internal/atproto"
12+ "pkg.rbrt.fr/glean/internal/cluster"
12 "pkg.rbrt.fr/glean/internal/db"13 "pkg.rbrt.fr/glean/internal/db"
13 "pkg.rbrt.fr/glean/internal/feed"14 "pkg.rbrt.fr/glean/internal/feed"
14 )15 )
@@ -28,6 +29,14 @@ func (s *Server) handleFeeds(w http.ResponseWriter, r *http.Request) {
28 allSubs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 1000, 0)29 allSubs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 1000, 0)
29 feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 10)30 feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 10)
30 peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5)31 peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5)
32+
33+ if len(feedRecs) > 0 {
34+ impressions := make([]cluster.Impression, len(feedRecs))
35+ for i, rec := range feedRecs {
36+ impressions[i] = cluster.Impression{TargetType: "feed", TargetID: rec.FeedURL}
37+ }
38+ _ = s.engine.RecordImpressions(r.Context(), user.DID, impressions)
39+ }
31 deadFeeds, _ := s.db.ListDeadFeeds(r.Context(), user.DID, 7)40 deadFeeds, _ := s.db.ListDeadFeeds(r.Context(), user.DID, 7)
32 41
33 categories, _ := s.db.GetCategories(r.Context(), user.DID)42 categories, _ := s.db.GetCategories(r.Context(), user.DID)
@@ -122,6 +131,10 @@ func (s *Server) handleAddFeed(w http.ResponseWriter, r *http.Request) {
122 return131 return
123 }132 }
124 133
134+ _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "feed", feedURL)
135+ sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID))
136+ s.engine.RewardSignal(r.Context(), user.DID, sig)
137+
125 sub, _ := s.db.GetSubscription(r.Context(), user.DID, feedURL)138 sub, _ := s.db.GetSubscription(r.Context(), user.DID, feedURL)
126 if sub == nil {139 if sub == nil {
127 w.WriteHeader(http.StatusNoContent)140 w.WriteHeader(http.StatusNoContent)
@@ -384,6 +397,28 @@ func (s *Server) handleDiscoverFeedURL(w http.ResponseWriter, r *http.Request) {
384 })397 })
385 }398 }
386 399
400+func (s *Server) handleDismissFeedRecommendation(w http.ResponseWriter, r *http.Request) {
401+ user := currentUser(r)
402+ feedURL := r.FormValue("feed_url")
403+ if feedURL == "" {
404+ http.Error(w, "feed_url required", http.StatusBadRequest)
405+ return
406+ }
407+
408+ reason := r.FormValue("reason")
409+ if reason == "" {
410+ reason = "not_interested"
411+ }
412+
413+ if err := s.engine.DismissFeed(r.Context(), user.DID, feedURL, reason); err != nil {
414+ s.logger.Error("failed to dismiss feed recommendation", "error", err)
415+ http.Error(w, err.Error(), http.StatusInternalServerError)
416+ return
417+ }
418+
419+ w.WriteHeader(http.StatusOK)
420+}
421+
387 func nullString(s string) sql.NullString {422 func nullString(s string) sql.NullString {
388 return sql.NullString{String: s, Valid: s != ""}423 return sql.NullString{String: s, Valid: s != ""}
389 }424 }
modified internal/server/server.go +3 -1
@@ -162,6 +162,7 @@ func (s *Server) setupRoutes() {
162162 r.Get("/list", s.handleFeedList)
163163 r.Get("/discover-url", s.handleDiscoverFeedURL)
164164 r.Post("/clear", s.handleClearAllSubscriptions)
165+ r.Post("/dismiss", s.handleDismissFeedRecommendation)
165166 })
166167
167168 s.router.Route("/articles", func(r chi.Router) {
@@ -174,6 +175,7 @@ func (s *Server) setupRoutes() {
174175 r.Post("/{id}/like", s.handleLikeArticle)
175176 r.Post("/{id}/fetch-content", s.handleFetchContent)
176177 r.Post("/mark-all-read", s.handleMarkAllRead)
178+ r.Post("/dismiss", s.handleDismissArticleRecommendation)
177179 })
178180
179181 s.router.Route("/trending", func(r chi.Router) {
@@ -198,7 +200,7 @@ func (s *Server) setupRoutes() {
198200 s.router.Post("/auth/logout", s.handleAuthLogout)
199201 s.router.Get("/oauth/client-metadata", s.handleOAuthClientMetadata)
200202
201- xrpc := atproto.NewXRPCHandler(s.db.DB)
203+ xrpc := atproto.NewXRPCHandler(s.db.DB, s.engine)
202204 s.router.Get("/xrpc/at.glean.listSubscriptions", xrpc.ListSubscriptions)
203205 s.router.Get("/xrpc/at.glean.listAnnotations", xrpc.ListAnnotations)
204206 s.router.Get("/xrpc/at.glean.listLikes", xrpc.ListLikes)
@@ -162,6 +162,7 @@ func (s *Server) setupRoutes() {
162 r.Get("/list", s.handleFeedList)162 r.Get("/list", s.handleFeedList)
163 r.Get("/discover-url", s.handleDiscoverFeedURL)163 r.Get("/discover-url", s.handleDiscoverFeedURL)
164 r.Post("/clear", s.handleClearAllSubscriptions)164 r.Post("/clear", s.handleClearAllSubscriptions)
165+ r.Post("/dismiss", s.handleDismissFeedRecommendation)
165 })166 })
166 167
167 s.router.Route("/articles", func(r chi.Router) {168 s.router.Route("/articles", func(r chi.Router) {
@@ -174,6 +175,7 @@ func (s *Server) setupRoutes() {
174 r.Post("/{id}/like", s.handleLikeArticle)175 r.Post("/{id}/like", s.handleLikeArticle)
175 r.Post("/{id}/fetch-content", s.handleFetchContent)176 r.Post("/{id}/fetch-content", s.handleFetchContent)
176 r.Post("/mark-all-read", s.handleMarkAllRead)177 r.Post("/mark-all-read", s.handleMarkAllRead)
178+ r.Post("/dismiss", s.handleDismissArticleRecommendation)
177 })179 })
178 180
179 s.router.Route("/trending", func(r chi.Router) {181 s.router.Route("/trending", func(r chi.Router) {
@@ -198,7 +200,7 @@ func (s *Server) setupRoutes() {
198 s.router.Post("/auth/logout", s.handleAuthLogout)200 s.router.Post("/auth/logout", s.handleAuthLogout)
199 s.router.Get("/oauth/client-metadata", s.handleOAuthClientMetadata)201 s.router.Get("/oauth/client-metadata", s.handleOAuthClientMetadata)
200 202
201- xrpc := atproto.NewXRPCHandler(s.db.DB)203+ xrpc := atproto.NewXRPCHandler(s.db.DB, s.engine)
202 s.router.Get("/xrpc/at.glean.listSubscriptions", xrpc.ListSubscriptions)204 s.router.Get("/xrpc/at.glean.listSubscriptions", xrpc.ListSubscriptions)
203 s.router.Get("/xrpc/at.glean.listAnnotations", xrpc.ListAnnotations)205 s.router.Get("/xrpc/at.glean.listAnnotations", xrpc.ListAnnotations)
204 s.router.Get("/xrpc/at.glean.listLikes", xrpc.ListLikes)206 s.router.Get("/xrpc/at.glean.listLikes", xrpc.ListLikes)
modified internal/tmpl/partials/recommendation-card.html +9 -2
@@ -1,5 +1,5 @@
11 {{define "recommendation-card.html"}}
2-<div class="bg-spot-surface rounded-xl p-3 hover:bg-spot-hover-50 transition">
2+<div class="recommendation-card bg-spot-surface rounded-xl p-3 hover:bg-spot-hover-50 transition">
33 <div class="flex items-center justify-between gap-2">
44 <a href="/articles?feed={{.feed_url}}" class="min-w-0 flex items-center gap-2 flex-1">
55 {{if .favicon_url}}<img src="{{.favicon_url}}" class="w-4 h-4 rounded shrink-0" loading="lazy">{{end}}
@@ -10,7 +10,14 @@
1010 </a>
1111 <div class="flex items-center gap-2 shrink-0">
1212 <span class="text-xs text-spot-secondary">{{.subscriber_count}} subs</span>
13- <form hx-post="/feeds/add" hx-target="closest .bg-spot-surface" hx-swap="outerHTML" class="inline">
13+ <form hx-post="/feeds/dismiss" hx-target="closest .recommendation-card" hx-swap="outerHTML" class="inline">
14+ {{csrfInput .CSRFToken}}
15+ <input type="hidden" name="feed_url" value="{{.feed_url}}">
16+ <button type="submit" title="Not interested" class="text-spot-muted hover:text-spot-red transition -m-1 p-1 rounded-full hover:bg-spot-hover-50 flex items-center">
17+ <svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="w-4 h-4"><path stroke-linecap="round" stroke-linejoin="round" d="M6 18 18 6M6 6l12 12"/></svg>
18+ </button>
19+ </form>
20+ <form hx-post="/feeds/add" hx-target="closest .recommendation-card" hx-swap="outerHTML" class="inline">
1421 {{csrfInput .CSRFToken}}
1522 <input type="hidden" name="feed_url" value="{{.feed_url}}">
1623 <button type="submit" class="text-xs font-bold uppercase tracking-button text-spot-green hover:brightness-110 transition">Subscribe</button>
@@ -1,5 +1,5 @@
1 {{define "recommendation-card.html"}}1 {{define "recommendation-card.html"}}
2-<div class="bg-spot-surface rounded-xl p-3 hover:bg-spot-hover-50 transition">2+<div class="recommendation-card bg-spot-surface rounded-xl p-3 hover:bg-spot-hover-50 transition">
3 <div class="flex items-center justify-between gap-2">3 <div class="flex items-center justify-between gap-2">
4 <a href="/articles?feed={{.feed_url}}" class="min-w-0 flex items-center gap-2 flex-1">4 <a href="/articles?feed={{.feed_url}}" class="min-w-0 flex items-center gap-2 flex-1">
5 {{if .favicon_url}}<img src="{{.favicon_url}}" class="w-4 h-4 rounded shrink-0" loading="lazy">{{end}}5 {{if .favicon_url}}<img src="{{.favicon_url}}" class="w-4 h-4 rounded shrink-0" loading="lazy">{{end}}
@@ -10,7 +10,14 @@
10 </a>10 </a>
11 <div class="flex items-center gap-2 shrink-0">11 <div class="flex items-center gap-2 shrink-0">
12 <span class="text-xs text-spot-secondary">{{.subscriber_count}} subs</span>12 <span class="text-xs text-spot-secondary">{{.subscriber_count}} subs</span>
13- <form hx-post="/feeds/add" hx-target="closest .bg-spot-surface" hx-swap="outerHTML" class="inline">13+ <form hx-post="/feeds/dismiss" hx-target="closest .recommendation-card" hx-swap="outerHTML" class="inline">
14+ {{csrfInput .CSRFToken}}
15+ <input type="hidden" name="feed_url" value="{{.feed_url}}">
16+ <button type="submit" title="Not interested" class="text-spot-muted hover:text-spot-red transition -m-1 p-1 rounded-full hover:bg-spot-hover-50 flex items-center">
17+ <svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="w-4 h-4"><path stroke-linecap="round" stroke-linejoin="round" d="M6 18 18 6M6 6l12 12"/></svg>
18+ </button>
19+ </form>
20+ <form hx-post="/feeds/add" hx-target="closest .recommendation-card" hx-swap="outerHTML" class="inline">
14 {{csrfInput .CSRFToken}}21 {{csrfInput .CSRFToken}}
15 <input type="hidden" name="feed_url" value="{{.feed_url}}">22 <input type="hidden" name="feed_url" value="{{.feed_url}}">
16 <button type="submit" class="text-xs font-bold uppercase tracking-button text-spot-green hover:brightness-110 transition">Subscribe</button>23 <button type="submit" class="text-xs font-bold uppercase tracking-button text-spot-green hover:brightness-110 transition">Subscribe</button>