Refactor recommendation engine and add dismissal logicUnverified
b8b7aa8 parent: c5ad568 modified
docs/specs.md +144 -67 | @@ -143,13 +143,13 @@ Glean also indexes records from the Skyreader lexicon (`app.skyreader.feed.subsc | ||
| 143 | 143 | |
| 144 | 144 | The mapping from Skyreader subscription to Glean subscription: |
| 145 | 145 | |
| 146 | -| Skyreader field | Glean field | Notes | | |
| 147 | -| --------------- | ------------- | ------------------------------------ | | |
| 148 | -| `feedUrl` | `feed_url` | Direct mapping | | |
| 149 | -| `title` | `title` | Direct mapping | | |
| 150 | -| `siteUrl` | `site_url` | Stored on the feed record | | |
| 151 | -| `createdAt` | `added_at` | Direct mapping | | |
| 152 | -| _(none)_ | `category` | Empty (Skyreader has no categories) | | |
| 146 | +| Skyreader field | Glean field | Notes | | |
| 147 | +| --------------- | ----------- | ----------------------------------- | | |
| 148 | +| `feedUrl` | `feed_url` | Direct mapping | | |
| 149 | +| `title` | `title` | Direct mapping | | |
| 150 | +| `siteUrl` | `site_url` | Stored on the feed record | | |
| 151 | +| `createdAt` | `added_at` | Direct mapping | | |
| 152 | +| _(none)_ | `category` | Empty (Skyreader has no categories) | | |
| 153 | 153 | |
| 154 | 154 | If a Glean subscription already exists for the same `feed_url`, the existing one is kept. If the existing subscription has no URI (was created locally without PDS sync), the Skyreader URI/CID is backfilled. |
| 155 | 155 | |
| @@ -661,12 +661,20 @@ CREATE TABLE oauth_sessions ( | ||
| 661 | 661 | ); |
| 662 | 662 | ``` |
| 663 | 663 | |
| 664 | -Glean has two complementary recommendation signals: | |
| 664 | +Glean uses a multi-signal recommendation system that combines subscription overlap, like patterns, social graph distance, and user behavior feedback. | |
| 665 | 665 | |
| 666 | -- **Subscriptions** (Jaccard similarity): "Who reads the same feeds?" → feed and people discovery | |
| 667 | -- **Likes** (co-occurrence): "Who likes the same articles?" → article and feed discovery | |
| 666 | +### 7.1 Signals | |
| 668 | 667 | |
| 669 | -### 7.1 Feed Co-occurrence (Jaccard Similarity) | |
| 668 | +| Signal | Source | Weight (default) | Description | | |
| 669 | +|--------|--------|-------------------|-------------| | |
| 670 | +| Subscription | `subscriptions` | 1.0 | Jaccard over subscriber sets between similar users | | |
| 671 | +| Like | `likes` | 0.5 | Time-decayed like co-occurrence (30-day half-life) | | |
| 672 | +| Tag | `annotations.tags` | 0.3 | Jaccard over annotation tag sets | | |
| 673 | +| Social | `follow_distances` | 0.7 | Follow distance: 1-hop=1.0, 2-hop=0.3 | | |
| 674 | +| Popularity | `feeds.subscriber_count` | 0.2 | `log(1 + subscribers) / log(1 + max)` | | |
| 675 | +| Category | `subscriptions.category` | 0.4 | Boost feeds matching user's existing categories | | |
| 676 | + | |
| 677 | +### 7.2 Feed Co-occurrence (Jaccard Similarity) | |
| 670 | 678 | |
| 671 | 679 | For any two feeds, the similarity is the Jaccard index of their subscriber sets: |
| 672 | 680 | |
| @@ -674,89 +682,153 @@ For any two feeds, the similarity is the Jaccard index of their subscriber sets: | ||
| 674 | 682 | J(A, B) = |subscribers(A) ∩ subscribers(B)| / |subscribers(A) ∪ subscribers(B)| |
| 675 | 683 | ``` |
| 676 | 684 | |
| 677 | -This is recomputed periodically (cron job) or incrementally when subscriptions change. | |
| 685 | +Feed description text similarity is also computed (word overlap after stopword removal) and added as a boost. | |
| 678 | 686 | |
| 679 | -### 7.2 User Similarity | |
| 687 | +### 7.3 User Similarity | |
| 680 | 688 | |
| 681 | -For any two users, compute Jaccard over their subscription sets: | |
| 689 | +For any two users, compute Jaccard over their subscription sets, plus like co-occurrence (time-decayed) and tag overlap: | |
| 682 | 690 | |
| 683 | 691 | ``` |
| 684 | -J(U1, U2) = |feeds(U1) ∩ feeds(U2)| / |feeds(U1) ∪ feeds(U2)| | |
| 692 | +J(U1, U2) = jaccard_subscriptions + 0.3 * jaccard_likes + 0.2 * jaccard_tags + follow_boost | |
| 685 | 693 | ``` |
| 686 | 694 | |
| 687 | -### 7.3 Recommendation Algorithms | |
| 695 | +Like overlap uses exponential time decay: `EXP(-0.023 * age_days)` (30-day half-life). | |
| 696 | + | |
| 697 | +### 7.4 On-Demand Scoring | |
| 688 | 698 | |
| 689 | -**Feed recommendations (on glean.at):** | |
| 699 | +Recommendations are computed **on-demand** at query time, not pre-materialized. This avoids write amplification on every cron run. | |
| 690 | 700 | |
| 691 | -1. Find users with Jaccard > 0.2 (similar readers) | |
| 692 | -2. Collect feeds those users subscribe to that the target user does not | |
| 693 | -3. Rank by frequency (how many similar users subscribe) and average similarity | |
| 694 | -4. Return top N feeds as recommendations | |
| 701 | +**Feed recommendation score** (computed in SQL): | |
| 695 | 702 | |
| 696 | 703 | ``` |
| 697 | -score(feed) = Σ J(target, U) for each user U subscribed to feed | |
| 704 | +score = sub_signal * w_sub | |
| 705 | + + like_signal * w_like | |
| 706 | + + social_signal * w_social | |
| 707 | + + pop_signal * w_pop | |
| 708 | + + category_signal * w_category | |
| 698 | 709 | ``` |
| 699 | 710 | |
| 700 | -**Article recommendations (on glean.at, from likes):** | |
| 711 | +Where: | |
| 712 | +- `sub_signal = SUM(jaccard(target, U))` for similar users U subscribed to feed | |
| 713 | +- `like_signal = SUM(jaccard(target, U) * time_decay)` for likes in that feed by similar users | |
| 714 | +- `social_signal = SUM(distance_weight)` from follow_distances | |
| 715 | +- `pop_signal = log(1 + subscriber_count) / log(1 + max_subscribers)` | |
| 716 | +- `category_signal = 1` if feed description matches user's top categories | |
| 701 | 717 | |
| 702 | -1. Find users who liked articles that the target user also liked | |
| 703 | -2. Collect articles those users liked that the target has not | |
| 704 | -3. Rank by frequency and recency | |
| 705 | -4. Return top N articles as recommendations | |
| 718 | +**Article recommendation score**: | |
| 706 | 719 | |
| 707 | 720 | ``` |
| 708 | -score(article) = Σ J(target, U) for each similar user U who liked the article | |
| 721 | +score = like_signal * w_like | |
| 722 | + + social_signal * w_social | |
| 723 | + + recency_signal * 0.2 | |
| 709 | 724 | ``` |
| 710 | 725 | |
| 711 | -**People recommendations (to follow on Bluesky):** | |
| 726 | +### 7.5 User Feedback (Dismiss) | |
| 712 | 727 | |
| 713 | -1. Compute user similarity for all pairs | |
| 714 | -2. Return users with highest Jaccard, linking to their Bluesky profile for follow | |
| 728 | +Users can dismiss recommendations they don't want to see again: | |
| 715 | 729 | |
| 716 | -### 7.4 Implementation | |
| 730 | +- `POST /feeds/dismiss` — dismiss a feed recommendation | |
| 731 | +- `POST /articles/dismiss` — dismiss an article recommendation | |
| 732 | +- Dismissals are stored locally in `dismissed_recommendations` (not on PDS) | |
| 733 | +- Dismissed items are excluded from all future recommendation queries | |
| 734 | +- Auto-dismiss: items shown >15 times over >30 days without action are auto-dismissed | |
| 717 | 735 | |
| 718 | -For the initial version, brute-force Jaccard with SQLite is sufficient (scale: ~10k users, ~100k subscriptions). The query is: | |
| 736 | +Impression tracking (`recommendation_impressions`) records how many times each recommendation was shown and whether the user acted on it. | |
| 719 | 737 | |
| 720 | -```sql | |
| 721 | -SELECT s2.feed_url, COUNT(*) as overlap_count | |
| 722 | -FROM subscriptions s1 | |
| 723 | -JOIN subscriptions s2 ON s1.feed_url = s2.feed_url | |
| 724 | -WHERE s1.user_did = ? AND s2.user_did != ? | |
| 725 | -AND s2.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) | |
| 726 | -GROUP BY s2.feed_url | |
| 727 | -ORDER BY overlap_count DESC | |
| 728 | -LIMIT 20; | |
| 738 | +### 7.6 Auto-Tuned Signal Weights | |
| 739 | + | |
| 740 | +Each user has a row in `user_signal_weights` with per-signal weights. When a user acts on a recommendation (subscribes, likes), the dominant signal that produced that recommendation is rewarded: | |
| 741 | + | |
| 742 | +``` | |
| 743 | +new_weight = MAX(0.1, MIN(3.0, old_weight * (1 + learning_rate * delta))) | |
| 729 | 744 | ``` |
| 730 | 745 | |
| 731 | -For larger scale, move to MinHash + LSH (banded hashing) to approximate Jaccard in sub-linear time. | |
| 746 | +- `learning_rate = 0.1`, `delta = +1` for reward, `-1` for penalty | |
| 747 | +- Only activates after `minActionsTune = 5` positive actions | |
| 748 | +- Defaults are used when no row exists for a user | |
| 749 | + | |
| 750 | +### 7.7 Social Graph | |
| 751 | + | |
| 752 | +Follow distances (1-hop and 2-hop) are pre-computed in `follow_distances` during the cron job: | |
| 753 | + | |
| 754 | +- 1-hop: direct follows (weight 1.0) | |
| 755 | +- 2-hop: friends-of-friends (weight 0.3) | |
| 756 | +- 3-hop is excluded due to noise and computational cost | |
| 757 | + | |
| 758 | +### 7.8 Diversity & Freshness | |
| 759 | + | |
| 760 | +After scoring, diversity filtering is applied in Go (not SQL): | |
| 761 | + | |
| 762 | +- **Domain diversity**: max 2 feeds from the same domain in results | |
| 763 | +- **Category diversity**: max 3 feeds from the same category in results | |
| 764 | +- This prevents recommendation clustering on a single source | |
| 765 | + | |
| 766 | +### 7.9 Cold Start | |
| 767 | + | |
| 768 | +New users with <5 subscriptions get a fallback strategy: | |
| 769 | + | |
| 770 | +1. Feeds from 1-hop followed users (70% weight) | |
| 771 | +2. Globally popular feeds by subscriber count (30% weight) | |
| 732 | 772 | |
| 733 | -### 7.5 Clustering Engine (Cron) | |
| 773 | +### 7.10 Clustering Engine (Cron) | |
| 734 | 774 | |
| 735 | 775 | A background goroutine runs on a configurable schedule (`GLEAN_CLUSTER_INTERVAL`, default 10m): |
| 736 | 776 | |
| 737 | -1. **Compute feed similarity**: Batch-update the `feed_similarity` table (Jaccard over subscriber sets) | |
| 738 | -2. **Compute user similarity**: Batch-update the `user_similarity` table (Jaccard over subscription sets, boosted by follow relationships) | |
| 739 | -3. **Generate feed recommendations**: Materialize top feed recommendations per user into `user_feed_recommendations` | |
| 740 | -4. **Generate article recommendations**: Materialize top article recommendations per user into `user_article_recommendations` | |
| 777 | +1. **Compute feed similarity**: Batch-update `feed_similarity` table (Jaccard over subscriber sets + description similarity) | |
| 778 | +2. **Compute user similarity**: Batch-update `user_similarity` table (subscription Jaccard + time-decayed likes + tags + follow boost) | |
| 779 | +3. **Compute follow distances**: 1-hop and 2-hop from `follows` table | |
| 780 | +4. **Compute signal profiles**: Per-user category/tag/like summaries | |
| 781 | +5. **Auto-dismiss stale**: Dismiss items shown >15 times over >30 days without action | |
| 741 | 782 | |
| 742 | 783 | Jetstream ingestion and record indexing happen in a separate persistent goroutine (the Jetstream consumer), not in the cron. |
| 743 | 784 | |
| 785 | +### 7.11 New Database Tables | |
| 786 | + | |
| 744 | 787 | ```sql |
| 745 | -CREATE TABLE user_feed_recommendations ( | |
| 746 | - user_did TEXT NOT NULL REFERENCES users(did), | |
| 747 | - feed_url TEXT NOT NULL REFERENCES feeds(feed_url), | |
| 748 | - score REAL NOT NULL, | |
| 749 | - computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | |
| 750 | - PRIMARY KEY (user_did, feed_url) | |
| 788 | +CREATE TABLE dismissed_recommendations ( | |
| 789 | + user_did TEXT NOT NULL REFERENCES users(did), | |
| 790 | + target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')), | |
| 791 | + target_id TEXT NOT NULL, | |
| 792 | + reason TEXT, | |
| 793 | + dismissed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | |
| 794 | + PRIMARY KEY (user_did, target_type, target_id) | |
| 751 | 795 | ); |
| 752 | 796 | |
| 753 | -CREATE TABLE user_article_recommendations ( | |
| 754 | - user_did TEXT NOT NULL REFERENCES users(did), | |
| 755 | - feed_url TEXT NOT NULL, | |
| 756 | - article_url TEXT NOT NULL, | |
| 757 | - score REAL NOT NULL, | |
| 758 | - computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | |
| 759 | - PRIMARY KEY (user_did, feed_url, article_url) | |
| 797 | +CREATE TABLE recommendation_impressions ( | |
| 798 | + user_did TEXT NOT NULL REFERENCES users(did), | |
| 799 | + target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')), | |
| 800 | + target_id TEXT NOT NULL, | |
| 801 | + first_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | |
| 802 | + last_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | |
| 803 | + shown_count INTEGER NOT NULL DEFAULT 1, | |
| 804 | + acted BOOLEAN NOT NULL DEFAULT 0, | |
| 805 | + PRIMARY KEY (user_did, target_type, target_id) | |
| 806 | +); | |
| 807 | + | |
| 808 | +CREATE TABLE follow_distances ( | |
| 809 | + user_a TEXT NOT NULL, | |
| 810 | + user_b TEXT NOT NULL, | |
| 811 | + distance INTEGER NOT NULL CHECK(distance IN (1, 2)), | |
| 812 | + PRIMARY KEY (user_a, user_b) | |
| 813 | +); | |
| 814 | + | |
| 815 | +CREATE TABLE user_signal_weights ( | |
| 816 | + user_did TEXT PRIMARY KEY REFERENCES users(did), | |
| 817 | + w_sub REAL NOT NULL DEFAULT 1.0, | |
| 818 | + w_like REAL NOT NULL DEFAULT 0.5, | |
| 819 | + w_tag REAL NOT NULL DEFAULT 0.3, | |
| 820 | + w_social REAL NOT NULL DEFAULT 0.7, | |
| 821 | + w_pop REAL NOT NULL DEFAULT 0.2, | |
| 822 | + w_category REAL NOT NULL DEFAULT 0.4, | |
| 823 | + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP | |
| 824 | +); | |
| 825 | + | |
| 826 | +CREATE TABLE user_signal_profiles ( | |
| 827 | + user_did TEXT PRIMARY KEY REFERENCES users(did), | |
| 828 | + total_likes INTEGER NOT NULL DEFAULT 0, | |
| 829 | + total_tags INTEGER NOT NULL DEFAULT 0, | |
| 830 | + top_categories TEXT, | |
| 831 | + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP | |
| 760 | 832 | ); |
| 761 | 833 | ``` |
| 762 | 834 | |
| @@ -778,15 +850,17 @@ The server renders HTML fragments that htmx swaps into the page. No JSON API nee | ||
| 778 | 850 | | `/feeds/add` | POST | Add a single feed URL | |
| 779 | 851 | | `/feeds/remove` | DELETE | Remove a feed | |
| 780 | 852 | | `/feeds/refresh` | POST | Refresh all subscribed feeds | |
| 781 | -| `/feeds/clear` | POST | Clear all subscriptions | | |
| 853 | +| `/feeds/clear` | POST | Clear all subscriptions | | |
| 854 | +| `/feeds/dismiss` | POST | Dismiss a feed recommendation | | |
| 782 | 855 | | `/articles` | GET | Read articles (paginated, filterable by feed) | |
| 783 | 856 | | `/articles/{id}` | GET | Article detail view | |
| 784 | 857 | | `/articles/{id}/read` | POST | Mark article as read | |
| 785 | 858 | | `/articles/{id}/unread` | POST | Mark article as unread | |
| 786 | 859 | | `/articles/{id}/like` | POST | Like an article | |
| 787 | 860 | | `/articles/{id}/fetch-content` | POST | Fetch full article content from original URL | |
| 788 | -| `/articles/mark-all-read` | POST | Mark all articles as read | | |
| 789 | -| `/trending` | GET | Community feed: articles ranked by likes | | |
| 861 | +| `/articles/mark-all-read` | POST | Mark all articles as read | | |
| 862 | +| `/articles/dismiss` | POST | Dismiss an article recommendation | | |
| 863 | +| `/trending` | GET | Community feed: articles ranked by likes | | |
| 790 | 864 | | `/library` | GET | Liked articles and annotations | |
| 791 | 865 | | `/library/create` | POST | Create annotation on an article | |
| 792 | 866 | | `/library/{id}/delete` | POST | Delete an annotation | |
| @@ -846,7 +920,12 @@ glean/ | ||
| 846 | 920 | │ │ └── metrics.go # Prometheus metrics definitions |
| 847 | 921 | │ ├── cluster/ |
| 848 | 922 | │ │ ├── jaccard.go # Jaccard similarity computation |
| 849 | -│ │ ├── recommender.go # Feed + people recommendation queries | |
| 923 | +│ │ ├── recommender.go # Feed + people recommendation queries (on-demand) | |
| 924 | +│ │ ├── scoring.go # Multi-signal composite scoring queries | |
| 925 | +│ │ ├── social.go # Follow-distance computation (1-2 hop) | |
| 926 | +│ │ ├── dismiss.go # Dismiss + impression tracking | |
| 927 | +│ │ ├── weights.go # Bandit-style signal weight auto-tuning | |
| 928 | +│ │ ├── diversity.go # Post-query domain/category diversity filtering | |
| 850 | 929 | │ │ └── cron.go # Background recomputation scheduler |
| 851 | 930 | │ ├── server/ |
| 852 | 931 | │ │ ├── server.go # HTTP server, router setup |
| @@ -993,6 +1072,4 @@ All PDS records are public. There is no notion of private data on the AT Protoco | ||
| 993 | 1072 | |
| 994 | 1073 | ## 13. Future Considerations |
| 995 | 1074 | |
| 996 | -- **MinHash/LSH**: Replace brute-force Jaccard when user count exceeds ~50k | |
| 997 | -- **Full-text search**: Add FTS5 virtual table on articles for search | |
| 998 | 1075 | - **Email digest**: Periodic email with top articles from subscribed feeds |
| @@ -143,13 +143,13 @@ Glean also indexes records from the Skyreader lexicon (`app.skyreader.feed.subsc | |||
| 143 | 143 | ||
| 144 | The mapping from Skyreader subscription to Glean subscription: | 144 | The mapping from Skyreader subscription to Glean subscription: |
| 145 | 145 | ||
| 146 | -| Skyreader field | Glean field | Notes | | 146 | +| Skyreader field | Glean field | Notes | |
| 147 | -| --------------- | ------------- | ------------------------------------ | | 147 | +| --------------- | ----------- | ----------------------------------- | |
| 148 | -| `feedUrl` | `feed_url` | Direct mapping | | 148 | +| `feedUrl` | `feed_url` | Direct mapping | |
| 149 | -| `title` | `title` | Direct mapping | | 149 | +| `title` | `title` | Direct mapping | |
| 150 | -| `siteUrl` | `site_url` | Stored on the feed record | | 150 | +| `siteUrl` | `site_url` | Stored on the feed record | |
| 151 | -| `createdAt` | `added_at` | Direct mapping | | 151 | +| `createdAt` | `added_at` | Direct mapping | |
| 152 | -| _(none)_ | `category` | Empty (Skyreader has no categories) | | 152 | +| _(none)_ | `category` | Empty (Skyreader has no categories) | |
| 153 | 153 | ||
| 154 | If a Glean subscription already exists for the same `feed_url`, the existing one is kept. If the existing subscription has no URI (was created locally without PDS sync), the Skyreader URI/CID is backfilled. | 154 | If a Glean subscription already exists for the same `feed_url`, the existing one is kept. If the existing subscription has no URI (was created locally without PDS sync), the Skyreader URI/CID is backfilled. |
| 155 | 155 | ||
| @@ -661,12 +661,20 @@ CREATE TABLE oauth_sessions ( | |||
| 661 | ); | 661 | ); |
| 662 | ``` | 662 | ``` |
| 663 | 663 | ||
| 664 | -Glean has two complementary recommendation signals: | 664 | +Glean uses a multi-signal recommendation system that combines subscription overlap, like patterns, social graph distance, and user behavior feedback. |
| 665 | 665 | ||
| 666 | -- **Subscriptions** (Jaccard similarity): "Who reads the same feeds?" → feed and people discovery | 666 | +### 7.1 Signals |
| 667 | -- **Likes** (co-occurrence): "Who likes the same articles?" → article and feed discovery | ||
| 668 | 667 | ||
| 669 | -### 7.1 Feed Co-occurrence (Jaccard Similarity) | 668 | +| Signal | Source | Weight (default) | Description | |
| 669 | +|--------|--------|-------------------|-------------| | ||
| 670 | +| Subscription | `subscriptions` | 1.0 | Jaccard over subscriber sets between similar users | | ||
| 671 | +| Like | `likes` | 0.5 | Time-decayed like co-occurrence (30-day half-life) | | ||
| 672 | +| Tag | `annotations.tags` | 0.3 | Jaccard over annotation tag sets | | ||
| 673 | +| Social | `follow_distances` | 0.7 | Follow distance: 1-hop=1.0, 2-hop=0.3 | | ||
| 674 | +| Popularity | `feeds.subscriber_count` | 0.2 | `log(1 + subscribers) / log(1 + max)` | | ||
| 675 | +| Category | `subscriptions.category` | 0.4 | Boost feeds matching user's existing categories | | ||
| 676 | + | ||
| 677 | +### 7.2 Feed Co-occurrence (Jaccard Similarity) | ||
| 670 | 678 | ||
| 671 | For any two feeds, the similarity is the Jaccard index of their subscriber sets: | 679 | For any two feeds, the similarity is the Jaccard index of their subscriber sets: |
| 672 | 680 | ||
| @@ -674,89 +682,153 @@ For any two feeds, the similarity is the Jaccard index of their subscriber sets: | |||
| 674 | J(A, B) = |subscribers(A) ∩ subscribers(B)| / |subscribers(A) ∪ subscribers(B)| | 682 | J(A, B) = |subscribers(A) ∩ subscribers(B)| / |subscribers(A) ∪ subscribers(B)| |
| 675 | ``` | 683 | ``` |
| 676 | 684 | ||
| 677 | -This is recomputed periodically (cron job) or incrementally when subscriptions change. | 685 | +Feed description text similarity is also computed (word overlap after stopword removal) and added as a boost. |
| 678 | 686 | ||
| 679 | -### 7.2 User Similarity | 687 | +### 7.3 User Similarity |
| 680 | 688 | ||
| 681 | -For any two users, compute Jaccard over their subscription sets: | 689 | +For any two users, compute Jaccard over their subscription sets, plus like co-occurrence (time-decayed) and tag overlap: |
| 682 | 690 | ||
| 683 | ``` | 691 | ``` |
| 684 | -J(U1, U2) = |feeds(U1) ∩ feeds(U2)| / |feeds(U1) ∪ feeds(U2)| | 692 | +J(U1, U2) = jaccard_subscriptions + 0.3 * jaccard_likes + 0.2 * jaccard_tags + follow_boost |
| 685 | ``` | 693 | ``` |
| 686 | 694 | ||
| 687 | -### 7.3 Recommendation Algorithms | 695 | +Like overlap uses exponential time decay: `EXP(-0.023 * age_days)` (30-day half-life). |
| 696 | + | ||
| 697 | +### 7.4 On-Demand Scoring | ||
| 688 | 698 | ||
| 689 | -**Feed recommendations (on glean.at):** | 699 | +Recommendations are computed **on-demand** at query time, not pre-materialized. This avoids write amplification on every cron run. |
| 690 | 700 | ||
| 691 | -1. Find users with Jaccard > 0.2 (similar readers) | 701 | +**Feed recommendation score** (computed in SQL): |
| 692 | -2. Collect feeds those users subscribe to that the target user does not | ||
| 693 | -3. Rank by frequency (how many similar users subscribe) and average similarity | ||
| 694 | -4. Return top N feeds as recommendations | ||
| 695 | 702 | ||
| 696 | ``` | 703 | ``` |
| 697 | -score(feed) = Σ J(target, U) for each user U subscribed to feed | 704 | +score = sub_signal * w_sub |
| 705 | + + like_signal * w_like | ||
| 706 | + + social_signal * w_social | ||
| 707 | + + pop_signal * w_pop | ||
| 708 | + + category_signal * w_category | ||
| 698 | ``` | 709 | ``` |
| 699 | 710 | ||
| 700 | -**Article recommendations (on glean.at, from likes):** | 711 | +Where: |
| 712 | +- `sub_signal = SUM(jaccard(target, U))` for similar users U subscribed to feed | ||
| 713 | +- `like_signal = SUM(jaccard(target, U) * time_decay)` for likes in that feed by similar users | ||
| 714 | +- `social_signal = SUM(distance_weight)` from follow_distances | ||
| 715 | +- `pop_signal = log(1 + subscriber_count) / log(1 + max_subscribers)` | ||
| 716 | +- `category_signal = 1` if feed description matches user's top categories | ||
| 701 | 717 | ||
| 702 | -1. Find users who liked articles that the target user also liked | 718 | +**Article recommendation score**: |
| 703 | -2. Collect articles those users liked that the target has not | ||
| 704 | -3. Rank by frequency and recency | ||
| 705 | -4. Return top N articles as recommendations | ||
| 706 | 719 | ||
| 707 | ``` | 720 | ``` |
| 708 | -score(article) = Σ J(target, U) for each similar user U who liked the article | 721 | +score = like_signal * w_like |
| 722 | + + social_signal * w_social | ||
| 723 | + + recency_signal * 0.2 | ||
| 709 | ``` | 724 | ``` |
| 710 | 725 | ||
| 711 | -**People recommendations (to follow on Bluesky):** | 726 | +### 7.5 User Feedback (Dismiss) |
| 712 | 727 | ||
| 713 | -1. Compute user similarity for all pairs | 728 | +Users can dismiss recommendations they don't want to see again: |
| 714 | -2. Return users with highest Jaccard, linking to their Bluesky profile for follow | ||
| 715 | 729 | ||
| 716 | -### 7.4 Implementation | 730 | +- `POST /feeds/dismiss` — dismiss a feed recommendation |
| 731 | +- `POST /articles/dismiss` — dismiss an article recommendation | ||
| 732 | +- Dismissals are stored locally in `dismissed_recommendations` (not on PDS) | ||
| 733 | +- Dismissed items are excluded from all future recommendation queries | ||
| 734 | +- Auto-dismiss: items shown >15 times over >30 days without action are auto-dismissed | ||
| 717 | 735 | ||
| 718 | -For the initial version, brute-force Jaccard with SQLite is sufficient (scale: ~10k users, ~100k subscriptions). The query is: | 736 | +Impression tracking (`recommendation_impressions`) records how many times each recommendation was shown and whether the user acted on it. |
| 719 | 737 | ||
| 720 | -```sql | 738 | +### 7.6 Auto-Tuned Signal Weights |
| 721 | -SELECT s2.feed_url, COUNT(*) as overlap_count | 739 | + |
| 722 | -FROM subscriptions s1 | 740 | +Each user has a row in `user_signal_weights` with per-signal weights. When a user acts on a recommendation (subscribes, likes), the dominant signal that produced that recommendation is rewarded: |
| 723 | -JOIN subscriptions s2 ON s1.feed_url = s2.feed_url | 741 | + |
| 724 | -WHERE s1.user_did = ? AND s2.user_did != ? | 742 | +``` |
| 725 | -AND s2.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) | 743 | +new_weight = MAX(0.1, MIN(3.0, old_weight * (1 + learning_rate * delta))) |
| 726 | -GROUP BY s2.feed_url | ||
| 727 | -ORDER BY overlap_count DESC | ||
| 728 | -LIMIT 20; | ||
| 729 | ``` | 744 | ``` |
| 730 | 745 | ||
| 731 | -For larger scale, move to MinHash + LSH (banded hashing) to approximate Jaccard in sub-linear time. | 746 | +- `learning_rate = 0.1`, `delta = +1` for reward, `-1` for penalty |
| 747 | +- Only activates after `minActionsTune = 5` positive actions | ||
| 748 | +- Defaults are used when no row exists for a user | ||
| 749 | + | ||
| 750 | +### 7.7 Social Graph | ||
| 751 | + | ||
| 752 | +Follow distances (1-hop and 2-hop) are pre-computed in `follow_distances` during the cron job: | ||
| 753 | + | ||
| 754 | +- 1-hop: direct follows (weight 1.0) | ||
| 755 | +- 2-hop: friends-of-friends (weight 0.3) | ||
| 756 | +- 3-hop is excluded due to noise and computational cost | ||
| 757 | + | ||
| 758 | +### 7.8 Diversity & Freshness | ||
| 759 | + | ||
| 760 | +After scoring, diversity filtering is applied in Go (not SQL): | ||
| 761 | + | ||
| 762 | +- **Domain diversity**: max 2 feeds from the same domain in results | ||
| 763 | +- **Category diversity**: max 3 feeds from the same category in results | ||
| 764 | +- This prevents recommendation clustering on a single source | ||
| 765 | + | ||
| 766 | +### 7.9 Cold Start | ||
| 767 | + | ||
| 768 | +New users with <5 subscriptions get a fallback strategy: | ||
| 769 | + | ||
| 770 | +1. Feeds from 1-hop followed users (70% weight) | ||
| 771 | +2. Globally popular feeds by subscriber count (30% weight) | ||
| 732 | 772 | ||
| 733 | -### 7.5 Clustering Engine (Cron) | 773 | +### 7.10 Clustering Engine (Cron) |
| 734 | 774 | ||
| 735 | A background goroutine runs on a configurable schedule (`GLEAN_CLUSTER_INTERVAL`, default 10m): | 775 | A background goroutine runs on a configurable schedule (`GLEAN_CLUSTER_INTERVAL`, default 10m): |
| 736 | 776 | ||
| 737 | -1. **Compute feed similarity**: Batch-update the `feed_similarity` table (Jaccard over subscriber sets) | 777 | +1. **Compute feed similarity**: Batch-update `feed_similarity` table (Jaccard over subscriber sets + description similarity) |
| 738 | -2. **Compute user similarity**: Batch-update the `user_similarity` table (Jaccard over subscription sets, boosted by follow relationships) | 778 | +2. **Compute user similarity**: Batch-update `user_similarity` table (subscription Jaccard + time-decayed likes + tags + follow boost) |
| 739 | -3. **Generate feed recommendations**: Materialize top feed recommendations per user into `user_feed_recommendations` | 779 | +3. **Compute follow distances**: 1-hop and 2-hop from `follows` table |
| 740 | -4. **Generate article recommendations**: Materialize top article recommendations per user into `user_article_recommendations` | 780 | +4. **Compute signal profiles**: Per-user category/tag/like summaries |
| 781 | +5. **Auto-dismiss stale**: Dismiss items shown >15 times over >30 days without action | ||
| 741 | 782 | ||
| 742 | Jetstream ingestion and record indexing happen in a separate persistent goroutine (the Jetstream consumer), not in the cron. | 783 | Jetstream ingestion and record indexing happen in a separate persistent goroutine (the Jetstream consumer), not in the cron. |
| 743 | 784 | ||
| 785 | +### 7.11 New Database Tables | ||
| 786 | + | ||
| 744 | ```sql | 787 | ```sql |
| 745 | -CREATE TABLE user_feed_recommendations ( | 788 | +CREATE TABLE dismissed_recommendations ( |
| 746 | - user_did TEXT NOT NULL REFERENCES users(did), | 789 | + user_did TEXT NOT NULL REFERENCES users(did), |
| 747 | - feed_url TEXT NOT NULL REFERENCES feeds(feed_url), | 790 | + target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')), |
| 748 | - score REAL NOT NULL, | 791 | + target_id TEXT NOT NULL, |
| 749 | - computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | 792 | + reason TEXT, |
| 750 | - PRIMARY KEY (user_did, feed_url) | 793 | + dismissed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, |
| 794 | + PRIMARY KEY (user_did, target_type, target_id) | ||
| 751 | ); | 795 | ); |
| 752 | 796 | ||
| 753 | -CREATE TABLE user_article_recommendations ( | 797 | +CREATE TABLE recommendation_impressions ( |
| 754 | - user_did TEXT NOT NULL REFERENCES users(did), | 798 | + user_did TEXT NOT NULL REFERENCES users(did), |
| 755 | - feed_url TEXT NOT NULL, | 799 | + target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')), |
| 756 | - article_url TEXT NOT NULL, | 800 | + target_id TEXT NOT NULL, |
| 757 | - score REAL NOT NULL, | 801 | + first_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, |
| 758 | - computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | 802 | + last_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, |
| 759 | - PRIMARY KEY (user_did, feed_url, article_url) | 803 | + shown_count INTEGER NOT NULL DEFAULT 1, |
| 804 | + acted BOOLEAN NOT NULL DEFAULT 0, | ||
| 805 | + PRIMARY KEY (user_did, target_type, target_id) | ||
| 806 | +); | ||
| 807 | + | ||
| 808 | +CREATE TABLE follow_distances ( | ||
| 809 | + user_a TEXT NOT NULL, | ||
| 810 | + user_b TEXT NOT NULL, | ||
| 811 | + distance INTEGER NOT NULL CHECK(distance IN (1, 2)), | ||
| 812 | + PRIMARY KEY (user_a, user_b) | ||
| 813 | +); | ||
| 814 | + | ||
| 815 | +CREATE TABLE user_signal_weights ( | ||
| 816 | + user_did TEXT PRIMARY KEY REFERENCES users(did), | ||
| 817 | + w_sub REAL NOT NULL DEFAULT 1.0, | ||
| 818 | + w_like REAL NOT NULL DEFAULT 0.5, | ||
| 819 | + w_tag REAL NOT NULL DEFAULT 0.3, | ||
| 820 | + w_social REAL NOT NULL DEFAULT 0.7, | ||
| 821 | + w_pop REAL NOT NULL DEFAULT 0.2, | ||
| 822 | + w_category REAL NOT NULL DEFAULT 0.4, | ||
| 823 | + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP | ||
| 824 | +); | ||
| 825 | + | ||
| 826 | +CREATE TABLE user_signal_profiles ( | ||
| 827 | + user_did TEXT PRIMARY KEY REFERENCES users(did), | ||
| 828 | + total_likes INTEGER NOT NULL DEFAULT 0, | ||
| 829 | + total_tags INTEGER NOT NULL DEFAULT 0, | ||
| 830 | + top_categories TEXT, | ||
| 831 | + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP | ||
| 760 | ); | 832 | ); |
| 761 | ``` | 833 | ``` |
| 762 | 834 | ||
| @@ -778,15 +850,17 @@ The server renders HTML fragments that htmx swaps into the page. No JSON API nee | |||
| 778 | | `/feeds/add` | POST | Add a single feed URL | | 850 | | `/feeds/add` | POST | Add a single feed URL | |
| 779 | | `/feeds/remove` | DELETE | Remove a feed | | 851 | | `/feeds/remove` | DELETE | Remove a feed | |
| 780 | | `/feeds/refresh` | POST | Refresh all subscribed feeds | | 852 | | `/feeds/refresh` | POST | Refresh all subscribed feeds | |
| 781 | -| `/feeds/clear` | POST | Clear all subscriptions | | 853 | +| `/feeds/clear` | POST | Clear all subscriptions | |
| 854 | +| `/feeds/dismiss` | POST | Dismiss a feed recommendation | | ||
| 782 | | `/articles` | GET | Read articles (paginated, filterable by feed) | | 855 | | `/articles` | GET | Read articles (paginated, filterable by feed) | |
| 783 | | `/articles/{id}` | GET | Article detail view | | 856 | | `/articles/{id}` | GET | Article detail view | |
| 784 | | `/articles/{id}/read` | POST | Mark article as read | | 857 | | `/articles/{id}/read` | POST | Mark article as read | |
| 785 | | `/articles/{id}/unread` | POST | Mark article as unread | | 858 | | `/articles/{id}/unread` | POST | Mark article as unread | |
| 786 | | `/articles/{id}/like` | POST | Like an article | | 859 | | `/articles/{id}/like` | POST | Like an article | |
| 787 | | `/articles/{id}/fetch-content` | POST | Fetch full article content from original URL | | 860 | | `/articles/{id}/fetch-content` | POST | Fetch full article content from original URL | |
| 788 | -| `/articles/mark-all-read` | POST | Mark all articles as read | | 861 | +| `/articles/mark-all-read` | POST | Mark all articles as read | |
| 789 | -| `/trending` | GET | Community feed: articles ranked by likes | | 862 | +| `/articles/dismiss` | POST | Dismiss an article recommendation | |
| 863 | +| `/trending` | GET | Community feed: articles ranked by likes | | ||
| 790 | | `/library` | GET | Liked articles and annotations | | 864 | | `/library` | GET | Liked articles and annotations | |
| 791 | | `/library/create` | POST | Create annotation on an article | | 865 | | `/library/create` | POST | Create annotation on an article | |
| 792 | | `/library/{id}/delete` | POST | Delete an annotation | | 866 | | `/library/{id}/delete` | POST | Delete an annotation | |
| @@ -846,7 +920,12 @@ glean/ | |||
| 846 | │ │ └── metrics.go # Prometheus metrics definitions | 920 | │ │ └── metrics.go # Prometheus metrics definitions |
| 847 | │ ├── cluster/ | 921 | │ ├── cluster/ |
| 848 | │ │ ├── jaccard.go # Jaccard similarity computation | 922 | │ │ ├── jaccard.go # Jaccard similarity computation |
| 849 | -│ │ ├── recommender.go # Feed + people recommendation queries | 923 | +│ │ ├── recommender.go # Feed + people recommendation queries (on-demand) |
| 924 | +│ │ ├── scoring.go # Multi-signal composite scoring queries | ||
| 925 | +│ │ ├── social.go # Follow-distance computation (1-2 hop) | ||
| 926 | +│ │ ├── dismiss.go # Dismiss + impression tracking | ||
| 927 | +│ │ ├── weights.go # Bandit-style signal weight auto-tuning | ||
| 928 | +│ │ ├── diversity.go # Post-query domain/category diversity filtering | ||
| 850 | │ │ └── cron.go # Background recomputation scheduler | 929 | │ │ └── cron.go # Background recomputation scheduler |
| 851 | │ ├── server/ | 930 | │ ├── server/ |
| 852 | │ │ ├── server.go # HTTP server, router setup | 931 | │ │ ├── server.go # HTTP server, router setup |
| @@ -993,6 +1072,4 @@ All PDS records are public. There is no notion of private data on the AT Protoco | |||
| 993 | 1072 | ||
| 994 | ## 13. Future Considerations | 1073 | ## 13. Future Considerations |
| 995 | 1074 | ||
| 996 | -- **MinHash/LSH**: Replace brute-force Jaccard when user count exceeds ~50k | ||
| 997 | -- **Full-text search**: Add FTS5 virtual table on articles for search | ||
| 998 | - **Email digest**: Periodic email with top articles from subscribed feeds | 1075 | - **Email digest**: Periodic email with top articles from subscribed feeds |
modified
internal/atproto/xrpc.go +26 -54 | @@ -8,14 +8,17 @@ import ( | ||
| 8 | 8 | "strings" |
| 9 | 9 | |
| 10 | 10 | "github.com/go-chi/chi/v5" |
| 11 | + | |
| 12 | + "pkg.rbrt.fr/glean/internal/cluster" | |
| 11 | 13 | ) |
| 12 | 14 | |
| 13 | 15 | type XRPCHandler struct { |
| 14 | - db *sql.DB | |
| 16 | + db *sql.DB | |
| 17 | + engine *cluster.Engine | |
| 15 | 18 | } |
| 16 | 19 | |
| 17 | -func NewXRPCHandler(db *sql.DB) *XRPCHandler { | |
| 18 | - return &XRPCHandler{db: db} | |
| 20 | +func NewXRPCHandler(db *sql.DB, engine *cluster.Engine) *XRPCHandler { | |
| 21 | + return &XRPCHandler{db: db, engine: engine} | |
| 19 | 22 | } |
| 20 | 23 | |
| 21 | 24 | func (h *XRPCHandler) ListSubscriptions(w http.ResponseWriter, r *http.Request) { |
| @@ -303,72 +306,41 @@ func (h *XRPCHandler) GetRecommendations(w http.ResponseWriter, r *http.Request) | ||
| 303 | 306 | repo := r.URL.Query().Get("repo") |
| 304 | 307 | limit := min(parseIntParam(r, "limit", 20), 50) |
| 305 | 308 | |
| 306 | - feedRows, err := h.db.QueryContext(r.Context(), ` | |
| 307 | - SELECT r.feed_url, f.title, f.site_url, f.description, f.subscriber_count, r.score | |
| 308 | - FROM user_feed_recommendations r | |
| 309 | - JOIN feeds f ON r.feed_url = f.feed_url | |
| 310 | - WHERE r.user_did = ? | |
| 311 | - ORDER BY r.score DESC | |
| 312 | - LIMIT ? | |
| 313 | - `, repo, limit) | |
| 309 | + ctx := r.Context() | |
| 310 | + | |
| 311 | + feedRecs, err := h.engine.GetFeedRecommendations(ctx, repo, limit) | |
| 314 | 312 | if err != nil { |
| 315 | 313 | http.Error(w, err.Error(), http.StatusInternalServerError) |
| 316 | 314 | return |
| 317 | 315 | } |
| 318 | - defer feedRows.Close() | |
| 319 | 316 | |
| 320 | - feeds := make([]RecommendedFeed, 0) | |
| 321 | - for feedRows.Next() { | |
| 322 | - var feedURL, title, siteURL, description string | |
| 323 | - var subscriberCount int | |
| 324 | - var score float64 | |
| 325 | - if err := feedRows.Scan(&feedURL, &title, &siteURL, &description, &subscriberCount, &score); err != nil { | |
| 326 | - http.Error(w, err.Error(), http.StatusInternalServerError) | |
| 327 | - return | |
| 328 | - } | |
| 317 | + feeds := make([]RecommendedFeed, 0, len(feedRecs)) | |
| 318 | + for _, rec := range feedRecs { | |
| 329 | 319 | feeds = append(feeds, RecommendedFeed{ |
| 330 | - FeedURL: feedURL, | |
| 331 | - Title: title, | |
| 332 | - SiteURL: siteURL, | |
| 333 | - Description: description, | |
| 334 | - SubscriberCount: subscriberCount, | |
| 335 | - Score: score, | |
| 320 | + FeedURL: rec.FeedURL, | |
| 321 | + Title: rec.Title, | |
| 322 | + SiteURL: rec.SiteURL, | |
| 323 | + Description: rec.Description, | |
| 324 | + SubscriberCount: rec.SubscriberCount, | |
| 325 | + Score: rec.Score, | |
| 336 | 326 | }) |
| 337 | 327 | } |
| 338 | 328 | |
| 339 | - peopleRows, err := h.db.QueryContext(r.Context(), ` | |
| 340 | - SELECT u.did, u.handle, u.display_name, u.avatar_url, s.jaccard, s.common_feeds | |
| 341 | - FROM user_similarity s | |
| 342 | - JOIN users u ON ( | |
| 343 | - CASE WHEN s.user_a = ? THEN s.user_b ELSE s.user_a END | |
| 344 | - ) = u.did | |
| 345 | - WHERE s.user_a = ? OR s.user_b = ? | |
| 346 | - ORDER BY s.jaccard DESC | |
| 347 | - LIMIT ? | |
| 348 | - `, repo, repo, repo, limit) | |
| 329 | + peopleRecs, err := h.engine.GetPeopleRecommendations(ctx, repo, limit) | |
| 349 | 330 | if err != nil { |
| 350 | 331 | http.Error(w, err.Error(), http.StatusInternalServerError) |
| 351 | 332 | return |
| 352 | 333 | } |
| 353 | - defer peopleRows.Close() | |
| 354 | 334 | |
| 355 | - people := make([]RecommendedPerson, 0) | |
| 356 | - for peopleRows.Next() { | |
| 357 | - var did, handle string | |
| 358 | - var displayName, avatar sql.NullString | |
| 359 | - var jaccard float64 | |
| 360 | - var commonFeeds int | |
| 361 | - if err := peopleRows.Scan(&did, &handle, &displayName, &avatar, &jaccard, &commonFeeds); err != nil { | |
| 362 | - http.Error(w, err.Error(), http.StatusInternalServerError) | |
| 363 | - return | |
| 364 | - } | |
| 335 | + people := make([]RecommendedPerson, 0, len(peopleRecs)) | |
| 336 | + for _, rec := range peopleRecs { | |
| 365 | 337 | people = append(people, RecommendedPerson{ |
| 366 | - DID: did, | |
| 367 | - Handle: handle, | |
| 368 | - DisplayName: displayName.String, | |
| 369 | - Avatar: avatar.String, | |
| 370 | - Jaccard: jaccard, | |
| 371 | - CommonFeeds: commonFeeds, | |
| 338 | + DID: rec.DID, | |
| 339 | + Handle: rec.Handle, | |
| 340 | + DisplayName: rec.DisplayName, | |
| 341 | + Avatar: rec.AvatarURL, | |
| 342 | + Jaccard: rec.Jaccard, | |
| 343 | + CommonFeeds: rec.CommonFeeds, | |
| 372 | 344 | }) |
| 373 | 345 | } |
| 374 | 346 | |
| @@ -8,14 +8,17 @@ import ( | |||
| 8 | "strings" | 8 | "strings" |
| 9 | 9 | ||
| 10 | "github.com/go-chi/chi/v5" | 10 | "github.com/go-chi/chi/v5" |
| 11 | + | ||
| 12 | + "pkg.rbrt.fr/glean/internal/cluster" | ||
| 11 | ) | 13 | ) |
| 12 | 14 | ||
| 13 | type XRPCHandler struct { | 15 | type XRPCHandler struct { |
| 14 | - db *sql.DB | 16 | + db *sql.DB |
| 17 | + engine *cluster.Engine | ||
| 15 | } | 18 | } |
| 16 | 19 | ||
| 17 | -func NewXRPCHandler(db *sql.DB) *XRPCHandler { | 20 | +func NewXRPCHandler(db *sql.DB, engine *cluster.Engine) *XRPCHandler { |
| 18 | - return &XRPCHandler{db: db} | 21 | + return &XRPCHandler{db: db, engine: engine} |
| 19 | } | 22 | } |
| 20 | 23 | ||
| 21 | func (h *XRPCHandler) ListSubscriptions(w http.ResponseWriter, r *http.Request) { | 24 | func (h *XRPCHandler) ListSubscriptions(w http.ResponseWriter, r *http.Request) { |
| @@ -303,72 +306,41 @@ func (h *XRPCHandler) GetRecommendations(w http.ResponseWriter, r *http.Request) | |||
| 303 | repo := r.URL.Query().Get("repo") | 306 | repo := r.URL.Query().Get("repo") |
| 304 | limit := min(parseIntParam(r, "limit", 20), 50) | 307 | limit := min(parseIntParam(r, "limit", 20), 50) |
| 305 | 308 | ||
| 306 | - feedRows, err := h.db.QueryContext(r.Context(), ` | 309 | + ctx := r.Context() |
| 307 | - SELECT r.feed_url, f.title, f.site_url, f.description, f.subscriber_count, r.score | 310 | + |
| 308 | - FROM user_feed_recommendations r | 311 | + feedRecs, err := h.engine.GetFeedRecommendations(ctx, repo, limit) |
| 309 | - JOIN feeds f ON r.feed_url = f.feed_url | ||
| 310 | - WHERE r.user_did = ? | ||
| 311 | - ORDER BY r.score DESC | ||
| 312 | - LIMIT ? | ||
| 313 | - `, repo, limit) | ||
| 314 | if err != nil { | 312 | if err != nil { |
| 315 | http.Error(w, err.Error(), http.StatusInternalServerError) | 313 | http.Error(w, err.Error(), http.StatusInternalServerError) |
| 316 | return | 314 | return |
| 317 | } | 315 | } |
| 318 | - defer feedRows.Close() | ||
| 319 | 316 | ||
| 320 | - feeds := make([]RecommendedFeed, 0) | 317 | + feeds := make([]RecommendedFeed, 0, len(feedRecs)) |
| 321 | - for feedRows.Next() { | 318 | + for _, rec := range feedRecs { |
| 322 | - var feedURL, title, siteURL, description string | ||
| 323 | - var subscriberCount int | ||
| 324 | - var score float64 | ||
| 325 | - if err := feedRows.Scan(&feedURL, &title, &siteURL, &description, &subscriberCount, &score); err != nil { | ||
| 326 | - http.Error(w, err.Error(), http.StatusInternalServerError) | ||
| 327 | - return | ||
| 328 | - } | ||
| 329 | feeds = append(feeds, RecommendedFeed{ | 319 | feeds = append(feeds, RecommendedFeed{ |
| 330 | - FeedURL: feedURL, | 320 | + FeedURL: rec.FeedURL, |
| 331 | - Title: title, | 321 | + Title: rec.Title, |
| 332 | - SiteURL: siteURL, | 322 | + SiteURL: rec.SiteURL, |
| 333 | - Description: description, | 323 | + Description: rec.Description, |
| 334 | - SubscriberCount: subscriberCount, | 324 | + SubscriberCount: rec.SubscriberCount, |
| 335 | - Score: score, | 325 | + Score: rec.Score, |
| 336 | }) | 326 | }) |
| 337 | } | 327 | } |
| 338 | 328 | ||
| 339 | - peopleRows, err := h.db.QueryContext(r.Context(), ` | 329 | + peopleRecs, err := h.engine.GetPeopleRecommendations(ctx, repo, limit) |
| 340 | - SELECT u.did, u.handle, u.display_name, u.avatar_url, s.jaccard, s.common_feeds | ||
| 341 | - FROM user_similarity s | ||
| 342 | - JOIN users u ON ( | ||
| 343 | - CASE WHEN s.user_a = ? THEN s.user_b ELSE s.user_a END | ||
| 344 | - ) = u.did | ||
| 345 | - WHERE s.user_a = ? OR s.user_b = ? | ||
| 346 | - ORDER BY s.jaccard DESC | ||
| 347 | - LIMIT ? | ||
| 348 | - `, repo, repo, repo, limit) | ||
| 349 | if err != nil { | 330 | if err != nil { |
| 350 | http.Error(w, err.Error(), http.StatusInternalServerError) | 331 | http.Error(w, err.Error(), http.StatusInternalServerError) |
| 351 | return | 332 | return |
| 352 | } | 333 | } |
| 353 | - defer peopleRows.Close() | ||
| 354 | 334 | ||
| 355 | - people := make([]RecommendedPerson, 0) | 335 | + people := make([]RecommendedPerson, 0, len(peopleRecs)) |
| 356 | - for peopleRows.Next() { | 336 | + for _, rec := range peopleRecs { |
| 357 | - var did, handle string | ||
| 358 | - var displayName, avatar sql.NullString | ||
| 359 | - var jaccard float64 | ||
| 360 | - var commonFeeds int | ||
| 361 | - if err := peopleRows.Scan(&did, &handle, &displayName, &avatar, &jaccard, &commonFeeds); err != nil { | ||
| 362 | - http.Error(w, err.Error(), http.StatusInternalServerError) | ||
| 363 | - return | ||
| 364 | - } | ||
| 365 | people = append(people, RecommendedPerson{ | 337 | people = append(people, RecommendedPerson{ |
| 366 | - DID: did, | 338 | + DID: rec.DID, |
| 367 | - Handle: handle, | 339 | + Handle: rec.Handle, |
| 368 | - DisplayName: displayName.String, | 340 | + DisplayName: rec.DisplayName, |
| 369 | - Avatar: avatar.String, | 341 | + Avatar: rec.AvatarURL, |
| 370 | - Jaccard: jaccard, | 342 | + Jaccard: rec.Jaccard, |
| 371 | - CommonFeeds: commonFeeds, | 343 | + CommonFeeds: rec.CommonFeeds, |
| 372 | }) | 344 | }) |
| 373 | } | 345 | } |
| 374 | 346 | ||
modified
internal/cluster/cron.go +8 -2 | @@ -32,8 +32,14 @@ func (c *Cron) Run(ctx context.Context) error { | ||
| 32 | 32 | if err := c.engine.ComputeUserSimilarity(ctx); err != nil { |
| 33 | 33 | c.engine.logger.Error("user similarity failed", "error", err) |
| 34 | 34 | } |
| 35 | - if err := c.engine.ComputeRecommendations(ctx); err != nil { | |
| 36 | - c.engine.logger.Error("recommendations failed", "error", err) | |
| 35 | + if err := c.engine.ComputeFollowDistances(ctx); err != nil { | |
| 36 | + c.engine.logger.Error("follow distances failed", "error", err) | |
| 37 | + } | |
| 38 | + if err := c.engine.ComputeSignalProfiles(ctx); err != nil { | |
| 39 | + c.engine.logger.Error("signal profiles failed", "error", err) | |
| 40 | + } | |
| 41 | + if err := c.engine.AutoDismissStale(ctx, 15, 30); err != nil { | |
| 42 | + c.engine.logger.Error("auto dismiss failed", "error", err) | |
| 37 | 43 | } |
| 38 | 44 | c.engine.mu.Unlock() |
| 39 | 45 | } |
| @@ -32,8 +32,14 @@ func (c *Cron) Run(ctx context.Context) error { | |||
| 32 | if err := c.engine.ComputeUserSimilarity(ctx); err != nil { | 32 | if err := c.engine.ComputeUserSimilarity(ctx); err != nil { |
| 33 | c.engine.logger.Error("user similarity failed", "error", err) | 33 | c.engine.logger.Error("user similarity failed", "error", err) |
| 34 | } | 34 | } |
| 35 | - if err := c.engine.ComputeRecommendations(ctx); err != nil { | 35 | + if err := c.engine.ComputeFollowDistances(ctx); err != nil { |
| 36 | - c.engine.logger.Error("recommendations failed", "error", err) | 36 | + c.engine.logger.Error("follow distances failed", "error", err) |
| 37 | + } | ||
| 38 | + if err := c.engine.ComputeSignalProfiles(ctx); err != nil { | ||
| 39 | + c.engine.logger.Error("signal profiles failed", "error", err) | ||
| 40 | + } | ||
| 41 | + if err := c.engine.AutoDismissStale(ctx, 15, 30); err != nil { | ||
| 42 | + c.engine.logger.Error("auto dismiss failed", "error", err) | ||
| 37 | } | 43 | } |
| 38 | c.engine.mu.Unlock() | 44 | c.engine.mu.Unlock() |
| 39 | } | 45 | } |
added
internal/cluster/dismiss.go +82 -0 | new file mode 100644 | ||
| @@ -0,0 +1,82 @@ | ||
| 1 | +package cluster | |
| 2 | + | |
| 3 | +import ( | |
| 4 | + "context" | |
| 5 | + "time" | |
| 6 | +) | |
| 7 | + | |
| 8 | +type Impression struct { | |
| 9 | + TargetType string | |
| 10 | + TargetID string | |
| 11 | +} | |
| 12 | + | |
| 13 | +func (e *Engine) DismissFeed(ctx context.Context, userDID, feedURL, reason string) error { | |
| 14 | + _, err := e.db.ExecContext(ctx, ` | |
| 15 | + INSERT INTO dismissed_recommendations (user_did, target_type, target_id, reason) | |
| 16 | + VALUES (?, 'feed', ?, ?) | |
| 17 | + ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP | |
| 18 | + `, userDID, feedURL, reason) | |
| 19 | + return err | |
| 20 | +} | |
| 21 | + | |
| 22 | +func (e *Engine) DismissArticle(ctx context.Context, userDID, articleURL, reason string) error { | |
| 23 | + _, err := e.db.ExecContext(ctx, ` | |
| 24 | + INSERT INTO dismissed_recommendations (user_did, target_type, target_id, reason) | |
| 25 | + VALUES (?, 'article', ?, ?) | |
| 26 | + ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP | |
| 27 | + `, userDID, articleURL, reason) | |
| 28 | + return err | |
| 29 | +} | |
| 30 | + | |
| 31 | +func (e *Engine) RecordImpressions(ctx context.Context, userDID string, impressions []Impression) error { | |
| 32 | + tx, err := e.db.BeginTx(ctx, nil) | |
| 33 | + if err != nil { | |
| 34 | + return err | |
| 35 | + } | |
| 36 | + defer func() { _ = tx.Rollback() }() | |
| 37 | + | |
| 38 | + for _, imp := range impressions { | |
| 39 | + _, err := tx.ExecContext(ctx, ` | |
| 40 | + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count) | |
| 41 | + VALUES (?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, 1) | |
| 42 | + ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET | |
| 43 | + last_shown_at = CURRENT_TIMESTAMP, | |
| 44 | + shown_count = shown_count + 1 | |
| 45 | + `, userDID, imp.TargetType, imp.TargetID) | |
| 46 | + if err != nil { | |
| 47 | + return err | |
| 48 | + } | |
| 49 | + } | |
| 50 | + return tx.Commit() | |
| 51 | +} | |
| 52 | + | |
| 53 | +func (e *Engine) MarkImpressionActed(ctx context.Context, userDID, targetType, targetID string) error { | |
| 54 | + _, err := e.db.ExecContext(ctx, ` | |
| 55 | + UPDATE recommendation_impressions SET acted = 1 | |
| 56 | + WHERE user_did = ? AND target_type = ? AND target_id = ? | |
| 57 | + `, userDID, targetType, targetID) | |
| 58 | + return err | |
| 59 | +} | |
| 60 | + | |
| 61 | +func (e *Engine) AutoDismissStale(ctx context.Context, minShownCount int, maxAgeDays int) error { | |
| 62 | + cutoff := time.Now().AddDate(0, 0, -maxAgeDays).Format(time.RFC3339) | |
| 63 | + | |
| 64 | + _, err := e.db.ExecContext(ctx, ` | |
| 65 | + INSERT OR IGNORE INTO dismissed_recommendations (user_did, target_type, target_id, reason, dismissed_at) | |
| 66 | + SELECT user_did, target_type, target_id, 'auto_stale', CURRENT_TIMESTAMP | |
| 67 | + FROM recommendation_impressions | |
| 68 | + WHERE acted = 0 | |
| 69 | + AND shown_count >= ? | |
| 70 | + AND first_shown_at < ? | |
| 71 | + `, minShownCount, cutoff) | |
| 72 | + return err | |
| 73 | +} | |
| 74 | + | |
| 75 | +func (e *Engine) IsFeedDismissed(ctx context.Context, userDID, feedURL string) (bool, error) { | |
| 76 | + var count int | |
| 77 | + err := e.db.QueryRowContext(ctx, ` | |
| 78 | + SELECT COUNT(1) FROM dismissed_recommendations | |
| 79 | + WHERE user_did = ? AND target_type = 'feed' AND target_id = ? | |
| 80 | + `, userDID, feedURL).Scan(&count) | |
| 81 | + return count > 0, err | |
| 82 | +} | |
| new file mode 100644 | |||
| @@ -0,0 +1,82 @@ | |||
| 1 | +package cluster | ||
| 2 | + | ||
| 3 | +import ( | ||
| 4 | + "context" | ||
| 5 | + "time" | ||
| 6 | +) | ||
| 7 | + | ||
| 8 | +type Impression struct { | ||
| 9 | + TargetType string | ||
| 10 | + TargetID string | ||
| 11 | +} | ||
| 12 | + | ||
| 13 | +func (e *Engine) DismissFeed(ctx context.Context, userDID, feedURL, reason string) error { | ||
| 14 | + _, err := e.db.ExecContext(ctx, ` | ||
| 15 | + INSERT INTO dismissed_recommendations (user_did, target_type, target_id, reason) | ||
| 16 | + VALUES (?, 'feed', ?, ?) | ||
| 17 | + ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP | ||
| 18 | + `, userDID, feedURL, reason) | ||
| 19 | + return err | ||
| 20 | +} | ||
| 21 | + | ||
| 22 | +func (e *Engine) DismissArticle(ctx context.Context, userDID, articleURL, reason string) error { | ||
| 23 | + _, err := e.db.ExecContext(ctx, ` | ||
| 24 | + INSERT INTO dismissed_recommendations (user_did, target_type, target_id, reason) | ||
| 25 | + VALUES (?, 'article', ?, ?) | ||
| 26 | + ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP | ||
| 27 | + `, userDID, articleURL, reason) | ||
| 28 | + return err | ||
| 29 | +} | ||
| 30 | + | ||
| 31 | +func (e *Engine) RecordImpressions(ctx context.Context, userDID string, impressions []Impression) error { | ||
| 32 | + tx, err := e.db.BeginTx(ctx, nil) | ||
| 33 | + if err != nil { | ||
| 34 | + return err | ||
| 35 | + } | ||
| 36 | + defer func() { _ = tx.Rollback() }() | ||
| 37 | + | ||
| 38 | + for _, imp := range impressions { | ||
| 39 | + _, err := tx.ExecContext(ctx, ` | ||
| 40 | + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count) | ||
| 41 | + VALUES (?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, 1) | ||
| 42 | + ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET | ||
| 43 | + last_shown_at = CURRENT_TIMESTAMP, | ||
| 44 | + shown_count = shown_count + 1 | ||
| 45 | + `, userDID, imp.TargetType, imp.TargetID) | ||
| 46 | + if err != nil { | ||
| 47 | + return err | ||
| 48 | + } | ||
| 49 | + } | ||
| 50 | + return tx.Commit() | ||
| 51 | +} | ||
| 52 | + | ||
| 53 | +func (e *Engine) MarkImpressionActed(ctx context.Context, userDID, targetType, targetID string) error { | ||
| 54 | + _, err := e.db.ExecContext(ctx, ` | ||
| 55 | + UPDATE recommendation_impressions SET acted = 1 | ||
| 56 | + WHERE user_did = ? AND target_type = ? AND target_id = ? | ||
| 57 | + `, userDID, targetType, targetID) | ||
| 58 | + return err | ||
| 59 | +} | ||
| 60 | + | ||
| 61 | +func (e *Engine) AutoDismissStale(ctx context.Context, minShownCount int, maxAgeDays int) error { | ||
| 62 | + cutoff := time.Now().AddDate(0, 0, -maxAgeDays).Format(time.RFC3339) | ||
| 63 | + | ||
| 64 | + _, err := e.db.ExecContext(ctx, ` | ||
| 65 | + INSERT OR IGNORE INTO dismissed_recommendations (user_did, target_type, target_id, reason, dismissed_at) | ||
| 66 | + SELECT user_did, target_type, target_id, 'auto_stale', CURRENT_TIMESTAMP | ||
| 67 | + FROM recommendation_impressions | ||
| 68 | + WHERE acted = 0 | ||
| 69 | + AND shown_count >= ? | ||
| 70 | + AND first_shown_at < ? | ||
| 71 | + `, minShownCount, cutoff) | ||
| 72 | + return err | ||
| 73 | +} | ||
| 74 | + | ||
| 75 | +func (e *Engine) IsFeedDismissed(ctx context.Context, userDID, feedURL string) (bool, error) { | ||
| 76 | + var count int | ||
| 77 | + err := e.db.QueryRowContext(ctx, ` | ||
| 78 | + SELECT COUNT(1) FROM dismissed_recommendations | ||
| 79 | + WHERE user_did = ? AND target_type = 'feed' AND target_id = ? | ||
| 80 | + `, userDID, feedURL).Scan(&count) | ||
| 81 | + return count > 0, err | ||
| 82 | +} | ||
added
internal/cluster/diversity.go +68 -0 | new file mode 100644 | ||
| @@ -0,0 +1,68 @@ | ||
| 1 | +package cluster | |
| 2 | + | |
| 3 | +import ( | |
| 4 | + "net/url" | |
| 5 | + "strings" | |
| 6 | +) | |
| 7 | + | |
| 8 | +const maxPerDomain = 2 | |
| 9 | +const maxPerCategory = 3 | |
| 10 | + | |
| 11 | +func ApplyDiversity(candidates []*FeedRecommendation, topN int) []*FeedRecommendation { | |
| 12 | + domainCount := make(map[string]int, len(candidates)) | |
| 13 | + categoryCount := make(map[string]int) | |
| 14 | + result := make([]*FeedRecommendation, 0, topN) | |
| 15 | + | |
| 16 | + for _, c := range candidates { | |
| 17 | + if len(result) >= topN { | |
| 18 | + break | |
| 19 | + } | |
| 20 | + | |
| 21 | + domain := extractDomain(c.SiteURL) | |
| 22 | + if domain != "" && domainCount[domain] >= maxPerDomain { | |
| 23 | + continue | |
| 24 | + } | |
| 25 | + | |
| 26 | + cat := extractCategory(c.Description) | |
| 27 | + if cat != "" && categoryCount[cat] >= maxPerCategory { | |
| 28 | + continue | |
| 29 | + } | |
| 30 | + | |
| 31 | + if domain != "" { | |
| 32 | + domainCount[domain]++ | |
| 33 | + } | |
| 34 | + if cat != "" { | |
| 35 | + categoryCount[cat]++ | |
| 36 | + } | |
| 37 | + result = append(result, c) | |
| 38 | + } | |
| 39 | + | |
| 40 | + return result | |
| 41 | +} | |
| 42 | + | |
| 43 | +func extractDomain(siteURL string) string { | |
| 44 | + if siteURL == "" { | |
| 45 | + return "" | |
| 46 | + } | |
| 47 | + u, err := url.Parse(siteURL) | |
| 48 | + if err != nil { | |
| 49 | + return "" | |
| 50 | + } | |
| 51 | + host := u.Hostname() | |
| 52 | + parts := strings.Split(host, ".") | |
| 53 | + if len(parts) > 2 { | |
| 54 | + return strings.Join(parts[len(parts)-2:], ".") | |
| 55 | + } | |
| 56 | + return host | |
| 57 | +} | |
| 58 | + | |
| 59 | +func extractCategory(description string) string { | |
| 60 | + if description == "" { | |
| 61 | + return "" | |
| 62 | + } | |
| 63 | + words := strings.Fields(strings.ToLower(description)) | |
| 64 | + if len(words) == 0 { | |
| 65 | + return "" | |
| 66 | + } | |
| 67 | + return words[0] | |
| 68 | +} | |
| new file mode 100644 | |||
| @@ -0,0 +1,68 @@ | |||
| 1 | +package cluster | ||
| 2 | + | ||
| 3 | +import ( | ||
| 4 | + "net/url" | ||
| 5 | + "strings" | ||
| 6 | +) | ||
| 7 | + | ||
| 8 | +const maxPerDomain = 2 | ||
| 9 | +const maxPerCategory = 3 | ||
| 10 | + | ||
| 11 | +func ApplyDiversity(candidates []*FeedRecommendation, topN int) []*FeedRecommendation { | ||
| 12 | + domainCount := make(map[string]int, len(candidates)) | ||
| 13 | + categoryCount := make(map[string]int) | ||
| 14 | + result := make([]*FeedRecommendation, 0, topN) | ||
| 15 | + | ||
| 16 | + for _, c := range candidates { | ||
| 17 | + if len(result) >= topN { | ||
| 18 | + break | ||
| 19 | + } | ||
| 20 | + | ||
| 21 | + domain := extractDomain(c.SiteURL) | ||
| 22 | + if domain != "" && domainCount[domain] >= maxPerDomain { | ||
| 23 | + continue | ||
| 24 | + } | ||
| 25 | + | ||
| 26 | + cat := extractCategory(c.Description) | ||
| 27 | + if cat != "" && categoryCount[cat] >= maxPerCategory { | ||
| 28 | + continue | ||
| 29 | + } | ||
| 30 | + | ||
| 31 | + if domain != "" { | ||
| 32 | + domainCount[domain]++ | ||
| 33 | + } | ||
| 34 | + if cat != "" { | ||
| 35 | + categoryCount[cat]++ | ||
| 36 | + } | ||
| 37 | + result = append(result, c) | ||
| 38 | + } | ||
| 39 | + | ||
| 40 | + return result | ||
| 41 | +} | ||
| 42 | + | ||
| 43 | +func extractDomain(siteURL string) string { | ||
| 44 | + if siteURL == "" { | ||
| 45 | + return "" | ||
| 46 | + } | ||
| 47 | + u, err := url.Parse(siteURL) | ||
| 48 | + if err != nil { | ||
| 49 | + return "" | ||
| 50 | + } | ||
| 51 | + host := u.Hostname() | ||
| 52 | + parts := strings.Split(host, ".") | ||
| 53 | + if len(parts) > 2 { | ||
| 54 | + return strings.Join(parts[len(parts)-2:], ".") | ||
| 55 | + } | ||
| 56 | + return host | ||
| 57 | +} | ||
| 58 | + | ||
| 59 | +func extractCategory(description string) string { | ||
| 60 | + if description == "" { | ||
| 61 | + return "" | ||
| 62 | + } | ||
| 63 | + words := strings.Fields(strings.ToLower(description)) | ||
| 64 | + if len(words) == 0 { | ||
| 65 | + return "" | ||
| 66 | + } | ||
| 67 | + return words[0] | ||
| 68 | +} | ||
modified
internal/cluster/jaccard.go +14 -99 | @@ -9,20 +9,18 @@ import ( | ||
| 9 | 9 | ) |
| 10 | 10 | |
| 11 | 11 | type Config struct { |
| 12 | - SimilarityThreshold float64 | |
| 13 | - FollowBoost float64 | |
| 14 | - LikesWeight float64 | |
| 15 | - TagsWeight float64 | |
| 16 | - DescriptionWeight float64 | |
| 12 | + FollowBoost float64 | |
| 13 | + LikesWeight float64 | |
| 14 | + TagsWeight float64 | |
| 15 | + DescriptionWeight float64 | |
| 17 | 16 | } |
| 18 | 17 | |
| 19 | 18 | func DefaultConfig() Config { |
| 20 | 19 | return Config{ |
| 21 | - SimilarityThreshold: 0.2, | |
| 22 | - FollowBoost: 0.5, | |
| 23 | - LikesWeight: 0.3, | |
| 24 | - TagsWeight: 0.2, | |
| 25 | - DescriptionWeight: 0.15, | |
| 20 | + FollowBoost: 0.5, | |
| 21 | + LikesWeight: 0.3, | |
| 22 | + TagsWeight: 0.2, | |
| 23 | + DescriptionWeight: 0.15, | |
| 26 | 24 | } |
| 27 | 25 | } |
| 28 | 26 | |
| @@ -37,48 +35,6 @@ func NewEngine(db *sql.DB, logger *slog.Logger) *Engine { | ||
| 37 | 35 | return &Engine{db: db, logger: logger, config: DefaultConfig()} |
| 38 | 36 | } |
| 39 | 37 | |
| 40 | -func (e *Engine) ComputeArticleRecommendations(ctx context.Context) error { | |
| 41 | - tx, err := e.db.BeginTx(ctx, nil) | |
| 42 | - if err != nil { | |
| 43 | - return err | |
| 44 | - } | |
| 45 | - defer func() { _ = tx.Rollback() }() | |
| 46 | - | |
| 47 | - if _, err := tx.ExecContext(ctx, `DELETE FROM user_article_recommendations`); err != nil { | |
| 48 | - return err | |
| 49 | - } | |
| 50 | - | |
| 51 | - query := fmt.Sprintf(` | |
| 52 | - INSERT INTO user_article_recommendations (user_did, feed_url, article_url, score) | |
| 53 | - SELECT targets.target, l.feed_url, l.article_url, SUM(targets.jaccard) AS score | |
| 54 | - FROM ( | |
| 55 | - SELECT us.user_a AS target, us.user_b AS peer, us.jaccard | |
| 56 | - FROM user_similarity us | |
| 57 | - WHERE us.jaccard > %g | |
| 58 | - UNION ALL | |
| 59 | - SELECT us.user_b AS target, us.user_a AS peer, us.jaccard | |
| 60 | - FROM user_similarity us | |
| 61 | - WHERE us.jaccard > %g | |
| 62 | - ) targets | |
| 63 | - JOIN likes l ON l.author_did = targets.peer | |
| 64 | - WHERE NOT EXISTS ( | |
| 65 | - SELECT 1 FROM subscriptions sub WHERE sub.user_did = targets.target AND sub.feed_url = l.feed_url | |
| 66 | - ) | |
| 67 | - AND NOT EXISTS ( | |
| 68 | - SELECT 1 FROM likes ul WHERE ul.author_did = targets.target AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url | |
| 69 | - ) | |
| 70 | - GROUP BY targets.target, l.feed_url, l.article_url | |
| 71 | - ORDER BY score DESC | |
| 72 | - `, e.config.SimilarityThreshold, e.config.SimilarityThreshold) | |
| 73 | - | |
| 74 | - if _, err := tx.ExecContext(ctx, query); err != nil { | |
| 75 | - return err | |
| 76 | - } | |
| 77 | - | |
| 78 | - e.logger.Info("article recommendations computed") | |
| 79 | - return tx.Commit() | |
| 80 | -} | |
| 81 | - | |
| 82 | 38 | func (e *Engine) ComputeFeedSimilarity(ctx context.Context) error { |
| 83 | 39 | tx, err := e.db.BeginTx(ctx, nil) |
| 84 | 40 | if err != nil { |
| @@ -260,10 +216,15 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error { | ||
| 260 | 216 | } |
| 261 | 217 | if _, err := tx.ExecContext(ctx, ` |
| 262 | 218 | INSERT INTO _likes_overlap (user_a, user_b, common) |
| 263 | - SELECT l1.author_did, l2.author_did, COUNT(*) | |
| 219 | + SELECT l1.author_did, l2.author_did, | |
| 220 | + CAST(SUM( | |
| 221 | + EXP(-0.023 * CAST(julianday('now') - julianday(l1.created_at) AS REAL)) | |
| 222 | + * EXP(-0.023 * CAST(julianday('now') - julianday(l2.created_at) AS REAL)) | |
| 223 | + ) AS INTEGER) | |
| 264 | 224 | FROM likes l1 |
| 265 | 225 | JOIN likes l2 ON l1.feed_url = l2.feed_url AND l1.article_url = l2.article_url |
| 266 | 226 | AND l1.author_did < l2.author_did |
| 227 | + WHERE l1.created_at IS NOT NULL AND l2.created_at IS NOT NULL | |
| 267 | 228 | GROUP BY l1.author_did, l2.author_did |
| 268 | 229 | `); err != nil { |
| 269 | 230 | return err |
| @@ -400,49 +361,3 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error { | ||
| 400 | 361 | e.logger.Info("user similarity computed") |
| 401 | 362 | return tx.Commit() |
| 402 | 363 | } |
| 403 | - | |
| 404 | -func (e *Engine) ComputeRecommendations(ctx context.Context) error { | |
| 405 | - tx, err := e.db.BeginTx(ctx, nil) | |
| 406 | - if err != nil { | |
| 407 | - return err | |
| 408 | - } | |
| 409 | - defer func() { _ = tx.Rollback() }() | |
| 410 | - | |
| 411 | - if _, err := tx.ExecContext(ctx, `DELETE FROM user_feed_recommendations`); err != nil { | |
| 412 | - return err | |
| 413 | - } | |
| 414 | - | |
| 415 | - recQuery := fmt.Sprintf(` | |
| 416 | - INSERT INTO user_feed_recommendations (user_did, feed_url, score) | |
| 417 | - SELECT target, feed_url, SUM(jaccard) AS score | |
| 418 | - FROM ( | |
| 419 | - SELECT us.user_a AS target, s.feed_url, us.jaccard | |
| 420 | - FROM user_similarity us | |
| 421 | - JOIN subscriptions s ON s.user_did = us.user_b | |
| 422 | - WHERE us.jaccard > %g | |
| 423 | - AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = us.user_a) | |
| 424 | - | |
| 425 | - UNION ALL | |
| 426 | - | |
| 427 | - SELECT us.user_b AS target, s.feed_url, us.jaccard | |
| 428 | - FROM user_similarity us | |
| 429 | - JOIN subscriptions s ON s.user_did = us.user_a | |
| 430 | - WHERE us.jaccard > %g | |
| 431 | - AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = us.user_b) | |
| 432 | - ) | |
| 433 | - GROUP BY target, feed_url | |
| 434 | - ORDER BY score DESC | |
| 435 | - `, e.config.SimilarityThreshold, e.config.SimilarityThreshold) | |
| 436 | - | |
| 437 | - if _, err := tx.ExecContext(ctx, recQuery); err != nil { | |
| 438 | - return err | |
| 439 | - } | |
| 440 | - | |
| 441 | - e.logger.Info("feed recommendations computed") | |
| 442 | - | |
| 443 | - if err := tx.Commit(); err != nil { | |
| 444 | - return err | |
| 445 | - } | |
| 446 | - | |
| 447 | - return e.ComputeArticleRecommendations(ctx) | |
| 448 | -} | |
| @@ -9,20 +9,18 @@ import ( | |||
| 9 | ) | 9 | ) |
| 10 | 10 | ||
| 11 | type Config struct { | 11 | type Config struct { |
| 12 | - SimilarityThreshold float64 | 12 | + FollowBoost float64 |
| 13 | - FollowBoost float64 | 13 | + LikesWeight float64 |
| 14 | - LikesWeight float64 | 14 | + TagsWeight float64 |
| 15 | - TagsWeight float64 | 15 | + DescriptionWeight float64 |
| 16 | - DescriptionWeight float64 | ||
| 17 | } | 16 | } |
| 18 | 17 | ||
| 19 | func DefaultConfig() Config { | 18 | func DefaultConfig() Config { |
| 20 | return Config{ | 19 | return Config{ |
| 21 | - SimilarityThreshold: 0.2, | 20 | + FollowBoost: 0.5, |
| 22 | - FollowBoost: 0.5, | 21 | + LikesWeight: 0.3, |
| 23 | - LikesWeight: 0.3, | 22 | + TagsWeight: 0.2, |
| 24 | - TagsWeight: 0.2, | 23 | + DescriptionWeight: 0.15, |
| 25 | - DescriptionWeight: 0.15, | ||
| 26 | } | 24 | } |
| 27 | } | 25 | } |
| 28 | 26 | ||
| @@ -37,48 +35,6 @@ func NewEngine(db *sql.DB, logger *slog.Logger) *Engine { | |||
| 37 | return &Engine{db: db, logger: logger, config: DefaultConfig()} | 35 | return &Engine{db: db, logger: logger, config: DefaultConfig()} |
| 38 | } | 36 | } |
| 39 | 37 | ||
| 40 | -func (e *Engine) ComputeArticleRecommendations(ctx context.Context) error { | ||
| 41 | - tx, err := e.db.BeginTx(ctx, nil) | ||
| 42 | - if err != nil { | ||
| 43 | - return err | ||
| 44 | - } | ||
| 45 | - defer func() { _ = tx.Rollback() }() | ||
| 46 | - | ||
| 47 | - if _, err := tx.ExecContext(ctx, `DELETE FROM user_article_recommendations`); err != nil { | ||
| 48 | - return err | ||
| 49 | - } | ||
| 50 | - | ||
| 51 | - query := fmt.Sprintf(` | ||
| 52 | - INSERT INTO user_article_recommendations (user_did, feed_url, article_url, score) | ||
| 53 | - SELECT targets.target, l.feed_url, l.article_url, SUM(targets.jaccard) AS score | ||
| 54 | - FROM ( | ||
| 55 | - SELECT us.user_a AS target, us.user_b AS peer, us.jaccard | ||
| 56 | - FROM user_similarity us | ||
| 57 | - WHERE us.jaccard > %g | ||
| 58 | - UNION ALL | ||
| 59 | - SELECT us.user_b AS target, us.user_a AS peer, us.jaccard | ||
| 60 | - FROM user_similarity us | ||
| 61 | - WHERE us.jaccard > %g | ||
| 62 | - ) targets | ||
| 63 | - JOIN likes l ON l.author_did = targets.peer | ||
| 64 | - WHERE NOT EXISTS ( | ||
| 65 | - SELECT 1 FROM subscriptions sub WHERE sub.user_did = targets.target AND sub.feed_url = l.feed_url | ||
| 66 | - ) | ||
| 67 | - AND NOT EXISTS ( | ||
| 68 | - SELECT 1 FROM likes ul WHERE ul.author_did = targets.target AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url | ||
| 69 | - ) | ||
| 70 | - GROUP BY targets.target, l.feed_url, l.article_url | ||
| 71 | - ORDER BY score DESC | ||
| 72 | - `, e.config.SimilarityThreshold, e.config.SimilarityThreshold) | ||
| 73 | - | ||
| 74 | - if _, err := tx.ExecContext(ctx, query); err != nil { | ||
| 75 | - return err | ||
| 76 | - } | ||
| 77 | - | ||
| 78 | - e.logger.Info("article recommendations computed") | ||
| 79 | - return tx.Commit() | ||
| 80 | -} | ||
| 81 | - | ||
| 82 | func (e *Engine) ComputeFeedSimilarity(ctx context.Context) error { | 38 | func (e *Engine) ComputeFeedSimilarity(ctx context.Context) error { |
| 83 | tx, err := e.db.BeginTx(ctx, nil) | 39 | tx, err := e.db.BeginTx(ctx, nil) |
| 84 | if err != nil { | 40 | if err != nil { |
| @@ -260,10 +216,15 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error { | |||
| 260 | } | 216 | } |
| 261 | if _, err := tx.ExecContext(ctx, ` | 217 | if _, err := tx.ExecContext(ctx, ` |
| 262 | INSERT INTO _likes_overlap (user_a, user_b, common) | 218 | INSERT INTO _likes_overlap (user_a, user_b, common) |
| 263 | - SELECT l1.author_did, l2.author_did, COUNT(*) | 219 | + SELECT l1.author_did, l2.author_did, |
| 220 | + CAST(SUM( | ||
| 221 | + EXP(-0.023 * CAST(julianday('now') - julianday(l1.created_at) AS REAL)) | ||
| 222 | + * EXP(-0.023 * CAST(julianday('now') - julianday(l2.created_at) AS REAL)) | ||
| 223 | + ) AS INTEGER) | ||
| 264 | FROM likes l1 | 224 | FROM likes l1 |
| 265 | JOIN likes l2 ON l1.feed_url = l2.feed_url AND l1.article_url = l2.article_url | 225 | JOIN likes l2 ON l1.feed_url = l2.feed_url AND l1.article_url = l2.article_url |
| 266 | AND l1.author_did < l2.author_did | 226 | AND l1.author_did < l2.author_did |
| 227 | + WHERE l1.created_at IS NOT NULL AND l2.created_at IS NOT NULL | ||
| 267 | GROUP BY l1.author_did, l2.author_did | 228 | GROUP BY l1.author_did, l2.author_did |
| 268 | `); err != nil { | 229 | `); err != nil { |
| 269 | return err | 230 | return err |
| @@ -400,49 +361,3 @@ func (e *Engine) ComputeUserSimilarity(ctx context.Context) error { | |||
| 400 | e.logger.Info("user similarity computed") | 361 | e.logger.Info("user similarity computed") |
| 401 | return tx.Commit() | 362 | return tx.Commit() |
| 402 | } | 363 | } |
| 403 | - | ||
| 404 | -func (e *Engine) ComputeRecommendations(ctx context.Context) error { | ||
| 405 | - tx, err := e.db.BeginTx(ctx, nil) | ||
| 406 | - if err != nil { | ||
| 407 | - return err | ||
| 408 | - } | ||
| 409 | - defer func() { _ = tx.Rollback() }() | ||
| 410 | - | ||
| 411 | - if _, err := tx.ExecContext(ctx, `DELETE FROM user_feed_recommendations`); err != nil { | ||
| 412 | - return err | ||
| 413 | - } | ||
| 414 | - | ||
| 415 | - recQuery := fmt.Sprintf(` | ||
| 416 | - INSERT INTO user_feed_recommendations (user_did, feed_url, score) | ||
| 417 | - SELECT target, feed_url, SUM(jaccard) AS score | ||
| 418 | - FROM ( | ||
| 419 | - SELECT us.user_a AS target, s.feed_url, us.jaccard | ||
| 420 | - FROM user_similarity us | ||
| 421 | - JOIN subscriptions s ON s.user_did = us.user_b | ||
| 422 | - WHERE us.jaccard > %g | ||
| 423 | - AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = us.user_a) | ||
| 424 | - | ||
| 425 | - UNION ALL | ||
| 426 | - | ||
| 427 | - SELECT us.user_b AS target, s.feed_url, us.jaccard | ||
| 428 | - FROM user_similarity us | ||
| 429 | - JOIN subscriptions s ON s.user_did = us.user_a | ||
| 430 | - WHERE us.jaccard > %g | ||
| 431 | - AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = us.user_b) | ||
| 432 | - ) | ||
| 433 | - GROUP BY target, feed_url | ||
| 434 | - ORDER BY score DESC | ||
| 435 | - `, e.config.SimilarityThreshold, e.config.SimilarityThreshold) | ||
| 436 | - | ||
| 437 | - if _, err := tx.ExecContext(ctx, recQuery); err != nil { | ||
| 438 | - return err | ||
| 439 | - } | ||
| 440 | - | ||
| 441 | - e.logger.Info("feed recommendations computed") | ||
| 442 | - | ||
| 443 | - if err := tx.Commit(); err != nil { | ||
| 444 | - return err | ||
| 445 | - } | ||
| 446 | - | ||
| 447 | - return e.ComputeArticleRecommendations(ctx) | ||
| 448 | -} | ||
modified
internal/cluster/jaccard_test.go +323 -36 | @@ -2,6 +2,7 @@ package cluster | ||
| 2 | 2 | |
| 3 | 3 | import ( |
| 4 | 4 | "context" |
| 5 | + "fmt" | |
| 5 | 6 | "os" |
| 6 | 7 | "testing" |
| 7 | 8 | |
| @@ -64,6 +65,18 @@ func seedClusterData(t *testing.T, ctx context.Context, database *db.DB) { | ||
| 64 | 65 | } |
| 65 | 66 | } |
| 66 | 67 | |
| 68 | +func seedFollowData(t *testing.T, ctx context.Context, database *db.DB) { | |
| 69 | + t.Helper() | |
| 70 | + follows := []struct{ user, target string }{ | |
| 71 | + {"did:test:alice", "did:test:bob"}, | |
| 72 | + {"did:test:bob", "did:test:carol"}, | |
| 73 | + } | |
| 74 | + for _, f := range follows { | |
| 75 | + _, err := database.ExecContext(ctx, `INSERT OR IGNORE INTO follows (user_did, target_did) VALUES (?, ?)`, f.user, f.target) | |
| 76 | + assert.NilError(t, err) | |
| 77 | + } | |
| 78 | +} | |
| 79 | + | |
| 67 | 80 | func TestComputeFeedSimilarity(t *testing.T) { |
| 68 | 81 | ctx := context.Background() |
| 69 | 82 | database := setupClusterTestDB(t) |
| @@ -94,7 +107,7 @@ func TestComputeUserSimilarity(t *testing.T) { | ||
| 94 | 107 | assert.Assert(t, count > 0, "expected user similarity pairs") |
| 95 | 108 | } |
| 96 | 109 | |
| 97 | -func TestComputeRecommendations_GeneratesFeedRecsForNewUser(t *testing.T) { | |
| 110 | +func TestOnDemandFeedRecommendations(t *testing.T) { | |
| 98 | 111 | ctx := context.Background() |
| 99 | 112 | database := setupClusterTestDB(t) |
| 100 | 113 | seedClusterData(t, ctx, database) |
| @@ -102,7 +115,6 @@ func TestComputeRecommendations_GeneratesFeedRecsForNewUser(t *testing.T) { | ||
| 102 | 115 | engine := NewEngine(database.DB, slog.Default()) |
| 103 | 116 | assert.NilError(t, engine.ComputeFeedSimilarity(ctx)) |
| 104 | 117 | assert.NilError(t, engine.ComputeUserSimilarity(ctx)) |
| 105 | - assert.NilError(t, engine.ComputeRecommendations(ctx)) | |
| 106 | 118 | |
| 107 | 119 | recs, err := engine.GetFeedRecommendations(ctx, "did:test:carol", 10) |
| 108 | 120 | assert.NilError(t, err) |
| @@ -117,7 +129,7 @@ func TestComputeRecommendations_GeneratesFeedRecsForNewUser(t *testing.T) { | ||
| 117 | 129 | assert.Assert(t, found, "carol should be recommended feeds she doesn't subscribe to") |
| 118 | 130 | } |
| 119 | 131 | |
| 120 | -func TestComputeRecommendations_NoSelfRecommendations(t *testing.T) { | |
| 132 | +func TestNoSelfRecommendations(t *testing.T) { | |
| 121 | 133 | ctx := context.Background() |
| 122 | 134 | database := setupClusterTestDB(t) |
| 123 | 135 | seedClusterData(t, ctx, database) |
| @@ -125,7 +137,6 @@ func TestComputeRecommendations_NoSelfRecommendations(t *testing.T) { | ||
| 125 | 137 | engine := NewEngine(database.DB, slog.Default()) |
| 126 | 138 | assert.NilError(t, engine.ComputeFeedSimilarity(ctx)) |
| 127 | 139 | assert.NilError(t, engine.ComputeUserSimilarity(ctx)) |
| 128 | - assert.NilError(t, engine.ComputeRecommendations(ctx)) | |
| 129 | 140 | |
| 130 | 141 | recs, err := engine.GetFeedRecommendations(ctx, "did:test:alice", 10) |
| 131 | 142 | assert.NilError(t, err) |
| @@ -141,65 +152,341 @@ func TestComputeRecommendations_NoSelfRecommendations(t *testing.T) { | ||
| 141 | 152 | } |
| 142 | 153 | } |
| 143 | 154 | |
| 144 | -func TestLikesBasedSimilarity(t *testing.T) { | |
| 155 | +func TestDismissedFeedsExcluded(t *testing.T) { | |
| 145 | 156 | ctx := context.Background() |
| 146 | 157 | database := setupClusterTestDB(t) |
| 147 | 158 | seedClusterData(t, ctx, database) |
| 148 | 159 | |
| 149 | - _, err := database.ExecContext(ctx, `INSERT INTO articles (feed_url, guid, title, url) VALUES (?, ?, ?, ?)`, | |
| 150 | - "https://a.com/feed", "art1", "Article 1", "https://a.com/art1") | |
| 160 | + engine := NewEngine(database.DB, slog.Default()) | |
| 161 | + assert.NilError(t, engine.ComputeFeedSimilarity(ctx)) | |
| 162 | + assert.NilError(t, engine.ComputeUserSimilarity(ctx)) | |
| 163 | + | |
| 164 | + assert.NilError(t, engine.DismissFeed(ctx, "did:test:carol", "https://a.com/feed", "not_interested")) | |
| 165 | + | |
| 166 | + recs, err := engine.GetFeedRecommendations(ctx, "did:test:carol", 10) | |
| 151 | 167 | assert.NilError(t, err) |
| 152 | - _, err = database.ExecContext(ctx, `INSERT INTO articles (feed_url, guid, title, url) VALUES (?, ?, ?, ?)`, | |
| 153 | - "https://a.com/feed", "art2", "Article 2", "https://a.com/art2") | |
| 168 | + | |
| 169 | + for _, r := range recs { | |
| 170 | + assert.Assert(t, r.FeedURL != "https://a.com/feed", | |
| 171 | + "dismissed feed should not appear in recommendations") | |
| 172 | + } | |
| 173 | +} | |
| 174 | + | |
| 175 | +func TestIsFeedDismissed(t *testing.T) { | |
| 176 | + ctx := context.Background() | |
| 177 | + database := setupClusterTestDB(t) | |
| 178 | + seedClusterData(t, ctx, database) | |
| 179 | + | |
| 180 | + engine := NewEngine(database.DB, slog.Default()) | |
| 181 | + | |
| 182 | + dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://a.com/feed") | |
| 154 | 183 | assert.NilError(t, err) |
| 184 | + assert.Assert(t, !dismissed, "feed should not be dismissed initially") | |
| 185 | + | |
| 186 | + assert.NilError(t, engine.DismissFeed(ctx, "did:test:alice", "https://a.com/feed", "not_interested")) | |
| 155 | 187 | |
| 156 | - _, err = database.ExecContext(ctx, `INSERT INTO likes (uri, author_did, feed_url, article_url, created_at) VALUES (?, ?, ?, ?, datetime('now'))`, | |
| 157 | - "at://alice/like/1", "did:test:alice", "https://a.com/feed", "https://a.com/art1") | |
| 188 | + dismissed, err = engine.IsFeedDismissed(ctx, "did:test:alice", "https://a.com/feed") | |
| 158 | 189 | assert.NilError(t, err) |
| 159 | - _, err = database.ExecContext(ctx, `INSERT INTO likes (uri, author_did, feed_url, article_url, created_at) VALUES (?, ?, ?, ?, datetime('now'))`, | |
| 160 | - "at://alice/like/2", "did:test:alice", "https://a.com/feed", "https://a.com/art2") | |
| 190 | + assert.Assert(t, dismissed, "feed should be dismissed after dismiss call") | |
| 191 | +} | |
| 192 | + | |
| 193 | +func TestRecordImpressions(t *testing.T) { | |
| 194 | + ctx := context.Background() | |
| 195 | + database := setupClusterTestDB(t) | |
| 196 | + seedClusterData(t, ctx, database) | |
| 197 | + | |
| 198 | + engine := NewEngine(database.DB, slog.Default()) | |
| 199 | + | |
| 200 | + impressions := []Impression{ | |
| 201 | + {TargetType: "feed", TargetID: "https://a.com/feed"}, | |
| 202 | + {TargetType: "feed", TargetID: "https://b.com/feed"}, | |
| 203 | + } | |
| 204 | + assert.NilError(t, engine.RecordImpressions(ctx, "did:test:alice", impressions)) | |
| 205 | + | |
| 206 | + var count int | |
| 207 | + assert.NilError(t, database.QueryRowContext(ctx, | |
| 208 | + `SELECT COUNT(*) FROM recommendation_impressions WHERE user_did = 'did:test:alice'`).Scan(&count)) | |
| 209 | + assert.Equal(t, count, 2) | |
| 210 | + | |
| 211 | + assert.NilError(t, engine.RecordImpressions(ctx, "did:test:alice", impressions)) | |
| 212 | + | |
| 213 | + var shownCount int | |
| 214 | + assert.NilError(t, database.QueryRowContext(ctx, | |
| 215 | + `SELECT shown_count FROM recommendation_impressions WHERE user_did = 'did:test:alice' AND target_id = 'https://a.com/feed'`).Scan(&shownCount)) | |
| 216 | + assert.Equal(t, shownCount, 2, "shown_count should increment on repeated impression") | |
| 217 | +} | |
| 218 | + | |
| 219 | +func TestMarkImpressionActed(t *testing.T) { | |
| 220 | + ctx := context.Background() | |
| 221 | + database := setupClusterTestDB(t) | |
| 222 | + seedClusterData(t, ctx, database) | |
| 223 | + | |
| 224 | + engine := NewEngine(database.DB, slog.Default()) | |
| 225 | + | |
| 226 | + impressions := []Impression{{TargetType: "feed", TargetID: "https://a.com/feed"}} | |
| 227 | + assert.NilError(t, engine.RecordImpressions(ctx, "did:test:alice", impressions)) | |
| 228 | + | |
| 229 | + assert.NilError(t, engine.MarkImpressionActed(ctx, "did:test:alice", "feed", "https://a.com/feed")) | |
| 230 | + | |
| 231 | + var acted bool | |
| 232 | + assert.NilError(t, database.QueryRowContext(ctx, | |
| 233 | + `SELECT acted FROM recommendation_impressions WHERE user_did = 'did:test:alice' AND target_id = 'https://a.com/feed'`).Scan(&acted)) | |
| 234 | + assert.Assert(t, acted, "impression should be marked as acted") | |
| 235 | +} | |
| 236 | + | |
| 237 | +func TestComputeFollowDistances(t *testing.T) { | |
| 238 | + ctx := context.Background() | |
| 239 | + database := setupClusterTestDB(t) | |
| 240 | + seedClusterData(t, ctx, database) | |
| 241 | + seedFollowData(t, ctx, database) | |
| 242 | + | |
| 243 | + engine := NewEngine(database.DB, slog.Default()) | |
| 244 | + assert.NilError(t, engine.ComputeFollowDistances(ctx)) | |
| 245 | + | |
| 246 | + var d1, d2 int | |
| 247 | + assert.NilError(t, database.QueryRowContext(ctx, | |
| 248 | + `SELECT COUNT(*) FROM follow_distances WHERE distance = 1`).Scan(&d1)) | |
| 249 | + assert.NilError(t, database.QueryRowContext(ctx, | |
| 250 | + `SELECT COUNT(*) FROM follow_distances WHERE distance = 2`).Scan(&d2)) | |
| 251 | + assert.Assert(t, d1 >= 2, "expected at least 2 direct follow distances") | |
| 252 | + assert.Assert(t, d2 >= 1, "expected at least 1 two-hop distance (alice -> bob -> carol)") | |
| 253 | + | |
| 254 | + var dist int | |
| 255 | + assert.NilError(t, database.QueryRowContext(ctx, | |
| 256 | + `SELECT distance FROM follow_distances WHERE user_a = 'did:test:alice' AND user_b = 'did:test:carol'`).Scan(&dist)) | |
| 257 | + assert.Equal(t, dist, 2, "alice should be 2 hops from carol") | |
| 258 | +} | |
| 259 | + | |
| 260 | +func TestAutoDismissStale(t *testing.T) { | |
| 261 | + ctx := context.Background() | |
| 262 | + database := setupClusterTestDB(t) | |
| 263 | + seedClusterData(t, ctx, database) | |
| 264 | + | |
| 265 | + engine := NewEngine(database.DB, slog.Default()) | |
| 266 | + | |
| 267 | + _, err := database.ExecContext(ctx, ` | |
| 268 | + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted) | |
| 269 | + VALUES ('did:test:alice', 'feed', 'https://stale.com/feed', datetime('now', '-31 days'), datetime('now'), 20, 0) | |
| 270 | + `) | |
| 161 | 271 | assert.NilError(t, err) |
| 162 | - _, err = database.ExecContext(ctx, `INSERT INTO likes (uri, author_did, feed_url, article_url, created_at) VALUES (?, ?, ?, ?, datetime('now'))`, | |
| 163 | - "at://carol/like/1", "did:test:carol", "https://a.com/feed", "https://a.com/art1") | |
| 272 | + | |
| 273 | + assert.NilError(t, engine.AutoDismissStale(ctx, 15, 30)) | |
| 274 | + | |
| 275 | + dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://stale.com/feed") | |
| 164 | 276 | assert.NilError(t, err) |
| 277 | + assert.Assert(t, dismissed, "stale recommendation should be auto-dismissed") | |
| 278 | +} | |
| 279 | + | |
| 280 | +func TestAutoDismissStale_DoesNotDismissRecent(t *testing.T) { | |
| 281 | + ctx := context.Background() | |
| 282 | + database := setupClusterTestDB(t) | |
| 283 | + seedClusterData(t, ctx, database) | |
| 165 | 284 | |
| 166 | 285 | engine := NewEngine(database.DB, slog.Default()) |
| 167 | - assert.NilError(t, engine.ComputeUserSimilarity(ctx)) | |
| 168 | 286 | |
| 169 | - var jaccard float64 | |
| 170 | - var commonLikes int | |
| 171 | - assert.NilError(t, database.QueryRowContext(ctx, | |
| 172 | - `SELECT jaccard, common_likes FROM user_similarity WHERE user_a = ? AND user_b = ?`, | |
| 173 | - "did:test:alice", "did:test:carol").Scan(&jaccard, &commonLikes)) | |
| 174 | - assert.Equal(t, commonLikes, 1, "alice and carol share 1 liked article") | |
| 175 | - assert.Assert(t, jaccard > 0, "likes should contribute to similarity, got %f", jaccard) | |
| 287 | + _, err := database.ExecContext(ctx, ` | |
| 288 | + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted) | |
| 289 | + VALUES ('did:test:alice', 'feed', 'https://recent.com/feed', datetime('now'), datetime('now'), 5, 0) | |
| 290 | + `) | |
| 291 | + assert.NilError(t, err) | |
| 292 | + | |
| 293 | + assert.NilError(t, engine.AutoDismissStale(ctx, 15, 30)) | |
| 294 | + | |
| 295 | + dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://recent.com/feed") | |
| 296 | + assert.NilError(t, err) | |
| 297 | + assert.Assert(t, !dismissed, "recent impression should not be auto-dismissed") | |
| 298 | +} | |
| 299 | + | |
| 300 | +func TestAutoDismissStale_DoesNotDismissActed(t *testing.T) { | |
| 301 | + ctx := context.Background() | |
| 302 | + database := setupClusterTestDB(t) | |
| 303 | + seedClusterData(t, ctx, database) | |
| 304 | + | |
| 305 | + engine := NewEngine(database.DB, slog.Default()) | |
| 306 | + | |
| 307 | + _, err := database.ExecContext(ctx, ` | |
| 308 | + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted) | |
| 309 | + VALUES ('did:test:alice', 'feed', 'https://acted.com/feed', datetime('now', '-31 days'), datetime('now'), 20, 1) | |
| 310 | + `) | |
| 311 | + assert.NilError(t, err) | |
| 312 | + | |
| 313 | + assert.NilError(t, engine.AutoDismissStale(ctx, 15, 30)) | |
| 314 | + | |
| 315 | + dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://acted.com/feed") | |
| 316 | + assert.NilError(t, err) | |
| 317 | + assert.Assert(t, !dismissed, "acted recommendation should not be auto-dismissed") | |
| 318 | +} | |
| 319 | + | |
| 320 | +func TestDiversityFiltering(t *testing.T) { | |
| 321 | + candidates := []*FeedRecommendation{ | |
| 322 | + {FeedURL: "https://a.com/1", SiteURL: "https://a.com", Score: 1.0}, | |
| 323 | + {FeedURL: "https://a.com/2", SiteURL: "https://a.com", Score: 0.9}, | |
| 324 | + {FeedURL: "https://a.com/3", SiteURL: "https://a.com", Score: 0.8}, | |
| 325 | + {FeedURL: "https://b.com/1", SiteURL: "https://b.com", Score: 0.7}, | |
| 326 | + {FeedURL: "https://b.com/2", SiteURL: "https://b.com", Score: 0.6}, | |
| 327 | + {FeedURL: "https://c.com/1", SiteURL: "https://c.com", Score: 0.5}, | |
| 328 | + } | |
| 329 | + | |
| 330 | + result := ApplyDiversity(candidates, 6) | |
| 331 | + | |
| 332 | + aCount := 0 | |
| 333 | + bCount := 0 | |
| 334 | + cCount := 0 | |
| 335 | + for _, r := range result { | |
| 336 | + switch extractDomain(r.SiteURL) { | |
| 337 | + case "a.com": | |
| 338 | + aCount++ | |
| 339 | + case "b.com": | |
| 340 | + bCount++ | |
| 341 | + case "c.com": | |
| 342 | + cCount++ | |
| 343 | + } | |
| 344 | + } | |
| 345 | + assert.Assert(t, aCount <= maxPerDomain, "should limit feeds from same domain") | |
| 346 | + assert.Assert(t, len(result) <= 6, "should respect topN limit") | |
| 347 | + assert.Assert(t, cCount >= 1, "should include feeds from different domains") | |
| 348 | +} | |
| 349 | + | |
| 350 | +func TestDiversityFiltering_EmptySiteURL(t *testing.T) { | |
| 351 | + candidates := []*FeedRecommendation{ | |
| 352 | + {FeedURL: "https://a.com/1", SiteURL: "", Score: 1.0}, | |
| 353 | + {FeedURL: "https://b.com/1", SiteURL: "", Score: 0.9}, | |
| 354 | + } | |
| 355 | + result := ApplyDiversity(candidates, 5) | |
| 356 | + assert.Equal(t, len(result), 2, "feeds without site_url should not be filtered out") | |
| 357 | +} | |
| 358 | + | |
| 359 | +func TestSignalWeights_Default(t *testing.T) { | |
| 360 | + ctx := context.Background() | |
| 361 | + database := setupClusterTestDB(t) | |
| 362 | + seedClusterData(t, ctx, database) | |
| 363 | + | |
| 364 | + engine := NewEngine(database.DB, slog.Default()) | |
| 365 | + w := engine.GetWeights(ctx, "did:test:alice") | |
| 366 | + | |
| 367 | + assert.Equal(t, w.WSub, 1.0) | |
| 368 | + assert.Equal(t, w.WLike, 0.5) | |
| 369 | + assert.Equal(t, w.WTag, 0.3) | |
| 370 | + assert.Equal(t, w.WSocial, 0.7) | |
| 371 | + assert.Equal(t, w.WPop, 0.2) | |
| 372 | + assert.Equal(t, w.WCategory, 0.4) | |
| 373 | +} | |
| 374 | + | |
| 375 | +func TestSignalWeights_RewardPenalize(t *testing.T) { | |
| 376 | + ctx := context.Background() | |
| 377 | + database := setupClusterTestDB(t) | |
| 378 | + seedClusterData(t, ctx, database) | |
| 379 | + | |
| 380 | + engine := NewEngine(database.DB, slog.Default()) | |
| 381 | + | |
| 382 | + _, err := database.ExecContext(ctx, ` | |
| 383 | + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted) | |
| 384 | + VALUES ('did:test:alice', 'feed', 'https://a.com/feed', datetime('now'), datetime('now'), 1, 1) | |
| 385 | + `) | |
| 386 | + assert.NilError(t, err) | |
| 387 | + for i := range minActionsTune { | |
| 388 | + _, err = database.ExecContext(ctx, ` | |
| 389 | + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted) | |
| 390 | + VALUES ('did:test:alice', 'feed', ?, datetime('now'), datetime('now'), 1, 1) | |
| 391 | + `, fmt.Sprintf("https://%d.com/feed", i)) | |
| 392 | + assert.NilError(t, err) | |
| 393 | + } | |
| 394 | + | |
| 395 | + engine.RewardSignal(ctx, "did:test:alice", "social") | |
| 396 | + | |
| 397 | + w := engine.GetWeights(ctx, "did:test:alice") | |
| 398 | + assert.Assert(t, w.WSocial > 0.7, "rewarding social signal should increase w_social, got %f", w.WSocial) | |
| 176 | 399 | } |
| 177 | 400 | |
| 178 | -func TestTagsBasedSimilarity(t *testing.T) { | |
| 401 | +func TestColdStartRecommendations(t *testing.T) { | |
| 179 | 402 | ctx := context.Background() |
| 180 | 403 | database := setupClusterTestDB(t) |
| 181 | 404 | seedClusterData(t, ctx, database) |
| 405 | + seedFollowData(t, ctx, database) | |
| 406 | + | |
| 407 | + engine := NewEngine(database.DB, slog.Default()) | |
| 408 | + assert.NilError(t, engine.ComputeFollowDistances(ctx)) | |
| 182 | 409 | |
| 183 | - _, err := database.ExecContext(ctx, `INSERT INTO annotations (uri, author_did, feed_url, article_url, tags, created_at) VALUES (?, ?, ?, ?, ?, datetime('now'))`, | |
| 184 | - "at://alice/ann/1", "did:test:alice", "https://a.com/feed", "https://a.com/art1", "go,programming") | |
| 410 | + _, err := database.ExecContext(ctx, `UPDATE feeds SET subscriber_count = 2 WHERE feed_url = 'https://a.com/feed'`) | |
| 185 | 411 | assert.NilError(t, err) |
| 186 | - _, err = database.ExecContext(ctx, `INSERT INTO annotations (uri, author_did, feed_url, article_url, tags, created_at) VALUES (?, ?, ?, ?, ?, datetime('now'))`, | |
| 187 | - "at://alice/ann/2", "did:test:alice", "https://a.com/feed", "https://a.com/art2", "rust,programming") | |
| 412 | + _, err = database.ExecContext(ctx, `UPDATE feeds SET subscriber_count = 2 WHERE feed_url = 'https://b.com/feed'`) | |
| 188 | 413 | assert.NilError(t, err) |
| 189 | - _, err = database.ExecContext(ctx, `INSERT INTO annotations (uri, author_did, feed_url, article_url, tags, created_at) VALUES (?, ?, ?, ?, ?, datetime('now'))`, | |
| 190 | - "at://carol/ann/1", "did:test:carol", "https://c.com/feed", "https://c.com/art1", "go,web") | |
| 414 | + | |
| 415 | + _, err = database.ExecContext(ctx, `INSERT INTO users (did, handle) VALUES (?, ?)`, "did:test:newuser", "newuser") | |
| 191 | 416 | assert.NilError(t, err) |
| 192 | 417 | |
| 418 | + recs, err := engine.ColdStartRecommendations(ctx, "did:test:newuser", 10) | |
| 419 | + assert.NilError(t, err) | |
| 420 | + assert.Assert(t, len(recs) > 0, "new user should get cold start recommendations") | |
| 421 | +} | |
| 422 | + | |
| 423 | +func TestColdStartRecommendations_NotTriggeredForEstablishedUser(t *testing.T) { | |
| 424 | + ctx := context.Background() | |
| 425 | + database := setupClusterTestDB(t) | |
| 426 | + seedClusterData(t, ctx, database) | |
| 427 | + | |
| 428 | + engine := NewEngine(database.DB, slog.Default()) | |
| 429 | + | |
| 430 | + recs, err := engine.ColdStartRecommendations(ctx, "did:test:alice", 10) | |
| 431 | + assert.NilError(t, err) | |
| 432 | + assert.Assert(t, recs == nil, "established user should not get cold start recommendations") | |
| 433 | +} | |
| 434 | + | |
| 435 | +func TestOnDemandPeopleRecommendations(t *testing.T) { | |
| 436 | + ctx := context.Background() | |
| 437 | + database := setupClusterTestDB(t) | |
| 438 | + seedClusterData(t, ctx, database) | |
| 439 | + | |
| 193 | 440 | engine := NewEngine(database.DB, slog.Default()) |
| 194 | 441 | assert.NilError(t, engine.ComputeUserSimilarity(ctx)) |
| 195 | 442 | |
| 196 | - var jaccard float64 | |
| 197 | - var commonTags int | |
| 443 | + recs, err := engine.GetPeopleRecommendations(ctx, "did:test:carol", 10) | |
| 444 | + assert.NilError(t, err) | |
| 445 | + assert.Assert(t, len(recs) > 0, "carol should get people recommendations") | |
| 446 | +} | |
| 447 | + | |
| 448 | +func TestDismissArticle(t *testing.T) { | |
| 449 | + ctx := context.Background() | |
| 450 | + database := setupClusterTestDB(t) | |
| 451 | + seedClusterData(t, ctx, database) | |
| 452 | + | |
| 453 | + engine := NewEngine(database.DB, slog.Default()) | |
| 454 | + | |
| 455 | + assert.NilError(t, engine.DismissArticle(ctx, "did:test:alice", "https://a.com/article1", "not_interested")) | |
| 456 | + | |
| 457 | + var count int | |
| 458 | + assert.NilError(t, database.QueryRowContext(ctx, | |
| 459 | + `SELECT COUNT(*) FROM dismissed_recommendations WHERE user_did = 'did:test:alice' AND target_type = 'article'`).Scan(&count)) | |
| 460 | + assert.Equal(t, count, 1) | |
| 461 | +} | |
| 462 | + | |
| 463 | +func TestComputeSignalProfiles(t *testing.T) { | |
| 464 | + ctx := context.Background() | |
| 465 | + database := setupClusterTestDB(t) | |
| 466 | + seedClusterData(t, ctx, database) | |
| 467 | + | |
| 468 | + engine := NewEngine(database.DB, slog.Default()) | |
| 469 | + assert.NilError(t, engine.ComputeSignalProfiles(ctx)) | |
| 470 | + | |
| 471 | + var count int | |
| 472 | + assert.NilError(t, database.QueryRowContext(ctx, `SELECT COUNT(*) FROM user_signal_profiles`).Scan(&count)) | |
| 473 | + assert.Assert(t, count >= 3, "expected signal profiles for all users") | |
| 474 | +} | |
| 475 | + | |
| 476 | +func TestDismissFeed_Idempotent(t *testing.T) { | |
| 477 | + ctx := context.Background() | |
| 478 | + database := setupClusterTestDB(t) | |
| 479 | + seedClusterData(t, ctx, database) | |
| 480 | + | |
| 481 | + engine := NewEngine(database.DB, slog.Default()) | |
| 482 | + | |
| 483 | + assert.NilError(t, engine.DismissFeed(ctx, "did:test:alice", "https://a.com/feed", "reason1")) | |
| 484 | + assert.NilError(t, engine.DismissFeed(ctx, "did:test:alice", "https://a.com/feed", "reason2")) | |
| 485 | + | |
| 486 | + var count int | |
| 198 | 487 | assert.NilError(t, database.QueryRowContext(ctx, |
| 199 | - `SELECT jaccard, common_tags FROM user_similarity WHERE user_a = ? AND user_b = ?`, | |
| 200 | - "did:test:alice", "did:test:carol").Scan(&jaccard, &commonTags)) | |
| 201 | - assert.Equal(t, commonTags, 1, "alice and carol share 1 tag (go)") | |
| 202 | - assert.Assert(t, jaccard > 0, "tags should contribute to similarity, got %f", jaccard) | |
| 488 | + `SELECT COUNT(*) FROM dismissed_recommendations WHERE user_did = 'did:test:alice' AND target_type = 'feed'`).Scan(&count)) | |
| 489 | + assert.Equal(t, count, 1, "duplicate dismiss should not create extra rows") | |
| 203 | 490 | } |
| 204 | 491 | |
| 205 | 492 | func TestDescriptionBasedFeedSimilarity(t *testing.T) { |
| @@ -2,6 +2,7 @@ package cluster | |||
| 2 | 2 | ||
| 3 | import ( | 3 | import ( |
| 4 | "context" | 4 | "context" |
| 5 | + "fmt" | ||
| 5 | "os" | 6 | "os" |
| 6 | "testing" | 7 | "testing" |
| 7 | 8 | ||
| @@ -64,6 +65,18 @@ func seedClusterData(t *testing.T, ctx context.Context, database *db.DB) { | |||
| 64 | } | 65 | } |
| 65 | } | 66 | } |
| 66 | 67 | ||
| 68 | +func seedFollowData(t *testing.T, ctx context.Context, database *db.DB) { | ||
| 69 | + t.Helper() | ||
| 70 | + follows := []struct{ user, target string }{ | ||
| 71 | + {"did:test:alice", "did:test:bob"}, | ||
| 72 | + {"did:test:bob", "did:test:carol"}, | ||
| 73 | + } | ||
| 74 | + for _, f := range follows { | ||
| 75 | + _, err := database.ExecContext(ctx, `INSERT OR IGNORE INTO follows (user_did, target_did) VALUES (?, ?)`, f.user, f.target) | ||
| 76 | + assert.NilError(t, err) | ||
| 77 | + } | ||
| 78 | +} | ||
| 79 | + | ||
| 67 | func TestComputeFeedSimilarity(t *testing.T) { | 80 | func TestComputeFeedSimilarity(t *testing.T) { |
| 68 | ctx := context.Background() | 81 | ctx := context.Background() |
| 69 | database := setupClusterTestDB(t) | 82 | database := setupClusterTestDB(t) |
| @@ -94,7 +107,7 @@ func TestComputeUserSimilarity(t *testing.T) { | |||
| 94 | assert.Assert(t, count > 0, "expected user similarity pairs") | 107 | assert.Assert(t, count > 0, "expected user similarity pairs") |
| 95 | } | 108 | } |
| 96 | 109 | ||
| 97 | -func TestComputeRecommendations_GeneratesFeedRecsForNewUser(t *testing.T) { | 110 | +func TestOnDemandFeedRecommendations(t *testing.T) { |
| 98 | ctx := context.Background() | 111 | ctx := context.Background() |
| 99 | database := setupClusterTestDB(t) | 112 | database := setupClusterTestDB(t) |
| 100 | seedClusterData(t, ctx, database) | 113 | seedClusterData(t, ctx, database) |
| @@ -102,7 +115,6 @@ func TestComputeRecommendations_GeneratesFeedRecsForNewUser(t *testing.T) { | |||
| 102 | engine := NewEngine(database.DB, slog.Default()) | 115 | engine := NewEngine(database.DB, slog.Default()) |
| 103 | assert.NilError(t, engine.ComputeFeedSimilarity(ctx)) | 116 | assert.NilError(t, engine.ComputeFeedSimilarity(ctx)) |
| 104 | assert.NilError(t, engine.ComputeUserSimilarity(ctx)) | 117 | assert.NilError(t, engine.ComputeUserSimilarity(ctx)) |
| 105 | - assert.NilError(t, engine.ComputeRecommendations(ctx)) | ||
| 106 | 118 | ||
| 107 | recs, err := engine.GetFeedRecommendations(ctx, "did:test:carol", 10) | 119 | recs, err := engine.GetFeedRecommendations(ctx, "did:test:carol", 10) |
| 108 | assert.NilError(t, err) | 120 | assert.NilError(t, err) |
| @@ -117,7 +129,7 @@ func TestComputeRecommendations_GeneratesFeedRecsForNewUser(t *testing.T) { | |||
| 117 | assert.Assert(t, found, "carol should be recommended feeds she doesn't subscribe to") | 129 | assert.Assert(t, found, "carol should be recommended feeds she doesn't subscribe to") |
| 118 | } | 130 | } |
| 119 | 131 | ||
| 120 | -func TestComputeRecommendations_NoSelfRecommendations(t *testing.T) { | 132 | +func TestNoSelfRecommendations(t *testing.T) { |
| 121 | ctx := context.Background() | 133 | ctx := context.Background() |
| 122 | database := setupClusterTestDB(t) | 134 | database := setupClusterTestDB(t) |
| 123 | seedClusterData(t, ctx, database) | 135 | seedClusterData(t, ctx, database) |
| @@ -125,7 +137,6 @@ func TestComputeRecommendations_NoSelfRecommendations(t *testing.T) { | |||
| 125 | engine := NewEngine(database.DB, slog.Default()) | 137 | engine := NewEngine(database.DB, slog.Default()) |
| 126 | assert.NilError(t, engine.ComputeFeedSimilarity(ctx)) | 138 | assert.NilError(t, engine.ComputeFeedSimilarity(ctx)) |
| 127 | assert.NilError(t, engine.ComputeUserSimilarity(ctx)) | 139 | assert.NilError(t, engine.ComputeUserSimilarity(ctx)) |
| 128 | - assert.NilError(t, engine.ComputeRecommendations(ctx)) | ||
| 129 | 140 | ||
| 130 | recs, err := engine.GetFeedRecommendations(ctx, "did:test:alice", 10) | 141 | recs, err := engine.GetFeedRecommendations(ctx, "did:test:alice", 10) |
| 131 | assert.NilError(t, err) | 142 | assert.NilError(t, err) |
| @@ -141,65 +152,341 @@ func TestComputeRecommendations_NoSelfRecommendations(t *testing.T) { | |||
| 141 | } | 152 | } |
| 142 | } | 153 | } |
| 143 | 154 | ||
| 144 | -func TestLikesBasedSimilarity(t *testing.T) { | 155 | +func TestDismissedFeedsExcluded(t *testing.T) { |
| 145 | ctx := context.Background() | 156 | ctx := context.Background() |
| 146 | database := setupClusterTestDB(t) | 157 | database := setupClusterTestDB(t) |
| 147 | seedClusterData(t, ctx, database) | 158 | seedClusterData(t, ctx, database) |
| 148 | 159 | ||
| 149 | - _, err := database.ExecContext(ctx, `INSERT INTO articles (feed_url, guid, title, url) VALUES (?, ?, ?, ?)`, | 160 | + engine := NewEngine(database.DB, slog.Default()) |
| 150 | - "https://a.com/feed", "art1", "Article 1", "https://a.com/art1") | 161 | + assert.NilError(t, engine.ComputeFeedSimilarity(ctx)) |
| 162 | + assert.NilError(t, engine.ComputeUserSimilarity(ctx)) | ||
| 163 | + | ||
| 164 | + assert.NilError(t, engine.DismissFeed(ctx, "did:test:carol", "https://a.com/feed", "not_interested")) | ||
| 165 | + | ||
| 166 | + recs, err := engine.GetFeedRecommendations(ctx, "did:test:carol", 10) | ||
| 151 | assert.NilError(t, err) | 167 | assert.NilError(t, err) |
| 152 | - _, err = database.ExecContext(ctx, `INSERT INTO articles (feed_url, guid, title, url) VALUES (?, ?, ?, ?)`, | 168 | + |
| 153 | - "https://a.com/feed", "art2", "Article 2", "https://a.com/art2") | 169 | + for _, r := range recs { |
| 170 | + assert.Assert(t, r.FeedURL != "https://a.com/feed", | ||
| 171 | + "dismissed feed should not appear in recommendations") | ||
| 172 | + } | ||
| 173 | +} | ||
| 174 | + | ||
| 175 | +func TestIsFeedDismissed(t *testing.T) { | ||
| 176 | + ctx := context.Background() | ||
| 177 | + database := setupClusterTestDB(t) | ||
| 178 | + seedClusterData(t, ctx, database) | ||
| 179 | + | ||
| 180 | + engine := NewEngine(database.DB, slog.Default()) | ||
| 181 | + | ||
| 182 | + dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://a.com/feed") | ||
| 154 | assert.NilError(t, err) | 183 | assert.NilError(t, err) |
| 184 | + assert.Assert(t, !dismissed, "feed should not be dismissed initially") | ||
| 185 | + | ||
| 186 | + assert.NilError(t, engine.DismissFeed(ctx, "did:test:alice", "https://a.com/feed", "not_interested")) | ||
| 155 | 187 | ||
| 156 | - _, err = database.ExecContext(ctx, `INSERT INTO likes (uri, author_did, feed_url, article_url, created_at) VALUES (?, ?, ?, ?, datetime('now'))`, | 188 | + dismissed, err = engine.IsFeedDismissed(ctx, "did:test:alice", "https://a.com/feed") |
| 157 | - "at://alice/like/1", "did:test:alice", "https://a.com/feed", "https://a.com/art1") | ||
| 158 | assert.NilError(t, err) | 189 | assert.NilError(t, err) |
| 159 | - _, err = database.ExecContext(ctx, `INSERT INTO likes (uri, author_did, feed_url, article_url, created_at) VALUES (?, ?, ?, ?, datetime('now'))`, | 190 | + assert.Assert(t, dismissed, "feed should be dismissed after dismiss call") |
| 160 | - "at://alice/like/2", "did:test:alice", "https://a.com/feed", "https://a.com/art2") | 191 | +} |
| 192 | + | ||
| 193 | +func TestRecordImpressions(t *testing.T) { | ||
| 194 | + ctx := context.Background() | ||
| 195 | + database := setupClusterTestDB(t) | ||
| 196 | + seedClusterData(t, ctx, database) | ||
| 197 | + | ||
| 198 | + engine := NewEngine(database.DB, slog.Default()) | ||
| 199 | + | ||
| 200 | + impressions := []Impression{ | ||
| 201 | + {TargetType: "feed", TargetID: "https://a.com/feed"}, | ||
| 202 | + {TargetType: "feed", TargetID: "https://b.com/feed"}, | ||
| 203 | + } | ||
| 204 | + assert.NilError(t, engine.RecordImpressions(ctx, "did:test:alice", impressions)) | ||
| 205 | + | ||
| 206 | + var count int | ||
| 207 | + assert.NilError(t, database.QueryRowContext(ctx, | ||
| 208 | + `SELECT COUNT(*) FROM recommendation_impressions WHERE user_did = 'did:test:alice'`).Scan(&count)) | ||
| 209 | + assert.Equal(t, count, 2) | ||
| 210 | + | ||
| 211 | + assert.NilError(t, engine.RecordImpressions(ctx, "did:test:alice", impressions)) | ||
| 212 | + | ||
| 213 | + var shownCount int | ||
| 214 | + assert.NilError(t, database.QueryRowContext(ctx, | ||
| 215 | + `SELECT shown_count FROM recommendation_impressions WHERE user_did = 'did:test:alice' AND target_id = 'https://a.com/feed'`).Scan(&shownCount)) | ||
| 216 | + assert.Equal(t, shownCount, 2, "shown_count should increment on repeated impression") | ||
| 217 | +} | ||
| 218 | + | ||
| 219 | +func TestMarkImpressionActed(t *testing.T) { | ||
| 220 | + ctx := context.Background() | ||
| 221 | + database := setupClusterTestDB(t) | ||
| 222 | + seedClusterData(t, ctx, database) | ||
| 223 | + | ||
| 224 | + engine := NewEngine(database.DB, slog.Default()) | ||
| 225 | + | ||
| 226 | + impressions := []Impression{{TargetType: "feed", TargetID: "https://a.com/feed"}} | ||
| 227 | + assert.NilError(t, engine.RecordImpressions(ctx, "did:test:alice", impressions)) | ||
| 228 | + | ||
| 229 | + assert.NilError(t, engine.MarkImpressionActed(ctx, "did:test:alice", "feed", "https://a.com/feed")) | ||
| 230 | + | ||
| 231 | + var acted bool | ||
| 232 | + assert.NilError(t, database.QueryRowContext(ctx, | ||
| 233 | + `SELECT acted FROM recommendation_impressions WHERE user_did = 'did:test:alice' AND target_id = 'https://a.com/feed'`).Scan(&acted)) | ||
| 234 | + assert.Assert(t, acted, "impression should be marked as acted") | ||
| 235 | +} | ||
| 236 | + | ||
| 237 | +func TestComputeFollowDistances(t *testing.T) { | ||
| 238 | + ctx := context.Background() | ||
| 239 | + database := setupClusterTestDB(t) | ||
| 240 | + seedClusterData(t, ctx, database) | ||
| 241 | + seedFollowData(t, ctx, database) | ||
| 242 | + | ||
| 243 | + engine := NewEngine(database.DB, slog.Default()) | ||
| 244 | + assert.NilError(t, engine.ComputeFollowDistances(ctx)) | ||
| 245 | + | ||
| 246 | + var d1, d2 int | ||
| 247 | + assert.NilError(t, database.QueryRowContext(ctx, | ||
| 248 | + `SELECT COUNT(*) FROM follow_distances WHERE distance = 1`).Scan(&d1)) | ||
| 249 | + assert.NilError(t, database.QueryRowContext(ctx, | ||
| 250 | + `SELECT COUNT(*) FROM follow_distances WHERE distance = 2`).Scan(&d2)) | ||
| 251 | + assert.Assert(t, d1 >= 2, "expected at least 2 direct follow distances") | ||
| 252 | + assert.Assert(t, d2 >= 1, "expected at least 1 two-hop distance (alice -> bob -> carol)") | ||
| 253 | + | ||
| 254 | + var dist int | ||
| 255 | + assert.NilError(t, database.QueryRowContext(ctx, | ||
| 256 | + `SELECT distance FROM follow_distances WHERE user_a = 'did:test:alice' AND user_b = 'did:test:carol'`).Scan(&dist)) | ||
| 257 | + assert.Equal(t, dist, 2, "alice should be 2 hops from carol") | ||
| 258 | +} | ||
| 259 | + | ||
| 260 | +func TestAutoDismissStale(t *testing.T) { | ||
| 261 | + ctx := context.Background() | ||
| 262 | + database := setupClusterTestDB(t) | ||
| 263 | + seedClusterData(t, ctx, database) | ||
| 264 | + | ||
| 265 | + engine := NewEngine(database.DB, slog.Default()) | ||
| 266 | + | ||
| 267 | + _, err := database.ExecContext(ctx, ` | ||
| 268 | + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted) | ||
| 269 | + VALUES ('did:test:alice', 'feed', 'https://stale.com/feed', datetime('now', '-31 days'), datetime('now'), 20, 0) | ||
| 270 | + `) | ||
| 161 | assert.NilError(t, err) | 271 | assert.NilError(t, err) |
| 162 | - _, err = database.ExecContext(ctx, `INSERT INTO likes (uri, author_did, feed_url, article_url, created_at) VALUES (?, ?, ?, ?, datetime('now'))`, | 272 | + |
| 163 | - "at://carol/like/1", "did:test:carol", "https://a.com/feed", "https://a.com/art1") | 273 | + assert.NilError(t, engine.AutoDismissStale(ctx, 15, 30)) |
| 274 | + | ||
| 275 | + dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://stale.com/feed") | ||
| 164 | assert.NilError(t, err) | 276 | assert.NilError(t, err) |
| 277 | + assert.Assert(t, dismissed, "stale recommendation should be auto-dismissed") | ||
| 278 | +} | ||
| 279 | + | ||
| 280 | +func TestAutoDismissStale_DoesNotDismissRecent(t *testing.T) { | ||
| 281 | + ctx := context.Background() | ||
| 282 | + database := setupClusterTestDB(t) | ||
| 283 | + seedClusterData(t, ctx, database) | ||
| 165 | 284 | ||
| 166 | engine := NewEngine(database.DB, slog.Default()) | 285 | engine := NewEngine(database.DB, slog.Default()) |
| 167 | - assert.NilError(t, engine.ComputeUserSimilarity(ctx)) | ||
| 168 | 286 | ||
| 169 | - var jaccard float64 | 287 | + _, err := database.ExecContext(ctx, ` |
| 170 | - var commonLikes int | 288 | + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted) |
| 171 | - assert.NilError(t, database.QueryRowContext(ctx, | 289 | + VALUES ('did:test:alice', 'feed', 'https://recent.com/feed', datetime('now'), datetime('now'), 5, 0) |
| 172 | - `SELECT jaccard, common_likes FROM user_similarity WHERE user_a = ? AND user_b = ?`, | 290 | + `) |
| 173 | - "did:test:alice", "did:test:carol").Scan(&jaccard, &commonLikes)) | 291 | + assert.NilError(t, err) |
| 174 | - assert.Equal(t, commonLikes, 1, "alice and carol share 1 liked article") | 292 | + |
| 175 | - assert.Assert(t, jaccard > 0, "likes should contribute to similarity, got %f", jaccard) | 293 | + assert.NilError(t, engine.AutoDismissStale(ctx, 15, 30)) |
| 294 | + | ||
| 295 | + dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://recent.com/feed") | ||
| 296 | + assert.NilError(t, err) | ||
| 297 | + assert.Assert(t, !dismissed, "recent impression should not be auto-dismissed") | ||
| 298 | +} | ||
| 299 | + | ||
| 300 | +func TestAutoDismissStale_DoesNotDismissActed(t *testing.T) { | ||
| 301 | + ctx := context.Background() | ||
| 302 | + database := setupClusterTestDB(t) | ||
| 303 | + seedClusterData(t, ctx, database) | ||
| 304 | + | ||
| 305 | + engine := NewEngine(database.DB, slog.Default()) | ||
| 306 | + | ||
| 307 | + _, err := database.ExecContext(ctx, ` | ||
| 308 | + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted) | ||
| 309 | + VALUES ('did:test:alice', 'feed', 'https://acted.com/feed', datetime('now', '-31 days'), datetime('now'), 20, 1) | ||
| 310 | + `) | ||
| 311 | + assert.NilError(t, err) | ||
| 312 | + | ||
| 313 | + assert.NilError(t, engine.AutoDismissStale(ctx, 15, 30)) | ||
| 314 | + | ||
| 315 | + dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://acted.com/feed") | ||
| 316 | + assert.NilError(t, err) | ||
| 317 | + assert.Assert(t, !dismissed, "acted recommendation should not be auto-dismissed") | ||
| 318 | +} | ||
| 319 | + | ||
| 320 | +func TestDiversityFiltering(t *testing.T) { | ||
| 321 | + candidates := []*FeedRecommendation{ | ||
| 322 | + {FeedURL: "https://a.com/1", SiteURL: "https://a.com", Score: 1.0}, | ||
| 323 | + {FeedURL: "https://a.com/2", SiteURL: "https://a.com", Score: 0.9}, | ||
| 324 | + {FeedURL: "https://a.com/3", SiteURL: "https://a.com", Score: 0.8}, | ||
| 325 | + {FeedURL: "https://b.com/1", SiteURL: "https://b.com", Score: 0.7}, | ||
| 326 | + {FeedURL: "https://b.com/2", SiteURL: "https://b.com", Score: 0.6}, | ||
| 327 | + {FeedURL: "https://c.com/1", SiteURL: "https://c.com", Score: 0.5}, | ||
| 328 | + } | ||
| 329 | + | ||
| 330 | + result := ApplyDiversity(candidates, 6) | ||
| 331 | + | ||
| 332 | + aCount := 0 | ||
| 333 | + bCount := 0 | ||
| 334 | + cCount := 0 | ||
| 335 | + for _, r := range result { | ||
| 336 | + switch extractDomain(r.SiteURL) { | ||
| 337 | + case "a.com": | ||
| 338 | + aCount++ | ||
| 339 | + case "b.com": | ||
| 340 | + bCount++ | ||
| 341 | + case "c.com": | ||
| 342 | + cCount++ | ||
| 343 | + } | ||
| 344 | + } | ||
| 345 | + assert.Assert(t, aCount <= maxPerDomain, "should limit feeds from same domain") | ||
| 346 | + assert.Assert(t, len(result) <= 6, "should respect topN limit") | ||
| 347 | + assert.Assert(t, cCount >= 1, "should include feeds from different domains") | ||
| 348 | +} | ||
| 349 | + | ||
| 350 | +func TestDiversityFiltering_EmptySiteURL(t *testing.T) { | ||
| 351 | + candidates := []*FeedRecommendation{ | ||
| 352 | + {FeedURL: "https://a.com/1", SiteURL: "", Score: 1.0}, | ||
| 353 | + {FeedURL: "https://b.com/1", SiteURL: "", Score: 0.9}, | ||
| 354 | + } | ||
| 355 | + result := ApplyDiversity(candidates, 5) | ||
| 356 | + assert.Equal(t, len(result), 2, "feeds without site_url should not be filtered out") | ||
| 357 | +} | ||
| 358 | + | ||
| 359 | +func TestSignalWeights_Default(t *testing.T) { | ||
| 360 | + ctx := context.Background() | ||
| 361 | + database := setupClusterTestDB(t) | ||
| 362 | + seedClusterData(t, ctx, database) | ||
| 363 | + | ||
| 364 | + engine := NewEngine(database.DB, slog.Default()) | ||
| 365 | + w := engine.GetWeights(ctx, "did:test:alice") | ||
| 366 | + | ||
| 367 | + assert.Equal(t, w.WSub, 1.0) | ||
| 368 | + assert.Equal(t, w.WLike, 0.5) | ||
| 369 | + assert.Equal(t, w.WTag, 0.3) | ||
| 370 | + assert.Equal(t, w.WSocial, 0.7) | ||
| 371 | + assert.Equal(t, w.WPop, 0.2) | ||
| 372 | + assert.Equal(t, w.WCategory, 0.4) | ||
| 373 | +} | ||
| 374 | + | ||
| 375 | +func TestSignalWeights_RewardPenalize(t *testing.T) { | ||
| 376 | + ctx := context.Background() | ||
| 377 | + database := setupClusterTestDB(t) | ||
| 378 | + seedClusterData(t, ctx, database) | ||
| 379 | + | ||
| 380 | + engine := NewEngine(database.DB, slog.Default()) | ||
| 381 | + | ||
| 382 | + _, err := database.ExecContext(ctx, ` | ||
| 383 | + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted) | ||
| 384 | + VALUES ('did:test:alice', 'feed', 'https://a.com/feed', datetime('now'), datetime('now'), 1, 1) | ||
| 385 | + `) | ||
| 386 | + assert.NilError(t, err) | ||
| 387 | + for i := range minActionsTune { | ||
| 388 | + _, err = database.ExecContext(ctx, ` | ||
| 389 | + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted) | ||
| 390 | + VALUES ('did:test:alice', 'feed', ?, datetime('now'), datetime('now'), 1, 1) | ||
| 391 | + `, fmt.Sprintf("https://%d.com/feed", i)) | ||
| 392 | + assert.NilError(t, err) | ||
| 393 | + } | ||
| 394 | + | ||
| 395 | + engine.RewardSignal(ctx, "did:test:alice", "social") | ||
| 396 | + | ||
| 397 | + w := engine.GetWeights(ctx, "did:test:alice") | ||
| 398 | + assert.Assert(t, w.WSocial > 0.7, "rewarding social signal should increase w_social, got %f", w.WSocial) | ||
| 176 | } | 399 | } |
| 177 | 400 | ||
| 178 | -func TestTagsBasedSimilarity(t *testing.T) { | 401 | +func TestColdStartRecommendations(t *testing.T) { |
| 179 | ctx := context.Background() | 402 | ctx := context.Background() |
| 180 | database := setupClusterTestDB(t) | 403 | database := setupClusterTestDB(t) |
| 181 | seedClusterData(t, ctx, database) | 404 | seedClusterData(t, ctx, database) |
| 405 | + seedFollowData(t, ctx, database) | ||
| 406 | + | ||
| 407 | + engine := NewEngine(database.DB, slog.Default()) | ||
| 408 | + assert.NilError(t, engine.ComputeFollowDistances(ctx)) | ||
| 182 | 409 | ||
| 183 | - _, err := database.ExecContext(ctx, `INSERT INTO annotations (uri, author_did, feed_url, article_url, tags, created_at) VALUES (?, ?, ?, ?, ?, datetime('now'))`, | 410 | + _, err := database.ExecContext(ctx, `UPDATE feeds SET subscriber_count = 2 WHERE feed_url = 'https://a.com/feed'`) |
| 184 | - "at://alice/ann/1", "did:test:alice", "https://a.com/feed", "https://a.com/art1", "go,programming") | ||
| 185 | assert.NilError(t, err) | 411 | assert.NilError(t, err) |
| 186 | - _, err = database.ExecContext(ctx, `INSERT INTO annotations (uri, author_did, feed_url, article_url, tags, created_at) VALUES (?, ?, ?, ?, ?, datetime('now'))`, | 412 | + _, err = database.ExecContext(ctx, `UPDATE feeds SET subscriber_count = 2 WHERE feed_url = 'https://b.com/feed'`) |
| 187 | - "at://alice/ann/2", "did:test:alice", "https://a.com/feed", "https://a.com/art2", "rust,programming") | ||
| 188 | assert.NilError(t, err) | 413 | assert.NilError(t, err) |
| 189 | - _, err = database.ExecContext(ctx, `INSERT INTO annotations (uri, author_did, feed_url, article_url, tags, created_at) VALUES (?, ?, ?, ?, ?, datetime('now'))`, | 414 | + |
| 190 | - "at://carol/ann/1", "did:test:carol", "https://c.com/feed", "https://c.com/art1", "go,web") | 415 | + _, err = database.ExecContext(ctx, `INSERT INTO users (did, handle) VALUES (?, ?)`, "did:test:newuser", "newuser") |
| 191 | assert.NilError(t, err) | 416 | assert.NilError(t, err) |
| 192 | 417 | ||
| 418 | + recs, err := engine.ColdStartRecommendations(ctx, "did:test:newuser", 10) | ||
| 419 | + assert.NilError(t, err) | ||
| 420 | + assert.Assert(t, len(recs) > 0, "new user should get cold start recommendations") | ||
| 421 | +} | ||
| 422 | + | ||
| 423 | +func TestColdStartRecommendations_NotTriggeredForEstablishedUser(t *testing.T) { | ||
| 424 | + ctx := context.Background() | ||
| 425 | + database := setupClusterTestDB(t) | ||
| 426 | + seedClusterData(t, ctx, database) | ||
| 427 | + | ||
| 428 | + engine := NewEngine(database.DB, slog.Default()) | ||
| 429 | + | ||
| 430 | + recs, err := engine.ColdStartRecommendations(ctx, "did:test:alice", 10) | ||
| 431 | + assert.NilError(t, err) | ||
| 432 | + assert.Assert(t, recs == nil, "established user should not get cold start recommendations") | ||
| 433 | +} | ||
| 434 | + | ||
| 435 | +func TestOnDemandPeopleRecommendations(t *testing.T) { | ||
| 436 | + ctx := context.Background() | ||
| 437 | + database := setupClusterTestDB(t) | ||
| 438 | + seedClusterData(t, ctx, database) | ||
| 439 | + | ||
| 193 | engine := NewEngine(database.DB, slog.Default()) | 440 | engine := NewEngine(database.DB, slog.Default()) |
| 194 | assert.NilError(t, engine.ComputeUserSimilarity(ctx)) | 441 | assert.NilError(t, engine.ComputeUserSimilarity(ctx)) |
| 195 | 442 | ||
| 196 | - var jaccard float64 | 443 | + recs, err := engine.GetPeopleRecommendations(ctx, "did:test:carol", 10) |
| 197 | - var commonTags int | 444 | + assert.NilError(t, err) |
| 445 | + assert.Assert(t, len(recs) > 0, "carol should get people recommendations") | ||
| 446 | +} | ||
| 447 | + | ||
| 448 | +func TestDismissArticle(t *testing.T) { | ||
| 449 | + ctx := context.Background() | ||
| 450 | + database := setupClusterTestDB(t) | ||
| 451 | + seedClusterData(t, ctx, database) | ||
| 452 | + | ||
| 453 | + engine := NewEngine(database.DB, slog.Default()) | ||
| 454 | + | ||
| 455 | + assert.NilError(t, engine.DismissArticle(ctx, "did:test:alice", "https://a.com/article1", "not_interested")) | ||
| 456 | + | ||
| 457 | + var count int | ||
| 458 | + assert.NilError(t, database.QueryRowContext(ctx, | ||
| 459 | + `SELECT COUNT(*) FROM dismissed_recommendations WHERE user_did = 'did:test:alice' AND target_type = 'article'`).Scan(&count)) | ||
| 460 | + assert.Equal(t, count, 1) | ||
| 461 | +} | ||
| 462 | + | ||
| 463 | +func TestComputeSignalProfiles(t *testing.T) { | ||
| 464 | + ctx := context.Background() | ||
| 465 | + database := setupClusterTestDB(t) | ||
| 466 | + seedClusterData(t, ctx, database) | ||
| 467 | + | ||
| 468 | + engine := NewEngine(database.DB, slog.Default()) | ||
| 469 | + assert.NilError(t, engine.ComputeSignalProfiles(ctx)) | ||
| 470 | + | ||
| 471 | + var count int | ||
| 472 | + assert.NilError(t, database.QueryRowContext(ctx, `SELECT COUNT(*) FROM user_signal_profiles`).Scan(&count)) | ||
| 473 | + assert.Assert(t, count >= 3, "expected signal profiles for all users") | ||
| 474 | +} | ||
| 475 | + | ||
| 476 | +func TestDismissFeed_Idempotent(t *testing.T) { | ||
| 477 | + ctx := context.Background() | ||
| 478 | + database := setupClusterTestDB(t) | ||
| 479 | + seedClusterData(t, ctx, database) | ||
| 480 | + | ||
| 481 | + engine := NewEngine(database.DB, slog.Default()) | ||
| 482 | + | ||
| 483 | + assert.NilError(t, engine.DismissFeed(ctx, "did:test:alice", "https://a.com/feed", "reason1")) | ||
| 484 | + assert.NilError(t, engine.DismissFeed(ctx, "did:test:alice", "https://a.com/feed", "reason2")) | ||
| 485 | + | ||
| 486 | + var count int | ||
| 198 | assert.NilError(t, database.QueryRowContext(ctx, | 487 | assert.NilError(t, database.QueryRowContext(ctx, |
| 199 | - `SELECT jaccard, common_tags FROM user_similarity WHERE user_a = ? AND user_b = ?`, | 488 | + `SELECT COUNT(*) FROM dismissed_recommendations WHERE user_did = 'did:test:alice' AND target_type = 'feed'`).Scan(&count)) |
| 200 | - "did:test:alice", "did:test:carol").Scan(&jaccard, &commonTags)) | 489 | + assert.Equal(t, count, 1, "duplicate dismiss should not create extra rows") |
| 201 | - assert.Equal(t, commonTags, 1, "alice and carol share 1 tag (go)") | ||
| 202 | - assert.Assert(t, jaccard > 0, "tags should contribute to similarity, got %f", jaccard) | ||
| 203 | } | 490 | } |
| 204 | 491 | ||
| 205 | func TestDescriptionBasedFeedSimilarity(t *testing.T) { | 492 | func TestDescriptionBasedFeedSimilarity(t *testing.T) { |
deleted
internal/cluster/recommender.go +0 -165 | deleted file mode 100644 | ||
| @@ -1,165 +0,0 @@ | ||
| 1 | -package cluster | |
| 2 | - | |
| 3 | -import ( | |
| 4 | - "context" | |
| 5 | - "database/sql" | |
| 6 | -) | |
| 7 | - | |
| 8 | -type FeedRecommendation struct { | |
| 9 | - FeedURL string | |
| 10 | - Title string | |
| 11 | - SiteURL string | |
| 12 | - Description string | |
| 13 | - SubscriberCount int | |
| 14 | - FaviconURL string | |
| 15 | - Score float64 | |
| 16 | -} | |
| 17 | - | |
| 18 | -type PersonRecommendation struct { | |
| 19 | - DID string | |
| 20 | - Handle string | |
| 21 | - DisplayName string | |
| 22 | - AvatarURL string | |
| 23 | - Jaccard float64 | |
| 24 | - CommonFeeds int | |
| 25 | - CommonLikes int | |
| 26 | - CommonTags int | |
| 27 | -} | |
| 28 | - | |
| 29 | -type ArticleRecommendation struct { | |
| 30 | - ArticleID int64 | |
| 31 | - Title string | |
| 32 | - URL string | |
| 33 | - FeedURL string | |
| 34 | - FeedTitle string | |
| 35 | - Author string | |
| 36 | - Summary string | |
| 37 | - Published sql.NullTime | |
| 38 | - Score float64 | |
| 39 | -} | |
| 40 | - | |
| 41 | -type SimilarFeed struct { | |
| 42 | - FeedURL string | |
| 43 | - Title string | |
| 44 | - SiteURL string | |
| 45 | - Description string | |
| 46 | - FeedType string | |
| 47 | - Jaccard float64 | |
| 48 | -} | |
| 49 | - | |
| 50 | -func (e *Engine) GetFeedRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) { | |
| 51 | - rows, err := e.db.QueryContext(ctx, ` | |
| 52 | - SELECT r.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''), | |
| 53 | - COALESCE(f.description, ''), f.subscriber_count, COALESCE(f.favicon_url, ''), r.score | |
| 54 | - FROM user_feed_recommendations r | |
| 55 | - JOIN feeds f ON f.feed_url = r.feed_url | |
| 56 | - WHERE r.user_did = ? | |
| 57 | - AND r.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) | |
| 58 | - ORDER BY r.score DESC | |
| 59 | - LIMIT ? | |
| 60 | - `, userDID, userDID, limit) | |
| 61 | - if err != nil { | |
| 62 | - return nil, err | |
| 63 | - } | |
| 64 | - defer rows.Close() | |
| 65 | - | |
| 66 | - var results []*FeedRecommendation | |
| 67 | - for rows.Next() { | |
| 68 | - rec := &FeedRecommendation{} | |
| 69 | - if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description, | |
| 70 | - &rec.SubscriberCount, &rec.FaviconURL, &rec.Score); err != nil { | |
| 71 | - return nil, err | |
| 72 | - } | |
| 73 | - results = append(results, rec) | |
| 74 | - } | |
| 75 | - return results, rows.Err() | |
| 76 | -} | |
| 77 | - | |
| 78 | -func (e *Engine) GetPeopleRecommendations(ctx context.Context, userDID string, limit int) ([]*PersonRecommendation, error) { | |
| 79 | - rows, err := e.db.QueryContext(ctx, ` | |
| 80 | - SELECT u.did, u.handle, COALESCE(u.display_name, ''), COALESCE(u.avatar_url, ''), | |
| 81 | - sim.jaccard, sim.common_feeds, COALESCE(sim.common_likes, 0), COALESCE(sim.common_tags, 0) | |
| 82 | - FROM ( | |
| 83 | - SELECT user_b AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_a = ? | |
| 84 | - UNION ALL | |
| 85 | - SELECT user_a AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_b = ? | |
| 86 | - ) sim | |
| 87 | - JOIN users u ON u.did = sim.peer_did | |
| 88 | - WHERE u.handle IS NOT NULL AND u.handle != '' | |
| 89 | - ORDER BY sim.jaccard DESC | |
| 90 | - LIMIT ? | |
| 91 | - `, userDID, userDID, limit) | |
| 92 | - if err != nil { | |
| 93 | - return nil, err | |
| 94 | - } | |
| 95 | - defer rows.Close() | |
| 96 | - | |
| 97 | - var results []*PersonRecommendation | |
| 98 | - for rows.Next() { | |
| 99 | - rec := &PersonRecommendation{} | |
| 100 | - if err := rows.Scan(&rec.DID, &rec.Handle, &rec.DisplayName, &rec.AvatarURL, | |
| 101 | - &rec.Jaccard, &rec.CommonFeeds, &rec.CommonLikes, &rec.CommonTags); err != nil { | |
| 102 | - return nil, err | |
| 103 | - } | |
| 104 | - results = append(results, rec) | |
| 105 | - } | |
| 106 | - return results, rows.Err() | |
| 107 | -} | |
| 108 | - | |
| 109 | -func (e *Engine) GetArticleRecommendations(ctx context.Context, userDID string, limit int) ([]*ArticleRecommendation, error) { | |
| 110 | - rows, err := e.db.QueryContext(ctx, ` | |
| 111 | - SELECT a.id, a.title, COALESCE(a.url, ''), r.feed_url, COALESCE(f.title, ''), | |
| 112 | - COALESCE(a.author, ''), COALESCE(a.summary, ''), a.published, r.score | |
| 113 | - FROM user_article_recommendations r | |
| 114 | - JOIN articles a ON a.feed_url = r.feed_url AND a.url = r.article_url | |
| 115 | - LEFT JOIN feeds f ON f.feed_url = r.feed_url | |
| 116 | - WHERE r.user_did = ? | |
| 117 | - ORDER BY r.score DESC | |
| 118 | - LIMIT ? | |
| 119 | - `, userDID, limit) | |
| 120 | - if err != nil { | |
| 121 | - return nil, err | |
| 122 | - } | |
| 123 | - defer rows.Close() | |
| 124 | - | |
| 125 | - var recs []*ArticleRecommendation | |
| 126 | - for rows.Next() { | |
| 127 | - rec := &ArticleRecommendation{} | |
| 128 | - if err := rows.Scan(&rec.ArticleID, &rec.Title, &rec.URL, &rec.FeedURL, &rec.FeedTitle, | |
| 129 | - &rec.Author, &rec.Summary, &rec.Published, &rec.Score); err != nil { | |
| 130 | - return nil, err | |
| 131 | - } | |
| 132 | - recs = append(recs, rec) | |
| 133 | - } | |
| 134 | - return recs, rows.Err() | |
| 135 | -} | |
| 136 | - | |
| 137 | -func (e *Engine) GetSimilarFeeds(ctx context.Context, feedURL string, limit int) ([]*SimilarFeed, error) { | |
| 138 | - rows, err := e.db.QueryContext(ctx, ` | |
| 139 | - SELECT f.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''), | |
| 140 | - COALESCE(f.description, ''), COALESCE(f.feed_type, ''), sim.jaccard | |
| 141 | - FROM ( | |
| 142 | - SELECT feed_b AS peer_url, jaccard FROM feed_similarity WHERE feed_a = ? | |
| 143 | - UNION ALL | |
| 144 | - SELECT feed_a AS peer_url, jaccard FROM feed_similarity WHERE feed_b = ? | |
| 145 | - ) sim | |
| 146 | - JOIN feeds f ON f.feed_url = sim.peer_url | |
| 147 | - ORDER BY sim.jaccard DESC | |
| 148 | - LIMIT ? | |
| 149 | - `, feedURL, feedURL, limit) | |
| 150 | - if err != nil { | |
| 151 | - return nil, err | |
| 152 | - } | |
| 153 | - defer rows.Close() | |
| 154 | - | |
| 155 | - var results []*SimilarFeed | |
| 156 | - for rows.Next() { | |
| 157 | - rec := &SimilarFeed{} | |
| 158 | - if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description, | |
| 159 | - &rec.FeedType, &rec.Jaccard); err != nil { | |
| 160 | - return nil, err | |
| 161 | - } | |
| 162 | - results = append(results, rec) | |
| 163 | - } | |
| 164 | - return results, rows.Err() | |
| 165 | -} | |
| deleted file mode 100644 | |||
| @@ -1,165 +0,0 @@ | |||
| 1 | -package cluster | ||
| 2 | - | ||
| 3 | -import ( | ||
| 4 | - "context" | ||
| 5 | - "database/sql" | ||
| 6 | -) | ||
| 7 | - | ||
| 8 | -type FeedRecommendation struct { | ||
| 9 | - FeedURL string | ||
| 10 | - Title string | ||
| 11 | - SiteURL string | ||
| 12 | - Description string | ||
| 13 | - SubscriberCount int | ||
| 14 | - FaviconURL string | ||
| 15 | - Score float64 | ||
| 16 | -} | ||
| 17 | - | ||
| 18 | -type PersonRecommendation struct { | ||
| 19 | - DID string | ||
| 20 | - Handle string | ||
| 21 | - DisplayName string | ||
| 22 | - AvatarURL string | ||
| 23 | - Jaccard float64 | ||
| 24 | - CommonFeeds int | ||
| 25 | - CommonLikes int | ||
| 26 | - CommonTags int | ||
| 27 | -} | ||
| 28 | - | ||
| 29 | -type ArticleRecommendation struct { | ||
| 30 | - ArticleID int64 | ||
| 31 | - Title string | ||
| 32 | - URL string | ||
| 33 | - FeedURL string | ||
| 34 | - FeedTitle string | ||
| 35 | - Author string | ||
| 36 | - Summary string | ||
| 37 | - Published sql.NullTime | ||
| 38 | - Score float64 | ||
| 39 | -} | ||
| 40 | - | ||
| 41 | -type SimilarFeed struct { | ||
| 42 | - FeedURL string | ||
| 43 | - Title string | ||
| 44 | - SiteURL string | ||
| 45 | - Description string | ||
| 46 | - FeedType string | ||
| 47 | - Jaccard float64 | ||
| 48 | -} | ||
| 49 | - | ||
| 50 | -func (e *Engine) GetFeedRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) { | ||
| 51 | - rows, err := e.db.QueryContext(ctx, ` | ||
| 52 | - SELECT r.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''), | ||
| 53 | - COALESCE(f.description, ''), f.subscriber_count, COALESCE(f.favicon_url, ''), r.score | ||
| 54 | - FROM user_feed_recommendations r | ||
| 55 | - JOIN feeds f ON f.feed_url = r.feed_url | ||
| 56 | - WHERE r.user_did = ? | ||
| 57 | - AND r.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) | ||
| 58 | - ORDER BY r.score DESC | ||
| 59 | - LIMIT ? | ||
| 60 | - `, userDID, userDID, limit) | ||
| 61 | - if err != nil { | ||
| 62 | - return nil, err | ||
| 63 | - } | ||
| 64 | - defer rows.Close() | ||
| 65 | - | ||
| 66 | - var results []*FeedRecommendation | ||
| 67 | - for rows.Next() { | ||
| 68 | - rec := &FeedRecommendation{} | ||
| 69 | - if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description, | ||
| 70 | - &rec.SubscriberCount, &rec.FaviconURL, &rec.Score); err != nil { | ||
| 71 | - return nil, err | ||
| 72 | - } | ||
| 73 | - results = append(results, rec) | ||
| 74 | - } | ||
| 75 | - return results, rows.Err() | ||
| 76 | -} | ||
| 77 | - | ||
| 78 | -func (e *Engine) GetPeopleRecommendations(ctx context.Context, userDID string, limit int) ([]*PersonRecommendation, error) { | ||
| 79 | - rows, err := e.db.QueryContext(ctx, ` | ||
| 80 | - SELECT u.did, u.handle, COALESCE(u.display_name, ''), COALESCE(u.avatar_url, ''), | ||
| 81 | - sim.jaccard, sim.common_feeds, COALESCE(sim.common_likes, 0), COALESCE(sim.common_tags, 0) | ||
| 82 | - FROM ( | ||
| 83 | - SELECT user_b AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_a = ? | ||
| 84 | - UNION ALL | ||
| 85 | - SELECT user_a AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_b = ? | ||
| 86 | - ) sim | ||
| 87 | - JOIN users u ON u.did = sim.peer_did | ||
| 88 | - WHERE u.handle IS NOT NULL AND u.handle != '' | ||
| 89 | - ORDER BY sim.jaccard DESC | ||
| 90 | - LIMIT ? | ||
| 91 | - `, userDID, userDID, limit) | ||
| 92 | - if err != nil { | ||
| 93 | - return nil, err | ||
| 94 | - } | ||
| 95 | - defer rows.Close() | ||
| 96 | - | ||
| 97 | - var results []*PersonRecommendation | ||
| 98 | - for rows.Next() { | ||
| 99 | - rec := &PersonRecommendation{} | ||
| 100 | - if err := rows.Scan(&rec.DID, &rec.Handle, &rec.DisplayName, &rec.AvatarURL, | ||
| 101 | - &rec.Jaccard, &rec.CommonFeeds, &rec.CommonLikes, &rec.CommonTags); err != nil { | ||
| 102 | - return nil, err | ||
| 103 | - } | ||
| 104 | - results = append(results, rec) | ||
| 105 | - } | ||
| 106 | - return results, rows.Err() | ||
| 107 | -} | ||
| 108 | - | ||
| 109 | -func (e *Engine) GetArticleRecommendations(ctx context.Context, userDID string, limit int) ([]*ArticleRecommendation, error) { | ||
| 110 | - rows, err := e.db.QueryContext(ctx, ` | ||
| 111 | - SELECT a.id, a.title, COALESCE(a.url, ''), r.feed_url, COALESCE(f.title, ''), | ||
| 112 | - COALESCE(a.author, ''), COALESCE(a.summary, ''), a.published, r.score | ||
| 113 | - FROM user_article_recommendations r | ||
| 114 | - JOIN articles a ON a.feed_url = r.feed_url AND a.url = r.article_url | ||
| 115 | - LEFT JOIN feeds f ON f.feed_url = r.feed_url | ||
| 116 | - WHERE r.user_did = ? | ||
| 117 | - ORDER BY r.score DESC | ||
| 118 | - LIMIT ? | ||
| 119 | - `, userDID, limit) | ||
| 120 | - if err != nil { | ||
| 121 | - return nil, err | ||
| 122 | - } | ||
| 123 | - defer rows.Close() | ||
| 124 | - | ||
| 125 | - var recs []*ArticleRecommendation | ||
| 126 | - for rows.Next() { | ||
| 127 | - rec := &ArticleRecommendation{} | ||
| 128 | - if err := rows.Scan(&rec.ArticleID, &rec.Title, &rec.URL, &rec.FeedURL, &rec.FeedTitle, | ||
| 129 | - &rec.Author, &rec.Summary, &rec.Published, &rec.Score); err != nil { | ||
| 130 | - return nil, err | ||
| 131 | - } | ||
| 132 | - recs = append(recs, rec) | ||
| 133 | - } | ||
| 134 | - return recs, rows.Err() | ||
| 135 | -} | ||
| 136 | - | ||
| 137 | -func (e *Engine) GetSimilarFeeds(ctx context.Context, feedURL string, limit int) ([]*SimilarFeed, error) { | ||
| 138 | - rows, err := e.db.QueryContext(ctx, ` | ||
| 139 | - SELECT f.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''), | ||
| 140 | - COALESCE(f.description, ''), COALESCE(f.feed_type, ''), sim.jaccard | ||
| 141 | - FROM ( | ||
| 142 | - SELECT feed_b AS peer_url, jaccard FROM feed_similarity WHERE feed_a = ? | ||
| 143 | - UNION ALL | ||
| 144 | - SELECT feed_a AS peer_url, jaccard FROM feed_similarity WHERE feed_b = ? | ||
| 145 | - ) sim | ||
| 146 | - JOIN feeds f ON f.feed_url = sim.peer_url | ||
| 147 | - ORDER BY sim.jaccard DESC | ||
| 148 | - LIMIT ? | ||
| 149 | - `, feedURL, feedURL, limit) | ||
| 150 | - if err != nil { | ||
| 151 | - return nil, err | ||
| 152 | - } | ||
| 153 | - defer rows.Close() | ||
| 154 | - | ||
| 155 | - var results []*SimilarFeed | ||
| 156 | - for rows.Next() { | ||
| 157 | - rec := &SimilarFeed{} | ||
| 158 | - if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description, | ||
| 159 | - &rec.FeedType, &rec.Jaccard); err != nil { | ||
| 160 | - return nil, err | ||
| 161 | - } | ||
| 162 | - results = append(results, rec) | ||
| 163 | - } | ||
| 164 | - return results, rows.Err() | ||
| 165 | -} | ||
added
internal/cluster/scoring.go +376 -0 | new file mode 100644 | ||
| @@ -0,0 +1,376 @@ | ||
| 1 | +package cluster | |
| 2 | + | |
| 3 | +import ( | |
| 4 | + "context" | |
| 5 | + "database/sql" | |
| 6 | +) | |
| 7 | + | |
| 8 | +type FeedRecommendation struct { | |
| 9 | + FeedURL string | |
| 10 | + Title string | |
| 11 | + SiteURL string | |
| 12 | + Description string | |
| 13 | + SubscriberCount int | |
| 14 | + FaviconURL string | |
| 15 | + Score float64 | |
| 16 | +} | |
| 17 | + | |
| 18 | +type PersonRecommendation struct { | |
| 19 | + DID string | |
| 20 | + Handle string | |
| 21 | + DisplayName string | |
| 22 | + AvatarURL string | |
| 23 | + Jaccard float64 | |
| 24 | + CommonFeeds int | |
| 25 | + CommonLikes int | |
| 26 | + CommonTags int | |
| 27 | +} | |
| 28 | + | |
| 29 | +type ArticleRecommendation struct { | |
| 30 | + ArticleID int64 | |
| 31 | + Title string | |
| 32 | + URL string | |
| 33 | + FeedURL string | |
| 34 | + FeedTitle string | |
| 35 | + Author string | |
| 36 | + Summary string | |
| 37 | + Published sql.NullTime | |
| 38 | + Score float64 | |
| 39 | +} | |
| 40 | + | |
| 41 | +func (e *Engine) GetFeedRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) { | |
| 42 | + subCount := 0 | |
| 43 | + _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM subscriptions WHERE user_did = ?`, userDID).Scan(&subCount) | |
| 44 | + | |
| 45 | + if subCount < 5 { | |
| 46 | + recs, err := e.ColdStartRecommendations(ctx, userDID, limit*2) | |
| 47 | + if err == nil && len(recs) > 0 { | |
| 48 | + return ApplyDiversity(recs, limit), nil | |
| 49 | + } | |
| 50 | + } | |
| 51 | + | |
| 52 | + recs, err := e.ComputeFeedRecommendationsOnDemand(ctx, userDID, limit*2) | |
| 53 | + if err != nil { | |
| 54 | + return nil, err | |
| 55 | + } | |
| 56 | + | |
| 57 | + return ApplyDiversity(recs, limit), nil | |
| 58 | +} | |
| 59 | + | |
| 60 | +func (e *Engine) GetPeopleRecommendations(ctx context.Context, userDID string, limit int) ([]*PersonRecommendation, error) { | |
| 61 | + return e.ComputePeopleRecommendationsOnDemand(ctx, userDID, limit) | |
| 62 | +} | |
| 63 | + | |
| 64 | +func (e *Engine) GetArticleRecommendations(ctx context.Context, userDID string, limit int) ([]*ArticleRecommendation, error) { | |
| 65 | + return e.ComputeArticleRecommendationsOnDemand(ctx, userDID, limit) | |
| 66 | +} | |
| 67 | + | |
| 68 | +type SignalWeights struct { | |
| 69 | + WSub float64 | |
| 70 | + WLike float64 | |
| 71 | + WTag float64 | |
| 72 | + WSocial float64 | |
| 73 | + WPop float64 | |
| 74 | + WCategory float64 | |
| 75 | +} | |
| 76 | + | |
| 77 | +func defaultWeights() SignalWeights { | |
| 78 | + return SignalWeights{ | |
| 79 | + WSub: 1.0, | |
| 80 | + WLike: 0.5, | |
| 81 | + WTag: 0.3, | |
| 82 | + WSocial: 0.7, | |
| 83 | + WPop: 0.2, | |
| 84 | + WCategory: 0.4, | |
| 85 | + } | |
| 86 | +} | |
| 87 | + | |
| 88 | +func (e *Engine) GetWeights(ctx context.Context, userDID string) SignalWeights { | |
| 89 | + w := defaultWeights() | |
| 90 | + var dbW SignalWeights | |
| 91 | + err := e.db.QueryRowContext(ctx, ` | |
| 92 | + SELECT w_sub, w_like, w_tag, w_social, w_pop, w_category | |
| 93 | + FROM user_signal_weights WHERE user_did = ? | |
| 94 | + `, userDID).Scan(&dbW.WSub, &dbW.WLike, &dbW.WTag, &dbW.WSocial, &dbW.WPop, &dbW.WCategory) | |
| 95 | + if err == nil { | |
| 96 | + return dbW | |
| 97 | + } | |
| 98 | + return w | |
| 99 | +} | |
| 100 | + | |
| 101 | +func (e *Engine) ComputeFeedRecommendationsOnDemand(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) { | |
| 102 | + w := e.GetWeights(ctx, userDID) | |
| 103 | + | |
| 104 | + rows, err := e.db.QueryContext(ctx, ` | |
| 105 | + WITH similar_users AS ( | |
| 106 | + SELECT user_b AS peer, jaccard FROM user_similarity WHERE user_a = ? AND jaccard > 0.15 | |
| 107 | + UNION ALL | |
| 108 | + SELECT user_a AS peer, jaccard FROM user_similarity WHERE user_b = ? AND jaccard > 0.15 | |
| 109 | + ), | |
| 110 | + candidate_feeds AS ( | |
| 111 | + SELECT s.feed_url, | |
| 112 | + SUM(su.jaccard) AS sub_signal | |
| 113 | + FROM similar_users su | |
| 114 | + JOIN subscriptions s ON s.user_did = su.peer | |
| 115 | + WHERE s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) | |
| 116 | + AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed') | |
| 117 | + GROUP BY s.feed_url | |
| 118 | + ), | |
| 119 | + like_signals AS ( | |
| 120 | + SELECT s.feed_url, | |
| 121 | + SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal | |
| 122 | + FROM similar_users su | |
| 123 | + JOIN likes l ON l.author_did = su.peer | |
| 124 | + JOIN subscriptions s ON s.feed_url = l.feed_url | |
| 125 | + WHERE s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) | |
| 126 | + AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed') | |
| 127 | + GROUP BY s.feed_url | |
| 128 | + ), | |
| 129 | + social_boost AS ( | |
| 130 | + SELECT s.feed_url, | |
| 131 | + SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social | |
| 132 | + FROM follow_distances fd | |
| 133 | + JOIN subscriptions s ON s.user_did = fd.user_b | |
| 134 | + WHERE fd.user_a = ? | |
| 135 | + AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) | |
| 136 | + AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed') | |
| 137 | + GROUP BY s.feed_url | |
| 138 | + ), | |
| 139 | + category_counts AS ( | |
| 140 | + SELECT category, COUNT(*) AS cnt | |
| 141 | + FROM subscriptions WHERE user_did = ? AND category IS NOT NULL AND category != '' | |
| 142 | + GROUP BY category | |
| 143 | + ), | |
| 144 | + max_subs AS ( | |
| 145 | + SELECT CAST(COALESCE(MAX(subscriber_count), 1) AS REAL) AS m FROM feeds | |
| 146 | + ) | |
| 147 | + SELECT cf.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''), | |
| 148 | + COALESCE(f.description, ''), f.subscriber_count, COALESCE(f.favicon_url, ''), | |
| 149 | + COALESCE(cf.sub_signal, 0) * ? | |
| 150 | + + COALESCE(ls.like_signal, 0) * ? | |
| 151 | + + COALESCE(sb.social, 0) * ? | |
| 152 | + + COALESCE(LOG(1 + CAST(f.subscriber_count AS REAL)) / LOG(1 + ms.m), 0) * ? | |
| 153 | + + CASE WHEN f.description IS NOT NULL AND EXISTS ( | |
| 154 | + SELECT 1 FROM category_counts cc | |
| 155 | + WHERE LOWER(f.description) LIKE '%' || LOWER(cc.category) || '%' | |
| 156 | + ) THEN ? ELSE 0 END | |
| 157 | + AS score | |
| 158 | + FROM candidate_feeds cf | |
| 159 | + JOIN feeds f ON f.feed_url = cf.feed_url | |
| 160 | + LEFT JOIN like_signals ls ON ls.feed_url = cf.feed_url | |
| 161 | + LEFT JOIN social_boost sb ON sb.feed_url = cf.feed_url | |
| 162 | + CROSS JOIN max_subs ms | |
| 163 | + ORDER BY score DESC | |
| 164 | + LIMIT ? | |
| 165 | + `, userDID, userDID, userDID, userDID, userDID, userDID, userDID, userDID, userDID, userDID, | |
| 166 | + w.WSub, w.WLike, w.WSocial, w.WPop, w.WCategory, limit) | |
| 167 | + if err != nil { | |
| 168 | + return nil, err | |
| 169 | + } | |
| 170 | + defer rows.Close() | |
| 171 | + | |
| 172 | + var results []*FeedRecommendation | |
| 173 | + for rows.Next() { | |
| 174 | + rec := &FeedRecommendation{} | |
| 175 | + if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description, | |
| 176 | + &rec.SubscriberCount, &rec.FaviconURL, &rec.Score); err != nil { | |
| 177 | + return nil, err | |
| 178 | + } | |
| 179 | + results = append(results, rec) | |
| 180 | + } | |
| 181 | + return results, rows.Err() | |
| 182 | +} | |
| 183 | + | |
| 184 | +func (e *Engine) ComputeArticleRecommendationsOnDemand(ctx context.Context, userDID string, limit int) ([]*ArticleRecommendation, error) { | |
| 185 | + w := e.GetWeights(ctx, userDID) | |
| 186 | + | |
| 187 | + rows, err := e.db.QueryContext(ctx, ` | |
| 188 | + WITH similar_users AS ( | |
| 189 | + SELECT user_b AS peer, jaccard FROM user_similarity WHERE user_a = ? AND jaccard > 0.15 | |
| 190 | + UNION ALL | |
| 191 | + SELECT user_a AS peer, jaccard FROM user_similarity WHERE user_b = ? AND jaccard > 0.15 | |
| 192 | + ), | |
| 193 | + liked_articles AS ( | |
| 194 | + SELECT l.feed_url, l.article_url, | |
| 195 | + SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal | |
| 196 | + FROM similar_users su | |
| 197 | + JOIN likes l ON l.author_did = su.peer | |
| 198 | + WHERE NOT EXISTS ( | |
| 199 | + SELECT 1 FROM likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url | |
| 200 | + ) | |
| 201 | + AND NOT EXISTS ( | |
| 202 | + SELECT 1 FROM dismissed_recommendations d WHERE d.user_did = ? AND d.target_type = 'article' AND d.target_id = l.article_url | |
| 203 | + ) | |
| 204 | + GROUP BY l.feed_url, l.article_url | |
| 205 | + ), | |
| 206 | + social_likes AS ( | |
| 207 | + SELECT l.feed_url, l.article_url, | |
| 208 | + SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social | |
| 209 | + FROM follow_distances fd | |
| 210 | + JOIN likes l ON l.author_did = fd.user_b | |
| 211 | + WHERE fd.user_a = ? | |
| 212 | + AND NOT EXISTS ( | |
| 213 | + SELECT 1 FROM likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url | |
| 214 | + ) | |
| 215 | + GROUP BY l.feed_url, l.article_url | |
| 216 | + ) | |
| 217 | + SELECT a.id, a.title, COALESCE(a.url, ''), la.feed_url, COALESCE(f.title, ''), | |
| 218 | + COALESCE(a.author, ''), COALESCE(a.summary, ''), a.published, | |
| 219 | + COALESCE(la.like_signal, 0) * ? | |
| 220 | + + COALESCE(sl.social, 0) * ? | |
| 221 | + + EXP(-0.023 * CAST(julianday('now') - julianday(a.published) AS REAL)) * 0.2 | |
| 222 | + AS score | |
| 223 | + FROM liked_articles la | |
| 224 | + JOIN articles a ON a.feed_url = la.feed_url AND a.url = la.article_url | |
| 225 | + LEFT JOIN feeds f ON f.feed_url = la.feed_url | |
| 226 | + LEFT JOIN social_likes sl ON sl.feed_url = la.feed_url AND sl.article_url = la.article_url | |
| 227 | + ORDER BY score DESC | |
| 228 | + LIMIT ? | |
| 229 | + `, userDID, userDID, userDID, userDID, userDID, userDID, w.WLike, w.WSocial, limit) | |
| 230 | + if err != nil { | |
| 231 | + return nil, err | |
| 232 | + } | |
| 233 | + defer rows.Close() | |
| 234 | + | |
| 235 | + var recs []*ArticleRecommendation | |
| 236 | + for rows.Next() { | |
| 237 | + rec := &ArticleRecommendation{} | |
| 238 | + if err := rows.Scan(&rec.ArticleID, &rec.Title, &rec.URL, &rec.FeedURL, &rec.FeedTitle, | |
| 239 | + &rec.Author, &rec.Summary, &rec.Published, &rec.Score); err != nil { | |
| 240 | + return nil, err | |
| 241 | + } | |
| 242 | + recs = append(recs, rec) | |
| 243 | + } | |
| 244 | + return recs, rows.Err() | |
| 245 | +} | |
| 246 | + | |
| 247 | +func (e *Engine) ComputePeopleRecommendationsOnDemand(ctx context.Context, userDID string, limit int) ([]*PersonRecommendation, error) { | |
| 248 | + rows, err := e.db.QueryContext(ctx, ` | |
| 249 | + SELECT u.did, u.handle, COALESCE(u.display_name, ''), COALESCE(u.avatar_url, ''), | |
| 250 | + sim.jaccard, sim.common_feeds, COALESCE(sim.common_likes, 0), COALESCE(sim.common_tags, 0) | |
| 251 | + FROM ( | |
| 252 | + SELECT user_b AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_a = ? | |
| 253 | + UNION ALL | |
| 254 | + SELECT user_a AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_b = ? | |
| 255 | + ) sim | |
| 256 | + JOIN users u ON u.did = sim.peer_did | |
| 257 | + WHERE u.handle IS NOT NULL AND u.handle != '' | |
| 258 | + ORDER BY sim.jaccard DESC | |
| 259 | + LIMIT ? | |
| 260 | + `, userDID, userDID, limit) | |
| 261 | + if err != nil { | |
| 262 | + return nil, err | |
| 263 | + } | |
| 264 | + defer rows.Close() | |
| 265 | + | |
| 266 | + var results []*PersonRecommendation | |
| 267 | + for rows.Next() { | |
| 268 | + rec := &PersonRecommendation{} | |
| 269 | + if err := rows.Scan(&rec.DID, &rec.Handle, &rec.DisplayName, &rec.AvatarURL, | |
| 270 | + &rec.Jaccard, &rec.CommonFeeds, &rec.CommonLikes, &rec.CommonTags); err != nil { | |
| 271 | + return nil, err | |
| 272 | + } | |
| 273 | + results = append(results, rec) | |
| 274 | + } | |
| 275 | + return results, rows.Err() | |
| 276 | +} | |
| 277 | + | |
| 278 | +func (e *Engine) ComputeSignalProfiles(ctx context.Context) error { | |
| 279 | + tx, err := e.db.BeginTx(ctx, nil) | |
| 280 | + if err != nil { | |
| 281 | + return err | |
| 282 | + } | |
| 283 | + defer func() { _ = tx.Rollback() }() | |
| 284 | + | |
| 285 | + if _, err := tx.ExecContext(ctx, `DELETE FROM user_signal_profiles`); err != nil { | |
| 286 | + return err | |
| 287 | + } | |
| 288 | + | |
| 289 | + _, err = tx.ExecContext(ctx, ` | |
| 290 | + INSERT INTO user_signal_profiles (user_did, total_likes, total_tags, top_categories) | |
| 291 | + SELECT | |
| 292 | + u.did, | |
| 293 | + (SELECT COUNT(*) FROM likes WHERE author_did = u.did), | |
| 294 | + COALESCE((SELECT COUNT(DISTINCT TRIM(value)) | |
| 295 | + FROM annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]') | |
| 296 | + WHERE author_did = u.did AND tags IS NOT NULL AND tags != '' | |
| 297 | + ), 0), | |
| 298 | + (SELECT '[' || GROUP_CONCAT('{"c":"' || category || '","n":"' || CAST(cnt AS TEXT) || '}') || ']' | |
| 299 | + FROM ( | |
| 300 | + SELECT category, COUNT(*) AS cnt | |
| 301 | + FROM subscriptions WHERE user_did = u.did AND category IS NOT NULL AND category != '' | |
| 302 | + GROUP BY category ORDER BY cnt DESC LIMIT 5 | |
| 303 | + ) | |
| 304 | + ) | |
| 305 | + FROM users u | |
| 306 | + `) | |
| 307 | + if err != nil { | |
| 308 | + return err | |
| 309 | + } | |
| 310 | + | |
| 311 | + e.logger.Info("signal profiles computed") | |
| 312 | + return tx.Commit() | |
| 313 | +} | |
| 314 | + | |
| 315 | +func (e *Engine) ColdStartRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) { | |
| 316 | + subCount := 0 | |
| 317 | + _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM subscriptions WHERE user_did = ?`, userDID).Scan(&subCount) | |
| 318 | + if subCount >= 5 { | |
| 319 | + return nil, nil | |
| 320 | + } | |
| 321 | + | |
| 322 | + rows, err := e.db.QueryContext(ctx, ` | |
| 323 | + WITH followed_feeds AS ( | |
| 324 | + SELECT s.feed_url, 1.0 AS weight | |
| 325 | + FROM follow_distances fd | |
| 326 | + JOIN subscriptions s ON s.user_did = fd.user_b | |
| 327 | + WHERE fd.user_a = ? AND fd.distance = 1 | |
| 328 | + AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) | |
| 329 | + AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed') | |
| 330 | + ), | |
| 331 | + popular_feeds AS ( | |
| 332 | + SELECT feed_url, subscriber_count, | |
| 333 | + LOG(1 + CAST(subscriber_count AS REAL)) / LOG(1 + CAST((SELECT COALESCE(MAX(subscriber_count), 1) FROM feeds) AS REAL)) AS pop_score | |
| 334 | + FROM feeds | |
| 335 | + WHERE subscriber_count > 0 | |
| 336 | + AND feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) | |
| 337 | + AND feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed') | |
| 338 | + ORDER BY subscriber_count DESC | |
| 339 | + LIMIT 50 | |
| 340 | + ), | |
| 341 | + all_candidates AS ( | |
| 342 | + SELECT feed_url, MAX(weight) AS weight FROM ( | |
| 343 | + SELECT feed_url, weight FROM followed_feeds | |
| 344 | + UNION ALL | |
| 345 | + SELECT feed_url, pop_score AS weight FROM popular_feeds | |
| 346 | + ) | |
| 347 | + GROUP BY feed_url | |
| 348 | + ) | |
| 349 | + SELECT ac.feed_url, | |
| 350 | + COALESCE(f.title, ''), | |
| 351 | + COALESCE(f.site_url, ''), | |
| 352 | + COALESCE(f.description, ''), | |
| 353 | + f.subscriber_count, | |
| 354 | + COALESCE(f.favicon_url, ''), | |
| 355 | + ac.weight AS score | |
| 356 | + FROM all_candidates ac | |
| 357 | + JOIN feeds f ON f.feed_url = ac.feed_url | |
| 358 | + ORDER BY score DESC | |
| 359 | + LIMIT ? | |
| 360 | + `, userDID, userDID, userDID, userDID, userDID, limit) | |
| 361 | + if err != nil { | |
| 362 | + return nil, err | |
| 363 | + } | |
| 364 | + defer rows.Close() | |
| 365 | + | |
| 366 | + var results []*FeedRecommendation | |
| 367 | + for rows.Next() { | |
| 368 | + rec := &FeedRecommendation{} | |
| 369 | + if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description, | |
| 370 | + &rec.SubscriberCount, &rec.FaviconURL, &rec.Score); err != nil { | |
| 371 | + return nil, err | |
| 372 | + } | |
| 373 | + results = append(results, rec) | |
| 374 | + } | |
| 375 | + return results, rows.Err() | |
| 376 | +} | |
| new file mode 100644 | |||
| @@ -0,0 +1,376 @@ | |||
| 1 | +package cluster | ||
| 2 | + | ||
| 3 | +import ( | ||
| 4 | + "context" | ||
| 5 | + "database/sql" | ||
| 6 | +) | ||
| 7 | + | ||
| 8 | +type FeedRecommendation struct { | ||
| 9 | + FeedURL string | ||
| 10 | + Title string | ||
| 11 | + SiteURL string | ||
| 12 | + Description string | ||
| 13 | + SubscriberCount int | ||
| 14 | + FaviconURL string | ||
| 15 | + Score float64 | ||
| 16 | +} | ||
| 17 | + | ||
| 18 | +type PersonRecommendation struct { | ||
| 19 | + DID string | ||
| 20 | + Handle string | ||
| 21 | + DisplayName string | ||
| 22 | + AvatarURL string | ||
| 23 | + Jaccard float64 | ||
| 24 | + CommonFeeds int | ||
| 25 | + CommonLikes int | ||
| 26 | + CommonTags int | ||
| 27 | +} | ||
| 28 | + | ||
| 29 | +type ArticleRecommendation struct { | ||
| 30 | + ArticleID int64 | ||
| 31 | + Title string | ||
| 32 | + URL string | ||
| 33 | + FeedURL string | ||
| 34 | + FeedTitle string | ||
| 35 | + Author string | ||
| 36 | + Summary string | ||
| 37 | + Published sql.NullTime | ||
| 38 | + Score float64 | ||
| 39 | +} | ||
| 40 | + | ||
| 41 | +func (e *Engine) GetFeedRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) { | ||
| 42 | + subCount := 0 | ||
| 43 | + _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM subscriptions WHERE user_did = ?`, userDID).Scan(&subCount) | ||
| 44 | + | ||
| 45 | + if subCount < 5 { | ||
| 46 | + recs, err := e.ColdStartRecommendations(ctx, userDID, limit*2) | ||
| 47 | + if err == nil && len(recs) > 0 { | ||
| 48 | + return ApplyDiversity(recs, limit), nil | ||
| 49 | + } | ||
| 50 | + } | ||
| 51 | + | ||
| 52 | + recs, err := e.ComputeFeedRecommendationsOnDemand(ctx, userDID, limit*2) | ||
| 53 | + if err != nil { | ||
| 54 | + return nil, err | ||
| 55 | + } | ||
| 56 | + | ||
| 57 | + return ApplyDiversity(recs, limit), nil | ||
| 58 | +} | ||
| 59 | + | ||
| 60 | +func (e *Engine) GetPeopleRecommendations(ctx context.Context, userDID string, limit int) ([]*PersonRecommendation, error) { | ||
| 61 | + return e.ComputePeopleRecommendationsOnDemand(ctx, userDID, limit) | ||
| 62 | +} | ||
| 63 | + | ||
| 64 | +func (e *Engine) GetArticleRecommendations(ctx context.Context, userDID string, limit int) ([]*ArticleRecommendation, error) { | ||
| 65 | + return e.ComputeArticleRecommendationsOnDemand(ctx, userDID, limit) | ||
| 66 | +} | ||
| 67 | + | ||
| 68 | +type SignalWeights struct { | ||
| 69 | + WSub float64 | ||
| 70 | + WLike float64 | ||
| 71 | + WTag float64 | ||
| 72 | + WSocial float64 | ||
| 73 | + WPop float64 | ||
| 74 | + WCategory float64 | ||
| 75 | +} | ||
| 76 | + | ||
| 77 | +func defaultWeights() SignalWeights { | ||
| 78 | + return SignalWeights{ | ||
| 79 | + WSub: 1.0, | ||
| 80 | + WLike: 0.5, | ||
| 81 | + WTag: 0.3, | ||
| 82 | + WSocial: 0.7, | ||
| 83 | + WPop: 0.2, | ||
| 84 | + WCategory: 0.4, | ||
| 85 | + } | ||
| 86 | +} | ||
| 87 | + | ||
| 88 | +func (e *Engine) GetWeights(ctx context.Context, userDID string) SignalWeights { | ||
| 89 | + w := defaultWeights() | ||
| 90 | + var dbW SignalWeights | ||
| 91 | + err := e.db.QueryRowContext(ctx, ` | ||
| 92 | + SELECT w_sub, w_like, w_tag, w_social, w_pop, w_category | ||
| 93 | + FROM user_signal_weights WHERE user_did = ? | ||
| 94 | + `, userDID).Scan(&dbW.WSub, &dbW.WLike, &dbW.WTag, &dbW.WSocial, &dbW.WPop, &dbW.WCategory) | ||
| 95 | + if err == nil { | ||
| 96 | + return dbW | ||
| 97 | + } | ||
| 98 | + return w | ||
| 99 | +} | ||
| 100 | + | ||
| 101 | +func (e *Engine) ComputeFeedRecommendationsOnDemand(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) { | ||
| 102 | + w := e.GetWeights(ctx, userDID) | ||
| 103 | + | ||
| 104 | + rows, err := e.db.QueryContext(ctx, ` | ||
| 105 | + WITH similar_users AS ( | ||
| 106 | + SELECT user_b AS peer, jaccard FROM user_similarity WHERE user_a = ? AND jaccard > 0.15 | ||
| 107 | + UNION ALL | ||
| 108 | + SELECT user_a AS peer, jaccard FROM user_similarity WHERE user_b = ? AND jaccard > 0.15 | ||
| 109 | + ), | ||
| 110 | + candidate_feeds AS ( | ||
| 111 | + SELECT s.feed_url, | ||
| 112 | + SUM(su.jaccard) AS sub_signal | ||
| 113 | + FROM similar_users su | ||
| 114 | + JOIN subscriptions s ON s.user_did = su.peer | ||
| 115 | + WHERE s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) | ||
| 116 | + AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed') | ||
| 117 | + GROUP BY s.feed_url | ||
| 118 | + ), | ||
| 119 | + like_signals AS ( | ||
| 120 | + SELECT s.feed_url, | ||
| 121 | + SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal | ||
| 122 | + FROM similar_users su | ||
| 123 | + JOIN likes l ON l.author_did = su.peer | ||
| 124 | + JOIN subscriptions s ON s.feed_url = l.feed_url | ||
| 125 | + WHERE s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) | ||
| 126 | + AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed') | ||
| 127 | + GROUP BY s.feed_url | ||
| 128 | + ), | ||
| 129 | + social_boost AS ( | ||
| 130 | + SELECT s.feed_url, | ||
| 131 | + SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social | ||
| 132 | + FROM follow_distances fd | ||
| 133 | + JOIN subscriptions s ON s.user_did = fd.user_b | ||
| 134 | + WHERE fd.user_a = ? | ||
| 135 | + AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) | ||
| 136 | + AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed') | ||
| 137 | + GROUP BY s.feed_url | ||
| 138 | + ), | ||
| 139 | + category_counts AS ( | ||
| 140 | + SELECT category, COUNT(*) AS cnt | ||
| 141 | + FROM subscriptions WHERE user_did = ? AND category IS NOT NULL AND category != '' | ||
| 142 | + GROUP BY category | ||
| 143 | + ), | ||
| 144 | + max_subs AS ( | ||
| 145 | + SELECT CAST(COALESCE(MAX(subscriber_count), 1) AS REAL) AS m FROM feeds | ||
| 146 | + ) | ||
| 147 | + SELECT cf.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''), | ||
| 148 | + COALESCE(f.description, ''), f.subscriber_count, COALESCE(f.favicon_url, ''), | ||
| 149 | + COALESCE(cf.sub_signal, 0) * ? | ||
| 150 | + + COALESCE(ls.like_signal, 0) * ? | ||
| 151 | + + COALESCE(sb.social, 0) * ? | ||
| 152 | + + COALESCE(LOG(1 + CAST(f.subscriber_count AS REAL)) / LOG(1 + ms.m), 0) * ? | ||
| 153 | + + CASE WHEN f.description IS NOT NULL AND EXISTS ( | ||
| 154 | + SELECT 1 FROM category_counts cc | ||
| 155 | + WHERE LOWER(f.description) LIKE '%' || LOWER(cc.category) || '%' | ||
| 156 | + ) THEN ? ELSE 0 END | ||
| 157 | + AS score | ||
| 158 | + FROM candidate_feeds cf | ||
| 159 | + JOIN feeds f ON f.feed_url = cf.feed_url | ||
| 160 | + LEFT JOIN like_signals ls ON ls.feed_url = cf.feed_url | ||
| 161 | + LEFT JOIN social_boost sb ON sb.feed_url = cf.feed_url | ||
| 162 | + CROSS JOIN max_subs ms | ||
| 163 | + ORDER BY score DESC | ||
| 164 | + LIMIT ? | ||
| 165 | + `, userDID, userDID, userDID, userDID, userDID, userDID, userDID, userDID, userDID, userDID, | ||
| 166 | + w.WSub, w.WLike, w.WSocial, w.WPop, w.WCategory, limit) | ||
| 167 | + if err != nil { | ||
| 168 | + return nil, err | ||
| 169 | + } | ||
| 170 | + defer rows.Close() | ||
| 171 | + | ||
| 172 | + var results []*FeedRecommendation | ||
| 173 | + for rows.Next() { | ||
| 174 | + rec := &FeedRecommendation{} | ||
| 175 | + if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description, | ||
| 176 | + &rec.SubscriberCount, &rec.FaviconURL, &rec.Score); err != nil { | ||
| 177 | + return nil, err | ||
| 178 | + } | ||
| 179 | + results = append(results, rec) | ||
| 180 | + } | ||
| 181 | + return results, rows.Err() | ||
| 182 | +} | ||
| 183 | + | ||
| 184 | +func (e *Engine) ComputeArticleRecommendationsOnDemand(ctx context.Context, userDID string, limit int) ([]*ArticleRecommendation, error) { | ||
| 185 | + w := e.GetWeights(ctx, userDID) | ||
| 186 | + | ||
| 187 | + rows, err := e.db.QueryContext(ctx, ` | ||
| 188 | + WITH similar_users AS ( | ||
| 189 | + SELECT user_b AS peer, jaccard FROM user_similarity WHERE user_a = ? AND jaccard > 0.15 | ||
| 190 | + UNION ALL | ||
| 191 | + SELECT user_a AS peer, jaccard FROM user_similarity WHERE user_b = ? AND jaccard > 0.15 | ||
| 192 | + ), | ||
| 193 | + liked_articles AS ( | ||
| 194 | + SELECT l.feed_url, l.article_url, | ||
| 195 | + SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal | ||
| 196 | + FROM similar_users su | ||
| 197 | + JOIN likes l ON l.author_did = su.peer | ||
| 198 | + WHERE NOT EXISTS ( | ||
| 199 | + SELECT 1 FROM likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url | ||
| 200 | + ) | ||
| 201 | + AND NOT EXISTS ( | ||
| 202 | + SELECT 1 FROM dismissed_recommendations d WHERE d.user_did = ? AND d.target_type = 'article' AND d.target_id = l.article_url | ||
| 203 | + ) | ||
| 204 | + GROUP BY l.feed_url, l.article_url | ||
| 205 | + ), | ||
| 206 | + social_likes AS ( | ||
| 207 | + SELECT l.feed_url, l.article_url, | ||
| 208 | + SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social | ||
| 209 | + FROM follow_distances fd | ||
| 210 | + JOIN likes l ON l.author_did = fd.user_b | ||
| 211 | + WHERE fd.user_a = ? | ||
| 212 | + AND NOT EXISTS ( | ||
| 213 | + SELECT 1 FROM likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url | ||
| 214 | + ) | ||
| 215 | + GROUP BY l.feed_url, l.article_url | ||
| 216 | + ) | ||
| 217 | + SELECT a.id, a.title, COALESCE(a.url, ''), la.feed_url, COALESCE(f.title, ''), | ||
| 218 | + COALESCE(a.author, ''), COALESCE(a.summary, ''), a.published, | ||
| 219 | + COALESCE(la.like_signal, 0) * ? | ||
| 220 | + + COALESCE(sl.social, 0) * ? | ||
| 221 | + + EXP(-0.023 * CAST(julianday('now') - julianday(a.published) AS REAL)) * 0.2 | ||
| 222 | + AS score | ||
| 223 | + FROM liked_articles la | ||
| 224 | + JOIN articles a ON a.feed_url = la.feed_url AND a.url = la.article_url | ||
| 225 | + LEFT JOIN feeds f ON f.feed_url = la.feed_url | ||
| 226 | + LEFT JOIN social_likes sl ON sl.feed_url = la.feed_url AND sl.article_url = la.article_url | ||
| 227 | + ORDER BY score DESC | ||
| 228 | + LIMIT ? | ||
| 229 | + `, userDID, userDID, userDID, userDID, userDID, userDID, w.WLike, w.WSocial, limit) | ||
| 230 | + if err != nil { | ||
| 231 | + return nil, err | ||
| 232 | + } | ||
| 233 | + defer rows.Close() | ||
| 234 | + | ||
| 235 | + var recs []*ArticleRecommendation | ||
| 236 | + for rows.Next() { | ||
| 237 | + rec := &ArticleRecommendation{} | ||
| 238 | + if err := rows.Scan(&rec.ArticleID, &rec.Title, &rec.URL, &rec.FeedURL, &rec.FeedTitle, | ||
| 239 | + &rec.Author, &rec.Summary, &rec.Published, &rec.Score); err != nil { | ||
| 240 | + return nil, err | ||
| 241 | + } | ||
| 242 | + recs = append(recs, rec) | ||
| 243 | + } | ||
| 244 | + return recs, rows.Err() | ||
| 245 | +} | ||
| 246 | + | ||
| 247 | +func (e *Engine) ComputePeopleRecommendationsOnDemand(ctx context.Context, userDID string, limit int) ([]*PersonRecommendation, error) { | ||
| 248 | + rows, err := e.db.QueryContext(ctx, ` | ||
| 249 | + SELECT u.did, u.handle, COALESCE(u.display_name, ''), COALESCE(u.avatar_url, ''), | ||
| 250 | + sim.jaccard, sim.common_feeds, COALESCE(sim.common_likes, 0), COALESCE(sim.common_tags, 0) | ||
| 251 | + FROM ( | ||
| 252 | + SELECT user_b AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_a = ? | ||
| 253 | + UNION ALL | ||
| 254 | + SELECT user_a AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_b = ? | ||
| 255 | + ) sim | ||
| 256 | + JOIN users u ON u.did = sim.peer_did | ||
| 257 | + WHERE u.handle IS NOT NULL AND u.handle != '' | ||
| 258 | + ORDER BY sim.jaccard DESC | ||
| 259 | + LIMIT ? | ||
| 260 | + `, userDID, userDID, limit) | ||
| 261 | + if err != nil { | ||
| 262 | + return nil, err | ||
| 263 | + } | ||
| 264 | + defer rows.Close() | ||
| 265 | + | ||
| 266 | + var results []*PersonRecommendation | ||
| 267 | + for rows.Next() { | ||
| 268 | + rec := &PersonRecommendation{} | ||
| 269 | + if err := rows.Scan(&rec.DID, &rec.Handle, &rec.DisplayName, &rec.AvatarURL, | ||
| 270 | + &rec.Jaccard, &rec.CommonFeeds, &rec.CommonLikes, &rec.CommonTags); err != nil { | ||
| 271 | + return nil, err | ||
| 272 | + } | ||
| 273 | + results = append(results, rec) | ||
| 274 | + } | ||
| 275 | + return results, rows.Err() | ||
| 276 | +} | ||
| 277 | + | ||
| 278 | +func (e *Engine) ComputeSignalProfiles(ctx context.Context) error { | ||
| 279 | + tx, err := e.db.BeginTx(ctx, nil) | ||
| 280 | + if err != nil { | ||
| 281 | + return err | ||
| 282 | + } | ||
| 283 | + defer func() { _ = tx.Rollback() }() | ||
| 284 | + | ||
| 285 | + if _, err := tx.ExecContext(ctx, `DELETE FROM user_signal_profiles`); err != nil { | ||
| 286 | + return err | ||
| 287 | + } | ||
| 288 | + | ||
| 289 | + _, err = tx.ExecContext(ctx, ` | ||
| 290 | + INSERT INTO user_signal_profiles (user_did, total_likes, total_tags, top_categories) | ||
| 291 | + SELECT | ||
| 292 | + u.did, | ||
| 293 | + (SELECT COUNT(*) FROM likes WHERE author_did = u.did), | ||
| 294 | + COALESCE((SELECT COUNT(DISTINCT TRIM(value)) | ||
| 295 | + FROM annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]') | ||
| 296 | + WHERE author_did = u.did AND tags IS NOT NULL AND tags != '' | ||
| 297 | + ), 0), | ||
| 298 | + (SELECT '[' || GROUP_CONCAT('{"c":"' || category || '","n":"' || CAST(cnt AS TEXT) || '}') || ']' | ||
| 299 | + FROM ( | ||
| 300 | + SELECT category, COUNT(*) AS cnt | ||
| 301 | + FROM subscriptions WHERE user_did = u.did AND category IS NOT NULL AND category != '' | ||
| 302 | + GROUP BY category ORDER BY cnt DESC LIMIT 5 | ||
| 303 | + ) | ||
| 304 | + ) | ||
| 305 | + FROM users u | ||
| 306 | + `) | ||
| 307 | + if err != nil { | ||
| 308 | + return err | ||
| 309 | + } | ||
| 310 | + | ||
| 311 | + e.logger.Info("signal profiles computed") | ||
| 312 | + return tx.Commit() | ||
| 313 | +} | ||
| 314 | + | ||
| 315 | +func (e *Engine) ColdStartRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) { | ||
| 316 | + subCount := 0 | ||
| 317 | + _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM subscriptions WHERE user_did = ?`, userDID).Scan(&subCount) | ||
| 318 | + if subCount >= 5 { | ||
| 319 | + return nil, nil | ||
| 320 | + } | ||
| 321 | + | ||
| 322 | + rows, err := e.db.QueryContext(ctx, ` | ||
| 323 | + WITH followed_feeds AS ( | ||
| 324 | + SELECT s.feed_url, 1.0 AS weight | ||
| 325 | + FROM follow_distances fd | ||
| 326 | + JOIN subscriptions s ON s.user_did = fd.user_b | ||
| 327 | + WHERE fd.user_a = ? AND fd.distance = 1 | ||
| 328 | + AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) | ||
| 329 | + AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed') | ||
| 330 | + ), | ||
| 331 | + popular_feeds AS ( | ||
| 332 | + SELECT feed_url, subscriber_count, | ||
| 333 | + LOG(1 + CAST(subscriber_count AS REAL)) / LOG(1 + CAST((SELECT COALESCE(MAX(subscriber_count), 1) FROM feeds) AS REAL)) AS pop_score | ||
| 334 | + FROM feeds | ||
| 335 | + WHERE subscriber_count > 0 | ||
| 336 | + AND feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) | ||
| 337 | + AND feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed') | ||
| 338 | + ORDER BY subscriber_count DESC | ||
| 339 | + LIMIT 50 | ||
| 340 | + ), | ||
| 341 | + all_candidates AS ( | ||
| 342 | + SELECT feed_url, MAX(weight) AS weight FROM ( | ||
| 343 | + SELECT feed_url, weight FROM followed_feeds | ||
| 344 | + UNION ALL | ||
| 345 | + SELECT feed_url, pop_score AS weight FROM popular_feeds | ||
| 346 | + ) | ||
| 347 | + GROUP BY feed_url | ||
| 348 | + ) | ||
| 349 | + SELECT ac.feed_url, | ||
| 350 | + COALESCE(f.title, ''), | ||
| 351 | + COALESCE(f.site_url, ''), | ||
| 352 | + COALESCE(f.description, ''), | ||
| 353 | + f.subscriber_count, | ||
| 354 | + COALESCE(f.favicon_url, ''), | ||
| 355 | + ac.weight AS score | ||
| 356 | + FROM all_candidates ac | ||
| 357 | + JOIN feeds f ON f.feed_url = ac.feed_url | ||
| 358 | + ORDER BY score DESC | ||
| 359 | + LIMIT ? | ||
| 360 | + `, userDID, userDID, userDID, userDID, userDID, limit) | ||
| 361 | + if err != nil { | ||
| 362 | + return nil, err | ||
| 363 | + } | ||
| 364 | + defer rows.Close() | ||
| 365 | + | ||
| 366 | + var results []*FeedRecommendation | ||
| 367 | + for rows.Next() { | ||
| 368 | + rec := &FeedRecommendation{} | ||
| 369 | + if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description, | ||
| 370 | + &rec.SubscriberCount, &rec.FaviconURL, &rec.Score); err != nil { | ||
| 371 | + return nil, err | ||
| 372 | + } | ||
| 373 | + results = append(results, rec) | ||
| 374 | + } | ||
| 375 | + return results, rows.Err() | ||
| 376 | +} | ||
added
internal/cluster/social.go +63 -0 | new file mode 100644 | ||
| @@ -0,0 +1,63 @@ | ||
| 1 | +package cluster | |
| 2 | + | |
| 3 | +import ( | |
| 4 | + "context" | |
| 5 | +) | |
| 6 | + | |
| 7 | +func (e *Engine) ComputeFollowDistances(ctx context.Context) error { | |
| 8 | + tx, err := e.db.BeginTx(ctx, nil) | |
| 9 | + if err != nil { | |
| 10 | + return err | |
| 11 | + } | |
| 12 | + defer func() { _ = tx.Rollback() }() | |
| 13 | + | |
| 14 | + if _, err := tx.ExecContext(ctx, `DELETE FROM follow_distances`); err != nil { | |
| 15 | + return err | |
| 16 | + } | |
| 17 | + | |
| 18 | + if _, err := tx.ExecContext(ctx, ` | |
| 19 | + INSERT INTO follow_distances (user_a, user_b, distance) | |
| 20 | + SELECT user_did, target_did, 1 | |
| 21 | + FROM follows | |
| 22 | + WHERE user_did != target_did | |
| 23 | + `); err != nil { | |
| 24 | + return err | |
| 25 | + } | |
| 26 | + | |
| 27 | + if _, err := tx.ExecContext(ctx, ` | |
| 28 | + INSERT OR IGNORE INTO follow_distances (user_a, user_b, distance) | |
| 29 | + SELECT f1.user_did, f2.target_did, 2 | |
| 30 | + FROM follows f1 | |
| 31 | + JOIN follows f2 ON f1.target_did = f2.user_did | |
| 32 | + WHERE f1.user_did != f2.target_did | |
| 33 | + `); err != nil { | |
| 34 | + return err | |
| 35 | + } | |
| 36 | + | |
| 37 | + e.logger.Info("follow distances computed") | |
| 38 | + return tx.Commit() | |
| 39 | +} | |
| 40 | + | |
| 41 | +func (e *Engine) ComputeFollowDistancesIncremental(ctx context.Context) error { | |
| 42 | + var maxFollowed string | |
| 43 | + err := e.db.QueryRowContext(ctx, ` | |
| 44 | + SELECT COALESCE(MAX(followed_at), '1970-01-01') FROM follows | |
| 45 | + `).Scan(&maxFollowed) | |
| 46 | + if err != nil { | |
| 47 | + return err | |
| 48 | + } | |
| 49 | + | |
| 50 | + var lastComputed string | |
| 51 | + err = e.db.QueryRowContext(ctx, ` | |
| 52 | + SELECT COALESCE(MAX(updated_at), '1970-01-01') FROM user_similarity | |
| 53 | + `).Scan(&lastComputed) | |
| 54 | + if err != nil { | |
| 55 | + return err | |
| 56 | + } | |
| 57 | + | |
| 58 | + if maxFollowed <= lastComputed { | |
| 59 | + return nil | |
| 60 | + } | |
| 61 | + | |
| 62 | + return e.ComputeFollowDistances(ctx) | |
| 63 | +} | |
| new file mode 100644 | |||
| @@ -0,0 +1,63 @@ | |||
| 1 | +package cluster | ||
| 2 | + | ||
| 3 | +import ( | ||
| 4 | + "context" | ||
| 5 | +) | ||
| 6 | + | ||
| 7 | +func (e *Engine) ComputeFollowDistances(ctx context.Context) error { | ||
| 8 | + tx, err := e.db.BeginTx(ctx, nil) | ||
| 9 | + if err != nil { | ||
| 10 | + return err | ||
| 11 | + } | ||
| 12 | + defer func() { _ = tx.Rollback() }() | ||
| 13 | + | ||
| 14 | + if _, err := tx.ExecContext(ctx, `DELETE FROM follow_distances`); err != nil { | ||
| 15 | + return err | ||
| 16 | + } | ||
| 17 | + | ||
| 18 | + if _, err := tx.ExecContext(ctx, ` | ||
| 19 | + INSERT INTO follow_distances (user_a, user_b, distance) | ||
| 20 | + SELECT user_did, target_did, 1 | ||
| 21 | + FROM follows | ||
| 22 | + WHERE user_did != target_did | ||
| 23 | + `); err != nil { | ||
| 24 | + return err | ||
| 25 | + } | ||
| 26 | + | ||
| 27 | + if _, err := tx.ExecContext(ctx, ` | ||
| 28 | + INSERT OR IGNORE INTO follow_distances (user_a, user_b, distance) | ||
| 29 | + SELECT f1.user_did, f2.target_did, 2 | ||
| 30 | + FROM follows f1 | ||
| 31 | + JOIN follows f2 ON f1.target_did = f2.user_did | ||
| 32 | + WHERE f1.user_did != f2.target_did | ||
| 33 | + `); err != nil { | ||
| 34 | + return err | ||
| 35 | + } | ||
| 36 | + | ||
| 37 | + e.logger.Info("follow distances computed") | ||
| 38 | + return tx.Commit() | ||
| 39 | +} | ||
| 40 | + | ||
| 41 | +func (e *Engine) ComputeFollowDistancesIncremental(ctx context.Context) error { | ||
| 42 | + var maxFollowed string | ||
| 43 | + err := e.db.QueryRowContext(ctx, ` | ||
| 44 | + SELECT COALESCE(MAX(followed_at), '1970-01-01') FROM follows | ||
| 45 | + `).Scan(&maxFollowed) | ||
| 46 | + if err != nil { | ||
| 47 | + return err | ||
| 48 | + } | ||
| 49 | + | ||
| 50 | + var lastComputed string | ||
| 51 | + err = e.db.QueryRowContext(ctx, ` | ||
| 52 | + SELECT COALESCE(MAX(updated_at), '1970-01-01') FROM user_similarity | ||
| 53 | + `).Scan(&lastComputed) | ||
| 54 | + if err != nil { | ||
| 55 | + return err | ||
| 56 | + } | ||
| 57 | + | ||
| 58 | + if maxFollowed <= lastComputed { | ||
| 59 | + return nil | ||
| 60 | + } | ||
| 61 | + | ||
| 62 | + return e.ComputeFollowDistances(ctx) | ||
| 63 | +} | ||
added
internal/cluster/weights.go +93 -0 | new file mode 100644 | ||
| @@ -0,0 +1,93 @@ | ||
| 1 | +package cluster | |
| 2 | + | |
| 3 | +import ( | |
| 4 | + "context" | |
| 5 | +) | |
| 6 | + | |
| 7 | +const ( | |
| 8 | + learningRate = 0.1 | |
| 9 | + weightMin = 0.1 | |
| 10 | + weightMax = 3.0 | |
| 11 | + minActionsTune = 5 | |
| 12 | +) | |
| 13 | + | |
| 14 | +func (e *Engine) RewardSignal(ctx context.Context, userDID string, signal string) { | |
| 15 | + e.adjustWeight(ctx, userDID, signal, 1.0) | |
| 16 | +} | |
| 17 | + | |
| 18 | +func (e *Engine) PenalizeSignal(ctx context.Context, userDID string, signal string) { | |
| 19 | + e.adjustWeight(ctx, userDID, signal, -1.0) | |
| 20 | +} | |
| 21 | + | |
| 22 | +func (e *Engine) adjustWeight(ctx context.Context, userDID string, signal string, delta float64) { | |
| 23 | + var actedCount int | |
| 24 | + _ = e.db.QueryRowContext(ctx, ` | |
| 25 | + SELECT COUNT(*) FROM recommendation_impressions WHERE user_did = ? AND acted = 1 | |
| 26 | + `, userDID).Scan(&actedCount) | |
| 27 | + if actedCount < minActionsTune { | |
| 28 | + return | |
| 29 | + } | |
| 30 | + | |
| 31 | + var exists int | |
| 32 | + _ = e.db.QueryRowContext(ctx, `SELECT 1 FROM user_signal_weights WHERE user_did = ?`, userDID).Scan(&exists) | |
| 33 | + | |
| 34 | + if exists == 0 { | |
| 35 | + _, _ = e.db.ExecContext(ctx, ` | |
| 36 | + INSERT INTO user_signal_weights (user_did, w_sub, w_like, w_tag, w_social, w_pop, w_category) | |
| 37 | + VALUES (?, 1.0, 0.5, 0.3, 0.7, 0.2, 0.4) | |
| 38 | + `, userDID) | |
| 39 | + } | |
| 40 | + | |
| 41 | + column := signalToColumn(signal) | |
| 42 | + if column == "" { | |
| 43 | + return | |
| 44 | + } | |
| 45 | + | |
| 46 | + adj := learningRate * delta | |
| 47 | + _, _ = e.db.ExecContext(ctx, ` | |
| 48 | + UPDATE user_signal_weights SET | |
| 49 | + `+column+` = MAX(?, MIN(?, `+column+` * (1 + ?))), | |
| 50 | + updated_at = CURRENT_TIMESTAMP | |
| 51 | + WHERE user_did = ? | |
| 52 | + `, weightMin, weightMax, adj, userDID) | |
| 53 | +} | |
| 54 | + | |
| 55 | +func signalToColumn(signal string) string { | |
| 56 | + switch signal { | |
| 57 | + case "sub": | |
| 58 | + return "w_sub" | |
| 59 | + case "like": | |
| 60 | + return "w_like" | |
| 61 | + case "tag": | |
| 62 | + return "w_tag" | |
| 63 | + case "social": | |
| 64 | + return "w_social" | |
| 65 | + case "pop": | |
| 66 | + return "w_pop" | |
| 67 | + case "category": | |
| 68 | + return "w_category" | |
| 69 | + default: | |
| 70 | + return "" | |
| 71 | + } | |
| 72 | +} | |
| 73 | + | |
| 74 | +func (e *Engine) GetDominantSignal(w SignalWeights) string { | |
| 75 | + signals := map[string]float64{ | |
| 76 | + "sub": w.WSub, | |
| 77 | + "like": w.WLike, | |
| 78 | + "tag": w.WTag, | |
| 79 | + "social": w.WSocial, | |
| 80 | + "pop": w.WPop, | |
| 81 | + "category": w.WCategory, | |
| 82 | + } | |
| 83 | + | |
| 84 | + var best string | |
| 85 | + bestVal := -1.0 | |
| 86 | + for s, v := range signals { | |
| 87 | + if v > bestVal { | |
| 88 | + bestVal = v | |
| 89 | + best = s | |
| 90 | + } | |
| 91 | + } | |
| 92 | + return best | |
| 93 | +} | |
| new file mode 100644 | |||
| @@ -0,0 +1,93 @@ | |||
| 1 | +package cluster | ||
| 2 | + | ||
| 3 | +import ( | ||
| 4 | + "context" | ||
| 5 | +) | ||
| 6 | + | ||
| 7 | +const ( | ||
| 8 | + learningRate = 0.1 | ||
| 9 | + weightMin = 0.1 | ||
| 10 | + weightMax = 3.0 | ||
| 11 | + minActionsTune = 5 | ||
| 12 | +) | ||
| 13 | + | ||
| 14 | +func (e *Engine) RewardSignal(ctx context.Context, userDID string, signal string) { | ||
| 15 | + e.adjustWeight(ctx, userDID, signal, 1.0) | ||
| 16 | +} | ||
| 17 | + | ||
| 18 | +func (e *Engine) PenalizeSignal(ctx context.Context, userDID string, signal string) { | ||
| 19 | + e.adjustWeight(ctx, userDID, signal, -1.0) | ||
| 20 | +} | ||
| 21 | + | ||
| 22 | +func (e *Engine) adjustWeight(ctx context.Context, userDID string, signal string, delta float64) { | ||
| 23 | + var actedCount int | ||
| 24 | + _ = e.db.QueryRowContext(ctx, ` | ||
| 25 | + SELECT COUNT(*) FROM recommendation_impressions WHERE user_did = ? AND acted = 1 | ||
| 26 | + `, userDID).Scan(&actedCount) | ||
| 27 | + if actedCount < minActionsTune { | ||
| 28 | + return | ||
| 29 | + } | ||
| 30 | + | ||
| 31 | + var exists int | ||
| 32 | + _ = e.db.QueryRowContext(ctx, `SELECT 1 FROM user_signal_weights WHERE user_did = ?`, userDID).Scan(&exists) | ||
| 33 | + | ||
| 34 | + if exists == 0 { | ||
| 35 | + _, _ = e.db.ExecContext(ctx, ` | ||
| 36 | + INSERT INTO user_signal_weights (user_did, w_sub, w_like, w_tag, w_social, w_pop, w_category) | ||
| 37 | + VALUES (?, 1.0, 0.5, 0.3, 0.7, 0.2, 0.4) | ||
| 38 | + `, userDID) | ||
| 39 | + } | ||
| 40 | + | ||
| 41 | + column := signalToColumn(signal) | ||
| 42 | + if column == "" { | ||
| 43 | + return | ||
| 44 | + } | ||
| 45 | + | ||
| 46 | + adj := learningRate * delta | ||
| 47 | + _, _ = e.db.ExecContext(ctx, ` | ||
| 48 | + UPDATE user_signal_weights SET | ||
| 49 | + `+column+` = MAX(?, MIN(?, `+column+` * (1 + ?))), | ||
| 50 | + updated_at = CURRENT_TIMESTAMP | ||
| 51 | + WHERE user_did = ? | ||
| 52 | + `, weightMin, weightMax, adj, userDID) | ||
| 53 | +} | ||
| 54 | + | ||
| 55 | +func signalToColumn(signal string) string { | ||
| 56 | + switch signal { | ||
| 57 | + case "sub": | ||
| 58 | + return "w_sub" | ||
| 59 | + case "like": | ||
| 60 | + return "w_like" | ||
| 61 | + case "tag": | ||
| 62 | + return "w_tag" | ||
| 63 | + case "social": | ||
| 64 | + return "w_social" | ||
| 65 | + case "pop": | ||
| 66 | + return "w_pop" | ||
| 67 | + case "category": | ||
| 68 | + return "w_category" | ||
| 69 | + default: | ||
| 70 | + return "" | ||
| 71 | + } | ||
| 72 | +} | ||
| 73 | + | ||
| 74 | +func (e *Engine) GetDominantSignal(w SignalWeights) string { | ||
| 75 | + signals := map[string]float64{ | ||
| 76 | + "sub": w.WSub, | ||
| 77 | + "like": w.WLike, | ||
| 78 | + "tag": w.WTag, | ||
| 79 | + "social": w.WSocial, | ||
| 80 | + "pop": w.WPop, | ||
| 81 | + "category": w.WCategory, | ||
| 82 | + } | ||
| 83 | + | ||
| 84 | + var best string | ||
| 85 | + bestVal := -1.0 | ||
| 86 | + for s, v := range signals { | ||
| 87 | + if v > bestVal { | ||
| 88 | + bestVal = v | ||
| 89 | + best = s | ||
| 90 | + } | ||
| 91 | + } | ||
| 92 | + return best | ||
| 93 | +} | ||
modified
internal/db/db.go +83 -16 | @@ -1,10 +1,13 @@ | ||
| 1 | 1 | package db |
| 2 | 2 | |
| 3 | 3 | import ( |
| 4 | + "context" | |
| 4 | 5 | "database/sql" |
| 5 | - _ "github.com/mattn/go-sqlite3" | |
| 6 | + "math" | |
| 6 | 7 | "strings" |
| 7 | 8 | "time" |
| 9 | + | |
| 10 | + "github.com/mattn/go-sqlite3" | |
| 8 | 11 | ) |
| 9 | 12 | |
| 10 | 13 | func NullStr(s string) sql.NullString { |
| @@ -40,6 +43,31 @@ func Open(path string) (*DB, error) { | ||
| 40 | 43 | db.SetMaxIdleConns(2) |
| 41 | 44 | db.SetConnMaxLifetime(30 * time.Minute) |
| 42 | 45 | |
| 46 | + conn, err := db.Conn(context.Background()) | |
| 47 | + if err != nil { | |
| 48 | + db.Close() | |
| 49 | + return nil, err | |
| 50 | + } | |
| 51 | + | |
| 52 | + err = conn.Raw(func(driverConn any) error { | |
| 53 | + sqliteConn, ok := driverConn.(*sqlite3.SQLiteConn) | |
| 54 | + if !ok { | |
| 55 | + return nil | |
| 56 | + } | |
| 57 | + if err := sqliteConn.RegisterFunc("exp", func(x float64) float64 { return math.Exp(x) }, true); err != nil { | |
| 58 | + return err | |
| 59 | + } | |
| 60 | + if err := sqliteConn.RegisterFunc("log", func(x float64) float64 { return math.Log(x) }, true); err != nil { | |
| 61 | + return err | |
| 62 | + } | |
| 63 | + return nil | |
| 64 | + }) | |
| 65 | + _ = conn.Close() | |
| 66 | + if err != nil { | |
| 67 | + db.Close() | |
| 68 | + return nil, err | |
| 69 | + } | |
| 70 | + | |
| 43 | 71 | if err := initSchema(db); err != nil { |
| 44 | 72 | db.Close() |
| 45 | 73 | return nil, err |
| @@ -158,21 +186,6 @@ var schema = []string{ | ||
| 158 | 186 | PRIMARY KEY (user_a, user_b), |
| 159 | 187 | CHECK(user_a < user_b) |
| 160 | 188 | )`, |
| 161 | - `CREATE TABLE IF NOT EXISTS user_feed_recommendations ( | |
| 162 | - user_did TEXT NOT NULL REFERENCES users(did), | |
| 163 | - feed_url TEXT NOT NULL REFERENCES feeds(feed_url), | |
| 164 | - score REAL NOT NULL, | |
| 165 | - computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | |
| 166 | - PRIMARY KEY (user_did, feed_url) | |
| 167 | - )`, | |
| 168 | - `CREATE TABLE IF NOT EXISTS user_article_recommendations ( | |
| 169 | - user_did TEXT NOT NULL REFERENCES users(did), | |
| 170 | - feed_url TEXT NOT NULL, | |
| 171 | - article_url TEXT NOT NULL, | |
| 172 | - score REAL NOT NULL, | |
| 173 | - computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | |
| 174 | - PRIMARY KEY (user_did, feed_url, article_url) | |
| 175 | - )`, | |
| 176 | 189 | `CREATE TABLE IF NOT EXISTS follows ( |
| 177 | 190 | user_did TEXT NOT NULL REFERENCES users(did), |
| 178 | 191 | target_did TEXT NOT NULL, |
| @@ -208,6 +221,60 @@ var schema = []string{ | ||
| 208 | 221 | `CREATE INDEX IF NOT EXISTS idx_follows_target ON follows(target_did)`, |
| 209 | 222 | `CREATE INDEX IF NOT EXISTS idx_follows_uri ON follows(uri)`, |
| 210 | 223 | `CREATE INDEX IF NOT EXISTS idx_user_similarity_b ON user_similarity(user_b)`, |
| 224 | + | |
| 225 | + `CREATE TABLE IF NOT EXISTS dismissed_recommendations ( | |
| 226 | + user_did TEXT NOT NULL REFERENCES users(did), | |
| 227 | + target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')), | |
| 228 | + target_id TEXT NOT NULL, | |
| 229 | + reason TEXT, | |
| 230 | + dismissed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | |
| 231 | + PRIMARY KEY (user_did, target_type, target_id) | |
| 232 | + )`, | |
| 233 | + | |
| 234 | + `CREATE TABLE IF NOT EXISTS recommendation_impressions ( | |
| 235 | + user_did TEXT NOT NULL REFERENCES users(did), | |
| 236 | + target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')), | |
| 237 | + target_id TEXT NOT NULL, | |
| 238 | + first_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | |
| 239 | + last_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | |
| 240 | + shown_count INTEGER NOT NULL DEFAULT 1, | |
| 241 | + acted BOOLEAN NOT NULL DEFAULT 0, | |
| 242 | + PRIMARY KEY (user_did, target_type, target_id) | |
| 243 | + )`, | |
| 244 | + | |
| 245 | + `CREATE TABLE IF NOT EXISTS follow_distances ( | |
| 246 | + user_a TEXT NOT NULL, | |
| 247 | + user_b TEXT NOT NULL, | |
| 248 | + distance INTEGER NOT NULL CHECK(distance IN (1, 2)), | |
| 249 | + PRIMARY KEY (user_a, user_b) | |
| 250 | + )`, | |
| 251 | + | |
| 252 | + `CREATE TABLE IF NOT EXISTS user_signal_weights ( | |
| 253 | + user_did TEXT PRIMARY KEY REFERENCES users(did), | |
| 254 | + w_sub REAL NOT NULL DEFAULT 1.0, | |
| 255 | + w_like REAL NOT NULL DEFAULT 0.5, | |
| 256 | + w_tag REAL NOT NULL DEFAULT 0.3, | |
| 257 | + w_social REAL NOT NULL DEFAULT 0.7, | |
| 258 | + w_pop REAL NOT NULL DEFAULT 0.2, | |
| 259 | + w_category REAL NOT NULL DEFAULT 0.4, | |
| 260 | + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP | |
| 261 | + )`, | |
| 262 | + | |
| 263 | + `CREATE TABLE IF NOT EXISTS user_signal_profiles ( | |
| 264 | + user_did TEXT PRIMARY KEY REFERENCES users(did), | |
| 265 | + total_likes INTEGER NOT NULL DEFAULT 0, | |
| 266 | + total_tags INTEGER NOT NULL DEFAULT 0, | |
| 267 | + top_categories TEXT, | |
| 268 | + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP | |
| 269 | + )`, | |
| 270 | + | |
| 271 | + `CREATE INDEX IF NOT EXISTS idx_dismissed_user_type ON dismissed_recommendations(user_did, target_type)`, | |
| 272 | + `CREATE INDEX IF NOT EXISTS idx_impressions_user_unacted ON recommendation_impressions(user_did, acted, shown_count)`, | |
| 273 | + `CREATE INDEX IF NOT EXISTS idx_impressions_last_shown ON recommendation_impressions(last_shown_at)`, | |
| 274 | + `CREATE INDEX IF NOT EXISTS idx_follow_distances_b ON follow_distances(user_b)`, | |
| 275 | + `CREATE INDEX IF NOT EXISTS idx_follow_distances_a_dist ON follow_distances(user_a, distance)`, | |
| 276 | + `CREATE INDEX IF NOT EXISTS idx_likes_author_feed ON likes(author_did, feed_url, created_at)`, | |
| 277 | + `CREATE INDEX IF NOT EXISTS idx_follows_followed_at ON follows(followed_at)`, | |
| 211 | 278 | `CREATE INDEX IF NOT EXISTS idx_users_handle ON users(handle)`, |
| 212 | 279 | `CREATE VIRTUAL TABLE IF NOT EXISTS articles_fts USING fts5(title, summary, content, author, content=articles, content_rowid=id)`, |
| 213 | 280 | `CREATE TRIGGER IF NOT EXISTS articles_ai AFTER INSERT ON articles BEGIN |
| @@ -1,10 +1,13 @@ | |||
| 1 | package db | 1 | package db |
| 2 | 2 | ||
| 3 | import ( | 3 | import ( |
| 4 | + "context" | ||
| 4 | "database/sql" | 5 | "database/sql" |
| 5 | - _ "github.com/mattn/go-sqlite3" | 6 | + "math" |
| 6 | "strings" | 7 | "strings" |
| 7 | "time" | 8 | "time" |
| 9 | + | ||
| 10 | + "github.com/mattn/go-sqlite3" | ||
| 8 | ) | 11 | ) |
| 9 | 12 | ||
| 10 | func NullStr(s string) sql.NullString { | 13 | func NullStr(s string) sql.NullString { |
| @@ -40,6 +43,31 @@ func Open(path string) (*DB, error) { | |||
| 40 | db.SetMaxIdleConns(2) | 43 | db.SetMaxIdleConns(2) |
| 41 | db.SetConnMaxLifetime(30 * time.Minute) | 44 | db.SetConnMaxLifetime(30 * time.Minute) |
| 42 | 45 | ||
| 46 | + conn, err := db.Conn(context.Background()) | ||
| 47 | + if err != nil { | ||
| 48 | + db.Close() | ||
| 49 | + return nil, err | ||
| 50 | + } | ||
| 51 | + | ||
| 52 | + err = conn.Raw(func(driverConn any) error { | ||
| 53 | + sqliteConn, ok := driverConn.(*sqlite3.SQLiteConn) | ||
| 54 | + if !ok { | ||
| 55 | + return nil | ||
| 56 | + } | ||
| 57 | + if err := sqliteConn.RegisterFunc("exp", func(x float64) float64 { return math.Exp(x) }, true); err != nil { | ||
| 58 | + return err | ||
| 59 | + } | ||
| 60 | + if err := sqliteConn.RegisterFunc("log", func(x float64) float64 { return math.Log(x) }, true); err != nil { | ||
| 61 | + return err | ||
| 62 | + } | ||
| 63 | + return nil | ||
| 64 | + }) | ||
| 65 | + _ = conn.Close() | ||
| 66 | + if err != nil { | ||
| 67 | + db.Close() | ||
| 68 | + return nil, err | ||
| 69 | + } | ||
| 70 | + | ||
| 43 | if err := initSchema(db); err != nil { | 71 | if err := initSchema(db); err != nil { |
| 44 | db.Close() | 72 | db.Close() |
| 45 | return nil, err | 73 | return nil, err |
| @@ -158,21 +186,6 @@ var schema = []string{ | |||
| 158 | PRIMARY KEY (user_a, user_b), | 186 | PRIMARY KEY (user_a, user_b), |
| 159 | CHECK(user_a < user_b) | 187 | CHECK(user_a < user_b) |
| 160 | )`, | 188 | )`, |
| 161 | - `CREATE TABLE IF NOT EXISTS user_feed_recommendations ( | ||
| 162 | - user_did TEXT NOT NULL REFERENCES users(did), | ||
| 163 | - feed_url TEXT NOT NULL REFERENCES feeds(feed_url), | ||
| 164 | - score REAL NOT NULL, | ||
| 165 | - computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | ||
| 166 | - PRIMARY KEY (user_did, feed_url) | ||
| 167 | - )`, | ||
| 168 | - `CREATE TABLE IF NOT EXISTS user_article_recommendations ( | ||
| 169 | - user_did TEXT NOT NULL REFERENCES users(did), | ||
| 170 | - feed_url TEXT NOT NULL, | ||
| 171 | - article_url TEXT NOT NULL, | ||
| 172 | - score REAL NOT NULL, | ||
| 173 | - computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | ||
| 174 | - PRIMARY KEY (user_did, feed_url, article_url) | ||
| 175 | - )`, | ||
| 176 | `CREATE TABLE IF NOT EXISTS follows ( | 189 | `CREATE TABLE IF NOT EXISTS follows ( |
| 177 | user_did TEXT NOT NULL REFERENCES users(did), | 190 | user_did TEXT NOT NULL REFERENCES users(did), |
| 178 | target_did TEXT NOT NULL, | 191 | target_did TEXT NOT NULL, |
| @@ -208,6 +221,60 @@ var schema = []string{ | |||
| 208 | `CREATE INDEX IF NOT EXISTS idx_follows_target ON follows(target_did)`, | 221 | `CREATE INDEX IF NOT EXISTS idx_follows_target ON follows(target_did)`, |
| 209 | `CREATE INDEX IF NOT EXISTS idx_follows_uri ON follows(uri)`, | 222 | `CREATE INDEX IF NOT EXISTS idx_follows_uri ON follows(uri)`, |
| 210 | `CREATE INDEX IF NOT EXISTS idx_user_similarity_b ON user_similarity(user_b)`, | 223 | `CREATE INDEX IF NOT EXISTS idx_user_similarity_b ON user_similarity(user_b)`, |
| 224 | + | ||
| 225 | + `CREATE TABLE IF NOT EXISTS dismissed_recommendations ( | ||
| 226 | + user_did TEXT NOT NULL REFERENCES users(did), | ||
| 227 | + target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')), | ||
| 228 | + target_id TEXT NOT NULL, | ||
| 229 | + reason TEXT, | ||
| 230 | + dismissed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | ||
| 231 | + PRIMARY KEY (user_did, target_type, target_id) | ||
| 232 | + )`, | ||
| 233 | + | ||
| 234 | + `CREATE TABLE IF NOT EXISTS recommendation_impressions ( | ||
| 235 | + user_did TEXT NOT NULL REFERENCES users(did), | ||
| 236 | + target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')), | ||
| 237 | + target_id TEXT NOT NULL, | ||
| 238 | + first_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | ||
| 239 | + last_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, | ||
| 240 | + shown_count INTEGER NOT NULL DEFAULT 1, | ||
| 241 | + acted BOOLEAN NOT NULL DEFAULT 0, | ||
| 242 | + PRIMARY KEY (user_did, target_type, target_id) | ||
| 243 | + )`, | ||
| 244 | + | ||
| 245 | + `CREATE TABLE IF NOT EXISTS follow_distances ( | ||
| 246 | + user_a TEXT NOT NULL, | ||
| 247 | + user_b TEXT NOT NULL, | ||
| 248 | + distance INTEGER NOT NULL CHECK(distance IN (1, 2)), | ||
| 249 | + PRIMARY KEY (user_a, user_b) | ||
| 250 | + )`, | ||
| 251 | + | ||
| 252 | + `CREATE TABLE IF NOT EXISTS user_signal_weights ( | ||
| 253 | + user_did TEXT PRIMARY KEY REFERENCES users(did), | ||
| 254 | + w_sub REAL NOT NULL DEFAULT 1.0, | ||
| 255 | + w_like REAL NOT NULL DEFAULT 0.5, | ||
| 256 | + w_tag REAL NOT NULL DEFAULT 0.3, | ||
| 257 | + w_social REAL NOT NULL DEFAULT 0.7, | ||
| 258 | + w_pop REAL NOT NULL DEFAULT 0.2, | ||
| 259 | + w_category REAL NOT NULL DEFAULT 0.4, | ||
| 260 | + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP | ||
| 261 | + )`, | ||
| 262 | + | ||
| 263 | + `CREATE TABLE IF NOT EXISTS user_signal_profiles ( | ||
| 264 | + user_did TEXT PRIMARY KEY REFERENCES users(did), | ||
| 265 | + total_likes INTEGER NOT NULL DEFAULT 0, | ||
| 266 | + total_tags INTEGER NOT NULL DEFAULT 0, | ||
| 267 | + top_categories TEXT, | ||
| 268 | + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP | ||
| 269 | + )`, | ||
| 270 | + | ||
| 271 | + `CREATE INDEX IF NOT EXISTS idx_dismissed_user_type ON dismissed_recommendations(user_did, target_type)`, | ||
| 272 | + `CREATE INDEX IF NOT EXISTS idx_impressions_user_unacted ON recommendation_impressions(user_did, acted, shown_count)`, | ||
| 273 | + `CREATE INDEX IF NOT EXISTS idx_impressions_last_shown ON recommendation_impressions(last_shown_at)`, | ||
| 274 | + `CREATE INDEX IF NOT EXISTS idx_follow_distances_b ON follow_distances(user_b)`, | ||
| 275 | + `CREATE INDEX IF NOT EXISTS idx_follow_distances_a_dist ON follow_distances(user_a, distance)`, | ||
| 276 | + `CREATE INDEX IF NOT EXISTS idx_likes_author_feed ON likes(author_did, feed_url, created_at)`, | ||
| 277 | + `CREATE INDEX IF NOT EXISTS idx_follows_followed_at ON follows(followed_at)`, | ||
| 211 | `CREATE INDEX IF NOT EXISTS idx_users_handle ON users(handle)`, | 278 | `CREATE INDEX IF NOT EXISTS idx_users_handle ON users(handle)`, |
| 212 | `CREATE VIRTUAL TABLE IF NOT EXISTS articles_fts USING fts5(title, summary, content, author, content=articles, content_rowid=id)`, | 279 | `CREATE VIRTUAL TABLE IF NOT EXISTS articles_fts USING fts5(title, summary, content, author, content=articles, content_rowid=id)`, |
| 213 | `CREATE TRIGGER IF NOT EXISTS articles_ai AFTER INSERT ON articles BEGIN | 280 | `CREATE TRIGGER IF NOT EXISTS articles_ai AFTER INSERT ON articles BEGIN |
modified
internal/server/articles_handler.go +6 -0 | @@ -269,6 +269,9 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) { | ||
| 269 | 269 | http.Error(w, err.Error(), http.StatusInternalServerError) |
| 270 | 270 | return |
| 271 | 271 | } |
| 272 | + _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "article", article.URL.String) | |
| 273 | + sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID)) | |
| 274 | + s.engine.RewardSignal(r.Context(), user.DID, sig) | |
| 272 | 275 | } else { |
| 273 | 276 | like := &db.Like{ |
| 274 | 277 | URI: fmt.Sprintf("glean:like:%d", time.Now().UnixNano()), |
| @@ -281,6 +284,9 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) { | ||
| 281 | 284 | http.Error(w, err.Error(), http.StatusInternalServerError) |
| 282 | 285 | return |
| 283 | 286 | } |
| 287 | + _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "article", article.URL.String) | |
| 288 | + sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID)) | |
| 289 | + s.engine.RewardSignal(r.Context(), user.DID, sig) | |
| 284 | 290 | } |
| 285 | 291 | } |
| 286 | 292 | |
| @@ -269,6 +269,9 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) { | |||
| 269 | http.Error(w, err.Error(), http.StatusInternalServerError) | 269 | http.Error(w, err.Error(), http.StatusInternalServerError) |
| 270 | return | 270 | return |
| 271 | } | 271 | } |
| 272 | + _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "article", article.URL.String) | ||
| 273 | + sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID)) | ||
| 274 | + s.engine.RewardSignal(r.Context(), user.DID, sig) | ||
| 272 | } else { | 275 | } else { |
| 273 | like := &db.Like{ | 276 | like := &db.Like{ |
| 274 | URI: fmt.Sprintf("glean:like:%d", time.Now().UnixNano()), | 277 | URI: fmt.Sprintf("glean:like:%d", time.Now().UnixNano()), |
| @@ -281,6 +284,9 @@ func (s *Server) handleLikeArticle(w http.ResponseWriter, r *http.Request) { | |||
| 281 | http.Error(w, err.Error(), http.StatusInternalServerError) | 284 | http.Error(w, err.Error(), http.StatusInternalServerError) |
| 282 | return | 285 | return |
| 283 | } | 286 | } |
| 287 | + _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "article", article.URL.String) | ||
| 288 | + sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID)) | ||
| 289 | + s.engine.RewardSignal(r.Context(), user.DID, sig) | ||
| 284 | } | 290 | } |
| 285 | } | 291 | } |
| 286 | 292 | ||
modified
internal/server/dashboard_handler.go +35 -0 | @@ -3,6 +3,8 @@ package server | ||
| 3 | 3 | import ( |
| 4 | 4 | "net/http" |
| 5 | 5 | "time" |
| 6 | + | |
| 7 | + "pkg.rbrt.fr/glean/internal/cluster" | |
| 6 | 8 | ) |
| 7 | 9 | |
| 8 | 10 | func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) { |
| @@ -23,6 +25,17 @@ func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) { | ||
| 23 | 25 | peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5) |
| 24 | 26 | feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 5) |
| 25 | 27 | |
| 28 | + var impressions []cluster.Impression | |
| 29 | + for _, rec := range feedRecs { | |
| 30 | + impressions = append(impressions, cluster.Impression{TargetType: "feed", TargetID: rec.FeedURL}) | |
| 31 | + } | |
| 32 | + for _, rec := range articleRecs { | |
| 33 | + impressions = append(impressions, cluster.Impression{TargetType: "article", TargetID: rec.URL}) | |
| 34 | + } | |
| 35 | + if len(impressions) > 0 { | |
| 36 | + _ = s.engine.RecordImpressions(r.Context(), user.DID, impressions) | |
| 37 | + } | |
| 38 | + | |
| 26 | 39 | since := time.Now().AddDate(0, 0, -7).Format(time.RFC3339) |
| 27 | 40 | personalTrending, _ := s.db.ListTrendingArticlesForUser(r.Context(), user.DID, since, 5, 0) |
| 28 | 41 | globalTrending, _ := s.db.ListTrendingArticles(r.Context(), user.DID, since, 10, 0) |
| @@ -43,3 +56,25 @@ func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) { | ||
| 43 | 56 | "Now": time.Now(), |
| 44 | 57 | }) |
| 45 | 58 | } |
| 59 | + | |
| 60 | +func (s *Server) handleDismissArticleRecommendation(w http.ResponseWriter, r *http.Request) { | |
| 61 | + user := currentUser(r) | |
| 62 | + articleURL := r.FormValue("article_url") | |
| 63 | + if articleURL == "" { | |
| 64 | + http.Error(w, "article_url required", http.StatusBadRequest) | |
| 65 | + return | |
| 66 | + } | |
| 67 | + | |
| 68 | + reason := r.FormValue("reason") | |
| 69 | + if reason == "" { | |
| 70 | + reason = "not_interested" | |
| 71 | + } | |
| 72 | + | |
| 73 | + if err := s.engine.DismissArticle(r.Context(), user.DID, articleURL, reason); err != nil { | |
| 74 | + s.logger.Error("failed to dismiss article recommendation", "error", err) | |
| 75 | + http.Error(w, err.Error(), http.StatusInternalServerError) | |
| 76 | + return | |
| 77 | + } | |
| 78 | + | |
| 79 | + w.WriteHeader(http.StatusOK) | |
| 80 | +} | |
| @@ -3,6 +3,8 @@ package server | |||
| 3 | import ( | 3 | import ( |
| 4 | "net/http" | 4 | "net/http" |
| 5 | "time" | 5 | "time" |
| 6 | + | ||
| 7 | + "pkg.rbrt.fr/glean/internal/cluster" | ||
| 6 | ) | 8 | ) |
| 7 | 9 | ||
| 8 | func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) { | 10 | func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) { |
| @@ -23,6 +25,17 @@ func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) { | |||
| 23 | peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5) | 25 | peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5) |
| 24 | feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 5) | 26 | feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 5) |
| 25 | 27 | ||
| 28 | + var impressions []cluster.Impression | ||
| 29 | + for _, rec := range feedRecs { | ||
| 30 | + impressions = append(impressions, cluster.Impression{TargetType: "feed", TargetID: rec.FeedURL}) | ||
| 31 | + } | ||
| 32 | + for _, rec := range articleRecs { | ||
| 33 | + impressions = append(impressions, cluster.Impression{TargetType: "article", TargetID: rec.URL}) | ||
| 34 | + } | ||
| 35 | + if len(impressions) > 0 { | ||
| 36 | + _ = s.engine.RecordImpressions(r.Context(), user.DID, impressions) | ||
| 37 | + } | ||
| 38 | + | ||
| 26 | since := time.Now().AddDate(0, 0, -7).Format(time.RFC3339) | 39 | since := time.Now().AddDate(0, 0, -7).Format(time.RFC3339) |
| 27 | personalTrending, _ := s.db.ListTrendingArticlesForUser(r.Context(), user.DID, since, 5, 0) | 40 | personalTrending, _ := s.db.ListTrendingArticlesForUser(r.Context(), user.DID, since, 5, 0) |
| 28 | globalTrending, _ := s.db.ListTrendingArticles(r.Context(), user.DID, since, 10, 0) | 41 | globalTrending, _ := s.db.ListTrendingArticles(r.Context(), user.DID, since, 10, 0) |
| @@ -43,3 +56,25 @@ func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) { | |||
| 43 | "Now": time.Now(), | 56 | "Now": time.Now(), |
| 44 | }) | 57 | }) |
| 45 | } | 58 | } |
| 59 | + | ||
| 60 | +func (s *Server) handleDismissArticleRecommendation(w http.ResponseWriter, r *http.Request) { | ||
| 61 | + user := currentUser(r) | ||
| 62 | + articleURL := r.FormValue("article_url") | ||
| 63 | + if articleURL == "" { | ||
| 64 | + http.Error(w, "article_url required", http.StatusBadRequest) | ||
| 65 | + return | ||
| 66 | + } | ||
| 67 | + | ||
| 68 | + reason := r.FormValue("reason") | ||
| 69 | + if reason == "" { | ||
| 70 | + reason = "not_interested" | ||
| 71 | + } | ||
| 72 | + | ||
| 73 | + if err := s.engine.DismissArticle(r.Context(), user.DID, articleURL, reason); err != nil { | ||
| 74 | + s.logger.Error("failed to dismiss article recommendation", "error", err) | ||
| 75 | + http.Error(w, err.Error(), http.StatusInternalServerError) | ||
| 76 | + return | ||
| 77 | + } | ||
| 78 | + | ||
| 79 | + w.WriteHeader(http.StatusOK) | ||
| 80 | +} | ||
modified
internal/server/feeds_handler.go +35 -0 | @@ -9,6 +9,7 @@ import ( | ||
| 9 | 9 | "time" |
| 10 | 10 | |
| 11 | 11 | "pkg.rbrt.fr/glean/internal/atproto" |
| 12 | + "pkg.rbrt.fr/glean/internal/cluster" | |
| 12 | 13 | "pkg.rbrt.fr/glean/internal/db" |
| 13 | 14 | "pkg.rbrt.fr/glean/internal/feed" |
| 14 | 15 | ) |
| @@ -28,6 +29,14 @@ func (s *Server) handleFeeds(w http.ResponseWriter, r *http.Request) { | ||
| 28 | 29 | allSubs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 1000, 0) |
| 29 | 30 | feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 10) |
| 30 | 31 | peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5) |
| 32 | + | |
| 33 | + if len(feedRecs) > 0 { | |
| 34 | + impressions := make([]cluster.Impression, len(feedRecs)) | |
| 35 | + for i, rec := range feedRecs { | |
| 36 | + impressions[i] = cluster.Impression{TargetType: "feed", TargetID: rec.FeedURL} | |
| 37 | + } | |
| 38 | + _ = s.engine.RecordImpressions(r.Context(), user.DID, impressions) | |
| 39 | + } | |
| 31 | 40 | deadFeeds, _ := s.db.ListDeadFeeds(r.Context(), user.DID, 7) |
| 32 | 41 | |
| 33 | 42 | categories, _ := s.db.GetCategories(r.Context(), user.DID) |
| @@ -122,6 +131,10 @@ func (s *Server) handleAddFeed(w http.ResponseWriter, r *http.Request) { | ||
| 122 | 131 | return |
| 123 | 132 | } |
| 124 | 133 | |
| 134 | + _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "feed", feedURL) | |
| 135 | + sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID)) | |
| 136 | + s.engine.RewardSignal(r.Context(), user.DID, sig) | |
| 137 | + | |
| 125 | 138 | sub, _ := s.db.GetSubscription(r.Context(), user.DID, feedURL) |
| 126 | 139 | if sub == nil { |
| 127 | 140 | w.WriteHeader(http.StatusNoContent) |
| @@ -384,6 +397,28 @@ func (s *Server) handleDiscoverFeedURL(w http.ResponseWriter, r *http.Request) { | ||
| 384 | 397 | }) |
| 385 | 398 | } |
| 386 | 399 | |
| 400 | +func (s *Server) handleDismissFeedRecommendation(w http.ResponseWriter, r *http.Request) { | |
| 401 | + user := currentUser(r) | |
| 402 | + feedURL := r.FormValue("feed_url") | |
| 403 | + if feedURL == "" { | |
| 404 | + http.Error(w, "feed_url required", http.StatusBadRequest) | |
| 405 | + return | |
| 406 | + } | |
| 407 | + | |
| 408 | + reason := r.FormValue("reason") | |
| 409 | + if reason == "" { | |
| 410 | + reason = "not_interested" | |
| 411 | + } | |
| 412 | + | |
| 413 | + if err := s.engine.DismissFeed(r.Context(), user.DID, feedURL, reason); err != nil { | |
| 414 | + s.logger.Error("failed to dismiss feed recommendation", "error", err) | |
| 415 | + http.Error(w, err.Error(), http.StatusInternalServerError) | |
| 416 | + return | |
| 417 | + } | |
| 418 | + | |
| 419 | + w.WriteHeader(http.StatusOK) | |
| 420 | +} | |
| 421 | + | |
| 387 | 422 | func nullString(s string) sql.NullString { |
| 388 | 423 | return sql.NullString{String: s, Valid: s != ""} |
| 389 | 424 | } |
| @@ -9,6 +9,7 @@ import ( | |||
| 9 | "time" | 9 | "time" |
| 10 | 10 | ||
| 11 | "pkg.rbrt.fr/glean/internal/atproto" | 11 | "pkg.rbrt.fr/glean/internal/atproto" |
| 12 | + "pkg.rbrt.fr/glean/internal/cluster" | ||
| 12 | "pkg.rbrt.fr/glean/internal/db" | 13 | "pkg.rbrt.fr/glean/internal/db" |
| 13 | "pkg.rbrt.fr/glean/internal/feed" | 14 | "pkg.rbrt.fr/glean/internal/feed" |
| 14 | ) | 15 | ) |
| @@ -28,6 +29,14 @@ func (s *Server) handleFeeds(w http.ResponseWriter, r *http.Request) { | |||
| 28 | allSubs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 1000, 0) | 29 | allSubs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 1000, 0) |
| 29 | feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 10) | 30 | feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 10) |
| 30 | peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5) | 31 | peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5) |
| 32 | + | ||
| 33 | + if len(feedRecs) > 0 { | ||
| 34 | + impressions := make([]cluster.Impression, len(feedRecs)) | ||
| 35 | + for i, rec := range feedRecs { | ||
| 36 | + impressions[i] = cluster.Impression{TargetType: "feed", TargetID: rec.FeedURL} | ||
| 37 | + } | ||
| 38 | + _ = s.engine.RecordImpressions(r.Context(), user.DID, impressions) | ||
| 39 | + } | ||
| 31 | deadFeeds, _ := s.db.ListDeadFeeds(r.Context(), user.DID, 7) | 40 | deadFeeds, _ := s.db.ListDeadFeeds(r.Context(), user.DID, 7) |
| 32 | 41 | ||
| 33 | categories, _ := s.db.GetCategories(r.Context(), user.DID) | 42 | categories, _ := s.db.GetCategories(r.Context(), user.DID) |
| @@ -122,6 +131,10 @@ func (s *Server) handleAddFeed(w http.ResponseWriter, r *http.Request) { | |||
| 122 | return | 131 | return |
| 123 | } | 132 | } |
| 124 | 133 | ||
| 134 | + _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "feed", feedURL) | ||
| 135 | + sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID)) | ||
| 136 | + s.engine.RewardSignal(r.Context(), user.DID, sig) | ||
| 137 | + | ||
| 125 | sub, _ := s.db.GetSubscription(r.Context(), user.DID, feedURL) | 138 | sub, _ := s.db.GetSubscription(r.Context(), user.DID, feedURL) |
| 126 | if sub == nil { | 139 | if sub == nil { |
| 127 | w.WriteHeader(http.StatusNoContent) | 140 | w.WriteHeader(http.StatusNoContent) |
| @@ -384,6 +397,28 @@ func (s *Server) handleDiscoverFeedURL(w http.ResponseWriter, r *http.Request) { | |||
| 384 | }) | 397 | }) |
| 385 | } | 398 | } |
| 386 | 399 | ||
| 400 | +func (s *Server) handleDismissFeedRecommendation(w http.ResponseWriter, r *http.Request) { | ||
| 401 | + user := currentUser(r) | ||
| 402 | + feedURL := r.FormValue("feed_url") | ||
| 403 | + if feedURL == "" { | ||
| 404 | + http.Error(w, "feed_url required", http.StatusBadRequest) | ||
| 405 | + return | ||
| 406 | + } | ||
| 407 | + | ||
| 408 | + reason := r.FormValue("reason") | ||
| 409 | + if reason == "" { | ||
| 410 | + reason = "not_interested" | ||
| 411 | + } | ||
| 412 | + | ||
| 413 | + if err := s.engine.DismissFeed(r.Context(), user.DID, feedURL, reason); err != nil { | ||
| 414 | + s.logger.Error("failed to dismiss feed recommendation", "error", err) | ||
| 415 | + http.Error(w, err.Error(), http.StatusInternalServerError) | ||
| 416 | + return | ||
| 417 | + } | ||
| 418 | + | ||
| 419 | + w.WriteHeader(http.StatusOK) | ||
| 420 | +} | ||
| 421 | + | ||
| 387 | func nullString(s string) sql.NullString { | 422 | func nullString(s string) sql.NullString { |
| 388 | return sql.NullString{String: s, Valid: s != ""} | 423 | return sql.NullString{String: s, Valid: s != ""} |
| 389 | } | 424 | } |
modified
internal/server/server.go +3 -1 | @@ -162,6 +162,7 @@ func (s *Server) setupRoutes() { | ||
| 162 | 162 | r.Get("/list", s.handleFeedList) |
| 163 | 163 | r.Get("/discover-url", s.handleDiscoverFeedURL) |
| 164 | 164 | r.Post("/clear", s.handleClearAllSubscriptions) |
| 165 | + r.Post("/dismiss", s.handleDismissFeedRecommendation) | |
| 165 | 166 | }) |
| 166 | 167 | |
| 167 | 168 | s.router.Route("/articles", func(r chi.Router) { |
| @@ -174,6 +175,7 @@ func (s *Server) setupRoutes() { | ||
| 174 | 175 | r.Post("/{id}/like", s.handleLikeArticle) |
| 175 | 176 | r.Post("/{id}/fetch-content", s.handleFetchContent) |
| 176 | 177 | r.Post("/mark-all-read", s.handleMarkAllRead) |
| 178 | + r.Post("/dismiss", s.handleDismissArticleRecommendation) | |
| 177 | 179 | }) |
| 178 | 180 | |
| 179 | 181 | s.router.Route("/trending", func(r chi.Router) { |
| @@ -198,7 +200,7 @@ func (s *Server) setupRoutes() { | ||
| 198 | 200 | s.router.Post("/auth/logout", s.handleAuthLogout) |
| 199 | 201 | s.router.Get("/oauth/client-metadata", s.handleOAuthClientMetadata) |
| 200 | 202 | |
| 201 | - xrpc := atproto.NewXRPCHandler(s.db.DB) | |
| 203 | + xrpc := atproto.NewXRPCHandler(s.db.DB, s.engine) | |
| 202 | 204 | s.router.Get("/xrpc/at.glean.listSubscriptions", xrpc.ListSubscriptions) |
| 203 | 205 | s.router.Get("/xrpc/at.glean.listAnnotations", xrpc.ListAnnotations) |
| 204 | 206 | s.router.Get("/xrpc/at.glean.listLikes", xrpc.ListLikes) |
| @@ -162,6 +162,7 @@ func (s *Server) setupRoutes() { | |||
| 162 | r.Get("/list", s.handleFeedList) | 162 | r.Get("/list", s.handleFeedList) |
| 163 | r.Get("/discover-url", s.handleDiscoverFeedURL) | 163 | r.Get("/discover-url", s.handleDiscoverFeedURL) |
| 164 | r.Post("/clear", s.handleClearAllSubscriptions) | 164 | r.Post("/clear", s.handleClearAllSubscriptions) |
| 165 | + r.Post("/dismiss", s.handleDismissFeedRecommendation) | ||
| 165 | }) | 166 | }) |
| 166 | 167 | ||
| 167 | s.router.Route("/articles", func(r chi.Router) { | 168 | s.router.Route("/articles", func(r chi.Router) { |
| @@ -174,6 +175,7 @@ func (s *Server) setupRoutes() { | |||
| 174 | r.Post("/{id}/like", s.handleLikeArticle) | 175 | r.Post("/{id}/like", s.handleLikeArticle) |
| 175 | r.Post("/{id}/fetch-content", s.handleFetchContent) | 176 | r.Post("/{id}/fetch-content", s.handleFetchContent) |
| 176 | r.Post("/mark-all-read", s.handleMarkAllRead) | 177 | r.Post("/mark-all-read", s.handleMarkAllRead) |
| 178 | + r.Post("/dismiss", s.handleDismissArticleRecommendation) | ||
| 177 | }) | 179 | }) |
| 178 | 180 | ||
| 179 | s.router.Route("/trending", func(r chi.Router) { | 181 | s.router.Route("/trending", func(r chi.Router) { |
| @@ -198,7 +200,7 @@ func (s *Server) setupRoutes() { | |||
| 198 | s.router.Post("/auth/logout", s.handleAuthLogout) | 200 | s.router.Post("/auth/logout", s.handleAuthLogout) |
| 199 | s.router.Get("/oauth/client-metadata", s.handleOAuthClientMetadata) | 201 | s.router.Get("/oauth/client-metadata", s.handleOAuthClientMetadata) |
| 200 | 202 | ||
| 201 | - xrpc := atproto.NewXRPCHandler(s.db.DB) | 203 | + xrpc := atproto.NewXRPCHandler(s.db.DB, s.engine) |
| 202 | s.router.Get("/xrpc/at.glean.listSubscriptions", xrpc.ListSubscriptions) | 204 | s.router.Get("/xrpc/at.glean.listSubscriptions", xrpc.ListSubscriptions) |
| 203 | s.router.Get("/xrpc/at.glean.listAnnotations", xrpc.ListAnnotations) | 205 | s.router.Get("/xrpc/at.glean.listAnnotations", xrpc.ListAnnotations) |
| 204 | s.router.Get("/xrpc/at.glean.listLikes", xrpc.ListLikes) | 206 | s.router.Get("/xrpc/at.glean.listLikes", xrpc.ListLikes) |
modified
internal/tmpl/partials/recommendation-card.html +9 -2 | @@ -1,5 +1,5 @@ | ||
| 1 | 1 | {{define "recommendation-card.html"}} |
| 2 | -<div class="bg-spot-surface rounded-xl p-3 hover:bg-spot-hover-50 transition"> | |
| 2 | +<div class="recommendation-card bg-spot-surface rounded-xl p-3 hover:bg-spot-hover-50 transition"> | |
| 3 | 3 | <div class="flex items-center justify-between gap-2"> |
| 4 | 4 | <a href="/articles?feed={{.feed_url}}" class="min-w-0 flex items-center gap-2 flex-1"> |
| 5 | 5 | {{if .favicon_url}}<img src="{{.favicon_url}}" class="w-4 h-4 rounded shrink-0" loading="lazy">{{end}} |
| @@ -10,7 +10,14 @@ | ||
| 10 | 10 | </a> |
| 11 | 11 | <div class="flex items-center gap-2 shrink-0"> |
| 12 | 12 | <span class="text-xs text-spot-secondary">{{.subscriber_count}} subs</span> |
| 13 | - <form hx-post="/feeds/add" hx-target="closest .bg-spot-surface" hx-swap="outerHTML" class="inline"> | |
| 13 | + <form hx-post="/feeds/dismiss" hx-target="closest .recommendation-card" hx-swap="outerHTML" class="inline"> | |
| 14 | + {{csrfInput .CSRFToken}} | |
| 15 | + <input type="hidden" name="feed_url" value="{{.feed_url}}"> | |
| 16 | + <button type="submit" title="Not interested" class="text-spot-muted hover:text-spot-red transition -m-1 p-1 rounded-full hover:bg-spot-hover-50 flex items-center"> | |
| 17 | + <svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="w-4 h-4"><path stroke-linecap="round" stroke-linejoin="round" d="M6 18 18 6M6 6l12 12"/></svg> | |
| 18 | + </button> | |
| 19 | + </form> | |
| 20 | + <form hx-post="/feeds/add" hx-target="closest .recommendation-card" hx-swap="outerHTML" class="inline"> | |
| 14 | 21 | {{csrfInput .CSRFToken}} |
| 15 | 22 | <input type="hidden" name="feed_url" value="{{.feed_url}}"> |
| 16 | 23 | <button type="submit" class="text-xs font-bold uppercase tracking-button text-spot-green hover:brightness-110 transition">Subscribe</button> |
| @@ -1,5 +1,5 @@ | |||
| 1 | {{define "recommendation-card.html"}} | 1 | {{define "recommendation-card.html"}} |
| 2 | -<div class="bg-spot-surface rounded-xl p-3 hover:bg-spot-hover-50 transition"> | 2 | +<div class="recommendation-card bg-spot-surface rounded-xl p-3 hover:bg-spot-hover-50 transition"> |
| 3 | <div class="flex items-center justify-between gap-2"> | 3 | <div class="flex items-center justify-between gap-2"> |
| 4 | <a href="/articles?feed={{.feed_url}}" class="min-w-0 flex items-center gap-2 flex-1"> | 4 | <a href="/articles?feed={{.feed_url}}" class="min-w-0 flex items-center gap-2 flex-1"> |
| 5 | {{if .favicon_url}}<img src="{{.favicon_url}}" class="w-4 h-4 rounded shrink-0" loading="lazy">{{end}} | 5 | {{if .favicon_url}}<img src="{{.favicon_url}}" class="w-4 h-4 rounded shrink-0" loading="lazy">{{end}} |
| @@ -10,7 +10,14 @@ | |||
| 10 | </a> | 10 | </a> |
| 11 | <div class="flex items-center gap-2 shrink-0"> | 11 | <div class="flex items-center gap-2 shrink-0"> |
| 12 | <span class="text-xs text-spot-secondary">{{.subscriber_count}} subs</span> | 12 | <span class="text-xs text-spot-secondary">{{.subscriber_count}} subs</span> |
| 13 | - <form hx-post="/feeds/add" hx-target="closest .bg-spot-surface" hx-swap="outerHTML" class="inline"> | 13 | + <form hx-post="/feeds/dismiss" hx-target="closest .recommendation-card" hx-swap="outerHTML" class="inline"> |
| 14 | + {{csrfInput .CSRFToken}} | ||
| 15 | + <input type="hidden" name="feed_url" value="{{.feed_url}}"> | ||
| 16 | + <button type="submit" title="Not interested" class="text-spot-muted hover:text-spot-red transition -m-1 p-1 rounded-full hover:bg-spot-hover-50 flex items-center"> | ||
| 17 | + <svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="w-4 h-4"><path stroke-linecap="round" stroke-linejoin="round" d="M6 18 18 6M6 6l12 12"/></svg> | ||
| 18 | + </button> | ||
| 19 | + </form> | ||
| 20 | + <form hx-post="/feeds/add" hx-target="closest .recommendation-card" hx-swap="outerHTML" class="inline"> | ||
| 14 | {{csrfInput .CSRFToken}} | 21 | {{csrfInput .CSRFToken}} |
| 15 | <input type="hidden" name="feed_url" value="{{.feed_url}}"> | 22 | <input type="hidden" name="feed_url" value="{{.feed_url}}"> |
| 16 | <button type="submit" class="text-xs font-bold uppercase tracking-button text-spot-green hover:brightness-110 transition">Subscribe</button> | 23 | <button type="submit" class="text-xs font-bold uppercase tracking-button text-spot-green hover:brightness-110 transition">Subscribe</button> |