Quellcode durchsuchen

feat:V564实验新增province_rovn召回并剔除9路冗余召回

- 新增 ProvinceRovnRecallStrategy,参照 city_rovn 实现按省份维度召回
- RecallService 在 564 实验门内挂载新召回 + removeIf 剔除 V564 不再 extract 的 9 路(v1/region系列5路/priori_province系列3路),减少无效 Redis 调用
- RankStrategy4RegionMergeModelV564 注释掉 v1 提取,新增 provinceRov(默认权重5)
- 新增 CLAUDE.md 记录项目提交规则、召回/排序架构、Apollo 约定,避免本地调试文件误入 PR

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
yangxiaohui vor 3 Tagen
Ursprung
Commit
8189e5450d

+ 47 - 0
CLAUDE.md

@@ -0,0 +1,47 @@
+# recommend-server 项目须知
+
+## 提交规则(最重要)
+
+以下文件/目录是**本地调试/同步用**,**不要 git add 进任何业务 PR**:
+
+- `recommend-server-service/src/main/resources/application-dev.yml`
+  - 用户本地会把 eureka / redis hostName / apollo meta 改成自己的调试地址
+  - 看到这个文件有 diff 时,**默认跳过它**(用户会自己处理)
+- `apollo/` 目录(如 `apollo/rank.score.merge.weightv564.json` 等)
+  - Git untracked,是 Apollo 后台配置的本地参考副本
+  - 改这个文件**不会**自动同步到 Apollo 后台,运维要单独操作
+  - **修改 Apollo JSON 永远不要 commit**
+- `docs/` 目录、`实验配置/` 目录 — 本地资料,同上
+
+`git add` 时手动列文件名,**不要用 `git add -A` 或 `git add .`**,避免把这些误带进去。
+
+## 召回 / 排序架构
+
+- **召回是公共池**:`RecallService.getStrategies()` 给所有用户挂载一组召回策略,并发跑 Redis 拿候选视频写入候选池
+- **Rank 实验各挑各的**:每个 `RankStrategy4RegionMergeModelVxxx` 在 `mergeAndRankRovRecall()` 里通过 `RecallUtils.extractRecall(weight, param, PUSH_FROM/FORM, ...)` 按 `PUSH_FROM` 字符串挑想要的召回路
+- **推论**:往 `RecallService` 加新召回**不会自动**进任何 rank 实验,必须同时改对应 rank 类的 `extractRecall` 调用才能生效
+- **实验门控的两种模式**:
+  - 加法门(主流):`if (isHitXxxExp) strategies.add(...)` — 命中实验才挂载(842/568 用此模式)
+  - 减法门(V564 新增):`if (isHit564Exp) { strategies.add(...); strategies.removeIf(...) }` — 命中实验时挂新召回 + 剔除该实验不要的召回,省 Redis 调用
+- **`judgeHitExp` 用 `rootSessionId` 分桶**:同一用户同一会话里要么始终命中要么始终不命中,召回/排序天然对齐
+
+## 命名陷阱
+
+- `PUSH_FORM`(拼错的,6+ 处)vs `PUSH_FROM`(正确,3 处)在 recall strategy 里并存
+  - 加新策略时**用 `PUSH_FROM`**,别复制旧的拼错版本
+  - 写过滤逻辑时**别用 `pushFrom()` 字符串过滤**,改用 `getClass().getSimpleName()`,绕开这个不一致
+- 新建 RecallStrategy 时只要 `@Component implements RecallStrategy`,Spring 会自动按 `getClass().getSimpleName()` 注册到 `RecallService` 的 `strategyMap`,然后再去 `getStrategies()` 里 `add` 一行就生效
+
+## Apollo 配置
+
+- 文件名约定:`apollo/rank.score.merge.weightv{NNN}.json` 对应 rank 类的 `@ApolloJsonValue("${rank.score.merge.weightv{NNN}:}")`
+- Key 命名约定:`xxxRov` / `xxxRos` / `xxxStr` 等,与 `mergeWeight.getOrDefault("xxx", 默认值)` 一致
+- **改了 rank 类的 `extractRecall` 列表后,Apollo JSON 要同步清理废弃 key**(不会报错,但是死配置)
+- 代码里 `getOrDefault(key, 默认值)` 提供兜底,所以 Apollo 没配也能跑
+
+## V564 实验当前状态(2026-05-20 改动后)
+
+- 基于 V569,相比 V569:
+  - **召回侧**:剔除 9 路(v1/region 系列 5 路 / priori_province 系列 3 路),新增 `province_rovn`
+  - **排序侧**:`provinceRov:5` 权重默认值(可在 Apollo JSON 覆盖)
+- 召回侧的剔除通过 `RecallService.java` 里 `if (isHit564Exp)` 块的 `removeIf` 实现,要回滚整块删除即可

+ 8 - 2
recommend-server-service/src/main/java/com/tzld/piaoquan/recommend/server/service/rank/strategy/RankStrategy4RegionMergeModelV564.java

@@ -25,7 +25,7 @@ import java.util.concurrent.Future;
 import java.util.concurrent.TimeUnit;
 
 /**
- * V564 实验:基于 V569,移除以下 8 路召回
+ * V564 实验:基于 V569,移除以下 9 路召回
  *   - recall_pool_region_h     (RegionHRecallStrategy)
  *   - recall_pool_24h          (RegionRelative24HRecallStrategy)
  *   - recall_pool_region_24h   (Region24HRecallStrategy)
@@ -34,6 +34,9 @@ import java.util.concurrent.TimeUnit;
  *   - priori_province_rovn     (PrioriProvinceRovnRecallStrategy)
  *   - priori_province_str      (PrioriProvinceStrRecallStrategy)
  *   - priori_province_ros      (PrioriProvinceRosRecallStrategy)
+ *   - v1                       (RegionRealtimeRecallStrategyV1)
+ * 新增 1 路召回:
+ *   - province_rovn            (ProvinceRovnRecallStrategy)
  */
 @Service
 @Slf4j
@@ -61,7 +64,8 @@ public class RankStrategy4RegionMergeModelV564 extends RankStrategy4RegionMergeM
         //-------------------return相似召回------------------
         RecallUtils.extractRecall(mergeWeight.getOrDefault("v6", 5.0).intValue(), param, ReturnVideoRecallStrategy.PUSH_FORM, setVideo, rovRecallRank);
         //-------------------新地域召回------------------
-        RecallUtils.extractRecall(mergeWeight.getOrDefault("v1", 5.0).intValue(), param, RegionRealtimeRecallStrategyV1.PUSH_FORM, setVideo, rovRecallRank);
+        // V564: 移除 v1 新地域召回
+        // RecallUtils.extractRecall(mergeWeight.getOrDefault("v1", 5.0).intValue(), param, RegionRealtimeRecallStrategyV1.PUSH_FORM, setVideo, rovRecallRank);
         //-------------------scene cf rovn------------------
         RecallUtils.extractRecall(mergeWeight.getOrDefault("sceneCFRovn", 5.0).intValue(), param, SceneCFRovnRecallStrategy.PUSH_FORM, setVideo, rovRecallRank);
         //-------------------scene cf rosn------------------
@@ -78,6 +82,8 @@ public class RankStrategy4RegionMergeModelV564 extends RankStrategy4RegionMergeM
         RecallUtils.extractRecall(mergeWeight.getOrDefault("headCate2Rov", 5.0).intValue(), param, HeadCate2RovRecallStrategy.PUSH_FROM, setVideo, rovRecallRank);
         //-------------------city rovn------------------
         RecallUtils.extractRecall(mergeWeight.getOrDefault("cityRov", 5.0).intValue(), param, CityRovnRecallStrategy.PUSH_FROM, setVideo, rovRecallRank);
+        //-------------------province rovn------------------
+        RecallUtils.extractRecall(mergeWeight.getOrDefault("provinceRov", 5.0).intValue(), param, ProvinceRovnRecallStrategy.PUSH_FROM, setVideo, rovRecallRank);
         // V564: 移除 priori_province_rovn / priori_province_str / priori_province_ros
         //-------------------return1 cate2 ros------------------
         RecallUtils.extractRecall(mergeWeight.getOrDefault("return1Cate2Ros", 5.0).intValue(), param, Return1Cate2RosRecallStrategy.PUSH_FORM, setVideo, rovRecallRank);

+ 18 - 0
recommend-server-service/src/main/java/com/tzld/piaoquan/recommend/server/service/recall/RecallService.java

@@ -153,6 +153,24 @@ public class RecallService implements ApplicationContextAware {
             strategies.add(strategyMap.get(YearReturnCate2RecallStrategy.class.getSimpleName()));
         }
 
+        boolean isHit564Exp = experimentService.judgeHitExp(param.getAppType(), param.getRootSessionId(), abExpCodes, "564");
+        if (isHit564Exp) {
+            strategies.add(strategyMap.get(ProvinceRovnRecallStrategy.class.getSimpleName()));
+            // V564: rank 侧不再 extract 以下 9 路召回,这里直接剔除避免无效 Redis 调用
+            Set<String> v564RemoveSet = new HashSet<>(Arrays.asList(
+                    RegionRealtimeRecallStrategyV1.class.getSimpleName(),
+                    RegionHRecallStrategy.class.getSimpleName(),
+                    Region24HRecallStrategy.class.getSimpleName(),
+                    RegionHDupRecallStrategy.class.getSimpleName(),
+                    RegionRelative24HRecallStrategy.class.getSimpleName(),
+                    RegionRelative24HDupRecallStrategy.class.getSimpleName(),
+                    PrioriProvinceRovnRecallStrategy.class.getSimpleName(),
+                    PrioriProvinceStrRecallStrategy.class.getSimpleName(),
+                    PrioriProvinceRosRecallStrategy.class.getSimpleName()
+            ));
+            strategies.removeIf(s -> s != null && v564RemoveSet.contains(s.getClass().getSimpleName()));
+        }
+
         // 命中用户黑名单不走流量池
         // 命中安全测试风险地域不走流量池
         if (!param.isRiskUser() && !param.isTestingRiskRegion()) {

+ 102 - 0
recommend-server-service/src/main/java/com/tzld/piaoquan/recommend/server/service/recall/strategy/ProvinceRovnRecallStrategy.java

@@ -0,0 +1,102 @@
+package com.tzld.piaoquan.recommend.server.service.recall.strategy;
+
+import com.tzld.piaoquan.recommend.server.model.Video;
+import com.tzld.piaoquan.recommend.server.service.filter.FilterParam;
+import com.tzld.piaoquan.recommend.server.service.filter.FilterResult;
+import com.tzld.piaoquan.recommend.server.service.filter.FilterService;
+import com.tzld.piaoquan.recommend.server.service.recall.FilterParamFactory;
+import com.tzld.piaoquan.recommend.server.service.recall.RecallParam;
+import com.tzld.piaoquan.recommend.server.service.recall.RecallStrategy;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.commons.collections4.CollectionUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.commons.lang3.tuple.MutablePair;
+import org.apache.commons.lang3.tuple.Pair;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.beans.factory.annotation.Qualifier;
+import org.springframework.data.redis.core.RedisTemplate;
+import org.springframework.stereotype.Component;
+
+import java.util.*;
+import java.util.stream.Collectors;
+
+@Component
+@Slf4j
+public class ProvinceRovnRecallStrategy implements RecallStrategy {
+    private final String CLASS_NAME = this.getClass().getSimpleName();
+    @Autowired
+    private FilterService filterService;
+    @Autowired
+    @Qualifier("redisTemplate")
+    public RedisTemplate<String, String> redisTemplate;
+
+    public static final String PUSH_FROM = "province_rovn";
+    public static final String redisKeyPrefix = "province_rovn_recall";
+
+    @Override
+    public String pushFrom() {
+        return PUSH_FROM;
+    }
+
+    @Override
+    public List<Video> recall(RecallParam param) {
+        List<Video> videosResult = new ArrayList<>();
+        try {
+            String province = param.getProvince().replaceAll("省$", "");
+            String redisKey = String.format("%s:%s", redisKeyPrefix, province);
+            String redisValue = redisTemplate.opsForValue().get(redisKey);
+            if (null == redisValue || redisValue.isEmpty()) {
+                return videosResult;
+            }
+            Pair<List<Long>, Map<Long, Double>> pair = parsePair(redisValue, param.getVideoId(), 100);
+            fillVideoResult(param, pair, videosResult);
+        } catch (Exception e) {
+            log.error("recall is wrong in {}, error={}", CLASS_NAME, e);
+        }
+        return videosResult;
+    }
+
+    private Pair<List<Long>, Map<Long, Double>> parsePair(String data, long headVid, int size) {
+        List<Long> idsList = new ArrayList<>();
+        Map<Long, Double> scoresMap = new HashMap<>();
+        if (!StringUtils.isBlank(data)) {
+            String[] cells = data.split("\t");
+            if (2 == cells.length) {
+                List<Long> ids = Arrays.stream(cells[0].split(",")).map(Long::valueOf).collect(Collectors.toList());
+                List<Double> scores = Arrays.stream(cells[1].split(",")).map(Double::valueOf).collect(Collectors.toList());
+                if (!ids.isEmpty() && ids.size() == scores.size()) {
+                    int minSize = Math.min(size, ids.size());
+                    for (int i = 0; i < minSize; ++i) {
+                        long id = ids.get(i);
+                        double score = scores.get(i);
+                        if (headVid != id && !scoresMap.containsKey(id)) {
+                            idsList.add(id);
+                            scoresMap.put(id, score);
+                        }
+                    }
+                }
+            }
+        }
+        return new MutablePair<>(idsList, scoresMap);
+    }
+
+    private void fillVideoResult(RecallParam param, Pair<List<Long>, Map<Long, Double>> pair, List<Video> videosResult) {
+        if (null != pair) {
+            List<Long> ids = pair.getLeft();
+            Map<Long, Double> scoresMap = pair.getRight();
+            if (null != ids && null != scoresMap && !ids.isEmpty()) {
+                FilterParam filterParam = FilterParamFactory.create(param, ids);
+                FilterResult filterResult = filterService.filter(filterParam);
+                if (null != filterResult && CollectionUtils.isNotEmpty(filterResult.getVideoIds())) {
+                    filterResult.getVideoIds().forEach(vid -> {
+                        Video video = new Video();
+                        video.setVideoId(vid);
+                        video.setRovScore(scoresMap.getOrDefault(vid, 0D));
+                        video.setPushFrom(pushFrom());
+                        videosResult.add(video);
+                    });
+                }
+            }
+        }
+    }
+}