|
|
@@ -0,0 +1,982 @@
|
|
|
+# DemandAgent 技术迭代 V2：DB 派生 Seed Terms 与 Query Seed Points
|
|
|
+
|
|
|
+## 1. 目标
|
|
|
+
|
|
|
+本轮只改 `examples/demand` 业务域，不改 `agent/` 通用框架。
|
|
|
+
|
|
|
+技术目标：
|
|
|
+
|
|
|
+1. `seed_terms` 不再读取 LLM 的 `evidence_refs.seed_terms`，改为从 PG `itemset_items` 确定性派生。
|
|
|
+2. 新增 `query_seed_points`，从当前 demand 的全部票圈支撑帖中筛出能命中当前 itemset 分类节点的点位，再按 `point_text` 覆盖度聚合 Top-K。
|
|
|
+3. 云端 MySQL 单表入口恢复高权重元素 / 分类工具，作为候选召回和排序线索。
|
|
|
+4. 恢复 Hive 供需缺口业务入口：只读 Hive 得到 `merge二级品类 + count`，自动接 PG 最新 success execution。
|
|
|
+5. `get_frequent_itemsets` / PG itemset 查询必须支持按 `merge_leve2/platform` 过滤 `pattern_itemset_post -> post` 的支撑帖，并用过滤后的 support 做候选排序和门槛判断。
|
|
|
+6. 保持 `demand_content` 顶层字段、MySQL 单表 schema、旧 `evidence_pack` 字段全部兼容。
|
|
|
+
|
|
|
+## 2. 当前代码证据
|
|
|
+
|
|
|
+### 2.1 当前 `seed_terms` 会受 LLM 影响
|
|
|
+
|
|
|
+文件：
|
|
|
+
|
|
|
+```text
|
|
|
+/Users/samlee/Documents/works/DemandAgentNew/examples/demand/evidence_pack_builder.py
|
|
|
+```
|
|
|
+
|
|
|
+现状：
|
|
|
+
|
|
|
+- `build_evidence_pack()` 在构造 `evidence_pack` 前调用 `_resolve_seed_terms(...)`。
|
|
|
+- `_resolve_seed_terms()` 会先读取 `evidence_refs.get("seed_terms")`。
|
|
|
+- 如果 LLM 提供的词能被 DB evidence 覆盖，就直接返回这组词。
|
|
|
+- 只有没有可用 LLM seed 时，才从 `itemset_items` 派生。
|
|
|
+
|
|
|
+需要调整为：
|
|
|
+
|
|
|
+- 完全忽略 `evidence_refs.seed_terms`。
|
|
|
+- 所有 `seed_terms` 都从 PG `itemset_items` 派生。
|
|
|
+
|
|
|
+### 2.2 当前 itemset items 已经能支持 DB 派生
|
|
|
+
|
|
|
+文件：
|
|
|
+
|
|
|
+```text
|
|
|
+/Users/samlee/Documents/works/DemandAgentNew/examples/demand/pg_pattern_repository.py
|
|
|
+```
|
|
|
+
|
|
|
+函数：
|
|
|
+
|
|
|
+```python
|
|
|
+query_itemset_items_with_categories(execution_id, itemset_ids)
|
|
|
+```
|
|
|
+
|
|
|
+已返回字段：
|
|
|
+
|
|
|
+- `itemset_item_id`
|
|
|
+- `itemset_id`
|
|
|
+- `point_type`
|
|
|
+- `dimension`
|
|
|
+- `category_id`
|
|
|
+- `category_path`
|
|
|
+- `element_name`
|
|
|
+- `category_name`
|
|
|
+- `category_full_path`
|
|
|
+- `category_level`
|
|
|
+- `category_nature`
|
|
|
+
|
|
|
+这些字段足够派生：
|
|
|
+
|
|
|
+```text
|
|
|
+seed_terms
|
|
|
+```
|
|
|
+
|
|
|
+### 2.3 当前 `element_bindings.sample_elements` 只是证据样例，不足以承担搜索点位池职责
|
|
|
+
|
|
|
+文件：
|
|
|
+
|
|
|
+```text
|
|
|
+/Users/samlee/Documents/works/DemandAgentNew/examples/demand/evidence_pack_builder.py
|
|
|
+```
|
|
|
+
|
|
|
+现状：
|
|
|
+
|
|
|
+```python
|
|
|
+query_element_bindings_for_items(
|
|
|
+ execution_id=execution_id,
|
|
|
+ itemset_items=itemset_items,
|
|
|
+ post_ids=[requested_source_post_id],
|
|
|
+)
|
|
|
+```
|
|
|
+
|
|
|
+它只围绕一个 `source_post_id` 查证据，主要用于证明 itemset item 能闭合到 PG topic element。它不是从全部 `matched_post_ids` 里聚合点位，也不是 DemandAgent 原本设计的搜索词池。
|
|
|
+
|
|
|
+CFA 当前临时把 `element_bindings.sample_elements` 当搜索种子，是因为 V1 还没有专门的 `query_seed_points` 字段；这不是它原本的设计职责。因此 V2 不是“把旧搜索词字段改成非搜索词”，而是补一个专门给 CFA 首轮搜索使用的新字段。
|
|
|
+
|
|
|
+V2 需要新增独立查询：
|
|
|
+
|
|
|
+```python
|
|
|
+query_seed_points_for_itemsets(...)
|
|
|
+```
|
|
|
+
|
|
|
+不要复用 `element_bindings.sample_elements` 当 Top-K 搜索点位。
|
|
|
+
|
|
|
+### 2.4 当前 MySQL 单表入口工具集太窄
|
|
|
+
|
|
|
+文件：
|
|
|
+
|
|
|
+```text
|
|
|
+/Users/samlee/Documents/works/DemandAgentNew/examples/demand/run.py
|
|
|
+```
|
|
|
+
|
|
|
+函数：
|
|
|
+
|
|
|
+```python
|
|
|
+_enabled_tools_for_run()
|
|
|
+```
|
|
|
+
|
|
|
+当前 MySQL 单表模式只允许：
|
|
|
+
|
|
|
+- `get_frequent_itemsets`
|
|
|
+- `get_itemset_detail`
|
|
|
+- `create_demand_item`
|
|
|
+- `create_demand_items`
|
|
|
+
|
|
|
+V2 需要允许：
|
|
|
+
|
|
|
+- `get_weight_score_topn`
|
|
|
+- `get_weight_score_by_name`
|
|
|
+
|
|
|
+并让 MySQL 单表模式也能生成 PG 权重 JSON。
|
|
|
+
|
|
|
+### 2.5 当前 Hive 缺口入口和 PG 品类过滤还没有接通
|
|
|
+
|
|
|
+Hive 供需缺口读取函数仍在：
|
|
|
+
|
|
|
+```text
|
|
|
+/Users/samlee/Documents/works/DemandAgentNew/examples/demand/data_query_tools.py
|
|
|
+```
|
|
|
+
|
|
|
+函数：
|
|
|
+
|
|
|
+```python
|
|
|
+get_demand_merge_level2_names()
|
|
|
+```
|
|
|
+
|
|
|
+它从 Hive/ODPS `loghubods.video_dimension_detail_add_column` 按 `merge二级品类` 聚合，筛选 `缺量 >= 50`，并返回：
|
|
|
+
|
|
|
+```json
|
|
|
+{
|
|
|
+ "cluster_name": "知识科普",
|
|
|
+ "platform_type": "piaoquan",
|
|
|
+ "count": 40
|
|
|
+}
|
|
|
+```
|
|
|
+
|
|
|
+注意：`loghubods.video_dimension_detail_add_column` 是缺口计算来源表；`loghubods.dwd_multi_demand_pool_di` 是旧链路输出/同步表，字段为 `strategy/demand_id/demand_name/weight/type/video_count/video_list/extend/dt`，不是 V2 缺口入口来源。
|
|
|
+
|
|
|
+但新版当前的 PG execution 选择逻辑仍是全局 fallback：
|
|
|
+
|
|
|
+```text
|
|
|
+/Users/samlee/Documents/works/DemandAgentNew/examples/demand/run.py
|
|
|
+```
|
|
|
+
|
|
|
+```python
|
|
|
+get_execution_id_by_merge_level2(cluster_name) -> query_latest_success_execution_id()
|
|
|
+```
|
|
|
+
|
|
|
+也就是说，`cluster_name/merge_leve2` 目前没有真正进入 PG itemset 候选过滤。
|
|
|
+
|
|
|
+工具层其实已经暴露了 `merge_leve2` 参数：
|
|
|
+
|
|
|
+```text
|
|
|
+/Users/samlee/Documents/works/DemandAgentNew/examples/demand/demand_pattern_tools.py
|
|
|
+```
|
|
|
+
|
|
|
+```python
|
|
|
+get_frequent_itemsets(..., account_name=None, merge_leve2=None)
|
|
|
+```
|
|
|
+
|
|
|
+但 PG service 当前直接丢弃：
|
|
|
+
|
|
|
+```text
|
|
|
+/Users/samlee/Documents/works/DemandAgentNew/examples/demand/pattern_builds/pg_pattern_service.py
|
|
|
+```
|
|
|
+
|
|
|
+```python
|
|
|
+del account_name, merge_leve2
|
|
|
+```
|
|
|
+
|
|
|
+V2 必须把这条参数真实接到 PG SQL：
|
|
|
+
|
|
|
+```text
|
|
|
+pattern_itemset -> pattern_itemset_post -> post.merge_leve2/platform
|
|
|
+```
|
|
|
+
|
|
|
+并用过滤后的支撑帖数量重算 `filtered_absolute_support/scoped_post_count`。这样才能恢复老版“从供需缺口品类出发”的业务入口，同时不恢复 `run_mining()`。
|
|
|
+
|
|
|
+## 3. 代码改动计划
|
|
|
+
|
|
|
+### 3.1 修改 prompt，收紧 LLM 职责
|
|
|
+
|
|
|
+涉及文件：
|
|
|
+
|
|
|
+- `/Users/samlee/Documents/works/DemandAgentNew/examples/demand/demand.md`
|
|
|
+- `/Users/samlee/Documents/works/DemandAgentNew/examples/demand/demand_mysql.md`
|
|
|
+
|
|
|
+修改点：
|
|
|
+
|
|
|
+1. 删除示例 JSON 中的 `evidence_refs.seed_terms`。
|
|
|
+2. 删除“LLM 需要填写 seed_terms”的描述。
|
|
|
+3. 新增说明：
|
|
|
+
|
|
|
+```text
|
|
|
+seed_terms、query_seed_points 均由代码从 PG DB 派生，LLM 不要填写。
|
|
|
+```
|
|
|
+
|
|
|
+4. MySQL 单表 prompt 不再写“不要查询权重文件”。
|
|
|
+5. MySQL 单表 prompt 的执行顺序改为：
|
|
|
+
|
|
|
+```text
|
|
|
+先用 get_weight_score_topn 看高权重元素/分类
|
|
|
+再用 get_frequent_itemsets 找能支撑这些方向的 topic itemset
|
|
|
+再用 get_itemset_detail 拿详情
|
|
|
+最后 create_demand_items
|
|
|
+```
|
|
|
+
|
|
|
+LLM 仍可输出：
|
|
|
+
|
|
|
+- `element_names`
|
|
|
+- `reason`
|
|
|
+- `desc`
|
|
|
+- `type`
|
|
|
+- `evidence_refs.source_kind`
|
|
|
+- `evidence_refs.source_tool`
|
|
|
+- `evidence_refs.itemset_ids`
|
|
|
+- `evidence_refs.source_post_id`
|
|
|
+- `evidence_refs.case_ids`
|
|
|
+
|
|
|
+### 3.2 恢复 MySQL 单表入口高权重工具
|
|
|
+
|
|
|
+涉及文件：
|
|
|
+
|
|
|
+- `/Users/samlee/Documents/works/DemandAgentNew/examples/demand/run.py`
|
|
|
+
|
|
|
+修改函数：
|
|
|
+
|
|
|
+```python
|
|
|
+_enabled_tools_for_run()
|
|
|
+```
|
|
|
+
|
|
|
+当前：
|
|
|
+
|
|
|
+```python
|
|
|
+allowed = {
|
|
|
+ "get_frequent_itemsets",
|
|
|
+ "get_itemset_detail",
|
|
|
+ "create_demand_item",
|
|
|
+ "create_demand_items",
|
|
|
+}
|
|
|
+```
|
|
|
+
|
|
|
+V2：
|
|
|
+
|
|
|
+```python
|
|
|
+allowed = {
|
|
|
+ "get_weight_score_topn",
|
|
|
+ "get_weight_score_by_name",
|
|
|
+ "get_frequent_itemsets",
|
|
|
+ "get_itemset_detail",
|
|
|
+ "create_demand_item",
|
|
|
+ "create_demand_items",
|
|
|
+}
|
|
|
+```
|
|
|
+
|
|
|
+同时修改 `_ensure_pg_weight_score_files()` 的调用条件。
|
|
|
+
|
|
|
+当前只在 `local_json` 模式调用：
|
|
|
+
|
|
|
+```python
|
|
|
+if _is_local_json_mode():
|
|
|
+ _ensure_pg_weight_score_files(int(execution_id))
|
|
|
+```
|
|
|
+
|
|
|
+V2 改为：
|
|
|
+
|
|
|
+```python
|
|
|
+if _is_local_json_mode() or _is_mysql_demand_content_mode():
|
|
|
+ _ensure_pg_weight_score_files(int(execution_id))
|
|
|
+```
|
|
|
+
|
|
|
+并调整 `_ensure_pg_weight_score_files()` 函数注释，不再说只服务 local JSON。
|
|
|
+
|
|
|
+### 3.3 恢复 Hive 供需缺口驱动入口，但不恢复 run_mining
|
|
|
+
|
|
|
+新增入口文件：
|
|
|
+
|
|
|
+```text
|
|
|
+/Users/samlee/Documents/works/DemandAgentNew/examples/demand/run_hive_gap_mysql.py
|
|
|
+```
|
|
|
+
|
|
|
+职责：
|
|
|
+
|
|
|
+1. 调用 `get_demand_merge_level2_names()` 只读 Hive/ODPS，拿到 `cluster_name/platform_type/count`。
|
|
|
+2. 调用 `query_latest_success_execution_id()` 选择最新 `status='success'` 且 topic itemset 非空的 PG Pattern V2 execution；不能只看 `is_current`。
|
|
|
+3. 对每个缺口品类调用现有 `run.main(...)`：
|
|
|
+
|
|
|
+```python
|
|
|
+await main(
|
|
|
+ cluster_name=item["cluster_name"],
|
|
|
+ platform_type=item["platform_type"],
|
|
|
+ count=item["count"],
|
|
|
+ execution_id=latest_success_execution_id,
|
|
|
+ task_id=None,
|
|
|
+)
|
|
|
+```
|
|
|
+
|
|
|
+4. 强制 `DEMAND_OUTPUT_MODE=mysql_demand_content`，只写 MySQL `demand_content`。
|
|
|
+5. 禁止调用 `piaoquan_prepare()`、`prepare()`、`run_mining()`。
|
|
|
+
|
|
|
+建议参数：
|
|
|
+
|
|
|
+```bash
|
|
|
+python -m examples.demand.run_hive_gap_mysql \
|
|
|
+ --run-label hive_gap_v2_YYYYMMDD \
|
|
|
+ --max-categories 10 \
|
|
|
+ --max-total-demands 100
|
|
|
+```
|
|
|
+
|
|
|
+这里的 `execution_id` 不是 DemandAgent 执行 ID，而是 PG `pattern_mining_execution.id`。Hive 只决定品类和数量，PG latest success execution 决定 Pattern 事实快照。
|
|
|
+
|
|
|
+### 3.4 接通 PG itemset 的 merge_leve2/platform 过滤
|
|
|
+
|
|
|
+涉及文件：
|
|
|
+
|
|
|
+- `/Users/samlee/Documents/works/DemandAgentNew/examples/demand/pg_pattern_repository.py`
|
|
|
+- `/Users/samlee/Documents/works/DemandAgentNew/examples/demand/pattern_builds/pg_pattern_service.py`
|
|
|
+- `/Users/samlee/Documents/works/DemandAgentNew/examples/demand/demand_pattern_tools.py`
|
|
|
+
|
|
|
+#### 3.4.1 repository 改造
|
|
|
+
|
|
|
+修改：
|
|
|
+
|
|
|
+```python
|
|
|
+def query_topic_itemsets(..., merge_leve2=None, platform=None, ...)
|
|
|
+```
|
|
|
+
|
|
|
+当传入 `merge_leve2/platform` 时，追加过滤并重算支撑帖：
|
|
|
+
|
|
|
+```sql
|
|
|
+LEFT JOIN LATERAL (
|
|
|
+ SELECT
|
|
|
+ array_agg(DISTINCT pip_scope.post_id ORDER BY pip_scope.post_id) AS filtered_matched_post_ids,
|
|
|
+ COUNT(DISTINCT pip_scope.post_id) AS filtered_absolute_support
|
|
|
+ FROM pattern_itemset_post pip_scope
|
|
|
+ JOIN post p_scope
|
|
|
+ ON p_scope.post_id = pip_scope.post_id
|
|
|
+ WHERE pip_scope.execution_id = i.execution_id
|
|
|
+ AND pip_scope.itemset_id = i.id
|
|
|
+ AND p_scope.merge_leve2 = ANY(%s)
|
|
|
+ AND p_scope.platform = ANY(%s)
|
|
|
+) scoped ON TRUE
|
|
|
+```
|
|
|
+
|
|
|
+过滤条件：
|
|
|
+
|
|
|
+```sql
|
|
|
+COALESCE(scoped.filtered_absolute_support, 0) >= min_support
|
|
|
+```
|
|
|
+
|
|
|
+注意：
|
|
|
+
|
|
|
+- `merge_leve2` 支持字符串或列表；列表为 OR。
|
|
|
+- `platform_type=piaoquan` 兼容 `piaoquan/票圈`。
|
|
|
+- 返回字段中增加 `scope_filter`、`filtered_matched_post_ids`、`filtered_absolute_support`。
|
|
|
+- 排序优先使用 `filtered_absolute_support DESC, absolute_support DESC`，避免全局高支持度但当前品类支撑很弱的 itemset 抢占候选。
|
|
|
+- 保留全局 `absolute_support/support`，但下游 evidence 默认使用过滤后的支撑帖。
|
|
|
+
|
|
|
+#### 3.4.2 service 改造
|
|
|
+
|
|
|
+当前：
|
|
|
+
|
|
|
+```python
|
|
|
+del account_name, merge_leve2
|
|
|
+```
|
|
|
+
|
|
|
+V2 改为：
|
|
|
+
|
|
|
+```python
|
|
|
+rows = repo.query_topic_itemsets(
|
|
|
+ execution_id=execution_id,
|
|
|
+ category_ids=category_ids,
|
|
|
+ dimension_mode=dimension_mode,
|
|
|
+ min_support=min_support,
|
|
|
+ min_item_count=min_item_count,
|
|
|
+ max_item_count=max_item_count,
|
|
|
+ sort_by=sort_by,
|
|
|
+ merge_leve2=merge_leve2,
|
|
|
+ platform=platform,
|
|
|
+ limit=max(int(top_n or 20) * 10, int(top_n or 20)),
|
|
|
+)
|
|
|
+```
|
|
|
+
|
|
|
+`demand_pattern_tools.get_frequent_itemsets()` 已经有 `merge_leve2` 参数；V2 需要补 `platform` 参数，prompt 要求 LLM 在当前品类任务中调用：
|
|
|
+
|
|
|
+```python
|
|
|
+get_frequent_itemsets(..., merge_leve2="%merge_level2%", platform="%platform_type%")
|
|
|
+```
|
|
|
+
|
|
|
+#### 3.4.3 detail 和 evidence scope 改造
|
|
|
+
|
|
|
+`get_itemset_detail()` 要返回同一 scope 下的过滤后帖子，或至少返回：
|
|
|
+
|
|
|
+```json
|
|
|
+{
|
|
|
+ "scope_filter": {
|
|
|
+ "merge_leve2": "知识科普",
|
|
|
+ "platform_type": "piaoquan"
|
|
|
+ },
|
|
|
+ "filtered_absolute_support": 59,
|
|
|
+ "filtered_matched_post_ids": ["..."]
|
|
|
+}
|
|
|
+```
|
|
|
+
|
|
|
+`evidence_pack` 增加业务 scope：
|
|
|
+
|
|
|
+```json
|
|
|
+{
|
|
|
+ "demand_scope": {
|
|
|
+ "source": "odps_gap",
|
|
|
+ "merge_leve2": "知识科普",
|
|
|
+ "platform_type": "piaoquan",
|
|
|
+ "gap_dt": "20260622",
|
|
|
+ "requested_count": 70,
|
|
|
+ "lack_count": 1443.21,
|
|
|
+ "pattern_execution_id": 581
|
|
|
+ }
|
|
|
+}
|
|
|
+```
|
|
|
+
|
|
|
+并要求：
|
|
|
+
|
|
|
+- `matched_post_ids/video_ids/case_ids` 优先使用当前 `merge_leve2/platform` scope 下的支撑帖。
|
|
|
+- `source_post_id` 必须来自 scoped `matched_post_ids`。
|
|
|
+- `query_seed_points` 必须基于 scoped `matched_post_ids` 聚合。
|
|
|
+- `filtered_absolute_support/scoped_post_count` 必填，并且必须小于等于全局 `absolute_support`。
|
|
|
+- 如果 itemset 全局存在但 scoped posts 为空，当前 DemandItem rejected。
|
|
|
+
|
|
|
+### 3.5 重写 seed_terms 派生函数
|
|
|
+
|
|
|
+涉及文件：
|
|
|
+
|
|
|
+- `/Users/samlee/Documents/works/DemandAgentNew/examples/demand/evidence_pack_builder.py`
|
|
|
+
|
|
|
+新增常量：
|
|
|
+
|
|
|
+```python
|
|
|
+SEED_TERM_MAX_COUNT = int(os.getenv("DEMAND_SEED_TERM_MAX_COUNT", "10"))
|
|
|
+SEED_TERM_STOPWORDS = {...}
|
|
|
+```
|
|
|
+
|
|
|
+建议先写死基础 stopwords，再允许环境变量追加：
|
|
|
+
|
|
|
+```text
|
|
|
+分享,内容,视频,表达,展示,讲述,记录,画面
|
|
|
+```
|
|
|
+
|
|
|
+新增函数：
|
|
|
+
|
|
|
+```python
|
|
|
+def _build_seed_terms_from_itemset_items(itemset_items: list[dict[str, Any]]) -> list[str]:
|
|
|
+ ...
|
|
|
+```
|
|
|
+
|
|
|
+派生规则：
|
|
|
+
|
|
|
+```text
|
|
|
+for item in itemset_items:
|
|
|
+ candidate = element_name
|
|
|
+ or category_name
|
|
|
+ or last(category_path)
|
|
|
+ or last(category_full_path)
|
|
|
+```
|
|
|
+
|
|
|
+要求：
|
|
|
+
|
|
|
+- 去重。
|
|
|
+- 保持 itemset item 顺序。
|
|
|
+- `seed_terms` 只从 DB itemset 字段过滤和裁剪。
|
|
|
+- 不新增完整词表字段。
|
|
|
+
|
|
|
+替换当前逻辑：
|
|
|
+
|
|
|
+```python
|
|
|
+seed_terms = _resolve_seed_terms(evidence_refs, itemset_items, element_bindings)
|
|
|
+```
|
|
|
+
|
|
|
+改为：
|
|
|
+
|
|
|
+```python
|
|
|
+seed_terms = _build_seed_terms_from_itemset_items(itemset_items)
|
|
|
+```
|
|
|
+
|
|
|
+只保留现有 `seed_terms` 字段，不新增完整词表字段：
|
|
|
+
|
|
|
+```python
|
|
|
+"seed_terms": seed_terms,
|
|
|
+```
|
|
|
+
|
|
|
+保留 `_validate_seed_terms()`，但语义改成校验：
|
|
|
+
|
|
|
+- `seed_terms` 非空。
|
|
|
+- `seed_terms` 必须能从 `itemset_items` 的 `element_name/category_name/category_path/category_full_path` 派生。
|
|
|
+
|
|
|
+### 3.6 新增 query_seed_points PG 查询
|
|
|
+
|
|
|
+涉及文件：
|
|
|
+
|
|
|
+- `/Users/samlee/Documents/works/DemandAgentNew/examples/demand/pg_pattern_repository.py`
|
|
|
+- `/Users/samlee/Documents/works/DemandAgentNew/examples/demand/db_manager.py`
|
|
|
+- `/Users/samlee/Documents/works/DemandAgentNew/examples/demand/evidence_pack_builder.py`
|
|
|
+
|
|
|
+#### 3.6.1 repository 新函数
|
|
|
+
|
|
|
+新增函数：
|
|
|
+
|
|
|
+```python
|
|
|
+def query_seed_points_for_itemsets(
|
|
|
+ execution_id: int,
|
|
|
+ itemset_ids: Iterable[Any],
|
|
|
+ matched_post_ids: Iterable[Any],
|
|
|
+ *,
|
|
|
+ platform: str = "piaoquan",
|
|
|
+ top_k: int = 30,
|
|
|
+) -> list[dict[str, Any]]:
|
|
|
+ ...
|
|
|
+```
|
|
|
+
|
|
|
+SQL 逻辑：
|
|
|
+
|
|
|
+```sql
|
|
|
+WITH item_cats AS (
|
|
|
+ SELECT DISTINCT category_id
|
|
|
+ FROM pattern_itemset_item
|
|
|
+ WHERE itemset_id = ANY(%(itemset_ids)s)
|
|
|
+ AND category_id IS NOT NULL
|
|
|
+),
|
|
|
+demand_posts AS (
|
|
|
+ SELECT DISTINCT ip.post_id
|
|
|
+ FROM pattern_itemset_post ip
|
|
|
+ JOIN post p ON p.post_id = ip.post_id
|
|
|
+ WHERE ip.execution_id = %(execution_id)s
|
|
|
+ AND ip.itemset_id = ANY(%(itemset_ids)s)
|
|
|
+ AND ip.post_id = ANY(%(matched_post_ids)s)
|
|
|
+ AND p.platform IN ('piaoquan', '票圈')
|
|
|
+),
|
|
|
+ranked AS (
|
|
|
+ SELECT
|
|
|
+ e.point_text,
|
|
|
+ e.point_type,
|
|
|
+ COUNT(DISTINCT e.post_id) AS coverage_post_count,
|
|
|
+ MIN(e.post_id) AS post_id,
|
|
|
+ MIN(e.id) AS id,
|
|
|
+ MIN(e.topic_point_id) AS topic_point_id,
|
|
|
+ MIN(e.source_element_id) AS source_element_id,
|
|
|
+ MIN(e.category_id) AS category_id,
|
|
|
+ MIN(e.category_path) AS category_path,
|
|
|
+ MIN(e.element_type) AS element_type,
|
|
|
+ MIN(e.name) AS name,
|
|
|
+ array_agg(DISTINCT e.name ORDER BY e.name) AS matched_element_names,
|
|
|
+ array_agg(DISTINCT e.category_id ORDER BY e.category_id) AS matched_category_ids
|
|
|
+ FROM pattern_mining_element e
|
|
|
+ JOIN demand_posts dp ON dp.post_id = e.post_id
|
|
|
+ JOIN item_cats ic ON ic.category_id = e.category_id
|
|
|
+ WHERE e.execution_id = %(execution_id)s
|
|
|
+ AND e.source_table = 'post_decode_topic_point_element'
|
|
|
+ AND e.point_type IN ('灵感点', '目的点')
|
|
|
+ AND e.point_text IS NOT NULL
|
|
|
+ AND e.point_text <> ''
|
|
|
+ GROUP BY e.point_text, e.point_type
|
|
|
+)
|
|
|
+SELECT *
|
|
|
+FROM ranked
|
|
|
+ORDER BY coverage_post_count DESC, point_text
|
|
|
+LIMIT %(top_k)s
|
|
|
+```
|
|
|
+
|
|
|
+Python 侧补 `rank`：
|
|
|
+
|
|
|
+```python
|
|
|
+for idx, row in enumerate(rows, start=1):
|
|
|
+ row["rank"] = idx
|
|
|
+```
|
|
|
+
|
|
|
+#### 3.6.2 db_manager re-export
|
|
|
+
|
|
|
+在 `/Users/samlee/Documents/works/DemandAgentNew/examples/demand/db_manager.py` 中导入并加入 `__all__`：
|
|
|
+
|
|
|
+```python
|
|
|
+query_seed_points_for_itemsets
|
|
|
+```
|
|
|
+
|
|
|
+#### 3.6.3 evidence_pack 写入
|
|
|
+
|
|
|
+在 `build_evidence_pack()` 中，在 `matched_post_ids` 和 `itemset_items` 都已校验后调用：
|
|
|
+
|
|
|
+```python
|
|
|
+query_seed_points = query_seed_points_for_itemsets(
|
|
|
+ execution_id=int(execution_id),
|
|
|
+ itemset_ids=itemset_ids,
|
|
|
+ matched_post_ids=matched_post_ids,
|
|
|
+ top_k=int(os.getenv("DEMAND_QUERY_SEED_POINTS_TOP_K", "30")),
|
|
|
+)
|
|
|
+```
|
|
|
+
|
|
|
+写入：
|
|
|
+
|
|
|
+```python
|
|
|
+"query_seed_points": query_seed_points,
|
|
|
+```
|
|
|
+
|
|
|
+边界：
|
|
|
+
|
|
|
+- 如果查询失败，当前建议 reject，因为它是下游搜索必需字段。
|
|
|
+- 如果查询成功但没有点位，写空数组并通过，前提是 evidence 本身完整；因为小需求可能没有足够点位。
|
|
|
+- 查询必须用 `pattern_itemset_item.category_id` 精确绑定当前 itemset item；不要默认取支撑帖里的全部点位。
|
|
|
+- 后续如要支持父类节点扩展到子类，必须显式改成 `category_path LIKE item.category_path || '/%'` 并单独验收，V2 不默认混入。
|
|
|
+
|
|
|
+建议实现为：
|
|
|
+
|
|
|
+```text
|
|
|
+DB 查询异常 -> reject
|
|
|
+查询结果为空 -> accepted，但 query_seed_points=[]
|
|
|
+```
|
|
|
+
|
|
|
+### 3.7 MySQL sink 校验增强
|
|
|
+
|
|
|
+涉及文件：
|
|
|
+
|
|
|
+- `/Users/samlee/Documents/works/DemandAgentNew/examples/demand/mysql_demand_content_sink.py`
|
|
|
+
|
|
|
+当前 `_validate_row()` 必填字段中没有 `query_seed_points`。
|
|
|
+
|
|
|
+V2 调整：
|
|
|
+
|
|
|
+- `query_seed_points` 字段必须存在，可以是空数组。
|
|
|
+- `seed_terms` 必填非空。
|
|
|
+- 校验 `seed_terms` 非空即可；其 DB 派生真实性由 `evidence_pack_builder` 保证。
|
|
|
+- 校验 `query_seed_points[*].point_type in {'灵感点', '目的点'}`。
|
|
|
+- 校验 `coverage_post_count` 是正整数。
|
|
|
+
|
|
|
+### 3.8 测试更新
|
|
|
+
|
|
|
+涉及文件：
|
|
|
+
|
|
|
+- `/Users/samlee/Documents/works/DemandAgentNew/examples/demand/tests/test_pg_evidence_builder.py`
|
|
|
+- `/Users/samlee/Documents/works/DemandAgentNew/examples/demand/tests/test_pg_pattern_repository.py`
|
|
|
+- `/Users/samlee/Documents/works/DemandAgentNew/examples/demand/tests/test_mysql_demand_content_sink.py`
|
|
|
+
|
|
|
+新增测试：
|
|
|
+
|
|
|
+1. `evidence_refs.seed_terms` 被忽略。
|
|
|
+2. `seed_terms` 从 itemset items 派生。
|
|
|
+3. stopwords 会过滤 `seed_terms`，但不能导致空值；空值时回退。
|
|
|
+4. `query_seed_points` 过滤关键点。
|
|
|
+5. `query_seed_points` 按 `coverage_post_count` 降序。
|
|
|
+6. `query_seed_points` 可以为空数组但字段必须存在。
|
|
|
+7. MySQL sink 缺 `query_seed_points` 字段时拒绝。
|
|
|
+8. Hive gap 入口禁止调用 `piaoquan_prepare/prepare/run_mining`。
|
|
|
+9. 最新 success execution 选择测试：failed/current 不能被选中，必须选最新 `status='success'` 且 topic itemset 非空。
|
|
|
+10. `get_frequent_itemsets(merge_leve2=..., platform=...)` 会过滤支撑帖并重算 filtered support。
|
|
|
+11. 未传 `merge_leve2/platform` 时保留全局查询语义。
|
|
|
+12. `get_itemset_detail()` 与 `evidence_pack.matched_post_ids` 使用同一套过滤范围。
|
|
|
+13. MySQL sink 校验 `evidence_pack.demand_scope.merge_leve2/platform`，或至少校验输出行 `merge_leve2` 与过滤参数一致。
|
|
|
+
|
|
|
+## 4. 真实 DB 验证计划
|
|
|
+
|
|
|
+使用真实 PG 只读样例：
|
|
|
+
|
|
|
+```text
|
|
|
+execution_id = 581
|
|
|
+itemset_id = 1608183
|
|
|
+```
|
|
|
+
|
|
|
+验证 SQL：
|
|
|
+
|
|
|
+```sql
|
|
|
+WITH item_cats AS (
|
|
|
+ SELECT DISTINCT category_id
|
|
|
+ FROM pattern_itemset_item
|
|
|
+ WHERE itemset_id = 1608183
|
|
|
+ AND category_id IS NOT NULL
|
|
|
+),
|
|
|
+matched_posts AS (
|
|
|
+ SELECT DISTINCT ip.post_id
|
|
|
+ FROM pattern_itemset_post ip
|
|
|
+ JOIN post p ON p.post_id = ip.post_id
|
|
|
+ WHERE ip.execution_id = 581
|
|
|
+ AND ip.itemset_id = 1608183
|
|
|
+ AND p.platform IN ('piaoquan', '票圈')
|
|
|
+),
|
|
|
+matched_points AS (
|
|
|
+ SELECT e.point_text, e.point_type, e.post_id, e.category_id, e.name
|
|
|
+ FROM pattern_mining_element e
|
|
|
+ JOIN matched_posts mp ON mp.post_id = e.post_id
|
|
|
+ JOIN item_cats ic ON ic.category_id = e.category_id
|
|
|
+ WHERE e.execution_id = 581
|
|
|
+ AND e.source_table = 'post_decode_topic_point_element'
|
|
|
+ AND e.point_type IN ('灵感点', '目的点')
|
|
|
+ AND COALESCE(e.point_text, '') <> ''
|
|
|
+)
|
|
|
+SELECT
|
|
|
+ point_text,
|
|
|
+ MIN(point_type) AS point_type,
|
|
|
+ COUNT(DISTINCT post_id) AS coverage_post_count,
|
|
|
+ array_agg(DISTINCT name ORDER BY name) AS matched_element_names,
|
|
|
+ array_agg(DISTINCT category_id ORDER BY category_id) AS matched_category_ids
|
|
|
+FROM matched_points
|
|
|
+GROUP BY point_text
|
|
|
+ORDER BY COUNT(DISTINCT post_id) DESC, point_text
|
|
|
+LIMIT 5;
|
|
|
+```
|
|
|
+
|
|
|
+当前真实返回头部：
|
|
|
+
|
|
|
+| point_text | point_type | coverage_post_count |
|
|
|
+| --- | --- | ---: |
|
|
|
+| 分享励志歌曲 | 目的点 | 12 |
|
|
|
+| 分享正能量歌曲 | 目的点 | 4 |
|
|
|
+| 好好爱自己 | 灵感点 | 4 |
|
|
|
+| 分享建党节祝福歌曲 | 目的点 | 3 |
|
|
|
+| 分享祝福歌曲 | 目的点 | 3 |
|
|
|
+
|
|
|
+### 4.1 已完成的真实只读校验记录
|
|
|
+
|
|
|
+当前文档更新时已用 `.venv` 和项目 `.env` 执行 PG 只读 SELECT，确认：
|
|
|
+
|
|
|
+```text
|
|
|
+latest_success_execution_id = 581
|
|
|
+```
|
|
|
+
|
|
|
+`post` 表字段存在：
|
|
|
+
|
|
|
+```text
|
|
|
+post_id, merge_leve2, platform
|
|
|
+```
|
|
|
+
|
|
|
+`pattern_itemset_post` 表字段存在：
|
|
|
+
|
|
|
+```text
|
|
|
+execution_id, itemset_id, post_id
|
|
|
+```
|
|
|
+
|
|
|
+`execution_id=581` 下，按 `pattern_itemset_post -> post.merge_leve2/platform` 能过滤出品类 itemset：
|
|
|
+
|
|
|
+| merge_leve2 | platform | itemset_count | post_count |
|
|
|
+| --- | --- | ---: | ---: |
|
|
|
+| 知识科普 | piaoquan | 16088 | 854 |
|
|
|
+| 历史名人 | piaoquan | 12571 | 738 |
|
|
|
+| 当代正能量人物 | piaoquan | 10675 | 582 |
|
|
|
+| 人生忠告 | piaoquan | 9308 | 554 |
|
|
|
+| 社会风气 | piaoquan | 5622 | 149 |
|
|
|
+
|
|
|
+样例 scoped itemset：
|
|
|
+
|
|
|
+| merge_leve2 | itemset_id | absolute_support | scoped_post_count |
|
|
|
+| --- | ---: | ---: | ---: |
|
|
|
+| 知识科普 | 1622463 | 175 | 59 |
|
|
|
+| 历史名人 | 1607313 | 579 | 111 |
|
|
|
+| 社会风气 | 1623199 | 261 | 36 |
|
|
|
+
|
|
|
+sub-agent 只读复核补充：
|
|
|
+
|
|
|
+- Hive 最新分区：`20260622`。
|
|
|
+- Hive 缺口样例：`知识科普` 缺量约 `1443.21`，`罕见画面` 约 `852.13`，`人生感悟音乐` 约 `564.34`，`民生政策` 约 `234.80`。
|
|
|
+- PG latest success execution：`id=581`，`snapshot_date=2026-05-09`，`post_count=13265`，`topic_itemset_count=35406`。
|
|
|
+- execution `581` 计数：`topic_itemsets=35406`，`topic_itemset_items=132904`，`itemset_posts=569849`。
|
|
|
+- `罕见画面/piaoquan` 可过滤命中 `4458` 个 itemset、`545` 个 post。
|
|
|
+- `itemset_id=1607313` 全局 `absolute_support=579`；过滤到 `当代正能量人物/piaoquan` 后 `filtered_absolute_support=115`，过滤到 `历史名人/piaoquan` 后 `111`。
|
|
|
+
|
|
|
+这证明 V2 不需要恢复 `run_mining()`；用最新 PG success execution 加品类过滤即可从真实 Pattern 池里拿到候选。
|
|
|
+
|
|
|
+### 4.2 必须补充的 DB 校验 SQL
|
|
|
+
|
|
|
+实现后必须加入只读 SQL smoke：
|
|
|
+
|
|
|
+```sql
|
|
|
+SELECT id, status, snapshot_date, topic_itemset_count
|
|
|
+FROM pattern_mining_execution
|
|
|
+WHERE status = 'success'
|
|
|
+ AND COALESCE(topic_itemset_count, 0) > 0
|
|
|
+ORDER BY snapshot_date DESC NULLS LAST, id DESC
|
|
|
+LIMIT 1;
|
|
|
+```
|
|
|
+
|
|
|
+```sql
|
|
|
+SELECT
|
|
|
+ i.id AS itemset_id,
|
|
|
+ i.absolute_support AS global_absolute_support,
|
|
|
+ COUNT(DISTINCT pip.post_id) AS filtered_absolute_support
|
|
|
+FROM pattern_itemset i
|
|
|
+JOIN pattern_itemset_post pip
|
|
|
+ ON pip.execution_id = i.execution_id
|
|
|
+ AND pip.itemset_id = i.id
|
|
|
+JOIN post p
|
|
|
+ ON p.post_id = pip.post_id
|
|
|
+WHERE i.execution_id = :execution_id
|
|
|
+ AND i.scope = 'topic'
|
|
|
+ AND p.merge_leve2 = :merge_leve2
|
|
|
+ AND p.platform IN ('piaoquan', '票圈')
|
|
|
+GROUP BY i.id, i.absolute_support
|
|
|
+HAVING COUNT(DISTINCT pip.post_id) >= :min_support
|
|
|
+ORDER BY COUNT(DISTINCT pip.post_id) DESC, i.absolute_support DESC
|
|
|
+LIMIT 20;
|
|
|
+```
|
|
|
+
|
|
|
+还应校验：
|
|
|
+
|
|
|
+- `pattern_itemset_post.post_id` 在 `post.post_id` 可 join。
|
|
|
+- `filtered_absolute_support <= pattern_itemset.absolute_support`。
|
|
|
+- `matched_post_ids` 必须全部满足目标 `merge_leve2/platform`。
|
|
|
+- `pattern_itemset_item.category_id/category_path` 能回连 `pattern_mining_category`；它表示 Pattern 特征分类，不等同于 Hive 的 `merge二级品类`。
|
|
|
+- 老 MySQL `topic_pattern_*` 只能作为 legacy 对照；不能用它承接 V2 的 PG scoped filtering。
|
|
|
+
|
|
|
+## 5. E2E 验证计划
|
|
|
+
|
|
|
+### 5.1 本地单条真实 LLM + 真实 PG + MySQL 测试
|
|
|
+
|
|
|
+命令形态：
|
|
|
+
|
|
|
+```bash
|
|
|
+python -m examples.demand.run_existing_execution_mysql \
|
|
|
+ --execution-id 581 \
|
|
|
+ --merge-level2 "PG Pattern V2 需求测试" \
|
|
|
+ --platform-type piaoquan \
|
|
|
+ --count 1 \
|
|
|
+ --run-label v2_seed_points_smoke
|
|
|
+```
|
|
|
+
|
|
|
+验收：
|
|
|
+
|
|
|
+- exit code 为 0。
|
|
|
+- MySQL `demand_content` 有对应 `run_label` 行。
|
|
|
+- `ext_data.evidence_pack.seed_terms` 非空。
|
|
|
+- `seed_terms` 不受 LLM `evidence_refs.seed_terms` 影响。
|
|
|
+- `ext_data.evidence_pack.query_seed_points` 字段存在。
|
|
|
+- `query_seed_points` 非空时只包含 `灵感点/目的点`。
|
|
|
+- `query_seed_points` 非空时 `coverage_post_count` 降序。
|
|
|
+- 不写 `demand_task`、Hive、PG Pattern、旧 MySQL Pattern。
|
|
|
+
|
|
|
+### 5.2 批量 100 条测试
|
|
|
+
|
|
|
+在单条 smoke 通过后再跑：
|
|
|
+
|
|
|
+```bash
|
|
|
+python -m examples.demand.run_existing_execution_mysql \
|
|
|
+ --execution-id 581 \
|
|
|
+ --merge-level2 "PG Pattern V2 需求测试" \
|
|
|
+ --platform-type piaoquan \
|
|
|
+ --count 10 \
|
|
|
+ --run-label v2_seed_points_100_batch01
|
|
|
+```
|
|
|
+
|
|
|
+重复补跑直到合格入库 100 条。
|
|
|
+
|
|
|
+验收 SQL：
|
|
|
+
|
|
|
+```sql
|
|
|
+SELECT COUNT(*)
|
|
|
+FROM demand_content
|
|
|
+WHERE JSON_UNQUOTE(JSON_EXTRACT(ext_data, '$.run_label')) LIKE 'v2_seed_points_100%';
|
|
|
+```
|
|
|
+
|
|
|
+逐条校验：
|
|
|
+
|
|
|
+```text
|
|
|
+pattern_source_system = pg_pattern_v2
|
|
|
+validation_status = passed
|
|
|
+source_certainty = db_validated
|
|
|
+seed_terms non-empty
|
|
|
+seed_terms ignored from LLM evidence_refs
|
|
|
+query_seed_points field exists
|
|
|
+```
|
|
|
+
|
|
|
+### 5.3 Hive gap 批量入口测试
|
|
|
+
|
|
|
+在单条和 100 条手动模式通过后，执行：
|
|
|
+
|
|
|
+```bash
|
|
|
+python -m examples.demand.run_hive_gap_mysql \
|
|
|
+ --run-label hive_gap_v2_smoke \
|
|
|
+ --max-categories 2 \
|
|
|
+ --max-total-demands 5
|
|
|
+```
|
|
|
+
|
|
|
+验收：
|
|
|
+
|
|
|
+- 真实调用 `get_demand_merge_level2_names()` 读取 Hive/ODPS。
|
|
|
+- 自动选择最新 PG `status='success'` 且 topic itemset 非空的 execution。
|
|
|
+- 每个品类调用 `get_frequent_itemsets(..., merge_leve2=当前品类, platform=当前平台)`。
|
|
|
+- MySQL `demand_content.merge_leve2` 等于 Hive 返回的 `cluster_name`。
|
|
|
+- `evidence_pack.demand_scope.merge_leve2` 等于同一个品类。
|
|
|
+- `source_post_id in matched_post_ids`。
|
|
|
+- `matched_post_ids` 均能在 PG `post` 表查到同一 `merge_leve2/platform`。
|
|
|
+- 不写 `demand_task`，不写 Hive，不卡回旧 MySQL Pattern。
|
|
|
+
|
|
|
+## 6. JSON 契约校验计划
|
|
|
+
|
|
|
+当前已有 V1 样例：
|
|
|
+
|
|
|
+```text
|
|
|
+examples/demand/test_output_data/pg_llm_e2e_581_20260605_100815/demand_content.json
|
|
|
+examples/demand/test_output_data/cfa_100_mix_20260604/combined/demand_content.json
|
|
|
+```
|
|
|
+
|
|
|
+真实检查结果：
|
|
|
+
|
|
|
+- 两份样例都有 `ext_data.evidence_pack`。
|
|
|
+- 两份样例都有 `seed_terms`。
|
|
|
+- 两份样例都没有 `query_seed_points`，因此不能作为 V2 合格输出交付 CFA。
|
|
|
+
|
|
|
+V2 JSON 校验脚本必须逐条检查：
|
|
|
+
|
|
|
+```text
|
|
|
+top-level: id/name/reason/suggestion/score/merge_leve2/dt/ext_data
|
|
|
+ext_data: reason/desc/type/video_ids/trace_id/run_label/evidence_pack
|
|
|
+evidence_pack: pattern_source_system=pg_pattern_v2
|
|
|
+evidence_pack: validation_status=passed
|
|
|
+evidence_pack: source_certainty=db_validated
|
|
|
+evidence_pack: demand_scope.merge_leve2 == demand_content.merge_leve2
|
|
|
+evidence_pack: demand_scope.scope_source == odps_gap
|
|
|
+evidence_pack: demand_scope.gap_dt/requested_count/lack_count present when generated from Hive gap
|
|
|
+evidence_pack: filtered_absolute_support or scoped_post_count present
|
|
|
+evidence_pack: filtered_absolute_support <= absolute_support
|
|
|
+evidence_pack: seed_terms non-empty
|
|
|
+evidence_pack: query_seed_points field exists
|
|
|
+evidence_pack: query_seed_points[*].point_type in {灵感点,目的点}
|
|
|
+evidence_pack: source_post_id in matched_post_ids
|
|
|
+evidence_pack: matched_post_ids scoped by post.merge_leve2/platform
|
|
|
+```
|
|
|
+
|
|
|
+旧 V1 JSON 如果缺 `query_seed_points`，校验必须失败。
|
|
|
+
|
|
|
+## 7. Sub-agent 交叉验证计划
|
|
|
+
|
|
|
+本轮文档更新必须使用三路只读 sub-agent 交叉验证：
|
|
|
+
|
|
|
+| sub-agent | 验证范围 |
|
|
|
+| --- | --- |
|
|
|
+| 代码链路审计 | 检查 Hive gap 入口、PG itemset 过滤、高权重工具、evidence builder、MySQL entrypoint 的真实调用链路 |
|
|
|
+| DB/样例数据验证 | 真实 SELECT PG schema、`post.merge_leve2/platform`、`pattern_itemset_post` 和 scoped itemset 样例 |
|
|
|
+| 输出契约审计 | 对照现有文档、sink 校验、测试 JSON，确认 `query_seed_points/demand_scope` 是纯增量且 V1 JSON 会失败 |
|
|
|
+
|
|
|
+实现阶段继续使用相同分工：
|
|
|
+
|
|
|
+| worker | 写入范围 |
|
|
|
+| --- | --- |
|
|
|
+| Worker A | `pg_pattern_repository.py`, `db_manager.py` |
|
|
|
+| Worker B | `evidence_pack_builder.py`, evidence builder tests |
|
|
|
+| Worker C | `run.py`, `run_hive_gap_mysql.py`, prompts, MySQL sink tests |
|
|
|
+
|
|
|
+主 agent 负责合并和最终验收，不把 sub-agent 结论当唯一证明。
|
|
|
+
|
|
|
+## 8. 风险与边界
|
|
|
+
|
|
|
+| 风险 | 处理方式 |
|
|
|
+| --- | --- |
|
|
|
+| `query_seed_points` 结果为空 | 字段保留为空数组；不伪造点位 |
|
|
|
+| PG `post.platform` 取值不统一 | 支持 `piaoquan` 和 `票圈`，后续可配置 |
|
|
|
+| Hive 品类在 PG latest execution 中没有 scoped itemset | 跳过该品类并记录 rejected/skip reason，不回退到 `run_mining` |
|
|
|
+| `seed_terms` 过滤过严 | 如果 stopwords 过滤后为空，回退到 DB itemset 词表前 N 个 |
|
|
|
+| LLM 仍输出 `seed_terms` | prompt 禁止；代码忽略 |
|
|
|
+| MySQL 单表 JSON 过大 | V2 先接受；如超限再进入 V2.1 拆表或压缩 |
|
|
|
+| CFA 仍用旧字段 | V2 纯增量，不破坏旧字段；CFA 可逐步切到 `query_seed_points` |
|
|
|
+
|
|
|
+## 9. 不处理事项
|
|
|
+
|
|
|
+- 不改 `agent/` 框架。
|
|
|
+- 不改 PG schema。
|
|
|
+- 不改 MySQL `demand_content` schema。
|
|
|
+- 不写 Hive，只读 Hive 供需缺口表。
|
|
|
+- 不恢复旧 MySQL Pattern evidence。
|
|
|
+- 不恢复旧版 `run_mining()`，不新建 MySQL `topic_pattern_execution`。
|
|
|
+- 不让 LLM 生成事实字段。
|
|
|
+- 不让 CFA 运行时反查 PG 才能拿首轮搜索点位。
|
SamLee