Parcourir la source

作用域动词剥离:③ LLM 名词化 + ① 窄表确定性兜底

值名词化放在 ②出候选 → ③回扣 之前(顺带提高树复用率):
- ③ 主力:prompts/normalize_scope.txt 批量把动宾改成名词核心,规范复合名词(氛围营造/行动引导)原样不动
- ① 兜底:strip_verb_tail 砍开头/结尾无歧义裸动词(寻找/定位/推导/核验/提取…14个),
  配名词语素保护表(营造/引导/表达/塑造…绝不砍)+ 砍到 <2 字回退

效果:寻找共识裂缝→共识裂缝、爆发力核验→爆发力、现实阻碍提取→现实阻碍;
86 个作用值刺眼动宾清零,仅余个别温和复合词(窄表有意取舍,避免过度剥离)。
全量重跑 19 颗 0 ERROR/0 WARN;抖音仍被语义闸判制作排除。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SamLee il y a 3 semaines
Parent
commit
19ffaede3b
4 fichiers modifiés avec 876 ajouts et 690 suppressions
  1. 8 0
      prompts/normalize_scope.txt
  2. 41 1
      scripts/decompose.py
  3. 478 462
      web/frameworks_v2.json
  4. 349 227
      web/payloads_v2.json

+ 8 - 0
prompts/normalize_scope.txt

@@ -0,0 +1,8 @@
+你在把一批「作用域标签值」规范成【纯名词】。用户消息是一个 JSON 字符串数组。
+
+规则:
+- 若值是"动宾/带动词"(如 寻找共识裂缝、爆发力核验、推导选题落点、完播率提升、现实阻碍提取),去掉动词、保留【名词核心】:共识裂缝、爆发力、选题落点、完播率、现实阻碍。
+- 若值本就是名词——**包括"氛围营造、行动引导、情感表达、推进节奏、形象塑造、概括叙述"这类规范的"名词+动词语素"复合名词——【原样不动】**。
+- 不要合并、不要新增、不要删项、不要解释。每个输入值都必须在输出里有对应(哪怕原样返回)。
+
+输出 JSON:{"映射": {"<原值>":"<规范名词>", "...":"..."}}

+ 41 - 1
scripts/decompose.py

@@ -31,6 +31,13 @@ PHASE2 = (SKILL / "extraction" / "phase2-scope.md").read_text(encoding="utf-8")
 GATE_ADMIT = load_prompt("gate_admit")        # ①.5 创作判定闸:判"是创作"
 GATE_REFUTE = load_prompt("gate_refute")      # ①.5:挑刺"是制作/越界"
 GATE_TIEBREAK = load_prompt("gate_tiebreak")  # ①.5:分歧裁决(边界倾向排除)
+NORMALIZE = load_prompt("normalize_scope")    # ③前:作用域值名词化(③LLM)
+
+# ① 窄表确定性兜底:只放无歧义裸动作动词(绝不放 营造/引导/表达 等名词语素)
+_STRIP_VERBS = ("寻找", "定位", "推导", "核验", "提取", "挖掘", "捕捉",
+                "识别", "梳理", "归纳", "判断", "验证", "确认", "复盘")
+_PROTECT = ("营造", "引导", "表达", "塑造", "叙述", "呈现", "刻画",
+            "升华", "控制", "推进", "转化", "传达")  # 名词语素,永不砍
 
 # from: fixture(读 tests/fixtures)/ live(实时 crawler 取数)
 SOURCES = [
@@ -140,6 +147,39 @@ def scope_candidates(knowledges: list[dict]) -> list:
     return chat_json(PHASE2, user, timeout=120).get("scopes") or []
 
 
+def strip_verb_tail(v: str) -> str:
+    """① 窄表确定性兜底:砍掉值开头/结尾的无歧义裸动词;砍到 <2 字则回退原值。"""
+    if not v or len(v) < 3:
+        return v
+    for verb in _STRIP_VERBS:                       # 开头裸动词
+        if v.startswith(verb) and len(v) - len(verb) >= 2:
+            v = v[len(verb):]
+            break
+    for verb in _STRIP_VERBS:                        # 结尾裸动词(_PROTECT 与之不相交,名词语素天然不在表里)
+        if v.endswith(verb) and len(v) - len(verb) >= 2:
+            v = v[:-len(verb)]
+            break
+    return v
+
+
+def nounify_scopes(scopes: list) -> list:
+    """作用域值名词化:③ 一次批量 LLM 名词化(语义,不误砍"氛围营造")→ ① 窄表兜底。在回扣前做。"""
+    vals = sorted({it["value"] for sc in scopes for it in (sc.get("items") or []) if it.get("value")})
+    if not vals:
+        return scopes
+    try:
+        mp = chat_json(NORMALIZE, json.dumps(vals, ensure_ascii=False), timeout=90).get("映射") or {}
+    except Exception:
+        mp = {}
+    for sc in scopes:
+        for it in sc.get("items") or []:
+            v = it.get("value")
+            if not v:
+                continue
+            it["value"] = strip_verb_tail(mp.get(v) or v)   # ③ 映射优先,再 ① 兜底
+    return scopes
+
+
 def link_scope(linker: ScopeLinker, scope_type: str, value: str) -> dict:
     try:
         hits = linker.link(value, source_type=SRC2CN.get(scope_type, scope_type), top_k=3)
@@ -279,7 +319,7 @@ def main() -> None:
             if k.get("role") == "组件" and (k.get("parent") or {}).get("how_id") not in how_ids:  # 守卫:组件 parent 必指向同帖 how
                 k["role"] = "主"; k["parent"] = None
         print(f"  ② {len(knowledges)} 颗:" + ", ".join(f"{k.get('type')}/{k.get('role')}" for k in knowledges))
-        apply_scopes(knowledges, scope_candidates(knowledges), linker)
+        apply_scopes(knowledges, nounify_scopes(scope_candidates(knowledges)), linker)  # ③名词化+①兜底 → 回扣
         print("  ③⑤ 作用域回扣 + 组装")
         posts_out.append({**meta, "knowledges": knowledges})
         payloads += [build_payload(post, k, how_titles) for k in knowledges]

Fichier diff supprimé car celui-ci est trop grand
+ 478 - 462
web/frameworks_v2.json


Fichier diff supprimé car celui-ci est trop grand
+ 349 - 227
web/payloads_v2.json


Certains fichiers n'ont pas été affichés car il y a eu trop de fichiers modifiés dans ce diff