Skip to content

Add versioned prompt profiles, output gates, bounded generation concurrency and env-gated LLM client patches - #194

Merged
ChenZiHong-Gavin merged 1 commit into
InternScience:mainfrom
Leon-Algo:pr-b-prompt-profiles-mechanisms
Nov 8, 2026
Merged

ChenZiHong-Gavin merged 1 commit into
InternScience:mainfrom
Leon-Algo:pr-b-prompt-profiles-mechanisms

Conversation

@Leon-Algo

Copy link
Copy Markdown
Contributor

Summary

  • Versioned prompt profiles: graphgen/templates/prompt_profiles.py + BaseGenerator.TEMPLATE_KEY/apply_prompt_profile/template(); all 11 generator build_prompt overrides become instance methods. Profiles are JSON directories structurally validated against the official template constants (fail-closed); built-in example.v1 ships byte-identical official templates as an example. No profile applied → byte-identical official behavior.
  • Generic output-gate registry: graphgen/operators/generate/gates.py with register_gate/GATES and a generic non_empty_answer gate. No domain-specific gates included.
  • Bounded generation concurrency: event-loop semaphore exposed as GenerateService(llm_concurrency=N); default unchanged.
  • GenerateService opt-in options: prompt_profile / output_gate / llm_concurrency — all default to upstream behavior.
  • Env-gated LLM client patches (all off by default): GRAPHGEN_EMPTY_RETRY bounded empty-response retry (rate-limiter re-applied per attempt), GRAPHGEN_EMPTY_EXTRACTION_PREFILL JSON-compatible assistant prefill { for the schema-guided extractor (the previous tuple-format prefill can never parse against json.loads), GRAPHGEN_DISABLE_THINKING for logprob calls, and GRAPHGEN_LLM_ACTOR_MAX_CONCURRENCY Ray actor option + logger-context fallback.
  • Packaging: package_data gains templates/profiles/*/*.json only. Upstream package identity (name/version/url/author) untouched.

Second of three PRs split from #192 per review; no PMS assets, no fork identity. PR A (partitioner test fixes) and PR C (event-entity pipeline, builds on this) follow separately.

Validation

  • pytest tests/unit_tests → 20 passed (profiles 8, gates 3, llm_concurrency 2, openai patches 4, log 1, plus collection checks).
  • pylint --rcfile=.pylintrc baselines/ graphgen/ webui/ → 10.00/10, zero messages (CI gate green; resolves the arguments-differ findings on all generator overrides noted in Add versioned prompt profiles and event-entity generation pipeline #192).

🤖 Generated with Claude Code

- prompt profile:BaseGenerator 新增 TEMPLATE_KEY/apply_prompt_profile/template,
  全部生成器 build_prompt 覆写改为实例方法;prompt_profiles.py 结构校验 +
  fail-closed;内置 example.v1 通用 profile(内容与官方模板逐字一致)
- 输出门注册表:gates.py register_gate/GATES + 内置 non_empty_answer 通用门,
  GenerateService 增加 prompt_profile/output_gate/llm_concurrency 选项
- 生成并发:llm_concurrency.py 事件循环内有界并发包装
- LLM 客户端 4 个 env 门控补丁:GRAPHGEN_EMPTY_RETRY 空响应有界重试、
  GRAPHGEN_EMPTY_EXTRACTION_PREFILL 空抽取 assistant prefill 重试、
  GRAPHGEN_DISABLE_THINKING logprob 调用关思考、可选 assistant_prefill 参数;
  GRAPHGEN_LLM_ACTOR_MAX_CONCURRENCY 提升 Ray actor 并发;logger 空上下文回退
- 未设置任何 env / 选项时行为与官方默认逐字一致;不改动包身份与版本号

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@ChenZiHong-Gavin ChenZiHong-Gavin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, merging. Checked that the default path stays byte-identical (template() returns the same official constant objects; all build_prompt callers already use self.), the RPM/TPM wait now sits inside the retry loop, and pylint is green on 3.10–3.12.

Two non-blocking nits for a follow-up:

  1. MaskedFillInBlankGenerator.TEMPLATE_KEY = "masked_fill_in_blank" but profiles/example.v1/ has no masked_fill_in_blank.json, so applying the shipped example profile to that generator raises prompt_profile_template_missing. Either add the file (same content as aggregated.json) or point the key at aggregated.
  2. In SchemaGuidedExtractor, the prefill retry passes assistant_prefill="{" but the response is fed to json.loads as-is. A backend that genuinely continues the assistant turn returns the text after {, so the retry still cannot parse; prepending the prefill when the response does not already start with it would make this robust for both backend behaviours.

@ChenZiHong-Gavin
ChenZiHong-Gavin merged commit 5689674 into InternScience:main Nov 8, 2026
6 checks passed
@Leon-Algo
Leon-Algo deleted the pr-b-prompt-profiles-mechanisms branch November 10, 2026 07:06
Leon-Algo pushed a commit to Leon-Algo/GraphGen that referenced this pull request Nov 10, 2026
…ce#195 + 维护者评审修复)

Co-Authored-By: Claude Code <noreply@anthropic.com>
Leon-Algo pushed a commit to Leon-Algo/GraphGen that referenced this pull request Nov 10, 2026
…nce#195 + 维护者评审修复)

冲突解决策略:
- 上游评审版为主:分区器默认翻转、构建器重构、profile loader、
  抽取器与 8 个生成器的签名/格式修复
- fork 生产特性保留:support 出处接线(生成器+服务+透传)、
  三个 PMS 出题门、中文事件描述模板、max_communities loss 选题、
  reload 有界重试、思考模型空响应容错(PMS_PATCHES InternScience#11)、
  pms 身份与 event_profiles 资源

Co-Authored-By: Claude Code <noreply@anthropic.com>
@Leon-Algo

Copy link
Copy Markdown
Contributor Author

Thanks for the quick review and merge, @ChenZiHong-Gavin! We really appreciate you double-checking that the default path stays byte-identical — that kind of care for backward compatibility is one of the things that makes GraphGen such a solid project to build on. Thanks also for the two nits you spotted in this review; they pointed at real gaps, and we have follow-up fixes for both (see #197).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants