# CLAUDE.md
??
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

DeerFlow is a LangGraph-based AI super agent system with a full-stack architecture. The backend provides a "super agent" with sandbox execution, persistent memory, subagent delegation, and extensible tool integration - all operating in per-thread isolated environments.

**Architecture**:
- **Gateway API** (port 8001): REST API plus embedded LangGraph-compatible agent runtime
- **Frontend** (port 3000): Next.js web interface
- **Nginx** (port 2026): Unified reverse proxy entry point
- **Provisioner** (port 8002, optional in Docker dev): Started only when sandbox is configured for provisioner/Kubernetes mode

**Runtime**:
- `make dev`, Docker dev, and production all run the agent runtime in Gateway via `RunManager` + `run_agent()` + `StreamBridge` (`packages/harness/deerflow/runtime/`). Nginx exposes that runtime at `/api/langgraph/*` and rewrites it to Gateway's native `/api/*` routers.

**Project Structure**:
```
deer-flow/
??? Makefile                    # Root commands (check, install, dev, stop)
??? config.yaml                 # Main application configuration
??? extensions_config.json      # MCP servers and skills configuration
??? backend/                    # Backend application (this directory)
?   ??? Makefile               # Backend-only commands (dev, gateway, lint)
?   ??? langgraph.json         # LangGraph Studio graph configuration
?   ??? packages/
?   ?   ??? harness/           # deerflow-harness package (import: deerflow.*)
?   ?       ??? pyproject.toml
?   ?       ??? deerflow/
?   ?           ??? agents/            # LangGraph agent system
?   ?           ?   ??? lead_agent/    # Main agent (factory + system prompt)
?   ?           ?   ??? middlewares/   # 10 middleware components
?   ?           ?   ??? memory/        # Memory V2: provider/manager/builtin/hindsight
?   ?           ?   ??? thread_state.py # ThreadState schema
?   ?           ??? sandbox/           # Sandbox execution system
?   ?           ?   ??? local/         # Local filesystem provider
?   ?           ?   ??? sandbox.py     # Abstract Sandbox interface
?   ?           ?   ??? tools.py       # bash, ls, read/write/str_replace
?   ?           ?   ??? middleware.py  # Sandbox lifecycle management
?   ?           ??? subagents/         # Subagent delegation system
?   ?           ?   ??? builtins/      # general-purpose, bash agents
?   ?           ?   ??? executor.py    # Background execution engine
?   ?           ?   ??? registry.py    # Agent registry
?   ?           ??? tools/builtins/    # Built-in tools (present_files, ask_clarification, view_image)
?   ?           ??? mcp/               # MCP integration (tools, cache, client)
?   ?           ??? models/            # Model factory with thinking/vision support
?   ?           ??? skills/            # Skills discovery, loading, parsing
?   ?           ??? config/            # Configuration system (app, model, sandbox, tool, etc.)
?   ?           ??? community/         # Community tools (tavily, jina_ai, firecrawl, image_search, aio_sandbox)
?   ?           ??? reflection/        # Dynamic module loading (resolve_variable, resolve_class)
?   ?           ??? utils/             # Utilities (network, readability)
?   ?           ??? client.py          # Embedded Python client (DeerFlowClient)
?   ??? app/                   # Application layer (import: app.*)
?   ?   ??? gateway/           # FastAPI Gateway API
?   ?   ?   ??? app.py         # FastAPI application
?   ?   ?   ??? routers/       # FastAPI route modules (models, mcp, memory, skills, uploads, threads, artifacts, agents, suggestions, channels)
?   ?   ??? channels/          # IM platform integrations
?   ??? tests/                 # Test suite
?   ??? docs/                  # Documentation
??? frontend/                   # Next.js frontend application
??? skills/                     # Agent skills directory
    ??? public/                # Public skills (committed)
    ??? custom/                # Custom skills (gitignored)
```

## Important Development Guidelines

### Documentation Update Policy
**CRITICAL: Always update README.md and CLAUDE.md after every code change**

When making code changes, you MUST update the relevant documentation:
- Update `README.md` for user-facing changes (features, setup, usage instructions)
- Update `CLAUDE.md` for development changes (architecture, commands, workflows, internal systems)
- Keep documentation synchronized with the codebase at all times
- Ensure accuracy and timeliness of all documentation

### Workflow Studio stream and canvas compatibility

- `app/gateway/workflow_agent_runner.py` is the boundary between LangGraph token chunks and durable workflow events. Provider reasoning in `AIMessage.additional_kwargs.reasoning_content`/`reasoning` is emitted to the UI as balanced inline `<think>` blocks; only visible answer content becomes the node's `text` output for downstream nodes. It must also emit every original `[AIMessage|ToolMessage, metadata]` tuple as `node.message.data.messages` with no whitelist, preview or length crop. Tool results and `additional_kwargs.clarification` are needed by the Studio's DeerFlow-compatible message UI and must survive durable SSE replay.
- An agent `ask_clarification` ToolMessage pauses the workflow at that agent node (`awaiting_input`), not as a terminal run. The pause records the source `nodeRunId`/`toolCallId`; resuming turns the submitted reply into the next user message on the same `wf-{run}-{node}` LangGraph thread. Keep the raw ToolMessage authoritative; do not invent a second card payload.
- `app/gateway/workflow_proposal_planner.py` converts the workflow controller's selected catalog roles into per-agent `mission`/`deliverable`/`scope`/`handoff` contracts. Only a selected visible `agentId` may contribute a contract; every omitted or malformed field falls back to a deterministic role-aware contract. Preserve this prompt-level worker boundary when adding strategies: researchers must not silently become final writers, and sequential roles must receive their upstream handoff binding plus an explicit next handoff.
- `app/gateway/routers/workflows_coze_compat.py` must return both `creator.self: true` and editable VCS metadata for an owned standalone workflow. Coze Playground derives its preview/read-only mode from those fields, so omitting `creator.self` disables node dragging even when the standalone page passes `readonly={false}`.

## Commands

**Root directory** (for full application):
```bash
make check      # Check system requirements
make install    # Install all dependencies (frontend + backend)
make dev        # Start all services (Gateway + Frontend + Nginx), with config.yaml preflight
make start      # Start production services locally
make stop       # Stop all services
```

**Backend directory** (for backend development only):
```bash
make install    # Install backend dependencies
make dev        # Run Gateway API with reload (port 8001)
make gateway    # Run Gateway API only (port 8001)
make test       # Run all backend tests
make lint       # Lint with ruff
make format     # Format code with ruff
```

Regression tests related to Docker/provisioner behavior:
- `tests/test_docker_sandbox_mode_detection.py` (mode detection from `config.yaml`)
- `tests/test_provisioner_kubeconfig.py` (kubeconfig file/directory handling)

Boundary check (harness ? app import firewall):
- `tests/test_harness_boundary.py` ? ensures `packages/harness/deerflow/` never imports from `app.*`

CI runs these regression tests for every pull request via [.github/workflows/backend-unit-tests.yml](../.github/workflows/backend-unit-tests.yml).

## Architecture

### Position roundtable recovery persistence

`/api/position-roundtable` uses the shared `/api/intent/*` LangGraph flow as
the live Step-1 execution source and stores an additional SQL recovery copy in
`position_roundtable_sessions`/`position_roundtable_nodes`. The recovery copy
contains complete intent/node/summary/action-plan conversations, unsent
composer drafts, workflow snapshots, and bounded full text for generated
Markdown/JSON artifacts. This lets history and downstream closing agents keep
working when container-local checkpoint or sandbox volumes are replaced.
Revision `20260723_05` adds the conversation columns. Session deletion
explicitly removes child nodes so SQLite deployments do not depend on FK
cascade pragmas. The router is mounted canonically below
`/api/position-roundtable` and, for intranet gateway allowlists, below the
schema-hidden compatibility prefix `/api/multi-agent/position-roundtable`.
Clients may retry the compatibility route only for a framework/proxy-level
`404 Not Found`; domain-level missing-session responses must remain real 404s.

The position-roundtable state machine is multi-worker safe: node commands
(`complete`, `bind`, conversation save, reject) and session commands
(`activate`, archive, delete) lock the parent session and affected nodes in one
transaction, then record a unique command id for retry replay.  The task and
intent snapshot writes use the same parent-row lock; they are permitted only
while `status=intent_pending`, so a late “confirm intent” write cannot change a
chain that another user has already activated.  Activation itself rechecks the
stored confirmed intent after taking the lock.  `tests/test_position_roundtable_
commands.py` and `tests/test_position_roundtable_concurrency.py` cover the
state transitions; `tests/test_roundtable_postgres_concurrency.py` is the real
PostgreSQL acceptance suite (set `DEERFLOW_POSTGRES_CONCURRENCY_TEST_URL`).

Roundtable backend jobs use a database unique active-dedupe key plus a leased
dispatcher, never process-local start deduplication.  Cancellation is two
phase (`cancel_requested` then `cancelled`): the lease owner’s watcher normally
finalizes it, while the dispatcher reaps a cancellation whose lease has
expired after a worker crash.  A worker that loses lease stops its local model
run immediately.  Do not reintroduce direct `executor.cancel()` from the HTTP
cancel route, because it can cancel the watcher before the database state is
finalized.  The boolean defaults on these new rows must use SQLAlchemy
`false()` rather than numeric `0`, so fresh PostgreSQL schemas can be created.

Position roundtable selects the separately seeded built-in agent
`position-roundtable-intent` by passing `intent_agent_id` to the shared
`/api/intent/init` and `/api/intent/stream` endpoints. Its request lifecycle,
files, and SSE protocol match `roundtable-intent`, but its cap is five
**user-assistance rounds**. A single round may emit several independent
`ask_clarification` tool calls; those calls are separate cards, not one card
containing several numbered questions. Count that as one round by the
tool-calling AI message, never by card count. Topic-only requests must request
all material missing decisions as separate cards rather than being treated as a
resolved intent; after the fifth set is answered it must emit completed final
JSON without unresolved placeholders. The required arrays are
`coreGoals`, `riskWarnings`, `keyPoints`, and `strategicSignificance`. Do not
treat a fresh dedicated-agent `ask_clarification` as stale: it must win over
a same-run premature `[INTENT_READY]` and leave the UI waiting for the user.
Conversely, the prior turn’s historical clarification must not block the user
reply from completing. Do not
change `roundtable-intent`'s `{objective,constraints,assumptions}` contract for
this feature. The position router accepts the new schema for activation and
injects all four sections into downstream position-node and closing-agent
context; legacy stored summaries are mapped read-only for compatibility.
`/api/intent/init` must create the empty Step-1 thread through
`multi_agent._create_thread_direct`, exactly like multi-agent initialization;
do not restore the old HTTP loopback to `/api/threads`, because deployed
Gateway ports/proxies may make the automatic page initialization fail while
later manual turns still work. If checkpoint creation fails or the parallel seat
task is cancelled, the helper compensates by deleting any partial checkpoint and
metadata row (the cancellation branch runs under `asyncio.shield` so a second
cancellation cannot abort it half-way). Streaming stays on the existing loopback
path.

`ThreadMetaStore.create` never issues a post-commit `session.refresh()`. Every
`ThreadMetaRow` column is assigned before commit and the session factory uses
`expire_on_commit=False`, so the returned dict is built from the in-memory row.
The removed read-back ran on a *different* pooled connection than the INSERT,
which a MySQL read/write-splitting endpoint can route to a lagging replica — the
row comes back empty and SQLAlchemy raises `Could not refresh instance`, surfacing
as `500 "Failed to create thread"`. Concurrency makes that far more likely, so a
single-user deployment never sees it. No caller uses the return value, and all
seven call sites (`multi_agent._create_thread_direct`, `services.start_run`,
`routers/threads.create_thread`, `roundtable_inprocess_gateway`,
`scheduler/service`, `thread_shares`, `authz`) benefit. Keep new `create`
implementations read-back-free; `tests/test_position_roundtable_sessions.py::
test_thread_meta_create_never_reads_back_after_commit` guards this.

### Harness / App Split

The backend is split into two layers with a strict dependency direction:

- **Harness** (`packages/harness/deerflow/`): Publishable agent framework package (`deerflow-harness`). Import prefix: `deerflow.*`. Contains agent orchestration, tools, sandbox, models, MCP, skills, config ? everything needed to build and run agents.
- **App** (`app/`): Unpublished application code. Import prefix: `app.*`. Contains the FastAPI Gateway API and IM channel integrations (Feishu, Slack, Telegram, DingTalk).

**Dependency rule**: App imports deerflow, but deerflow never imports app. This boundary is enforced by `tests/test_harness_boundary.py` which runs in CI.

### Report Collaboration (AgentScope, in progress)

Independent of Workflow Studio and the existing roundtable. Plan: repo-root `docs/agentscope-多智能体报告协作工作台-后端实施计划.md`（`RC-BE-000`～`018` 已落地，下一票 `RC-BE-019`）。Spike baseline: `docs/AGENTSCOPE_BASELINE.md`.

- **Config**: `config.yaml → report_collaboration` (`deerflow.config.report_collaboration_config.ReportCollaborationConfig`). Default **off** (`enabled` / `worker_enabled`).
- **Contracts**: `app/report_collaboration/contracts/` — snake_case Pydantic models matching `frontend-web/src/report-collaboration/api/types.ts`. SSE envelope uses camelCase (`eventId` / `sessionId` / `runId`). `node_run_id` is always `${node_id}-attempt{attempt}`.
- **Persistence**: `deerflow.persistence.report_collaboration` (`report_collaboration_*` tables, migrations `20260905_01` / `20260906_01` / `20260907_01`). SQL + memory stores; one active run per session via `active_dedupe_key`; command/message/run/version idempotency keys refuse cross-session reuse; event `seq` taken from the run row. Session delete removes children explicitly.
- **Gateway**: `/api/report-collaboration` (`app/gateway/routers/report_collaboration.py`). `enabled=false` → 503. Plan/run writes enqueue durable jobs and return immediately. Pre-run `POST /sessions/{id}/messages` runs a rule-based `IntentRouterAgent` + `RequirementResolver` (`app/report_collaboration/requirement/`): writes a requirement revision and clarification messages, never auto-starts planning. `POST /sessions/{id}/plan-requests` then runs `PlanProposalService` (`app/report_collaboration/planning/`): catalog projection (no SOUL/keys) → selector → 2–3 strategy RoleDemand graphs → assembler binds catalog agents/tools/quality gates → validator (acyclic DAG, artifact types, allowlisted tools). Select and start-run stay independent commands. `POST /runs` freezes role models, report-structure template and run budget. Rewrite / apply / restore are live (`reporting/`). `GET /runs/{id}/stream` is a read-only durable SSE tail (`execution/live_hub.py`): persist then publish, replay + `after_seq` / `Last-Event-ID`, heartbeat comments, token-delta coalesce before seq assignment. Disconnect does **not** cancel the run. Lifespan mounts `ReportCollaborationDispatcher` only when `enabled` **and** `worker_enabled` are both true; default both false.
- **Runtime**: AgentScope adapters live in `app/report_collaboration/agentscope_runtime/` (not the publishable harness). `ConversationReplica` maps AgentScope 2.0.5 events 1:1 onto collaboration SSE types; TeamSay is `HINT_BLOCK`. `DeerFlowChatModelAdapter` wraps `create_chat_model` (no second Provider config); role models are frozen onto node `agent_snapshot` at `POST /runs` and missing/incapable models fail with 422 before the run is created. Fallback is recorded on the snapshot plus a durable `heartbeat` audit event. Nodes complete only after a validated artifact (`app/report_collaboration/execution/completion.py`). `TaskLedger` (`execution/ledger.py`) owns ready waves, attempts and terminal node status; AgentScope / coordinator suggestions never pick the next node. A writer-bearing run is `completed` only after `ReportFinalizer` publishes a formal version. `WaveExecutor` + `QualityRoleKernel` (`execution/quality_roles.py`) remain for fixtures: Verifier marks stale/contested claims, Analyst/Writer only cite supported IDs, Reviewer `revise` replans only the target write node plus review. Production worker injects `AgentScopeMemberKernel` (`agentscope_runtime/member_kernel.py`) with DeerFlow tools and per-run budget. `QualityGate` (`execution/quality_gate.py`) is the claim-level gate: conflicts/missing units, unverified facts, coverage gaps, and unmapped review targets fail before `validated`; `ArtifactBoard.supersede_previous` hides old attempts from downstream. `ReportCollaborationLiveHub` (`execution/live_hub.py`) fans out persisted envelopes to SSE viewers; `PublishingReportCollaborationStore` sanitizes secrets/privacy/hidden reasoning without truncating visible tool results, and writes redaction + event audits. Dispatcher (`execution/dispatcher.py`) atomically claims queued/recovering/expired-lease runs, heartbeats via `renew_lease`, two-phase cancels (`cancel_requested` → `finalize_cancel`), and recovers orphans (`execution/recovery.py`): keep completed work, mark leftover in-flight `interrupted`/`superseded`, open a new `planned` attempt. Late artifacts after cancel/reclaim are `superseded` and must not change terminal run status. Tests drive `dispatch_once()`; do not rely on the background loop. Per-run budgets (`security/budget.py`) fail-closed; per-user active-run caps are enforced at `create_run`.
- **Quality eval**: `app/report_collaboration/eval/` (`RC-BE-018`) — 20 annotated tasks (policy/market/enterprise/event/comparison ×4) with required angles, key facts, forbidden fabrications, templates and review rules. Deterministic scorer (no LLM judge) gates citation/angle/template/directed-mod; gold reports must pass, canned failures (including a roundtable-style essay) must fail and stay reproducible. Recommended runtime knobs live in `eval/thresholds.py` and match current `ReportCollaborationConfig` defaults (`worker_enabled` remains false). Tests: `tests/test_report_collaboration_eval.py`.
- **Runtime intervention**: `app/report_collaboration/interventions/` owns run-time natural-language commands. `RuntimeIntentRouterAgent` uses explicit rules first and a tool-free coordinator-model fallback only for ambiguity; its Pydantic decision cannot dispatch or mutate anything. `ImpactAnalyzer` validates server-side node/agent ownership, resolves angle-specific lineage, and enforces confidence/cost/parallel-branch confirmation (`intent_confirmation_threshold`, default `0.8`). `RuntimeCommandService` mirrors the human message, emits frozen command events, CAS-claims accepted commands, and applies them only through `TaskLedger` safe points. Rework persists command overlays, supersedes affected attempts/artifacts, creates monotonic attempts, and prevents late output from entering downstream. Waiting-node answers resume only those nodes. `rerun_node` without a legal target does not expand to the whole graph; requirement patches apply only after rework succeeds; stuck `executing` commands are reclaimed on takeover or lease timeout; `command.cancelled` is a durable SSE event. In-run `POST /sessions/{id}/messages` returns 409 (`RUN_ACTIVE`). Formal rewrite candidates and version apply/restore live in `reporting/` (`RC-BE-015`).
- **Dependency**: optional extra `report-collaboration` → `agentscope==2.0.5`. Do not add AgentScope to the harness `pyproject.toml`.

Tests: `tests/test_report_collaboration_*.py`.

### Workflow Studio

Coze Studio is canvas-only; ChatDev is reference-only. Plan + status per phase: `docs/WORKFLOW_STUDIO_BACKEND_DEV_ZH.md`; decisions: `docs/adr/0001-workflow-studio-runtime.md`; contracts: `docs/WORKFLOW_SCHEMA_V1_ZH.md` + `docs/WORKFLOW_SSE_V1_ZH.md` + `docs/WORKFLOW_OPENAPI_DRAFT_V1.yaml`. Master switch + all limits: `config.yaml` → `workflows` (`deerflow.config.workflow_config.WorkflowConfig`). `workflows.agent_recursion_limit` defaults to 250: LangGraph counts each model→tool research exchange as about two super-steps, so workflow agent/skill nodes must not fall back to a chat-sized 60-step limit. In the local Studio proxy, keep Rsbuild `server.compress: false`; dev gzip buffers small SSE frames and makes live planning/run output arrive only at completion despite the backend's `no-transform` / `X-Accel-Buffering: no` headers. The composer’s optional `modelName` must be validated against `AppConfig.models`: it selects the controller model and is copied only into newly generated candidate agent configs (or the projected direct-agent task), never injected into an already-confirmed proposal/run.

**Execution kernel is a custom DAG scheduler, not a compiled LangGraph.** LangGraph stays the kernel *inside* an `agent`/`skill` node. The reason is single-source-of-truth recovery: a re-leased or resumed run replays completed node outputs from `workflow_node_runs` plus the `workflow_run_events` log, so a second (checkpoint) copy of workflow-level state would have no defensible tie-breaker. See the ADR's phase-2 amendment.

**Harness** (`packages/harness/deerflow/workflows/`, never imports `app.*`):
- `schemas` / `events` / `errors` / `ports` / `validator` / `samples` — frozen v1 contracts.
- `expressions.py` — `{{ inputs.x }}` / `{{ nodes.n.data.y }}` templates and conditions resolved by path walking + a small comparator set. **No `eval`.** Roots are restricted to `inputs` / `nodes` / `run` / `loop` / `env`.
- `runtime/engine.py` — the scheduler. Every edge is `pending` → `active` (source chose this port) or `pruned`; a node is ready when its non-back incoming edges are all resolved and at least one is `active`, and is skipped when all are `pruned` (skips propagate). That one rule covers fan-out, condition branching, and merge joins with no topological pre-pass. The `loop` node's `continue` back edge is the only permitted cycle; re-entry clears the body's results and invalidates its node runs.
- `runtime/{sink,sql_runner,code_runner,context}.py`, `nodes/*` (15 executors, including deterministic `evidence_normalizer` and durable `deep_research_write`), `security/{http_policy,sql_policy,secret_box}.py`.

**Persistence**: `workflow_definitions` / `workflow_versions` (migration `20260828_01`), `workflow_runs` / `workflow_node_runs` / `workflow_run_events` / `workflow_run_artifacts` (`20260828_02`), `workflow_data_sources` (`20260828_03`), and short-lived-but-durable `workflow_planning_sessions` / `workflow_planning_proposals` (`20260831_01`). Stores under `deerflow.persistence.workflow_{runs,events,data_sources,planning}` with SQL + in-memory implementations. Because planning proposal rows have a scalar FK rather than an ORM relationship, `SqlWorkflowPlanningStore.create_session` must flush the parent session before adding its children; otherwise SQLite can insert the child first and return a 500. A planning proposal is distinct from a run: its graph can be edited with optimistic `graph_revision` checks, but it becomes executable only once selected and atomically bound to one run.

**App layer**: `workflow_dispatcher.py` (lease scan → atomic claim; an expired lease is how a dead worker's run is reclaimed, so every worker can run the loop), `workflow_executor.py` (one lease's lifecycle: replay → engine under a run timeout with heartbeat + cancel watcher → exactly one terminal transition; also runs subworkflows inline, depth ≤ 3), `workflow_retention.py` (hourly sweep of `workflows.retention.event_retention_days` / `run_retention_days`), `workflow_live_hub.py` (in-process SSE fan-out), `workflow_agent_runner.py` (agent/skill nodes → lead agent; `disable_tools=True` means an empty bound-tool set for control-plane callers), `workflow_deep_research_adapter.py` (Evidence Pack → selected Deep Research sources/job/event projection; no internal HTTP call), and `workflow_proposal_planner.py` (calls the dedicated `workflow-planner` controller over visible-agent metadata, then constructs safe candidate graphs server-side). `workflow-planner` is a separate Workflow Studio control-plane agent; never reuse or alias `roundtable-coordinator` for workflow planning, and never let it dispatch a run, call research tools, or return arbitrary graph JSON. It always runs with tools and model thinking disabled.

**Gateway routes** (register static prefixes before `workflows.router`'s `/{workflow_id}` catch-alls): `/api/workflows` CRUD/draft/validate/publish/copy, `/api/workflows/node-types` + `/resources/*`, `/api/workflows/{id}/planning-sessions{/stream}` + `/api/workflows/planning-sessions/{session_id}{,/proposals/{proposal_id},/proposals/{proposal_id}/graph}`, `/api/workflows/{id}/runs` + `/api/workflows/runs/{run_id}{,/events,/stream,/cancel,/resume,/retry,/feedback,/artifacts}`, `/api/workflows/data-sources` (+ `/{id}/introspect`, `/{id}/validate-query`), `/api/workflows/embed/{tickets,verify}`, `/api/workflow_api/*` Coze compat. The planning `/stream` endpoint sends product-safe real controller milestones (`catalog`, `controller`, `selection`, `assembly`, `validation`, `complete`) then a final `result` session; never stream agent catalog descriptions or raw planner JSON. Formal runs accept the selected `planningSessionId` and `proposalId`; creating the same pair again returns the already-bound run rather than launching another execution.

**Guarantees worth not breaking**: events are persisted *then* published, and the DB is authoritative for `seq` ordering (live frames at or below the replay cursor are dropped and re-read on the next poll, so a reconnect sees no gap; `Last-Event-ID` wins over `?after=`); a run reaching a terminal status always gets a terminal event, synthesised if necessary, or SSE clients hang; `resumeToken` appears only in the `run.awaiting_input` event, never in the run view; `POST /retry` on a `failed` run copies completed node outputs onto a new run marked `retry_of_run_id` so the engine resumes from the failed node; `POST /feedback` never mutates its source run—on a terminal acyclic run it may target only a completed `agent`/`skill`, reuses completed nodes outside that node's downstream closure, records those `reusedNodeIds` alongside the feedback summary, and puts the bounded feedback only in `RunContext.env.nodeFeedback[target]` before creating a revision run; data-source secrets are write-only (encrypted via `WORKFLOW_SECRET_KEY`, decrypted only inside the executor, reads expose `masked_target`); the `http` node denies private/loopback/link-local targets by default (SSRF), `sql_read` allows one read-only statement with bound parameters, and the `code` node is **off by default** because its runner is a hardened subprocess rather than a container.

HTTP interface resources persist a non-secret base_url plus an allowed_methods allowlist (migration
20260829_01); request headers stay only in the encrypted secret blob. The resource catalog may return
the address and methods to the owning user, but must never return headers or any decrypted secret. The
executor receives HTTP metadata only through its data-source resolver.

The Workflow Studio agent/skill resource catalog is a read-only display projection over the existing
agent and skill ownership stores. It returns stable `agentId` / `skillId`, an agent's cached skill names,
and a `createdBy` display label. Resolve human labels from the immutable owner id at request time and
fall back to that id if lookup fails; built-in records must return `系统内置`. Do not create a parallel
creator table or accept creator data from the browser.

`evidence_normalizer` is a deterministic data node, not an LLM: it consumes only configured/upstream
node results and emits a bounded `evidencePack` of claims with node provenance, role labels and gaps.
The planner inserts it between parallel research roles and the synthesis Agent. It must not treat
source text as verified fact or fetch any external resource, and its explicit `sources[].nodeId` values
must stay upstream-only under `validate_workflow_graph`.

`deep_research_write` is a durable report node, not an Agent prompt shortcut. It accepts only an
Evidence Pack, derives a session identity from the workflow run/node plus the topic, evidence snapshot and writing config, writes
the pack as selected `other` source rows with upstream-node provenance, and queues/reuses the existing
Deep Research `regenerate` job. Its live `report_delta` and persisted `report_chunk` frames are
deduplicated before becoming workflow `node.output.delta` events; only report prose is forwarded (never
planning/reasoning deltas). Workflow cancellation requests cancellation on that same job. Do not make it
self-call `/api/deep-research/*`, use Deep Research `multi_agent` inside this node, or treat Evidence
Pack claims as verified facts. The parallel-research candidate grants the node an explicit 1800-second
timeout; deployments that override `workflows.node_timeout_seconds` must keep it at least that high.

Tests: `tests/test_workflow_schemas.py`, `tests/test_workflow_definitions.py`, `tests/test_workflow_engine.py`, `tests/test_workflow_runs.py`.

**Import conventions**:
```python
# Harness internal
from deerflow.agents import make_lead_agent
from deerflow.models import create_chat_model

# App internal
from app.gateway.app import app
from app.channels.service import start_channel_service

# App ? Harness (allowed)
from deerflow.config import get_app_config

# Harness ? App (FORBIDDEN ? enforced by test_harness_boundary.py)
# from app.gateway.routers.uploads import ...  # ? will fail CI
```

### Agent System

**Lead Agent** (`packages/harness/deerflow/agents/lead_agent/agent.py`):
- Entry point: `make_lead_agent(config: RunnableConfig)` registered in `langgraph.json`
- Dynamic model selection via `create_chat_model()` with thinking/vision support
- Tools loaded via `get_available_tools()` - combines sandbox, built-in, MCP, community, and subagent tools
- System prompt generated by `apply_prompt_template()` with skills, memory, and subagent instructions

**agentfx built-in analyst + report convert skills.** `agentfx-analyst` (「任务研判报告助手」) is seeded from `app/gateway/routers/_agentfx_seed_assets/` before `_sync_legacy_agents`. After mandatory `web_search`, the agent writes each tool JSON to `/mnt/user-data/workspace/hits/qN.json` and runs **one** command — `task-report-build` (group `all`) — to fill the 14-category `task-reports.json` for the dashboard slots; the four per-page skills `task-report-{enemy,our,env,judge}` remain for re-running a single dashboard page. All five are thin CLIs over the shared stdlib engine `skills/public/task-report-lib/convert_lib.py` (no SKILL.md on the lib, so it never enters the skill list). The model must not `write_file` that JSON. Because the sandbox maps `/mnt/user-data/...` onto a host dir, a virtual absolute hits path can be `isdir`-true yet list empty on a Windows host — `resolve_hit_files` therefore tries the path as given, then the cwd-mapped sandbox path (`discover_user_data_root` + `resolve_sandbox_path` — bash `cd`s into the thread `workspace`, so `/mnt/user-data/outputs/task-reports.json` must land in that thread's `outputs/`, never `F:\mnt\...` on a Windows host), then the virtual-prefix-stripped relative form, then the bare dir name, and picks the first candidate that actually contains `.json`; a miss returns a `warnings` entry listing every tried form so the agent never has to write a debug script. Assemble of the 14 categories is the same function (`assemble_records`); the model must not copy or merge files. If the model is interrupted after convert, `task-reports.json` is already in outputs and the frontend import card also scans `GET /api/threads/{id}/artifacts` so入库 still works without `present_files`. Import remains `task-report-import`. Existing agent/skill directories are never overwritten. Tests: `tests/test_task_report_convert.py`, `tests/test_agentfx_seed.py`.

**Forced-research built-in agent.** `forced-research-responder` (「强制检索输出助手」) is seeded from `app/gateway/routers/_forced_research_seed_assets/` before `_sync_legacy_agents`. `ForcedResearchMiddleware` is mounted only for this stable id. On every visible user turn it requires a successful information-gathering tool result before the model may finish; `tool_choice=required` is backed by an `after_model -> model` retry for gateways that ignore tool choice, with an eight-attempt failure cap that permits only an honest failure response. Skill discovery, reading `SKILL.md`, and writing a helper script do not satisfy the gate; web/knowledge/MCP results, non-skill uploaded-file reads, or successful Python skill-script execution do. Tool filtering removes `present_files`, `skill_manage`, task/schedule/memory/clarification surfaces. The middleware permits `write_file`/`str_replace` only for `/mnt/user-data/workspace/*.py` and permits `bash` only for `.py` execution without redirection/document paths, so skills can run scripts but the agent cannot create Markdown/document artifacts or write `outputs`. Tests: `tests/test_forced_research_agent.py`.

**ThreadState** (`packages/harness/deerflow/agents/thread_state.py`):
- Extends `AgentState` with: `sandbox`, `thread_data`, `title`, `artifacts`, `todos`, `uploaded_files`, `viewed_images`
- Uses custom reducers: `merge_artifacts` (deduplicate), `merge_viewed_images` (merge/clear)

**Runtime Configuration** (via `config.configurable`):
- `thinking_enabled` - Enable model's extended thinking
- `model_name` - Select specific LLM model
- `is_plan_mode` - Enable TodoList middleware
- `subagent_enabled` - Enable task delegation tool

### Middleware Chain

Lead-agent middlewares are assembled in strict append order across `packages/harness/deerflow/agents/middlewares/tool_error_handling_middleware.py` (`build_lead_runtime_middlewares`) and `packages/harness/deerflow/agents/lead_agent/agent.py` (`_build_middlewares`):

1. **ThreadDataMiddleware** - Creates per-thread directories under the creator's stable isolation scope (`backend/.deer-flow/users/{user_id}/threads/{thread_id}/user-data/{workspace,uploads,outputs}`); resolves persisted threads through `resolve_path_user_id(thread_id)` (falls back to `get_effective_user_id()` / `"default"` when metadata is unavailable); Web UI thread deletion now follows LangGraph thread removal with Gateway cleanup of the local thread directory
2. **UploadsMiddleware** - Tracks and injects newly uploaded files into conversation
3. **SandboxMiddleware** - Acquires sandbox, stores `sandbox_id` in state
4. **DanglingToolCallMiddleware** - Injects placeholder ToolMessages for AIMessage tool_calls that lack responses (e.g., due to user interruption), including raw provider tool-call payloads preserved only in `additional_kwargs["tool_calls"]`
5. **LLMErrorHandlingMiddleware** - Normalizes provider/model invocation failures into recoverable assistant-facing errors before later middleware/tool stages run
6. **GuardrailMiddleware** - Pre-tool-call authorization via pluggable `GuardrailProvider` protocol (optional, if `guardrails.enabled` in config). Evaluates each tool call and returns error ToolMessage on deny. Three provider options: built-in `AllowlistProvider` (zero deps), OAP policy providers (e.g. `aport-agent-guardrails`), or custom providers. See [docs/GUARDRAILS.md](docs/GUARDRAILS.md) for setup, usage, and how to implement a provider.
7. **SandboxAuditMiddleware** - Audits sandboxed shell/file operations for security logging before tool execution continues
8. **ToolErrorHandlingMiddleware** - Converts tool exceptions into error `ToolMessage`s so the run can continue instead of aborting
9. **SummarizationMiddleware** (`DeerFlowSummarizationMiddleware`) - Context reduction when approaching token limits (optional, if enabled). **Non-destructive**: overrides the stock `before_model` (which persists a `RemoveMessage(REMOVE_ALL_MESSAGES)`) to a no-op and instead compresses **transiently** in `wrap_model_call` ? the model request is replaced with `[summary, *recent]` while the full original conversation stays in state (so the frontend keeps rendering the user's real Q&A). The `summary` message is `hide_from_ui` and never persisted; summaries are cached **per-thread in a process-wide, thread-safe LRU** (`_GLOBAL_SUMMARY_CACHE`, bounded to 512 threads, sticky boundary). The cache is module-level — **not** on the middleware instance — because `make_lead_agent` rebuilds a fresh middleware on every run; an instance cache would be cold each turn and force a blocking summary LLM call on every turn of a long conversation. Keyed by `thread_id` (resolved like `ThreadDataMiddleware`), so a summary computed on one turn is reused on the next, paying for an LLM call only on the first threshold crossing or a genuine re-compaction. **Threshold-first gating**: `_plan_compression` returns passthrough (no compression at all) whenever the *full* current conversation is still under the trigger — the cache/summary substitution only engages once the real context crosses the configured trigger. **Stickiness headroom** (`_apply_stickiness_headroom`, `_KEEP_HEADROOM_FRACTION=0.75`): after the `keep` policy picks a cutoff, the kept suffix is forced below ~75% of *every* trigger before summarizing. Without it a `keep` in *messages* (e.g. 20) can itself exceed a *token* trigger (e.g. 60000) when recent messages are large (tool outputs / pasted docs), leaving the "compressed" context still over the trigger so it re-summarizes on every follow-up question; the headroom makes the sticky boundary actually stick (re-compaction only after genuinely new content). When the genuine (blocking) summarize path runs, it emits a `{"type":"context_compacting","message":"正在压缩上下文…"}` event on LangGraph's `custom` stream channel (the cheap reuse path stays silent); the chat frontend's `onCustomEvent` renders it as a transient toast. Tests: `tests/test_summarization_middleware.py`
10. **TodoListMiddleware** - Task tracking with `write_todos` tool (optional, if plan_mode)
11. **TokenUsageMiddleware** - Records token usage metrics when token tracking is enabled (optional)
12. **TitleMiddleware** - Two-phase auto-title (never shows "untitled"): `before_model` sets an instant question-derived *provisional* title the moment the first user message arrives (no LLM, zero latency), then `after_model` asynchronously generates the final LLM title and replaces it (tracked via the `title_provisional` state flag). Normalizes structured message content before prompting the title model
13. **MemoryMiddleware** - Memory V2: `wrap_model_call` injects Hindsight recall transiently; `after_agent` auto-retains the turn to Hindsight
14. **ViewImageMiddleware** - Injects base64 image data before LLM call (conditional on vision support)
15. **DeferredToolFilterMiddleware** - Hides deferred tool schemas from the bound model until tool search is enabled (optional)
16. **SubagentLimitMiddleware** - Truncates excess `task` tool calls from model response to enforce `MAX_CONCURRENT_SUBAGENTS` limit (optional, if `subagent_enabled`)
17. **LoopDetectionMiddleware** - Detects repeated tool-call loops; hard-stop responses clear both structured `tool_calls` and raw provider tool-call metadata before forcing a final text answer
18. **ClarificationMiddleware** - Intercepts `ask_clarification` tool calls, interrupts via `Command(goto=END)` (must be last)
8a. **ToolMetricsMiddleware** - Appended immediately after `ToolErrorHandlingMiddleware` in `_build_runtime_middlewares`, so it sits *deepest* in the `wrap_tool_call` chain. Records one `tool_call_metrics` row per tool/skill/MCP invocation ? timing duration and bucketing failures into a coarse `error_category` (rate_limited / ip_blocked / auth_error / timeout / network_error / not_found / empty_result / tool_error). It catches both raised exceptions (before #8 converts them) and *self-handled* errors returned as error `ToolMessage`s (search tools that swallow HTTP 429/403). It also captures the targeted Agent Skill into `tool_call_metrics.skill_name`: the lead agent invokes a skill by `read_file`-ing its `/mnt/skills/<category>/<name>/SKILL.md` path (per the `<skill_system>` prompt block ? there is no dedicated "use skill" tool), so `skill_name` is recovered by scanning every tool call's string arguments for the `skills/{public,custom}/{name}/` path shape (`_SKILL_PATH_RE`), plus the explicit `name` argument of `skill_view`/`skill_manage`/`skill_archive`. `skill_view`-specific success detection handles its swallowed failures (`Skill 'x' not found.` returned as a plain string rather than raised). Regression coverage: `tests/test_tool_metrics_middleware.py`. Writes are best-effort fire-and-forget. **Per-run skill merging**: `SqlToolMetricsStore.record` upserts skill rows by `(run_id, skill_name)` instead of inserting one row per call ? a skill touched by several tool calls / retries inside one run collapses to a single row. Any success in the run wins (a skill that failed then finally succeeded is recorded as one `success`); a skill that only ever failed yields one `error` row whose `error_message` accumulates every attempt's message; durations are summed. Non-skill tool calls, and skill calls with no `run_id`, keep one row each. The read?merge?write is guarded by a per-event-loop `asyncio.Lock`. Surfaced by the admin `GET /api/admin/tool-metrics` router (which has a `kind=skill` filter and a per-skill `/summary` breakdown).
9. **PromptPrefixMiddleware** - Injects the admin-configured prompt prefix into the model request only. The prefix is tagged onto the first human message's `additional_kwargs['prompt_prefix']` by the Gateway (`app.gateway.services`) and prepended to content here at call time, so the persisted/displayed message stays prefix-free
10. **SummarizationMiddleware** (`DeerFlowSummarizationMiddleware`) - Context reduction when approaching token limits (optional, if enabled). **Non-destructive**: overrides the stock `before_model` (which persists a `RemoveMessage(REMOVE_ALL_MESSAGES)`) to a no-op and instead compresses **transiently** in `wrap_model_call` ? the model request is replaced with `[summary, *recent]` while the full original conversation stays in state (so the frontend keeps rendering the user's real Q&A). The `summary` message is `hide_from_ui` and never persisted; summaries are cached **per-thread in a process-wide, thread-safe LRU** (`_GLOBAL_SUMMARY_CACHE`, bounded to 512 threads, sticky boundary). The cache is module-level — **not** on the middleware instance — because `make_lead_agent` rebuilds a fresh middleware on every run; an instance cache would be cold each turn and force a blocking summary LLM call on every turn of a long conversation. Keyed by `thread_id` (resolved like `ThreadDataMiddleware`), so a summary computed on one turn is reused on the next, paying for an LLM call only on the first threshold crossing or a genuine re-compaction. **Threshold-first gating**: `_plan_compression` returns passthrough (no compression at all) whenever the *full* current conversation is still under the trigger — the cache/summary substitution only engages once the real context crosses the configured trigger. **Stickiness headroom** (`_apply_stickiness_headroom`, `_KEEP_HEADROOM_FRACTION=0.75`): after the `keep` policy picks a cutoff, the kept suffix is forced below ~75% of *every* trigger before summarizing. Without it a `keep` in *messages* (e.g. 20) can itself exceed a *token* trigger (e.g. 60000) when recent messages are large (tool outputs / pasted docs), leaving the "compressed" context still over the trigger so it re-summarizes on every follow-up question; the headroom makes the sticky boundary actually stick (re-compaction only after genuinely new content). When the genuine (blocking) summarize path runs, it emits a `{"type":"context_compacting","message":"正在压缩上下文…"}` event on LangGraph's `custom` stream channel (the cheap reuse path stays silent); the chat frontend's `onCustomEvent` renders it as a transient toast. Tests: `tests/test_summarization_middleware.py`
11. **TodoListMiddleware** - Task tracking with `write_todos` tool (optional, if plan_mode)
12. **TokenUsageMiddleware** - Records token usage metrics when token tracking is enabled (optional)
13. **TitleMiddleware** - Two-phase auto-title (never shows "untitled"): `before_model` sets an instant question-derived *provisional* title the moment the first user message arrives (no LLM, zero latency), then `after_model` asynchronously generates the final LLM title and replaces it (tracked via the `title_provisional` state flag). Normalizes structured message content before prompting the title model
14. **MemoryMiddleware** - Queues conversations for async memory update (filters to user + final AI responses)
15. **ViewImageMiddleware** - Injects base64 image data before LLM call (conditional on vision support)
16. **DeferredToolFilterMiddleware** - Hides deferred tool schemas from the bound model until tool search is enabled (optional)
17. **SubagentLimitMiddleware** - Truncates excess `task` tool calls from model response to enforce `MAX_CONCURRENT_SUBAGENTS` limit (optional, if `subagent_enabled`)
18. **LoopDetectionMiddleware** - Detects repeated tool-call loops; hard-stop responses clear both structured `tool_calls` and raw provider tool-call metadata before forcing a final text answer
19. **ClarificationMiddleware** - Intercepts `ask_clarification` tool calls, interrupts via `Command(goto=END)` (must be last)

### Configuration System

**Main Configuration** (`config.yaml`):

Setup: Copy `config.example.yaml` to `config.yaml` in the **project root** directory.

**Config Versioning**: `config.example.yaml` has a `config_version` field. On startup, `AppConfig.from_file()` compares user version vs example version and emits a warning if outdated. Missing `config_version` = version 0. Run `make config-upgrade` to auto-merge missing fields. When changing the config schema, bump `config_version` in `config.example.yaml`.

**Config Caching**: `get_app_config()` caches the parsed config, but automatically reloads it when the resolved config path changes or the file's mtime increases. This keeps Gateway and LangGraph reads aligned with `config.yaml` edits without requiring a manual process restart.

Configuration priority:
1. Explicit `config_path` argument
2. `DEER_FLOW_CONFIG_PATH` environment variable
3. `config.yaml` in current directory (backend/)
4. `config.yaml` in parent directory (project root - **recommended location**)

Config values starting with `$` are resolved as environment variables (e.g., `$OPENAI_API_KEY`).
`ModelConfig` also declares `use_responses_api` and `output_version` so OpenAI `/v1/responses` can be enabled explicitly while still using `langchain_openai:ChatOpenAI`.

**Extensions Configuration** (`extensions_config.json`):

MCP servers and skills are configured together in `extensions_config.json` in project root:

Configuration priority:
1. Explicit `config_path` argument
2. `DEER_FLOW_EXTENSIONS_CONFIG_PATH` environment variable
3. `extensions_config.json` in current directory (backend/)
4. `extensions_config.json` in parent directory (project root - **recommended location**)

### Gateway API (`app/gateway/`)

FastAPI application on port 8001 with health checks at `GET|HEAD /health`, `/api/health`, and `/api/langgraph/health`. These exact paths are always public in `auth_middleware.PUBLIC_HEALTH_PATHS`, including when authentication is enabled; reverse-proxy prefixes and trailing slashes are normalized before the whitelist check. Set `GATEWAY_ENABLE_DOCS=false` to disable `/docs`, `/redoc`, and `/openapi.json` in production (default: enabled). Regression coverage: `tests/test_health_auth_bypass.py`.

**Routers**:

| Router | Endpoints |
|--------|-----------|
| **Models** (`/api/models`) | `GET /` - list models; `GET /{name}` - model details |
| **MCP** (`/api/mcp`) | `GET /config` - get config; `PUT /config` - update config (saves to extensions_config.json) |
| **Skills** (`/api/skills`) | `GET /` - list skills; `GET /{name}` - details; `PUT /{name}` - update enabled; `POST /install` - install from .skill archive (accepts standard optional frontmatter like `version`, `author`, `compatibility`) |
| **Memory V2** (`/api/memory/v2`) | `GET /` - builtin data (USER.md/MEMORY.md); `POST/PUT/DELETE /entries` - add/replace/remove entry; `GET/PATCH/DELETE /config` - effective config + per-user overrides; `POST /search` - Hindsight recall/reflect proxy |
| **Agents** (`/api/agents`) | `GET /` - list built-in, owned, and published agents; `POST /` - create user-owned agent; `GET/PUT/DELETE /{id}` - read/update/delete by stable agent id; `GET /check` - validate non-empty display name (duplicates allowed) |
| **Skills** (`/api/skills`) | `GET /` - list skills (optional `search` fuzzy-name + `tag_id` filters; each item carries `tags`); `GET /{name}` - details; `PUT /{name}` - update enabled; `POST /install` - install from .skill archive (accepts standard optional frontmatter like `version`, `author`, `compatibility`) |
| **Skills** (`/api/skills`) | `GET /` - list skills (optional `search` fuzzy-name + `tag_id` filters; each item carries `tags` + owner-authored `detail`); `GET /{name}` - details; `PUT /{name}` - update enabled; `PUT /{name}/detail` - set the long-form `detail` text (custom skill: owner or admin; built-in/public: admin only ? materializes a `skills` row on demand); `POST /install` - install from .skill archive (accepts standard optional frontmatter like `version`, `author`, `compatibility`) |
| **Skills** (`/api/skills`) | `GET /` - list skills (optional `search` fuzzy-name + `tag_id` filters; each item carries `tags` + owner-authored `detail`); `GET /{name}` - details; `PUT /{name}` - update enabled; `PUT /{name}/detail` - set the long-form `detail` text (custom skill: owner or admin; built-in/public: admin only ? materializes a `skills` row on demand); `GET /{name}/content` - raw SKILL.md source, restricted to an admin or the publisher (custom-skill owner); other users get 403 and see only description + detail; `POST /install` - install from .skill archive (accepts standard optional frontmatter like `version`, `author`, `compatibility`) |
| **Skills** (`/api/skills`) | `GET /` - list skills (optional `search` fuzzy-name + repeatable `tag_ids` multi-select filter, OR-matched; each item carries `tags` + owner-authored `detail`); `GET /{name}` - details; `PUT /{name}` - update enabled; `PUT /{name}/detail` - set the long-form `detail` text (custom skill: owner or admin; built-in/public: admin only ? materializes a `skills` row on demand); `GET /{name}/content` - raw SKILL.md source, restricted to an admin or the publisher (custom-skill owner); other users get 403 and see only description + detail; `POST /install` - install from .skill archive (accepts standard optional frontmatter like `version`, `author`, `compatibility`) |
| **Memory** (`/api/memory`) | `GET /` - memory data; `POST /reload` - force reload; `GET /config` - config; `GET /status` - config + data |
| **Agents** (`/api/agents`) | `GET /` - list built-in, owned, and published agents (optional `search` + repeatable `tag_ids` multi-select filter, OR-matched; each item carries `tags` and `featured_order`); `POST /` - create user-owned agent; `GET/PUT/DELETE /{id}` - read/update/delete by stable agent id; `GET /check` - validate non-empty display name (duplicates allowed); `GET /selector` - homepage agent picker (single admin-curated list, visibility-filtered, sorted by `featured_order` asc); `PUT /{id}/featured` - **admin** sets/clears the homepage `featured_order` (must be `published=true` to feature; passing `order: null` clears unconditionally); `PUT /featured/order` - **admin** bulk-reorders the homepage list (body `{agent_ids: [...]}` ? indices become 0..N-1) |
| **Tags** (`/api/tags`) | `GET /` - list tags (optional `tag_type` filter; open to all); `POST /` - create tag (admin); `PUT /{id}` / `DELETE /{id}` - update/delete tag (admin); `GET /assignments/{target_type}/{target_id}` - tags on a resource; `PUT /assignments/{target_type}/{target_id}` - replace a resource's tags (admin). Tags are typed `agent`/`skill`/`scheduled_task` and may only tag resources of their own type. |
| **Uploads** (`/api/threads/{id}/uploads`) | `POST /` - upload files (auto-converts PDF/PPT/Excel/Word); `GET /list` - list; `DELETE /{filename}` - delete |
| **Threads** (`/api/threads/{id}`) | `DELETE /` - remove DeerFlow-managed local thread data after LangGraph thread deletion; unexpected failures are logged server-side and return a generic 500 detail |
| **Threads search** (`POST /threads/search`, LangGraph-compat) | Conversation listing. **System threads are excluded by default**: rows whose `metadata.system` is truthy (scheduler runs, roundtable workers) never crowd the user's list; a caller whose `metadata` filter explicitly references `thread_type`/`system` (e.g. studio notebooks) opts out of the exclusion. New `query` field = case-insensitive title (`display_name`) substring search, used by the chats page's server-side pagination. `POST /threads/count` takes the same request body (limit/offset ignored) and returns `{total}` under identical filtering/exclusion semantics ? powers the chats page's ??/??? display without fetching rows (`ThreadMetaStore.count`). `ThreadMetaStore.search` gained `exclude_system`/`query` params; JSON-blob filters batch-scan until the requested `limit`/`offset` window is full (so pagination over a heavily-filtered set never under-fills), with a `thread_id` tie-breaker on the `updated_at DESC` ordering for stable OFFSET paging. Tests: `tests/test_thread_meta_search.py` |
| **Artifacts** (`/api/threads/{id}/artifacts`, `/api/artifact-library`) | `GET /{path}` - serve artifacts; active content types (`text/html`, `application/xhtml+xml`, `image/svg+xml`) are always forced as download attachments to reduce XSS risk; `?download=true` still forces download for other file types. `GET /api/artifact-library` scans every current-user normal thread's authoritative `outputs/` directory (including files omitted from `present_files`), excludes uploads and image/audio/video media, joins the owning conversation title/agent route metadata, and exposes search/filter/pagination with a bounded 15-second per-user cache plus `refresh=true` bypass. Frontend route `/page/workspace/library` links each item back to the source chat with `?artifact=<path>` so the chat opens the existing artifact panel directly. |
| **Word export** (`POST /api/writing/export/docx`) | Shared Markdown → DOCX generator used by conversation, artifact, scheduled-task, Deep Research, and AI-writing download surfaces. `app/gateway/word_export.py` builds OOXML with no Python conversion dependency: A4 portrait, 3.7/3.5/2.8/2.6 cm margins, real multilevel heading numbering (`一、` / `(一)` / `1.`), two-character first-line indent on heading levels 1–3, formal body/table styles, page numbers beginning at 1 with odd pages left-aligned and even pages right-aligned, and embedded `app/gateway/assets/fonts/方正小标宋简体.ttf`. Before numbering it removes complete manual heading prefixes (including `2.1` / `2.1.1`) and drops Markdown horizontal rules from Word output. The ordinary-Q&A Lead Agent can emit the matching Markdown heading forms and avoid unordered bullet lists when the admin enables `prompt_prefix.ordinary_qa_markdown_format_enabled` in `system_settings.json`; the switch defaults off, while writing setup, writing mode, notebook, scheduled runs, and SOUL-driven custom agents keep their independent prompts. The font loader validates its internal family, SHA-256, and OpenType embedding permission (`OS/2.fsType`) before export. Tests: `tests/test_word_export.py`, `tests/test_es_query_routing_prompt.py`. |
| **Deep Research** (`/api/deep-research`) | Durable, user-owned research sessions/jobs with replayable SSE events and hidden-thread artifacts. `basic`/`quick` use the compact runner; `detailed` creates bounded subtopics, searches/writes sections concurrently, and assembles a report; `deep` recursively searches follow-up questions under a deterministic breadth/depth query budget. `multi_agent` is a LangGraph 1.x editor/researcher/writer/reviewer graph: its initial plan emits `awaiting_input`, the recorder stores the interrupt and releases the lease, and `POST /jobs/{id}/resume` rebuilds the graph from the persisted approved/revised plan. A bounded `custom_outline` is frozen into the session; headings/list entries deterministically supply the detailed and multi-agent plan, and the shared writer receives it only as a guarded content constraint. Completed reports support owner-scoped follow-up (`GET /sessions/{id}/messages`, `POST /sessions/{id}/chat`): context is only the current report, selected sources and recent messages; source-id citations are persisted, while `allow_new_research=true` is explicitly rejected. Owners may change that evidence set only after the job stops via `PATCH /sessions/{id}/sources/{source_id}`; this applies to future follow-ups rather than rewriting a completed report. Optional report illustrations are gated by `config.yaml -> deep_research.image_generation`: the Harness `OpenAICompatibleImageProvider` requires base64 output, the shared runner step writes only bounded `research-image-01..04.(png|jpg|webp)` artifacts and Markdown uses private `deep-research://` links. `GET /sessions/{id}/artifacts/{name}` checks session ownership before serving an image; an unavailable provider yields a warning but never fails the text report. Runners live in `packages/harness/deerflow/agents/deep_research/runners/`, may never import `app.*`, and use only the `AdapterBundle`. |
| **Gateway auth** | Owner-scoped Deep Research routes await the platform's asynchronous `get_current_user` resolver before they read or write persistence, ensuring each session and job operation receives a concrete user ID. Deep Research artifacts are always resolved from a whitelisted filename via `/mnt/user-data/outputs/<name>` before mapping to a host path, preserving the sandbox guard on native Windows and Linux hosts. |
| **Suggestions** (`/api/threads/{id}/suggestions`) | `POST /` - generate follow-up questions; rich list/block model content is normalized before JSON parsing |
| **Thread Runs** (`/api/threads/{id}/runs`) | `POST /` - create background run; `POST /stream` - create + SSE stream; `POST /wait` - create + block; `GET /` - list runs; `GET /{rid}` - run details; `POST /{rid}/cancel` - cancel; `GET /{rid}/join` - join SSE; `GET /{rid}/messages` - paginated messages `{data, has_more}`; `GET /{rid}/events` - full event stream; `GET /../messages` - thread messages with feedback; `GET /../token-usage` - aggregate tokens |
| **Feedback** (`/api/threads/{id}/runs/{rid}/feedback`) | `PUT /` - upsert feedback; `DELETE /` - delete user feedback; `POST /` - create feedback; `GET /` - list feedback; `GET /stats` - aggregate stats; `DELETE /{fid}` - delete specific |
| **Runs** (`/api/runs`) | `POST /stream` - stateless run + SSE; `POST /wait` - stateless run + block; `GET /{rid}/messages` - paginated messages by run_id `{data, has_more}` (cursor: `after_seq`/`before_seq`); `GET /{rid}/feedback` - list feedback by run_id |
| **Open Chat** (`/api/open/chat`) | **Unauthenticated** open Q&A entry for external systems (the `/api/open/` prefix is whitelisted in `auth_middleware._PUBLIC_PATH_PREFIXES` + CSRF-exempt in `csrf_middleware.should_check_csrf`). `POST /chat` `{message \| messages[], model_name?, agent_name?, agent_id?, thinking_enabled=false, strip_think=true, thread_id?, recursion_limit=100}` → resolves the agent by **Chinese display name** (`agent_name`, matched against each `.deer-flow/agents/*/config.yaml` `name` via `_resolve_agent_id`; explicit `agent_id` wins if given), builds the lead agent via `make_lead_agent` (so the chosen **agent's** SOUL/tools/skills all apply), `ainvoke`s it once (stateless — no checkpointer, no thread persistence; runs under the `"default"` user bucket), and returns **only the final** `{answer, model_name, agent_name, agent_id, thinking_enabled, thread_id}` (no intermediate tool-call/process). **`POST /chat/stream`** is the **SSE streaming variant** (same request body) for **long tasks** — the blocking `/chat` produces no downstream bytes for the whole run, so a front nginx `proxy_read_timeout` (default 60s) cuts it off with a **504**; the stream keeps the connection warm (a `: keepalive` SSE comment every `_HEARTBEAT_SECONDS=15`s via a background producer-task + queue, plus `event: delta` AI-text increments and `event: progress`), and ends with **`event: result`** carrying the same final `{answer, …}` JSON (or `event: error`). Clients only need to read `event: result`; intermediate events are ignorable. Both endpoints share `_build_run_context` (config/agent/thread). `GET /agents` lists `{name, id}` so callers know what `agent_name` to send. **Thinking off is reliable**: when `thinking_enabled=false` the route also sets `thinking_force_disabled=true` so the lead agent calls `create_chat_model(force_disable_thinking=True)`, which injects **three** OpenAI-compatible "thinking off" shapes into `extra_body` — top-level `enable_thinking=false` (**SiliconFlow / DashScope hybrid models** like `deepseek-ai/DeepSeek-V4-*` on `api.siliconflow.cn` — the knob the nested `chat_template_kwargs` shape does NOT reach), `chat_template_kwargs.enable_thinking=false` (vLLM/Qwen native), and `thinking.type="disabled"` (Zhipu GLM) — even for models that only declare `supports_thinking: true`. For the **web-UI path** (which sends `thinking_enabled=false` but NOT `force_disable`), SiliconFlow models need `when_thinking_disabled: {extra_body: {enable_thinking: false}}` in their `config.yaml` entry (added to `deepseek-ai/DeepSeek-V4-Flash`) for thinking-off to take effect. Sets `is_scheduled_run=true` internally to skip the auto-title LLM call. **Generated-file return (`include_files=true`, default on)**: an agent that produces its report as a file (writes to `/mnt/user-data/outputs/` + `present_files`) returns only a short summary in `answer`; to recover the real deliverable both endpoints now also return `files: [{name, virtual_path, encoding(text|base64), content}]` — `_collect_output_files` scans the run's `state['thread_data']['outputs_path']` (the open thread is stateless so artifacts can't be fetched by thread, but the endpoint runs server-side and the sandbox already wrote the files to disk), text decoded as utf-8 else base64, capped by `_MAX_OUTPUT_FILES=20` / `_MAX_OUTPUT_FILE_BYTES=5MB`. `/chat` puts `files` on the response; `/chat/stream` puts it in the `event: result` payload. A run with no answer **and** no files now 502s ("…文本答案或文件"). Router `app/gateway/routers/open_chat.py`; standalone caller `test_open_chat_api.py` (repo root, streaming by default); tests `tests/test_open_chat.py`. |
| **Open 3qfx Ask** (`/api/open/3qfx/ask`) | **Unauthenticated** open entry that, given just a **任务id**, directly kicks off the 3qfx（三情分析）「第一次问答」as a **real persisted conversation** (sibling of `/api/open/chat`; same `/api/open/` auth-whitelist + CSRF-exempt). `POST /ask` `{task_id, message?, model_name?, thinking_enabled=false}` → (1) resolves the `3Q` agent from `app.state.business_mapping_store.get_mapping("3Q")` (the **same** agent the frontend deep-link `goPath=3qfx` uses; 503 if unconfigured); (2) **fetches the task detail server-side** (consumer `cop-task-detail`) and builds the opening text **identical to the frontend** `buildOpeningText` 3qfx branch — `任务名称：…\n任务内容：…\n任务id（task_id）为{task_id}` (an explicit `message` overrides and **skips** the fetch); (3) creates a fresh thread tagged `metadata.{taskId, agent_id}` (the exact filter `useTaskThreads` uses, and `metadata.taskId` makes it task-deeplink-shared / creator-path-resolved) and **starts a background persisted run** via `start_run(..., enforce_access=False)` (writes the checkpointer, runs under the `"default"` bucket); (4) **returns immediately** `{thread_id, run_id, agent_id, status, opening_text}` — the caller just needs "started", then watches the run in the 3qfx 任务工作区 conversation list. `start_run` gained a keyword-only `enforce_access` (default `True`); the open entry passes `False` so the admin-configured agent isn't rejected by the per-(missing)-login visibility check. **Consumer fetch config** lives in `config.yaml → task_deeplink`: `cop_task_detail_url` (full URL, appends `?taskId=`), `cop_task_detail_authorization` (default `Bearer admin`), `cop_task_detail_test` + `mock.cop_task_detail` (offline fake when the upstream is unreachable — on in this deployment). Router `app/gateway/routers/open_3qfx.py`; tests `tests/test_open_3qfx.py`. |
| **Thread Shares** (`/api/thread-shares`) | Per-conversation sharing with two modes (`thread_shares.mode`). **`import`**: `POST /` `{thread_id, mode}` - create (or reuse) a revocable share code; `GET /{code}/preview` + `POST /{code}/import` - copy that conversation into the caller's account (fresh `thread_id` via `app/gateway/thread_copy.py` + new `threads_meta` row + `user-data` copy; dedup on `metadata.shared_origin`). **`view`**: a public, read-only static page ? see Public Shares below. Both modes: `GET /` - list my codes; `DELETE /{code}` - revoke. |
| **Public Shares** (`/api/public/shares`) | **Unauthenticated** (the `/api/public/` prefix is whitelisted in `auth_middleware._PUBLIC_PATH_PREFIXES`). `GET /{code}` - read-only conversation snapshot (title + serialized messages + artifacts) for a `view`-mode share; messages are read straight from the latest checkpoint. `GET /{code}/files/{path}` - serves a file referenced by that conversation (images, Markdown, text, PDF, ?) so the shared page's links are openable; path resolution is confined to the owner's thread directory. Active content (`text/html`, `application/xhtml+xml`, `image/svg+xml`) is forced to download (`_public_share_disposition`); everything else is served inline. Missing/revoked/non-`view` codes all return 404. |
| **Roundtable Drafts** (`/api/roundtable-drafts`) | **Strictly per-user** persistence for the roundtable-planning page (replaces the old browser `localStorage`). `GET /` - list drafts (lightweight meta — incl. `chain_title`, the business-chain name used this session, cheaply regex-extracted from the in-memory `step2` blob in `_draft_to_meta`/`_extract_chain_title`; null = no chain / 自由模式; surfaced on the 第一步「分析记录」history card + sidebar list instead of the old「第 X 步」); `POST /` - create; `GET/PUT/DELETE /{id}` - full read / partial update / delete (all owner-scoped). A draft = one full session (Step 1 intent + Step 2 roundtable), step snapshots stored as JSON in `PortableLongText`. Recommendation history bound to a draft: `GET /{id}/recommendations` - list newest-first; `POST /{id}/recommendations` - append one run (objective/status/rationale/picks/candidates). taskId 深链会商记录 are **not** here — they live in the separate `/api/roundtable-task-drafts` store (see below) so personal history never mixes with task records. Backed by `deerflow.persistence.roundtable_drafts` (tables `roundtable_drafts`, `roundtable_recommend_history`); store wired in `deps.py` as `app.state.roundtable_draft_store`. Tests: `tests/test_roundtable_task_drafts.py`. |
| **Roundtable Task Drafts** (`/api/roundtable-task-drafts`) | **Task-scoped, unpartitioned** store for taskId-deeplink roundtable 聊天记录 (无界嵌入抽屉). Keyed **only by `task_id`** (one task → many records), **never partitioned by user** — any authenticated caller may read/write/delete a task's records (`created_by` is audit-only). Physically separate from `roundtable_drafts` so the two never mix. `GET /by-task/{task_id}` - all drafts for a task, newest-first (history dropdown), enriched with the latest background-job status per draft via `RoundtableJobRepository.status_map_by_drafts` (each meta also carries `chain_title`, same `_extract_chain_title` regex as the personal store); `GET /by-task/{task_id}/latest` - newest draft (回显最新一条); `POST /` - create (`task_id` required); `GET/PUT/DELETE /{id}` - full read / partial update / delete (all user-agnostic). Backed by `deerflow.persistence.roundtable_task_drafts` (table `roundtable_task_drafts`, auto-created by `create_all`; migration `20260618_01`); store wired in `deps.py` as `app.state.roundtable_task_draft_store`. Existing `task_id`-bearing `roundtable_drafts` rows are migrated over by `scripts/migrate_roundtable_task_drafts.py`. Tests: `tests/test_roundtable_task_drafts.py`, `tests/test_roundtable_drafts_by_task.py`. |
| **Roundtable Diagnostics** (`/api/roundtable-diagnostics`) | 会商全流程诊断日志 (报错 + 关键事件). One row per event in `roundtable_diagnostics`: `{scope (background = 后台挂起 / foreground = 前台服务端观测 / client = 前端上报), stage (step1_intent/step2_init/step2_leader/step2_seat/step2_recommend/step2_orchestration/step3_summary/step3_report/step3_dashboard/step3_structure/upstream/job), level (warning/error — **`info` 级运行日志不再收集**：`record_diagnostic` 在写库前丢弃 `level=="info"`，只留报错；begin/consensus/gather/job_started 等纯进度事件即使仍调 `.diag(level="info")` 也成 no-op，故诊断页 = 报错日志页), event, message, detail (JSON), job_id/draft_id/task_id/user_id/agent_id/agent_name/cycle, created_at}`. **Reads are admin-only**: `GET /` - filter (scope/stage/level/event/job_id/draft_id/task_id/user_id/q/since/until; `scope` accepts a comma-separated set so 前端「前台调用」= `foreground,client`) + paginate (newest-first), each item enriched with `userName` (user_id → 邮箱 `@`-prefix via `get_local_provider().get_user`, best-effort batched cache); `GET /facets` - distinct values for filter dropdowns; `POST /admin/cleanup` - retention purge. **`POST /report` is NOT admin-gated** — any authenticated user's frontend posts the errors *the user actually hit* (HTTP 502 before any SSE frame / network drop / SSE error frame) so they're collected too; stored as `scope="client"`. Backed by `deerflow.persistence.roundtable_diagnostics` (table `roundtable_diagnostics`, auto-created by `create_all`; migration `20260619_01`); store wired in `deps.py` as `app.state.roundtable_diagnostic_store` + a process-level default store (`register_default_store`) so the harness-side engine/gateway can write across the app/harness boundary. **Writers** (all best-effort — diagnostics never break a run): background path uses `DiagnosticsRecorder` (bound to job ctx) threaded through `RoundtableJobExecutor` → `JobProgressWriter.diag(...)` (engine transitions/step3 failures/empty-delivery) + `InProcessRoundtableGateway` (per-turn timeout/report·summary·dashboard truncation+missing); foreground path uses `app/gateway/roundtable_diag.py::record_foreground_diag` in `multi_agent.py`/`intent.py`/`recommend.py` error frames. **Bug-fix shipped alongside**: `InProcessRoundtableGateway._run_agent_turn` guards each agent run with an **inactivity (idle) timeout** (`_await_run_with_idle_guard`, `ROUNDTABLE_SEAT_IDLE_TIMEOUT_SECONDS` default 1200s = 20min, tuned high for unstable intranet models) — **not** a total-duration cap, so a slow-but-alive run that keeps streaming tokens runs as long as it needs; only a run that emits **zero StreamBridge events for `idle_timeout` seconds straight** (a true freeze: 内网模型连接池挂死 / sqlite 锁在挂载卷上挂死) is cancelled + raises (顶层把 job 标 error，而非永久冻结) + records a `seat_turn_timeout` diagnostic. Activity is observed via `app.state.stream_bridge` (the worker publishes every chunk there). Optional absolute cap `ROUNDTABLE_SEAT_MAX_TOTAL_SECONDS` (default 0 = off). Frontend: admin page `pages/RoundtableDiagnosticsPage.tsx` (route `/page/strategy/admin/roundtable-diagnostics`), opened from the **会商页右上角设置按钮** (admin only — `handleSettingsClick` in `roundtable-planning/pages/RoundtablePlanningPage.tsx`); admin read client `strategy-components/api/roundtable-diagnostics.ts`. **Client error reporter** `roundtable-planning/api/diagnostics-report.ts` (`reportRoundtableError` fire-and-forget + `keepalive`, `setRoundtableErrorContext` for draft/task attribution set by the page, `stageForRoundtableAgent`) is called at every client-observable failure: `multi-agent.ts` (`streamMultiAgent` — covers Step2 leader/seat AND Step3 report/summary/dashboard/structure — reports HTTP error + SSE `{status:"error"}` frame + worker `event:error` frame [模型调用没成功] + fetch-reject [网络不可达] + mid-stream drop [连接重置/超时], deduped via a `reported` flag; `initMultiAgent` HTTP/error-frame/incomplete/mid-stream), `intent.ts` (Step1 HTTP+error-frame), `recommend.ts` (Step2 recommend HTTP+error-frame), `roundtable-jobs.ts` (job api/stream). The admin page's first filter is a **后台挂起 / 前台调用** toggle (frontend = `foreground,client`); the stage dropdown options adapt to it (background → step2+step3+job; frontend → step1+step2+step3) and each row shows the reporting user's name + 👤. Tests: `tests/test_roundtable_diagnostics.py`, `tests/test_roundtable_inprocess_timeout.py`. **Seat skill-recognition (弱模型补偿)**: a seat = one custom agent run whose configured skills (`agent config.yaml → skills`) are injected **only** via the system prompt's `<skill_system>` block (model must `read_file` the SKILL.md itself) — weak models (deepseek-chat 等) ignore it and never use their skills. `app/gateway/roundtable_seat_skills.py::append_seat_skill_directive(agent_id, task, *, chain_enabled=False)` appends an explicit Chinese directive (each configured-and-enabled skill's name + description + SKILL.md container path + "先 read_file 再按其流程取信息/执行") to the **seat's per-turn task (human turn)** — far stronger pull on weak models than the system prompt. Wired into **both** seat paths: foreground `multi_agent._special_run` (only `role in {seat, seat_parallel}`; Step3 singletons report/summary/dashboard/ingest excluded) + background `InProcessRoundtableGateway.run_seat`. No-op (returns task unchanged) when the seat has no configured skills. **2026-06-20: default-off + two opt-in switches (OR semantics)** — was global-on, now injects only when **either** the seat agent's own switch (`AgentConfig.seat_skill_directive`, edited in the agent edit dialog `agent-card.tsx` + `agent-detail-panel.tsx`) **or** the **per-seat** switch on that agent **within the selected business chain** is on. The per-seat chain switch lives **in the chain's `seats` JSON** (`ChainSeatSchema.seat_skill_directive`, no DB column — rides the existing JSON like the per-seat `model` override), edited via a small ✨ icon → dropdown (Switch + description) on each agent card in `BusinessChainEditorPage`. It is threaded **per-seat** to each seat run: foreground `multi_agent` per-call field `seat_skill_directive` (orchestration looks it up per agentId from a `{agentId: bool}` session map) → `_special_run` → `chain_enabled`; background as the job field `seat_skill_directives: dict[str,bool]` (StartJobRequest → StartParams → deps factory → `InProcessRoundtableGateway._seat_skill_directives`) → `run_seat` looks up `agent_id` → `chain_enabled`. `ROUNDTABLE_SEAT_SKILL_DIRECTIVE` is only a **master kill-switch** (default allow; set `0/false` to hard-disable regardless of the switches — never force-enables). Tests: `tests/test_roundtable_seat_skills.py`, `tests/test_agents_backend.py`. **2026-06-20 Step2 提速三件套**：(1) **每席位推理深度覆盖** —— 业务链条 `seats` JSON 里逐席位可配 `seat_mode`（flash/thinking/pro/ultra，覆盖链条级 `seatMode`；编辑器席位卡右侧 ⏱Gauge 图标下拉，null=跟随链条默认）。前台经 `useStep2Orchestration` 三处 dispatch 的 `seatModeToRequestFields(seatModes[id] ?? selectedMode)` 透传；后台经 job `seat_modes: dict[str,str]` → `StartParams` → `InProcessRoundtableGateway._seat_modes`，`run_seat` 用 `_SEAT_MODE_PARAMS`（与前端 `seatModeToRunParams` 逐字对齐）解析覆盖作业级 seat_*（未命中→跟随作业级→policy 默认）。无 DB 列（ride `seats` JSON，但 `ChainSeatSchema.seat_mode` 须显式声明否则 Pydantic 剥掉）。Tests: `tests/test_roundtable_seat_modes.py`。(2) **recommend 批内并行** —— 后端 `engine._run_recommend` 由 for-await 串行改 `asyncio.gather`（对齐前端 2026-06-13 已并行；总控一轮派多席位=判定相互独立→并发；批内用派活前 deliveries 快照、批后统一并入，语义不变）。(3) **研讨前并行『取数』两段式**（业务链条 `gather_first` 列，逐条 opt-in，migration `20260620_02`，editor「研讨前并行取数」开关）—— `gather_first=True` 时 `run_orchestration` 在分发前先 `engine._run_gather_phase`（全席位 `asyncio.gather` 跑 `_GATHER_TASK`「只取数不研讨」），产出 `gathered: dict` 经 `_seat_message(gathered=...)` 作为「已收集资料」块注入 recommend/chain/dag 各研讨轮席位任务（与「前序交付」块分开），把最慢的技能延迟前置并行、给（尤其 chain 串行的）研讨段提速；研讨阶段不限制再调技能。前台对称实现：`useStep2Orchestration.runGatherPhase` 在 `runActiveOrchestration` 分发前跑一次（`gatherPhaseDoneRef` 闸门，resetInitGate 复位），产出存 `gatheredDeliveriesRef` 经 `peerDeliveries`/初始 `priorDeliveries` 注入研讨席位。链条级 `gatherFirst`(camel)↔`gather_first`(snake) 经 `api/chains.ts` 映射；job 经 `StartJobRequest.gather_first`→`StartParams`→`run_orchestration(gather_first=…)`。Tests: `tests/test_roundtable_gather_phase.py`。(4) **削减重复广播**（前端 only，多智能体 foreground 路径）—— `suppress_pre_broadcast`（仅 leader：抑制「总控本轮输入→各席位 thread」预广播 + 「总控给X派活」→兄弟席位的 post 广播，二者一个 flag）此前只在 DAG 派活轮开；现扩到 recommend（`safetyCounter > 1`，即第 2 轮起的续轮/综合）+ chain 逐席位（`i > 0`，第 2 个席位起）+ chain 收口轮（恒 true）。**保留每种模式的首轮预广播**（它为席位打底任务背景），只 suppress 重复的续轮广播（N 次读写巨大席位 thread 的纯浪费 + 弱模型模仿派活）；席位的前序交付另经「席位交付广播」`broadcastSeatDeliveryToOthers` 送达，不受影响。后台引擎路径无此广播（`_seat_message` 直接拼上下文），故 #4 仅前端。**2026-06-20 深链接业务链抽取规范**：深链接会商（taskId+goPath，businessCode 如 rwfx→6BF）下，结构化抽取(flow-json)与方案总结报告都按所选业务链**完整层级**强制校正。结构化抽取实为 `roundtable-structure` 智能体（`flow_extract.py` 的 `_system_prompt()=load_agent_soul(STRUCTURE_AGENT_ID)`，抽取→校验两段式 `model.astream`）；其 SOUL 全局一份，旧实现把某业务层级写死进 SOUL→所有业务都按「目的→行为体」抽（bug）。修法：**单份领域知识、两角色用法**的 `deerflow.agents.roundtable_orchestrator.extraction_spec`（放 harness 层，app+harness 都可 import；`build_structure_directive`/`build_summary_directive` 按 businessCode 取；rwfx=6BF 写实「任务→目的→行为体→预期行为→驱动因素→叙事，预期行为↔驱动因素 1:1、余 1:N、label 不含步骤/阶段名、优先报告回退席位」，其余业务通用规范=强约束完整性+命名+取数且明令不套固定模板；business_code 空→""零副作用）。注入：(1) 结构化抽取 `flow_extract._hierarchy_hint`+`_validate_messages` 用 `build_structure_directive` 权威覆盖默认层级（前端 `streamFlowExtract` 传 `business_code`，源 `ingestBusinessCode`）；(2) 方案总结报告 `build_summary_directive`——前台 `multi_agent._special_run`(role==summary 且 business_code) 追加、后台 `engine.build_summary_prompt(business_code=…)`（StartParams/StartJobRequest 透传）。**仅深链接生效**。种子 `roundtable-structure/SOUL.md` 层级顺序改 spec-driven、连接示例去 rwfx 写死（存量 live SOUL 不被 seed 覆盖，但注入规范权威可纠正）。Tests: `tests/test_roundtable_extraction_spec.py`。 |
| **Roundtable Draft Shares** (`/api/roundtable-draft-shares`) | Per-draft sharing for the roundtable-planning page, mirroring Thread Shares but the shared resource is a `roundtable_drafts` row. Two modes (`roundtable_draft_shares.mode`). **`import`**: `POST /` `{draft_id, mode}` - create (or reuse) a revocable code; `GET /{code}/preview` + `POST /{code}/import` - copy that draft (step1/2/3) into the caller's account under a deterministic per-(code,recipient) id (`imp_<sha1>` ? re-import is a no-op). **`view`**: a public, read-only result page ? see Public Roundtable Shares. Both: `GET /` - list my codes; `DELETE /{code}` - revoke. Backed by `deerflow.persistence.roundtable_draft_shares` (table `roundtable_draft_shares`, migration `20260606_01`); store wired in `deps.py` as `app.state.roundtable_draft_share_store`. Tests: `tests/test_roundtable_draft_shares.py`. |
| **Public Roundtable Shares** (`/api/public/roundtable-shares`) | **Unauthenticated** (`/api/public/` whitelisted). `GET /{code}` - read-only snapshot for a `view`-mode draft share: returns the Step 3??????report HTML (`step3.html`, a self-contained document) + title/summary/`has_report`. The frontend renders it in a sandboxed `iframe` (no `allow-scripts`) at `#/roundtable-share/<code>`. Missing/revoked/non-`view` codes all return 404. |
| **Scheduled Tasks** (`/api/scheduled-tasks`) | Per-user cron-like agent tasks. `GET/POST /` - list/create owned tasks; `PATCH/DELETE /{id}` - update/delete (owner); `POST /{id}/{pause,resume,trigger}`; `GET /{id}/runs`, `GET /runs/{rid}`, run-file preview/edit/download; `GET /runs/{rid}/messages` - agent messages for the run's underlying agent run (used to inspect a stuck/running run; authorized via `_load_run`); `GET /runs/{rid}/html` - generated HTML document for an `html_page` run; `DELETE /runs/{rid}` - owner-scoped, with an admin fallback so admins can delete any user's run; publish/subscribe under `/{id}/{publish,unpublish,subscribe}` and `GET /published`. **Admin (cross-user):** `GET /admin/all` - every task; `PATCH/DELETE /admin/{id}` - edit/delete any task; `POST /admin/{id}/trigger`; `GET /admin/{id}/runs`. Admin routes require `system_role == "admin"`. |
| **HTML Page Favorites** (`/api/html-page-favorites`) | Bookmarked generated HTML pages, owner-scoped. `GET /` - list; `POST /` `{page_id, title?, description?, tags?}` - bookmark a `scheduled_html_pages` row (extracts a reusable style/layout summary via GLM5 at favorite time); a page can only be favorited once per user (`DuplicateFavoriteError` ? HTTP 409). `GET/DELETE /{id}` - read/delete (the create-task reference picker deletes favorites here). A favorite can be selected as a style reference when creating a new `html_page` task. |
| **Public Scheduled Tasks** (`/api/public/scheduled-tasks`) | **Unauthenticated** mirror for embedding. Adds `GET /runs/{rid}/html` and `GET /{task_id}/latest-html` for the `html_page` preview; the run result keeps `result.html` (only file paths are redacted). |
| **Light Apps** (`/api/light-apps`) | Light-app registry (????? / ????), **global/shared** (no user scoping; `created_by`/`updated_by` audit-only). Two kinds: `window_open` (launched in a new tab from the ???? card grid) and `iframe` (embedded under a parent sidebar menu via `mount_location` and dynamically injected into the sidebar). Both carry `route_params` ? a list of `{key, paramType: login\|theme, source, customValue, themeValue}` descriptors the **frontend** resolves at launch time (login params read from `localStorage.userInfo`; theme params use a literal value) and appends to `url` as query string. `GET /` - list (filters `app_type`/`mount_location`/`enabled_only`/`search`); `GET /{app_id}` - read one (both open to any authenticated user); `POST /`, `PUT /{app_id}`, `DELETE /{app_id}` - **admin only**; iframe apps must carry a `mount_location` (400 otherwise); the unique `name` collides as HTTP 409. Backed by `deerflow.persistence.light_apps` (table `light_apps`, auto-created by `create_all`); store wired in `deps.py` as `app.state.light_app_store`. Tests: `tests/test_light_apps.py`. |
| **Menu Overrides** (`/api/menu-overrides`) | Sidebar-menu overrides (菜单管理), **global/shared** (no user scoping; `updated_by` audit-only). The static menu tree (icons/routes/`adminOnly`) lives in the **frontend** code (`core/page-layout/sidebar-menu.ts`); iframe light-apps are injected at render time. This router persists a per-node **overlay** the frontend merges onto that tree: `parent_id` (re-parent, moves the whole subtree; NULL = top level), `sort_order` (reorder among siblings), `label` (rename; NULL keeps code label), `disabled` (停用 — hides the node **and its subtree**). A node with no row renders exactly as code defines it, so new code menus / newly-registered iframe apps appear automatically and are immediately configurable. `GET /` - list all overrides (open to any authenticated user); `PUT /` - **admin only**, atomically **replaces the whole set** (`{overrides: [...]}`); rejects re-parenting that would exceed **3 levels** or form a cycle (`_validate_depth`, HTTP 400). Backed by `deerflow.persistence.menu_overrides` (table `menu_overrides`, auto-created by `create_all`); store wired in `deps.py` as `app.state.menu_override_store`. Frontend: merge logic `core/page-layout/menu-overrides.ts` (`applyMenuOverrides` in `use-sidebar-menu.ts`), client `strategy-components/api/menu-overrides.ts`, admin page `pages/MenuManagementPage.tsx` (@dnd-kit drag-reorder/re-parent + 「移动到」parent selector + ↑↓ arrows + 改名/停用), entry in the bottom-left settings dropdown (`components/page-sidebar.tsx`, admin-only). Tests: `tests/test_menu_overrides.py`, `frontend-web/src/core/page-layout/menu-overrides.test.ts`. |
| **Compact Menu Overrides** (`/api/compact-menu-overrides`) | Compact/sidebar-summary left-nav overlay (简介模式菜单), **global/shared**. Two independent sets keyed by `menu_type` (`A` default / `B`); saving one type does not wipe the other. Type B rows use a `B::` node_id prefix so they stay unique on databases that still use `node_id` as the sole PK. `GET ?menu_type=` lists one type (open); `PUT {menu_type, overrides}` admin-only replaces that type. Frontend: `MenuManagementPage` 简介模式 type switcher + `runtime-config.js` `VITE_COMPACT_MENU_TYPE`. Tests: `tests/test_compact_menu_overrides.py`. |
| **Sensitive Words** (`/api/sensitive-words`) | Sensitive-word / desensitize map (敏感词管理 / 脱敏词), **global/shared** (no user scoping; `updated_by` audit-only). Each row maps an original `word` → `replacement`, with `enabled` (停用 parks a word without deleting) + `sort_order`. The **frontend** ships a seed map in code (`src/lib/desensitize.ts` `SEED_DESENSITIZE_MAP`) used as the default on a fresh deployment; once an admin saves, this table is the source of truth and the frontend loads it at startup (`DesensitizeWordsLoader` → `replaceDesensitizeMap`, **enabled** rows only). `GET /` - list all words (open to any authenticated user — every page desensitizes); `PUT /` - **admin OR the `lqq` account** (`_require_manager`: `system_role == "admin"` or email `@`-prefix `== "lqq"`), atomically **replaces the whole set** (`{words: [...]}`, de-duped by `word`, last wins). Backed by `deerflow.persistence.sensitive_words` (table `sensitive_words`, PK `word` `String(191)`, auto-created by `create_all`); store wired in `deps.py` as `app.state.sensitive_word_store`. Frontend: client `strategy-components/api/sensitive-words.ts`, page `pages/SensitiveWordsPage.tsx` (route `/page/strategy/admin/sensitive-words`, entry in the bottom-left settings dropdown — **gated on username `lqq`, not admin role**, present in both sidebar modes; 「导入种子词」 merges missing seed entries). Tests: `tests/test_sensitive_words.py`. |
| **Sentiment Agent Proxy** (`/api/sentiment-agent`) | BFF for the frontend 舆情分析 virtual agent. `POST /stream` accepts the AG-UI body plus `apiUrl`/`authUsername`/`authPassword` from `runtime-config.js`, forwards to that URL with TLS verify off, and byte-pipes upstream SSE. Connection/HTTP failures return the full error in `detail`; mid-stream failures emit AG-UI `RUN_ERROR`. Distinct from `skill-packages/sentiment-analysis.skill`. Tests: `tests/test_sentiment_agent_proxy.py`. |
| **Dashboard Sessions** (`/api/dashboard-sessions`) | 「大屏绘制智能体」独立页面会话，**按外部 `task_id` 存取，不按 user 分权**（open read + shared writes；`user_id` audit-only）。一个 task 一行（upsert）：`transcript`（聊天记录 JSON）+ `report_json`（最新结构化数据，可解析 = 「有效结果」）。`GET /by-task/{task_id}` - 取该 task 会话（无则 404）；`PUT /by-task/{task_id}` - upsert（`{transcript?, report_json?}`，只覆盖传入项）。Backed by `deerflow.persistence.dashboard_sessions`（table `dashboard_sessions`, auto-created by `create_all`，无迁移）；store wired in `deps.py` as `app.state.dashboard_session_store`。前端独立页 `roundtable-planning/pages/DashboardAgentPage.tsx`（路由 `/page/strategy/dashboard-agent?taskId=`，侧边栏「问答管理 → 大屏绘制」）让智能体分析任务报告 JSON → report-json → 渲染 `DashboardApp`。Tests: `tests/test_dashboard_sessions.py`。 |
| **Task Reports** (`/api/task-reports`) | 任务级研判情报报告详情，**按 `task_id` 共享、不按 user 分权**（写入者只留 `updated_by` 审计）。`GET /by-task/{task_id}` 始终返回原 Consumer 兼容外壳 `{state:"200", msg:"操作成功！", data:[...]}`（未保存为 `data: []`）；`POST /by-task/{task_id}` 首次保存，已有报告返回 409；`PUT /by-task/{task_id}` 原子替换完整 `data` 数组。`POST /import/by-task/{task_id}` 面向 TaskCOP 导入页，接收 `sq-report-mock.json` 兼容的 `data`/`records` 数组并完整覆盖目标任务；路径中的选定任务 id 优先于文件内的 `taskId`，避免误写。每一项严格是 `{id, taskId, createTime, updateTime, sessionId, contentJson, categoryType}`；`contentJson` 以 JSON **字符串**原样落在 `records_json`，避免破坏大屏按类目解析的输入格式。保存或导入非空 `data` 后，router 会通过 `app.state.taskcop_task_store.mark_analysis_completed` 将同 id 的 TaskCOP 任务升级至状态 `25`，使旧任务列表显示并筛选为“已完成”；普通保存时不存在匹配任务的报告仍可独立保存，导入则要求目标任务存在。Backed by `deerflow.persistence.task_reports`（table `task_reports`, auto-created by `create_all`）；store wired in `deps.py` as `app.state.task_report_store`。前端客户端 `strategy-components/api/task-reports.ts`，`core/auth/dashboard-input.ts` 与 rwfx 3q 完成检查均直接读此接口，不再依赖 Consumer 或本地 sq-report mock。Tests: `tests/test_task_reports.py`。 |
| **Task Buttons** (`/api/task-buttons`) | Task-workspace configurable jump buttons (按钮管理), **global/shared** (no user scoping; `updated_by` audit-only). Each row = one jump button in a task deep-link workspace's bottom 快捷跳转 bar: `{id, business (rwfx/3qfx/xdfx), label, link_type (url/business/purpose/host), target, append_task_id, id_param (editable query-param name, e.g. id/task_id), id_kind (task/action — only xdfx differs, its deep-link id is an 行动id), open_mode (blank/self), login_params, append_auth, auth_token_param, auth_name_param, enabled, sort_order}`. **Two independent, coexisting login-append mechanisms for `url` buttons** — both resolved frontend-side: (1) `login_params` (JSON column `login_params_json`, `PortableJSON`, migration `20260625_01`) = a list of `{key, source, custom_value}` resolved at click time (`source` = a `userInfo` field or `other`→literal `custom_value`) and appended to the target URL — mirrors the light-app login params; empty = nothing appended. (2) `append_auth` + `auth_token_param` + `auth_name_param` (columns, migration `20260625_02`, default off) — appends `?{auth_token_param}={token1}&{auth_name_param}={username}` so the target system can identify the current user; gated on **token login** — the frontend only stashes `token1` (the `?authToken=`) + the raw `getTokenInfo` username (`login.token1`/`login.username`) when login went through `/login/token`, so a non-token-login session appends neither. The **frontend** holds code-seed defaults (rwfx's 4 historical buttons built from the live `task_deeplink` config); only when this table is empty do the workspace + admin page fall back to them, else this table is the sole source of truth. `GET /` - list all (open to any authenticated user — the workspace renders them); `PUT /` - **admin only** (`_require_admin`), atomically **replaces the whole set** (`{buttons: [...]}`, de-duped by `id`, last wins). Backed by `deerflow.persistence.task_buttons` (table `task_buttons`, auto-created by `create_all`); store wired in `deps.py` as `app.state.task_button_store`. Frontend: helpers `core/auth/task-buttons.ts`, client `strategy-components/api/task-buttons.ts`, admin page `pages/TaskButtonsPage.tsx` (route `/page/strategy/admin/task-buttons`, entry「按钮管理」in the bottom-left settings dropdown, admin-only, both sidebar modes; per-business tabs + 「导入默认」). Tests: `tests/test_task_buttons.py`. |
| **Fixed Questions** (`/api/fixed-questions`) | Administrator-managed exact-match fixed Q&A (问题配置), **global/shared**. Each row is `{id, question, answer, enabled, tokens_per_second, sort_order}`; stream speed is configurable per row from `1–1000 token/s` and defaults to `100`. `GET /` and whole-set atomic `PUT /` are both admin-only. At the start of an external chat run, `services._find_fixed_answer` checks the latest eligible plain-text human message with exact, case/space/punctuation-sensitive equality. Enabled matches switch the run to `make_fixed_answer_agent`: a local `BaseChatModel` tokenizes with `cl100k_base`, replays the answer in rate-limited token chunks, and keeps the existing LangGraph SSE `messages`/`values` protocol, run journal, checkpoint history, deterministic local title, and refresh behavior intact while no provider/model request or model-concurrency slot is used. File/multimodal/context-prefixed turns, `canvas_agent`, `ai_writing`, writing mode, resume commands, and internal orchestration/background child runs bypass matching. Storage failures fail open to the normal model path. Backed by `deerflow.persistence.fixed_questions` (table `fixed_questions`, SHA-256 indexed lookup plus exact original-string collision check, migrations `20260821_01` + `20260821_02`), wired as `app.state.fixed_question_store`. Frontend: `strategy-components/api/fixed-questions.ts`, admin page `pages/FixedQuestionsPage.tsx`, route `/page/strategy/admin/fixed-questions`, entries in both full-layout and compact-workspace settings menus. Tests: `tests/test_fixed_questions.py`. |
| **Knowledge** (`/api/knowledge`) | Product knowledge base, **global/shared** (no user isolation; writes record `created_by`/`updated_by` for audit). `POST /ingest/thread/{thread_id}` - sediment one thread into a note (reads messages from the checkpointer, `threads:read` owner-checked); `POST /ingest/threads/batch` - batch-sediment the caller's recent threads (`skip_existing` via `.manifest.json` dedup). **Defaults to `background=true`**: the slow per-thread extraction loop is deferred to a FastAPI background task and the endpoint returns `{queued:true, processed:N}` immediately (each thread's transcript is truncated to `max_input_chars` before the LLM call). Running the loop inline for a large batch outlives the nginx gateway timeout (504), which the frontend's `parseResponse` surfaces as a generic "Request failed"; `background=false` keeps the blocking path (full per-thread `created/skipped/failed` tally) for small batches/scripts. Shared loop: `_run_batch_ingest`. **Optional model selection**: both ingest endpoints accept `model_name` (overrides `knowledge.extract_model_name` for that capture; threaded through `KnowledgeService.capture_thread` ? `extract_thread_llm_multi`). **Model-error surfacing**: when the chosen model produces nothing usable (timeout / API error / non-JSON) capture still **falls back to rule-based extraction and lands a draft** (`llm_fallback:true`); the single-thread (non-background) response and the batch response (`llm_fallback` count) carry this so the UI tells the user "???????". The "??????" chat button stays background/fire-and-forget (passes `model_name`, no per-save error toast); the batch-import dialog is synchronous and shows the fallback count. `POST /ingest/file` - **import an uploaded file** (multipart: `file` + optional `title`/`tags`/`status`/`model_name`) into wiki note(s): the file is converted to Markdown (md/txt read directly, pdf/word/ppt/excel via `convert_file_to_markdown`), distilled by the same wiki-capture LLM pipeline as thread capture (`source_type="file"`), and falls back to a verbatim **draft** note when the model returns nothing usable (`llm_fallback:true`); `POST /ingest/search` - sediment search/tool results as a reference note; `POST /notes` - manual note; `GET /notes` (keyword/tag/source_type/status + pagination); `GET/PUT/DELETE /notes/{id}` (DELETE is soft = archive, `?hard=true` removes the vault file); `POST /search` - `mode=keyword\|vector\|hybrid` (vector?keyword fallback when no embedding endpoint); `GET /notes/{id}/versions` - edit history; `GET /duplicates` + `POST /merge` - dedup; `POST /reindex` - rebuild vector index; `GET /graph` - product graph data (notes + entities, graph-blacklist filtered); `GET/POST /graph/blacklist` + `DELETE /graph/blacklist/{id}` - graph node blacklist (labels = entity names / note titles, case-insensitive match; blacklisted nodes and their edges are dropped from `/graph` and the wiki-export artifacts; table `knowledge_graph_blacklist`, auto-created); `POST /export/graph` + `GET /export/files/{graph.json\|graph.html\|graph.graphml\|cypher.txt}` - obsidian-wiki `wiki-export` artifacts (self-contained `graph.html` for iframe); `POST /resolve` `{targets:[?]}` + `GET /notes/{id}/wikilinks` - resolve `[[wikilink]]` targets to a note id / vault page / missing (powers clickable wikilinks); `GET /vault-page?path=` - read a raw vault meta/hub Markdown page (no DB note). `GET/POST /folders` + `PUT/DELETE /folders/{id}` - user-managed directories (notes carry a `folder` path; `GET /notes?folder=` filters a dir + sub-dirs); `GET/POST /extract-templates` + `GET/PUT/DELETE /extract-templates/{id}` - editable extraction-prompt templates (thread/document ingest accept `template_id`; capture/PUT-note accept `folder`). Backed by the `deerflow.knowledge` module (see below). |

Proxied through nginx: `/api/langgraph/*` ? LangGraph, all other `/api/*` ? Gateway.

### TaskCOP Situation-Overview Compatibility

The standalone legacy Vue page calls the Gateway URL in its runtime `b1ConsumerUrl` configuration directly (no Vite `/api` reverse proxy). The Gateway CORS middleware merges standard local Vite origins (`localhost`/`127.0.0.1` on ports 5173, 5174, 3000, and 8080) even when `GATEWAY_CORS_ORIGINS` was set for another frontend. This avoids a restart regression when the existing Vite server is on 5173 but the launcher sets 5174. Deployments must set `GATEWAY_CORS_ORIGINS` to any additional exact frontend origins that are allowed to call the Gateway.

`app/gateway/routers/taskcop_tasks.py` is a temporary compatibility layer for the separately deployed legacy Vue page at `/situation-overview/action/sentiment` and its task import page at `/situation-overview/action/task/import`. It preserves the old Consumer paths and response envelopes: `POST /cop/saveSuperiorTask` creates a row with a sequential four-digit string id (`0001`, `0002`, ...) and returns `{state: 201, msg, data}`; `POST /api/taskcop/import/tasks` accepts a JSON array or `tasks`/`records`/legacy list envelope, uses each supplied `id` or `taskId` as the primary key, and completely replaces duplicate IDs; `DELETE /cop/tasks/{task_id}` removes that task and its shared report document; `GET /taskAnalyseSearch/cop-task-three-list` (all), `GET /taskAnalyseSearch/cop-my-tasks` (creator-scoped), and `GET /taskAnalyseSearch/cop-task-list` return `{state: 200, data: {total, records}}`; `GET /taskAnalyseSearch/cop-task-detail?taskId=` returns one task. Existing UUID-style task rows are retained, and do not affect the new numeric sequence. An empty TaskCOP store remains empty—there are no built-in task or report seed records. It supports the page's existing `content`, `taskStatus`, `taskDirection`, `startTime`, `endTime`, `pageNum`, and `pageSize` query fields. Tasks are stored in `taskcop_tasks` through `deerflow.persistence.taskcop_tasks`, registered by `persistence.models`, and initialized as `app.state.taskcop_task_store` in `deps.py`. A non-empty save/update/import through `/api/task-reports/*` calls `mark_analysis_completed(task_id)`: it changes only pre-completion task states to `25`, so the list renders and filters the task as “已完成” without downgrading later workflow states. `trendSucc` is derived from the same status for the legacy sentiment list. Zero-padded task ids stay strings in the report API, so `0001` is never converted to `1`. The Vue router calls DeerFlow's `/api/v1/auth/login/username` directly with an incoming `username`/`userId`/`yUserId` parameter; it does not depend on the retired Consumer login service, and sends the returned DeerFlow JWT to every TaskCOP call. The TaskCOP routes are no longer public or CSRF-exempt. The Gateway fallback CORS allow-list includes TaskCOP Vite origins `http://localhost:8080` and `http://127.0.0.1:8080`; deployments must use explicit `GATEWAY_CORS_ORIGINS`. Tests: `tests/test_taskcop_tasks.py` and `tests/test_taskcop_cors.py`.

### AI Writing Intent Router (对话驱动写作台意图路由)

Backs the frontend「AI 写作·对话版」page's unified bottom composer. `POST /api/ai-writing/sessions/{session_id}/intent` (owner-checked; in `routers/ai_writing.py`) reads the session's graph checkpoint via the shared `app.state.checkpointer` — the **pending interrupt is the authoritative pause point** (the frontend's claimed `pause_point` is only a fallback when the checkpoint can't be read) — then routes the user's free text through `deerflow/agents/ai_writing/intent_router.py::parse_user_intent`: an LLM call (session's `model_name` + `fallback_chain` model fault-tolerance) that returns `{intent, pause_point, action, payload, need_research, research_query, answer, confidence, clarify}`. Safety boundaries in `normalize_intent_result`: **per-pause action whitelist strictly aligned to graph.py's real resume actions** (`ALLOWED_ACTIONS`), no pause → no action, high-cost actions (`finalize` / `force_finalize` / query-less覆盖式 `re_search`) require confidence ≥ 0.85 else downgrade to `clarify`, payload key-whitelist cleaning (material-id filtering, review-issue index bounds, section-decision back-fill with the majority decision). QA intents return `answer` directly from the context snapshot (`build_context_snapshot` — stage-tailored: materials with ids / outline / draft body capped / review issues with indices). Model failures never 5xx — they degrade to clarify. Prompts in `agents/ai_writing/prompts/intent_router_prompts.py`; tests `tests/test_ai_writing_intent.py`.

### AI Writing Built-in Functional Agents (????????)

The AI-writing graph (`packages/harness/deerflow/agents/ai_writing/`) has four stage "brains" ? ?????? / ????? / ?? / ?? ? that can each run in one of two modes, switched per-stage by `config.yaml ? ai_writing.use_builtin_agents.{researcher,outliner,writer,editor}` (default **false**, mtime hot-reloaded, new sessions pick up changes immediately):

- **Legacy (flag off)**: hard-coded prompts in `prompts/*.py`, direct LLM call (unchanged behavior).
- **Builtin-agent (flag on)**: the node loads the corresponding functional agent from `.deer-flow/agents/ai-writing-<stage>/` ? `SOUL.md` becomes the system prompt and `config.yaml`'s `model` overrides the stage model (priority: `agent.model > node explicit fallback (keyword_model/rank_model) > state.model_name`). Admins edit SOUL/model from the agent management page. Runner: `nodes/_agent_runner.py` (`run_writing_agent` / `stream_writing_agent` / `parse_marker_json`). Output contracts (marker + ```json``` blocks, intent.py-style): researcher `[KEYWORDS_READY]` / `[MATERIALS_RANKED]`, outliner `[OUTLINE_READY]`, editor (per-dimension) `[REVIEW_READY]`; writer emits raw section Markdown (keeps per-section streaming) with the legacy `[[SECTION_BLOCKED]]` sentinel in strict mode. **Orchestration stays in the nodes** (search fan-out, per-section loop, 3-dimension review loop, all pause points) ? the agent only does single structured generations, and a missing/empty SOUL silently falls back to the legacy path (`WritingAgentUnavailable`).

Seeding mirrors the roundtable pattern: verified `config.yaml` + `SOUL.md` ship in `app/gateway/routers/_ai_writing_seed_assets/<id>/`; `_ai_writing_seed.py::ensure_ai_writing_functional_agents()` copies them to `.deer-flow/agents/` when missing (never overwrites local edits), called from the startup lifespan (before `_sync_legacy_agents`, so the agents appear in `/api/agents` on first boot) and self-heals from `GET /api/ai-writing/sessions` + `GET /api/ai-writing/article-types`. Tests: `tests/test_ai_writing_seed.py`, `tests/test_ai_writing_agent_runner.py`.

**Sample imitation mode (样文仿写).** The initial writing form supports `writing_mode_type=imitate`: the user can paste sample text or upload Word/PDF/Markdown/TXT. `POST /api/ai-writing/sample/extract` converts supported document files to Markdown via `deerflow.utils.file_conversion` and returns capped plain text to the frontend. The LangGraph entry point now starts at `sample_analyzer` before `intent_parser`; non-imitation sessions return immediately, while imitation sessions analyze up to 12k chars into `sample_style_profile` (`SampleStyleProfile` in `state.py`) and emit `sample_profile_ready`. `writer_outline` and `writer_draft` call `render_sample_style_section(state)` so both outline planning and section drafting receive the same style profile, imitation strength (`light|medium|strong`), optional structure-preservation preference, and guardrails: learn structure/tone/paragraph rhythm/sentence style, but do not copy sample sentences, proprietary facts, people, organizations, cases, or data. Frontend state mirrors this with `analyzing_sample`, uploaded/pasted sample controls, history/timeline restoration of `sampleStyleProfile`, and the setup-chat direct-start path preserves the same fields. Tests live with the knowledge-base AI-writing tests (`tests/test_ai_writing_knowledge_base.py`).

**Skill-driven retrieval (检索类型).** `material_source` has three values: `general` (通用检索, default), `knowledge_base` (知识库检索), and `notebook` (我的空间); legacy `skill` is normalized to `general` by the researcher node. **`knowledge_base` 直连内网 ES 接口（= 旧版「内网直连检索」）** — it bypasses skill orchestration entirely and calls `IntranetSearchProvider` over `ai_writing.intranet_search_url` (concurrent over the generated keywords; `intranet_knowledge_base`/`intranet_search_timeout` apply). **An empty/unconfigured `intranet_search_url` does NOT disable it** — the empty string is passed straight to `IntranetSearchProvider`, which falls back to its built-in default endpoint (`intranet_stub._DEFAULT_ENDPOINT`), exactly like the legacy `get_search_provider(search_provider=intranet)` path. So 内网直连检索 works out of the box without configuring an address (configure `intranet_search_url` only to point at a different endpoint). This is the "弱模型识别不了技能" escape hatch — the old no-skill retrieval path restored as an explicit option (weak models like deepseek-chat often fail to invoke configured skills). Tests: `tests/test_ai_writing_knowledge_base.py`. General retrieval is orchestrated from the **skills configured on the `ai-writing-researcher` agent** (`config.yaml → skills`, editable from the agent management page): the built-in `knowledge-base-search` skill (知识库检索) runs the native keyword fan-out web/intranet search (the old default path), while any other configured skill searches with a `description`-targeted query; skills run in configured order with early-stop at `max_materials`, and an empty/unresolvable skills config falls back to the built-in knowledge search so retrieval always works. The skill itself is seeded from `_ai_writing_seed_assets/skills/knowledge-base-search/SKILL.md` into `skills/public/` (never overwrites), and a pre-existing researcher `config.yaml` lacking the `skills` key is patched in place with the default (explicit `skills: []` is respected). **PAUSE-1 input append flow**: when the user types natural language into the material-confirm input (`re_search`/legacy `enable_skill_search` + `user_query`), the node matches the most relevant configured skill via `SkillPicker(candidates=…)` (skipped when only one skill is configured), searches with the user's words, and **appends** the new materials to the existing package (URL-dedup, original order preserved, no LLM re-rank/truncation, 100-item hard cap). **Skill-failure fault tolerance**: a configured skill that times out / raises / has all its keyword searches fail (vs. legitimately returning 0 results) no longer just silently moves on — the researcher node emits a `skill_search_step` of type `skill_error` (carrying the error `reason`), and if the configured skills collected **no** new materials at all it runs a `fallback` pass over other available skills (`_fallback_candidate_skills`: the built-in `knowledge-base-search` native path first, then other enabled catalog skills not yet tried) instead of erroring out. The frontend `SkillStepsCard` renders `skill_error` (rose) and `fallback` (amber) so the timeline shows "技能报错 → 改用其他技能搜集". Tests: `tests/test_ai_writing_material_source.py`.

The intranet client uses `trust_env=False` and defaults `intranet_verify_ssl=false`, so internal HTTPS endpoints with self-signed/internal-CA certificates do not fail on certificate verification; set `ai_writing.intranet_verify_ssl: true` only for environments that require strict certificate checks.

**Skill-call timeout** is hardcoded **300s (5 min)** — `researcher.py` `_SKILL_SEARCH_TIMEOUT_SECONDS` (per-skill wrapper) and `_skill_agent.py` `_REAL_AGENT_TIMEOUT_SECONDS` (whole real-agent skill run). The intranet ES fallback (`IntranetSearchProvider`) gets a longer **600s (10 min)** default (`intranet_stub.py`; overridable via `ai_writing.intranet_search_timeout`) — that interface is the slow one, so our own fallback timeout is generous to avoid being blamed for "检不出".

**Parallel intranet fallback (技能 0 条 → 直接用并行预跑的内网结果).** When `ai_writing.intranet_search_url` is configured, the researcher node launches the intranet ES search (`IntranetSearchProvider` over the generated keywords) as a **concurrent `asyncio.Task` the moment the skill loop starts** (not sequentially after all skills fail). If the configured + fallback skills collect **nothing**, the node awaits that already-running task and uses its results (`_collect`) — no extra wait. If the skills **did** get materials, the task is cancelled (`_cancel_intranet_task`, suppresses CancelledError) so the intranet call isn't wasted. URL empty → no task, behavior unchanged. Tests: `tests/test_ai_writing_intranet_parallel.py`.

**Partial-result harvest + query simplification (检不出/超时的容错).** When `researcher_skill_agent: true` runs a non-native skill via `run_skill_via_real_agent` (the deployment's real path — the skill, e.g. `knowledge-search-v2`, is invoked by the real agent through `bash python3 scripts/search.py "<query>"`), one writing run issues many queries and **some time out** (the skill script prints `{"success":false,"error":"请求超时…","count":0}`) while others succeed (`{"success":true,"count":30,"results":[…]}`). Two fixes so successful data is never dropped: (1) `_drive()` captures every **ToolMessage** stdout and `_harvest_tool_results()` parses the skill's JSON **by reusing the lead-agent 参考文献 mode's proven extractor** (`deerflow.runtime.references`: `_parse_json_like` + `_first_list` + `_normalize_reference_item`) — the same logic that already extracts skill JSON into citations 100% reliably on ChatPage/AgentChatPage, with its huge field-name union (title/m_title/name, content/page_content/content_preview/snippet/…, recUuid/uuid/docId/…) and per-skill display-config support — plus skips `success:false` outputs — so even if the model's final `{materials:[]}` summary gives up (confused by the timeouts), the successful queries' real results are still collected; the function merges model materials + harvested results (dedup by url/recUuid/title, capped at `max_results`). **The harvest+merge (`_finalize`) runs on EVERY exit path — normal end, overall timeout, AND `await task` exceptions (e.g. `GraphRecursionError`)** — so when the run is cut off before the model writes its summary (observed: 10 queries × ~2 graph steps > the old `recursion_limit`, so the agent errored out with the real 30+ results already in `tool_outputs`), those results are still salvaged instead of returning `[]`. `recursion_limit` was also raised from `_REAL_AGENT_MAX_TURNS` to `_REAL_AGENT_MAX_TURNS*2+4` (it counts graph super-steps ≈ 2 per model→tool round, not model turns) so the model normally has room to finish its summary. **Early-stop on `max_results`:** `_drive` re-harvests after every tool output and, once the deduped harvested count reaches `max_results` (= `per_skill` = `max_materials`), **breaks the nested `astream`** and emits a 「已获取 N 条素材，达到上限，停止本技能检索」 step — without it a weak model keeps issuing new queries forever (observed: 100+ step-bar entries, materials far over the limit). Because the skill now returns exactly `max_materials`, the researcher loop's own `collect_limit` early-stop fires right after, so later configured skills don't run either. (2) The model is instructed (in `_MATERIAL_OUTPUT_CONTRACT`) that a 0-result/timeout query must be **retried with a simpler, core query** (drop topic suffixes like 竞选/诉讼/政策, keep the core entity: 「特朗普竞选」→「特朗普」) and that any data already retrieved must always be kept. The same deterministic simplifier `search/base.py::simplify_query` is wired into `IntranetSearchProvider.search`: an empty first attempt retries once with the simplified query. **(3) Root cause of the instability — bash output middle-truncation.** The skill prints its results as JSON to stdout, and the `bash` tool **middle-truncates** any output over `sandbox.bash_output_max_chars` (default **20000**, inserting a `... [middle truncated …] ...` marker). A `count:30` skill response (~30 records × ~600 chars) exceeds 20000 → the JSON is cut in the middle → **unparseable as a whole** by both the model and `_try_load_json` → all materials lost (while a small `count:9` response parses fine — hence the flapping). Fix: when whole-JSON parse yields no items, `_harvest_tool_results` falls back to `_scan_balanced_json_objects` — a brace-depth/string-aware scanner that recovers **every individually-complete `{…}` record object** from the truncated text (the records on each side of the truncation marker are still valid JSON), so ~16+16 records survive a 20000-char cut, far above `max_materials`. This is why the lead-agent chat pages (`/page/workspace/chats/new`, agent chat) call the same skill with 100% success — they only need the model to summarize prose from whatever it sees, never to parse a structured list, so truncation doesn't hurt them; ai-writing needs the structured records, so it must salvage them itself. Tests: `tests/test_ai_writing_skill_real_agent.py`, `tests/test_ai_writing_query_simplify.py`.

**Agentic skill execution (`ai_writing.researcher_skill_agent`).** When this flag is on (the deployment `config.yaml` enables it), the directed-skill branch runs `nodes/_skill_agent.py::run_skill_via_real_agent`: it reuses `SubagentExecutor` with the full tool stack, sandbox middleware, and a one-skill whitelist so the skill can `read_file` its `SKILL.md`, call `bash`, MCP tools, or whatever the skill instructions require. The nested run now passes `agent_id=ai-writing-researcher` in both `config.configurable` and runtime `context`, matching normal AgentChat skill visibility/whitelist behavior. It also appends a human-turn hard directive listing the exact SKILL.md path and skill directory, requiring weak models to first `read_file` the SKILL.md and then `cd <skill-dir> && python/python3 scripts/*.py ...` when the skill is script-backed; the model is explicitly forbidden from returning `materials` before executing the skill's required tools. If the real-agent path fails or yields no materials, `_execute_skill` still falls back to the legacy description→query search so materials are never empty. The built-in `knowledge-base-search` native concurrent path is unchanged. Tests: `tests/test_ai_writing_skill_real_agent.py`.

**Targeted user revision (局部修改 — 只改命中章节).** When the user submits a free-text 修改意见 on the draft-confirm card (`action == "user_revise"`), `writer_draft_node` no longer always rewrites the whole article section-by-section. Before the full-rewrite loop it tries `_targeted_user_revision` (`nodes/writer_draft.py`): `_split_draft_sections` parses the previous `current_draft.full_markdown` into `(article_title, [(section_title, body)], references_block)` (level-1 `#` title, level-2 `##` sections, trailing `## 参考文献` pulled out and kept verbatim), then `_detect_revision_scope` runs a small `llm_json` classifier (prompts `DRAFT_REVISE_SCOPE_{SYSTEM,USER}`) returning **per-section targets** `[{index, new_title, rewrite_body}]`: it maps the note to the section indices it targets ("第一段/开头/引言"→0, "结尾/结论"→last, by title/topic) **and** distinguishes a **章节标题改名** (`new_title` set, `rewrite_body=false` → rename only, body untouched) from a **正文修改** (`rewrite_body=true`). Title-only renames apply directly to the assembled `## {new_title}` heading **and are written back into `current_outline.sections[i].section_title`** (so the renamed title persists through confirm/finalize/history — this fixes "改章节标题不生效"). Body rewrites run `_stream_revise_section` (prompt `DRAFT_REVISE_SECTION_USER`, streamed as `draft_chunk`). **Only hit sections are touched; unhit sections are preserved byte-for-byte and are NOT re-emitted to the stream** — the frontend clears the right canvas to its initial placeholder on entering `revising` and shows only the live rewrite of the changed section(s), so the user sees「清空 → 流式重写 → 终稿」rather than the whole article rebuilding. The classifier returns `whole_article=true` (整体性意见 like 「整体语气更正式」/「全文都改」, **structure changes** 加章/删章/调整顺序, or a **文章总标题** change) → `_detect_revision_scope` yields `None` → `_targeted_user_revision` returns `None` → the node **falls back to the whole-article rewrite** (which still runs `_revise_outline_with_notes` for title/structure changes). Also falls back when there is no prior draft, an empty note, or scope-detection errors. Editor-review revisions (`accept_review`) keep the holistic rewrite. Reassembly preserves the article title + structure and re-appends the original references block; word count is computed on the body (excluding references). Frontend: `isRevising` in `AIWritingState` (`types/ai-writing.ts`) is set in `useAIWritingStream.ts`'s `intervention_submitted` (`ns === 'revising'`) and cleared on `draft_ready`; `AIWritingDraftPanel.tsx` shows the initial `DraftWaitingPlaceholder` while `isRevising && streamingDraftSections.length === 0`, then the streaming preview once chunks arrive. Tests: `tests/test_ai_writing_targeted_revision.py`.

**History draft editing (历史回看可编辑成稿).** The right-side draft in 历史回看 is no longer read-only — the user can edit it like the live 写作完成态 (Tiptap + 目录 + 格式工具栏 + AI 润色) and **persist** the change back to the session. `PUT /api/ai-writing/sessions/{id}/draft` `{draft_markdown, draft_title?}` (owner-checked → 404 otherwise; `draft_title` omitted ⇒ keep原标题) updates the row via the existing `AIWritingSessionRepository.update`. Frontend: `AIWritingDraftPanel` now passes `readOnly={false}` + an `onSaveDraft` handler (only in history mode) → `TextArtifactRenderer` → `WritingToolbar` 「保存」button (loading + toast); the handler is `AIWritingHistoryContext.saveHistoryDraft(markdown, title?)` → `updateAIWritingDraft` (`api/ai-writing-sessions.ts`), which write-backs the returned summary to `historySession`. Edits to the **live** completed draft stay ephemeral (export-only) as before — only history mode shows 保存. Tests: `tests/test_ai_writing_draft_update.py`.

**Background suspend (后台挂起).** A writing session can be handed off to the server to auto-drive the remaining flow to finalize, surviving page leave / refresh / account switch. `POST /api/ai-writing/sessions/{id}/background` (owner-checked, idempotent) starts `AIWritingJobExecutor` (`app/gateway/ai_writing_job_executor.py`), wired in `deps.py` as `app.state.ai_writing_job_executor`. The executor **drives the same `ai_writing` graph in-process** — it builds `make_ai_writing_graph()`, attaches the shared `app.state.checkpointer` (the very saver the runtime worker uses, so it continues the real thread state the frontend already wrote), sets the owning user via `set_current_user`, and loops `aget_state` → `ainvoke(Command(resume=…))` until `status==done`. **Auto-action per interrupt**: material/outline confirm → `confirm`; section_help → every blocked section `loose` (continue with general knowledge, never re-loop to research); draft_confirm → `to_editor` until reviewed, then `finalize` (on pass, or when `revision_count >= max_revisions`); review_confirm → `accept_review` (revise & re-review until pass or budget exhausted, then forced finalize). To avoid racing a still-running runtime worker (suspend mid-node), the driver **waits for an actionable state** (`_wait_for_actionable`): it only acts when the graph is at an interrupt / done, or when the checkpoint stops advancing (no worker → abandoned/restart). No new table — the graph checkpoint + `ai_writing_sessions` already hold all state; `status="background"` marks 挂起中 (surfaced in the history dropdown), nodes write `draft`/`review_result`/`status=done` as usual, so the user reopens the finished article from history. Per-user concurrency cap `AI_WRITING_BACKGROUND_SLOTS_PER_USER` (default 2, same SQLite-lock rationale as the roundtable executor). Startup `reconcile` (in `deps.py`) relaunches drivers for sessions left `background` after a restart (`AIWritingSessionRepository.list_by_status`). **Progress flowchart popup**: `GET /api/ai-writing/sessions/{id}/progress` (owner-checked) reads the graph checkpoint via `executor.get_progress` and derives a canonical pipeline `stage` key (`intent/research/material_confirm/outline/outline_confirm/draft/draft_confirm/review/done`, from the pending interrupt's `pause_point` or `snapshot.next` node) + `note` (素材不足求助 / 编辑打回) + review progress + `isRunning`. Frontend: 「后台挂起」 button + auto-suspend on route unmount / `beforeunload` (`AIWritingPanel.tsx`, `suspendAIWritingToBackground[Keepalive]` in `api/ai-writing-sessions.ts`); 挂起中 badge in `AIWritingHistoryDropdown.tsx`, and **clicking a 挂起中 history record opens a flowchart dialog** (`AIWritingFlowDialog.tsx` polls `getAIWritingProgress` every 2.5s, renders the fixed vertical pipeline `AIWritingFlowChart.tsx` lit by the live `stage`) so the user sees how far the background run has gotten without reopening/taking over the session. **Transcript consistency**: the rich history timeline (`AIWritingHistoryView` → `AIWritingTimeline`) replays `transcript = {progressEvents, completedInterventions}`, normally assembled+saved client-side. A background run has no client, so the executor rebuilds it at the end (`_save_transcript`/`_build_transcript_from_checkpoint`) **entirely from the final checkpoint** — both `progressEvents` and `completedInterventions` are derived from the same set of `resumed` events (every pause-resume emits exactly one), so they are always 1:1 aligned. For each `resumed`: classify its message → `(pausePoint, action)` (`_classify_resumed`, matched against `graph.py`'s resume messages), insert an `await_user` (with the pause prompt) before it, and build a **rich** intervention card enriched from final state (material_confirm→`materials`, outline_confirm→`outline`, draft_confirm→`draftTitle`/`draftWordCount`/`reviewResult`, review_confirm→`reviewResult`); `materials_ready` is inserted before the first material-confirm. (Earlier code merged the frontend's pre-suspend `completedInterventions` with auto ones and aligned by index — but the frontend's last save can predate its most recent decision, so the counts drifted and cards rendered off-by-one with empty data; deriving everything from the checkpoint root-fixes both.) Raw (snake_case) events **and** intervention data are normalized on load by the **exported** `normalizeProgressDict` / `normalize{Material,Outline,ReviewResult}` (idempotent on already-camelCase live transcripts — one normalize path for both sources), so reopening a background-completed session shows the same cards as a frontend-driven one. Tests: `tests/test_ai_writing_background.py`.

### Admin Leaderboard Snapshots (`/api/admin/users/leaderboard*`)

The admin ????? is served from **precomputed per-day snapshots** in `admin_leaderboard_daily_stats` (keyed by `(stat_date, settings_hash)`), not by aggregating raw rows on every request. A background scheduler inside the Gateway (`_leaderboard_snapshot_scheduler_loop` in `app/gateway/app.py`, singleton file-locked) ticks every `ADMIN_LEADERBOARD_BACKGROUND_INTERVAL_SECONDS` (default 300s): it prewarms recent days, then builds any `queued`/`error`/stale-`running` day. `GET /leaderboard` merges the per-day snapshots for the requested range. **Today** is rebuilt whenever its snapshot is older than `ADMIN_LEADERBOARD_TODAY_TTL_SECONDS` (default 43200s = half a day, so the leaderboard refreshes ~twice daily); a past day, once `ready`, is treated as fresh forever.

Two on-demand/automatic refresh paths layer on top:
- **Manual "????"** ? `POST /leaderboard/refresh-today` force-rebuilds *today's* snapshot **synchronously** (bounded by the same semaphore/timeout the scheduler uses), then returns the merged range. Lets an admin pull the latest real-time records into the stats without waiting for the TTL. Frontend: a "????" button on `AdminUserLeaderboardPage` ? `useRefreshTodayLeaderboard` (`core/admin/hooks.ts`), which primes + invalidates the `["admin-activity","leaderboard"]` query.
- **Daily finalize of the previous day** ? once per day, on/after `ADMIN_LEADERBOARD_FINALIZE_HOUR` (local Beijing, default `1` = ?? 1 ?), the scheduler tick force-recomputes *yesterday* a single time so runs/tool-calls that only landed near midnight are fully captured. The decision is the pure `deerflow.persistence.admin_stats.finalize.should_finalize_previous_day` (status/generated_at/now/hour) ? the rebuild stamps `generated_at` to today, which self-debounces it for the rest of the day (no extra state file). Tests: `tests/test_leaderboard_finalize.py`.

### Admin Concurrency Monitor (实时并发监控 / 系统压力)

A live admin monitor embedded **at the top of `AdminUserLeaderboardPage`** (`components/workspace/admin/concurrency-monitor.tsx`, `ConcurrencyMonitorPanel`) that answers "当前有几个并发的大模型调用、是谁、能不能直接停掉" and charts system pressure over time. Built on the existing in-memory `RunManager` (no new run plumbing).

- **Live concurrency + 真正停止** — `RunManager.list_active()` returns every run whose status is `pending`/`running` (the registry keeps finished records around until `cleanup`, so it filters by status). `len(list_active())` is the current concurrent-call count. Router `app/gateway/routers/admin_active_runs.py` (`/api/admin/active-runs`, **admin only** via the shared `admin_users._require_admin`): `GET /` returns `{count, server_time, runs:[{run_id, thread_id, status, created_at, elapsed_seconds, user_id, user_name, agent_id, agent_name, thread_title, kind, multitask_strategy}]}` — each run best-effort enriched (batched DB) with the owning user's email (`threads_meta.user_id`→`users.email`) and the agent's display name (`agents.name`); `kind` is derived from thread metadata (圆桌会商 / 系统/后台 / 任务对话 / 对话). `POST /{run_id}/cancel` calls `RunManager.cancel(run_id, action="interrupt")`, which sets the abort event **and cancels the run's asyncio task** — i.e. it truly aborts the in-flight model stream, freeing capacity. `POST /cancel-all` stops every in-flight run.
- **并发量折线图 (按分钟/小时)** — a lightweight background **sampler** (`_concurrency_sampler_loop` in `app/gateway/app.py`, singleton file-locked, started in the lifespan) records `len(list_active())` every `CONCURRENCY_SAMPLE_INTERVAL_SECONDS` (default 15s) into the new `concurrency_samples` table (`deerflow.persistence.concurrency`, `ConcurrencySampleRow{sampled_at, active_count}`, auto-created by `create_all`, **no migration**; store wired as `app.state.concurrency_sample_store`). ~Hourly it purges rows older than `CONCURRENCY_SAMPLE_RETENTION_HOURS` (default 168h). `GET /concurrency?granularity=minute|hour&hours=N` aggregates samples into per-bucket **peak + avg** (`ConcurrencySampleStore.aggregate`, bucketed in Python so it's identical across sqlite/mysql/postgres; fetch capped at 60k rows). Default lookback: minute→3h, hour→48h.
- **大模型出字速度折线图 (tokens/sec, 按分钟/小时)** — `GET /token-speed?granularity=&hours=` aggregates the persisted `llm_call_metrics` (`status="success"`) into per-bucket avg/min/max tokens/sec (deriving tokens/sec from `output_tokens÷duration_ms` when the stored value is null). A dip pinpoints the model/provider as the bottleneck for that window — "知道具体原因出现在哪里".
- **On/off + 不影响线上** — the whole feature is designed to never impact production: sampling is one tiny INSERT, retention-purged, all paths best-effort/guarded, queries bounded. A **runtime switch** lives in `system_settings.json → concurrency_monitor.enabled` (`ConcurrencyMonitorSettings`, **default false / opt-in**, mtime-hot-reloaded): the background loop always runs (just sleeps) but only samples when an admin turns it on, and it re-reads the flag **every tick** so toggling it on/off **takes effect immediately without a restart** (the on-demand live snapshot + 停止 stay available regardless; only the historical chart needs sampling on). `GET|PUT /api/admin/active-runs/settings` reads/writes it (admin only); the panel's 「后台采样」 Switch drives it. Env `CONCURRENCY_SAMPLE_ENABLED=0` is a hard master kill (never starts the loop). Frontend client/hooks: `core/admin/active-runs.ts` + `core/admin/hooks.ts` (`useActiveRuns` polls 4s while the panel is open, `useConcurrencySeries`/`useTokenSpeedSeries` poll 15s, `useCancelActiveRun`/`useCancelAllActiveRuns`, `useMonitorSettings`/`useUpdateMonitorSettings`). Tests: `tests/test_concurrency_monitor.py`.

### Scheduled Task Runtime

`ScheduledTaskService` (`deerflow/runtime/scheduler/service.py`) polls due tasks every `poll_interval_seconds` and executes each in its own throwaway thread, serialized by a shared `_agent_run_lock`.

**Stuck-run recovery**: each execution attempt is bounded by `_RUN_ATTEMPT_TIMEOUT_SECONDS` (30 min) via `asyncio.wait_for`. On timeout the run is retried up to `_MAX_RUN_RETRIES` (3) times, then marked `failed`. A watchdog (`_reconcile_stuck_runs`, run every poll) recovers runs left in `running` with no in-process owner ? e.g. orphaned by a crash or restart ? re-driving them through the same retry budget so a task can never stay perpetually "executing". The `scheduled_task_runs.retry_count` column tracks consumed retries.

**Unique task names**: task names are unique per user (`uq_scheduled_tasks_user_name`). `create_task`/`update_task`/`admin_update_task` proactively check via `_assert_name_available()` and raise `DuplicateTaskNameError` on a collision; the router translates that into HTTP `409` instead of letting the raw DB `IntegrityError` surface as a `500`.

**HTML page tasks (`task_kind == "html_page"`)**: a second task kind that renders a full HTML document each run instead of a Markdown report. The kind + its config (`html_config`: `page_prompt`, `negative_prompt`, `reference_favorite_id`, `model_name`) live inside `execution_context` ? no schema change to `scheduled_tasks`. Data gathering is delegated to the agent's own search skills (driven by `page_prompt`), so there is no explicit search-query/data-source field; the render model is user-selectable (`model_name`, default `zai-org/GLM-5-FP8`). `_run_task_attempt()` dispatches by kind; `_run_html_page_attempt()` runs the pipeline: gather real data via the lead agent (its search tools) ? optionally load a reference favorite's reusable style/layout ? render with the chosen model (thinking off) ? fault-tolerant JSON parse ? sanitize (BeautifulSoup, strips `<script>`/`<iframe>`/`on*`/`javascript:`) ? structural validate ? one GLM5 repair pass if invalid ? **completeness gate**: if the HTML is still not a full valid document (missing doctype/html/head/body, empty body, or residual scripts) after the repair pass, raise `HtmlGenerationError` so the run is marked **failed** (a page only counts as successful when it is a complete, previewable HTML document) ? best-effort Playwright screenshot (no-op when Playwright absent) ? archive `index.html` ? persist a `scheduled_html_pages` row ? return `result` with `format: "html"` and the full HTML inline. Pure helpers live in `deerflow/runtime/scheduler/html_page.py`. The frontend previews the HTML in a sandboxed `iframe` (no `allow-scripts`) on the run-detail and public pages; a user can bookmark a page (style summary extracted via GLM5) and later pick it as a style reference. New tables `scheduled_html_pages` + `html_page_favorites` (`deerflow/persistence/html_pages/`, wired as `app.state.html_page_store`) are created by `Base.metadata.create_all` with no migration. Tests: `tests/test_html_page_tasks.py`.

**Unified output switches (`require_html` + `require_markdown`).** The task form no longer splits "???? / HTML ??" into separate kinds. `execution_context` now carries two **independent** booleans that may both be on: `require_html` (???? HTML) and `require_markdown` (???? Markdown). `_task_requires_html()` returns true for `require_html` **or** legacy `task_kind == "html_page"` (so old tasks need no data migration), and `_run_task_attempt()` dispatches to `_run_html_page_attempt` when it is true, else the regular agent attempt. When **both** switches are on, `_gather_html_data()` returns `(text, files)` ? the gather agent run is additionally told to save a `.md` file ? and `_run_html_page_attempt` validates it (`MarkdownRequirementError` on failure) and surfaces it alongside the generated page, so the run delivers both artifacts. The single main `prompt` doubles as the HTML page requirement (`html_config.page_prompt`); there is no separate page-prompt field. Tests: `tests/test_scheduled_task_require_html.py` + the markdown-coexistence cases in `tests/test_html_page_tasks.py`.

**Template-fill mode (JSON → 固定 HTML 模板填充).** A third execution mode, distinct from the free-form `require_html` render pipeline: convert a user-supplied JSON into a **fixed** HTML template's data structure and fill it. Triggered when `execution_context.template_config.template_id` is set (chosen in the task form after selecting the built-in **「模板数据转换器」** agent `template-json-builder`). `_task_template_id()` reads it and `_run_task_attempt()` dispatches to `_run_template_fill_attempt` **before** the `require_html` check. Pipeline: take the source (the task `prompt`, pasted or uploaded .json) — which may be **pure JSON, plain text, or text-with-embedded-JSON** (`template_fill.extract_embedded_json` pre-extracts any embedded `{…}`/`[…]` with a string-aware balanced scan and surfaces it as a labeled block while keeping the full original text; plain text → no block, the model reads it semantically) → load the selected agent's `SOUL.md`+model and run a **direct** `_invoke_conversion_model` call (system=SOUL, human=conversion prompt carrying the template's compact **schema skeleton** + the recognized-JSON block + source) → `extract_json_object` → `template_fill.validate_data` (top-level keys + container types) → **on failure feed the structure errors + `field_coverage_gaps` back and retry up to `_MAX_TEMPLATE_FILL_RETRIES`=2** → `template_fill.fill_template` replaces the template's `const DATA={…}` line (and refreshes `MANIFEST.系统.更新时间`) → archive `index.html` + persist a `scheduled_html_pages` row → return `format:"html"` (reusing the existing HTML preview at `/runs/{id}/html`). Any terminal failure raises `TemplateFillError` → run marked failed (no retry, message shown). **Pure engine** `deerflow/runtime/scheduler/template_fill.py` (harness, no `app.*`): the template registry (`TEMPLATES`: id=`no1` name=`no.1` 台湾滨海防卫大屏, and id=`no2` name=`no.2` 台湾飞行部队平台 with **`convertible=False`** — a 「只展示」 template that renders its existing HTML **as-is**, no model/validation/fill, until conversion is wired in later; `_run_template_fill_attempt` short-circuits to `read_template_text` for it, and `list_templates`/`is_convertible` expose the flag so the form hides the JSON input), `list_templates`/`reference_data`/`schema_skeleton_json`/`validate_data`/`field_coverage_gaps`/`fill_template` — **the template's own embedded `DATA` is the schema source of truth** (parsed at load, 1-sample-per-array skeleton fed to the LLM, top-level structure as the validation basis); template HTML files live under `runtime/scheduler/templates/<id>/`, add a `TemplateSpec` to register a no.2. The agent is seeded like the AI-writing ones: `app/gateway/routers/_template_builder_seed.py` + `_template_builder_seed_assets/template-json-builder/{config.yaml,SOUL.md}`, called from the lifespan before `_sync_legacy_agents` (which upserts it into the `agents` table as built-in/published so it shows in `/api/agents` → the task form's 执行智能体 dropdown). Template list API: `GET /api/scheduled-tasks/html-templates` → `{templates:[{id,name,description,convertible}]}` (open). **Debug-render API** `POST /api/scheduled-tasks/template-fill/render` `{template_id, text?|data?}` → `{ok, display_only?, errors, gaps, html}` (open): no model call — `extract_embedded_json(text)` (or `data`) → `validate_data` → `fill_template`, returning the filled HTML **plus** the validation errors/field-gaps (fills even when invalid so you can see the effect). Powers the chat-page debug button below; non-convertible templates return their raw HTML. **Templates as `require_html` 参考收藏页面 (use the REAL template, all sub-pages)**: the 参考收藏页面 list in the task form also offers the templates (value `tpl:<id>`, e.g. `tpl:no1`). These templates are **JS-driven multi-page SPAs** (sub-pages/tabs are rendered by script from `DATA`), so a free-form model render would only ever produce the main page and **drop every sub-page**. Therefore, when `_run_html_page_attempt` sees a `tpl:<id>` reference (`template_fill.reference_template_id` parses it; distinguishes a template marker from a real favorite id), it short-circuits **before** the free-form pipeline to `_run_html_template_reference`, which renders the **actual template file** (all sub-pages intact): convertible templates gather real data (`_gather_html_data`) → convert to the template `DATA` (`_convert_to_template_data`, shared with template-fill mode) → **`merge_with_reference`** (per top-level key = per sub-page: any sub-page/field with no extracted content falls back to the template's **original** data, so sub-pages never render blank) → `fill_template`; non-convertible templates (no.2) output the raw template as-is; if the task prompt **and** description are both empty, or gather/conversion yields nothing, it uses the template's original data outright. **数据抽取开关** `html_config.template_extract` (default **off**, shown in the form only when a `tpl:` reference is picked): it **only** controls whether the HTML uses **extracted** data vs the template's **default** data — it does **not** affect anything else. Off → HTML fills the template's default data (no convert); on → gather+convert+merge. **Markdown is independent**: when `require_markdown` is also on, the gather agent run still runs and the `.md` is produced/validated/surfaced **regardless** of the extract switch (so turning extraction off never blocks the Markdown). Display-only templates (no.2) never extract; they still honor `require_markdown`. `_run_html_template_reference` runs gather once when `do_extract` (extract on + has prompt/desc + convertible) **or** `require_markdown`, and threads any `.md` through `_finalize_template_html(extra_files=…)`. `merge_with_reference` is also applied in template-fill mode (`_run_template_fill_attempt`) so empty sub-pages there fall back too. Archive/persist/result are shared via `_finalize_template_html`. (The earlier `style_reference` CSS-extraction approach was removed — it lost the sub-pages.) Frontend: when the agent dropdown = `template-json-builder` the form hides the HTML/Markdown switches and shows a 「HTML模板」 select (`useHtmlTemplates`) + a .json upload (reads file text into the prompt box). Tests: `tests/test_template_fill.py`, `tests/test_template_fill_attempt.py`, `tests/test_template_builder_seed.py`.

### Per-user Prompts & UI Preferences

**Custom prompts can be scoped to one agent.** `user_custom_prompts` gains a nullable `agent_id` column (migration `20260603_02`): `NULL` = global (the chip shows in every chat), otherwise the prompt only surfaces when that agent is selected. The visibility filter is applied in the frontend `CustomPromptBar` (it receives the current agent id from `ChatPage`'s selector or `AgentChatPage`'s `assistantId`); the backend (`/api/user-prompts` CRUD) just stores/returns/clears `agent_id` (update accepts an explicit `null` to clear). Tests: `tests/test_user_prompts_agent_id.py`.

**Per-user UI preferences** (`deerflow/persistence/user_preferences/`, wired as `app.state.user_preferences_store`; migration `20260603_03`, also auto-created by `create_all`): one JSON-blob row per user so new toggles need no schema change. Router `/api/user-preferences` (`GET`/`PUT`, account-scoped). Current key: `hide_admin_prompt_prefix` ? when true the chat input hides the admin-configured prompt-prefix switch and never applies the prefix (the user uses only their own prompts; enforced in `ChatPage` for both `shouldApplyPrefix` and `promptPrefixProp`). Toggled from the "???????" dialog. Tests: `tests/test_user_preferences.py`.

**HTML skills (tools the render step uses).** Two offline skills under `skills/public/` shape the generated HTML and are toggled in the skills settings page (enabled by default): `html-page-builder` (a self-contained inline-CSS design system ? colors/cards/tables/KPI/responsive) and `html-charts` (inline-SVG charts that render in the no-`allow-scripts` preview iframe; optional local ECharts documented in its `assets/README.md`). A skill opts into HTML injection via `metadata.kind: html_page` in its SKILL.md frontmatter; `_collect_html_skill_instructions()` reads the bodies of **enabled** such skills and `build_generation_prompt(skill_instructions=?)` injects them, so the (raw-model) render step honors the user's skill selection. The data-gathering step and the render/repair/style passes all use the task's selected `model_name` (no silent fallback to the default model ? and there is **no hardcoded GLM-5**; an unset selection falls back to config `models[0]`). To stop auxiliary LLM calls from leaking onto the default model, **`TitleMiddleware` is skipped when `is_scheduled_run` is set** (`_build_middlewares` in `lead_agent/agent.py`) ? scheduled/HTML runs use throwaway threads whose titles are never shown, and `config.title.model_name` is usually unset (? would call `models[0]`, not the chosen model). The completeness gate (`validate_html`) also requires a non-empty `<title>` and forbids external stylesheets, on top of the doctype/html/head/body + non-empty-body + no-script checks.

### Knowledge Base (`packages/harness/deerflow/knowledge/`)

Self-contained product knowledge base. **All knowledge capability lives in this one module** (harness layer, `deerflow.*` only ? no `app.*` imports). Wired as `app.state.knowledge_service` in `deps.py` via `make_knowledge_service(sf, config.knowledge)` (returns `None` on the memory backend or when `knowledge.enabled=false` ? router 503). Surfaced by `app/gateway/routers/knowledge.py` (`/api/knowledge`). Config: `config.yaml` ? `knowledge` (`deerflow/config/knowledge_config.py`).

**Two stores, kept in sync on every write:**
- **DB mirror** ? tables `knowledge_notes` / `knowledge_sources` / `knowledge_tags` / `knowledge_note_tags` (`knowledge/models.py`, registered in `persistence/models/__init__.py`, auto-created by `Base.metadata.create_all`; portable types `PortableLongText`/`BeijingDateTime`). Powers listing/filter/search/audit. Global/shared ? no `user_id` column; `created_by`/`updated_by` are audit-only (resolved via `get_effective_user_id()`).
- **Markdown vault** ? obsidian-wiki compatible, default `{$DEER_FLOW_HOME}/data/knowledge/obsidian-vault`. `ObsidianWikiAdapter` (`obsidian_adapter.py`) writes wiki-capture frontmatter+body, and after **every** write syncs `index.md` / `log.md` (CAPTURE/INGEST/UPDATE/ARCHIVE/EXPORT) / `hot.md` / `.manifest.json` (source-hash dedup for `skip_existing`). `markdown_vault.py` handles frontmatter build/parse, slugging, wikilink extraction and **path-traversal-proof** resolution.

**Module layout:** `models.py` (ORM: notes/sources/tags/note_tags + embeddings/versions/entities/relations) ? `repository.py` (DB only, no LLM) ? `schemas.py` (NoteDraft/SourceDraft DTOs) ? `skill_templates.py` ? `markdown_vault.py` ? `manifest.py` ? `extractor.py` (rule-based fallback) ? `llm_extractor.py` (**default**: distills declarative knowledge per wiki-capture, structured JSON incl. entities/relations; falls back to `extractor.py`) ? `sources.py` ? `exporter.py` (`graph.json`/`graph.graphml`/`cypher.txt` + self-contained vanilla-JS `graph.html`; entity nodes + relation edges. **`graph.html` is theme-aware** ? it reads `?theme=light|dark` from its own URL since the embedding iframe can't inherit the app's CSS; `KnowledgeGraphPage.tsx` appends the active day/night mode and re-skins on toggle. The layout cools down and **freezes** so dense graphs stay clickable, node radius scales with degree, and hovering/clicking a node focuses it + its direct neighbours while dimming the rest + a detail panel ? taming heavy node/edge clutter) ? `chunking.py` + `embeddings.py` (OpenAI-compatible embedding client, keyword fallback) + `search.py` (keyword/vector/hybrid) ? `sensitive.py` (secret/PII redaction) ? `dedup.py` (Jaccard + hash dedup) ? `auto_ingest.py` (background queue/worker) ? `obsidian_adapter.py` ? `service.py` (`KnowledgeService` orchestrator ? the only thing the router touches; capture entry points `capture_thread` / `capture_search` / `capture_file` / `create_manual_note`, all converging on `_persist_draft`). Tests: `tests/test_knowledge.py` (20 cases). Frontend: `frontend-web/src/core/knowledge/*` + `pages/KnowledgeBasePage.tsx` / `KnowledgeGraphPage.tsx` + chat `knowledge-save-trigger.tsx` / `knowledge-rag-toggle.tsx`; spec at `frontend-web/docs/product-knowledge-base-dev.md`.

**Wikilinks & multi-note capture** (`knowledge.{related_notes_enabled,related_notes_limit,multi_note_enabled}`, all default-on; `project_name` defaults to `zncm`):
- **Navigable `[[wikilinks]]`** ? `service.resolve_wikilink(s)` maps a `[[target]]` to a knowledge note (by id ? `vault_path` ? title) or a vault meta page (`read_vault_page`, path-traversal-proof), else `missing`. `markdown_vault.normalize_wikilink_target` canonicalizes targets (drops `|alias`, slashes, `.md`). The frontend `MarkdownRenderer` renders non-numeric `[[...]]` as clickable links from the resolver map (numeric `[[123]]` stay citation badges); note links navigate, vault links open a `VaultPageDialog`, missing links are greyed.
- **Real related notes only (no structural meta-links)** ? `extractor.related_links()` now returns `[]`: notes no longer auto-link `index` / `_meta/taxonomy` / a project hub (they were noise in every `## Related`). On capture, `service._find_related_notes` cross-links **only by shared *distinctive* tags** (project/`knowledge-base` defaults excluded; free-text title/summary matching was removed because CJK tokenization linked unrelated notes via generic words like ??/????) and `_inject_related_notes` writes real `[[Title]]` links into `## Related` (plus note?note `related` graph edges); no distinctive tags ? no related notes. The renderers omit the `## Related` section entirely when there are no related notes and no sources. The vault's own `index.md`/`_meta/taxonomy.md` still exist (obsidian scaffold) and remain resolvable for manually-written links. The router enriches `GET /notes/{id}` with `created_by_name`/`updated_by_name` (auth-provider email lookup) so the UI shows a person, not a UUID. The frontend `MarkdownRenderer` renders wiki-capture provenance markers `^[inferred]`/`^[ambiguous]` as small "??"/"??" badges.
- **LLM-extraction outcomes** ? capture distinguishes three LLM outcomes: (1) **success** ? declarative note(s); (2) **deliberate rejection** ? the model returns `should_ingest:false` with no notes (greeting / meta chatter) ? `extract_thread_llm_multi` raises `LLMRejectedIngest` and `capture_thread` raises `EmptyThreadError` (skip ? nothing persisted, so worthless chats never enter the KB); (3) **genuine failure** ? API error / non-JSON ? returns `[]`, capture falls back to the rule-based verbatim extractor and lands the note as a **`draft`** (not `approved`), logs a WARNING, and returns `llm_fallback:true` (surfaced by the router). `llm_extractor` logs an actionable WARNING/INFO for each non-success path so extraction issues are diagnosable.
- **Multi-note split** ? `llm_extractor.extract_thread_llm_multi` lets one conversation distill into 1?4 cross-linked notes (`notes:[?]` JSON array; legacy single-object output still accepted). `capture_thread` persists each, sibling-cross-links them by title, and returns `{note, created, notes:[?]}`. Gated by `multi_note_enabled` (false ? first note only). Tests: `tests/test_knowledge_wikilinks.py`.

**Phases 2?4 (all config-gated, defaults safe):**
- **Phase 2 ? vector RAG**: `knowledge.embedding.{enabled,base_url,api_key,model,...}` enables an OpenAI-compatible embedding endpoint; notes are chunked + embedded on write (`knowledge_embeddings`), `POST /search` supports `mode=keyword|vector|hybrid` (vector degrades to keyword when unconfigured). `KnowledgeRagMiddleware` (lead-agent chain) injects a transient `<knowledge-context>` recall before the model call, gated **per-conversation** by the `knowledge_rag_enabled` runtime flag (frontend chat toggle) falling back to global `knowledge.rag_enabled`. `POST /reindex` rebuilds the index.
- **Phase 3 ? governance**: `sensitive.py` redacts secrets/PII before persistence (`sensitive_redaction_enabled`); every update snapshots `knowledge_note_versions` (`GET /notes/{id}/versions`); `GET /duplicates` + `POST /merge` for dedup; background `AutoIngestQueue` (wired in `deps.py`, started/stopped in the gateway lifespan) auto-sediments finished conversations **off the chat path** when `auto_ingest_enabled` ? enqueued from `KnowledgeRagMiddleware.after_agent`, value-judged by min-chars + `skip_existing`, landing as `draft`.
- **Phase 4 ? graph**: LLM extraction also yields entities/relations ? `knowledge_entities`/`knowledge_relations`; `exporter.build_graph` adds entity nodes + note?entity/entity?entity edges (reflected in `graph.html`/`graph.json`); `GET /graph` returns product graph data.

**User directories + editable extraction templates + display polish (knowledge UX overhaul):**
- **Custom directories (`knowledge_notes.folder` + `knowledge_folders`)** ? notes carry a free-form, slash-joined `folder` path (e.g. `投研/行业`) independent of the wiki-capture `category` and of `vault_path`. `knowledge_folders` is the manageable directory list (supports empty folders + rename/move that re-points descendant folders **and** filed notes). Router CRUD under `/api/knowledge/folders` (`GET`/`POST`/`PUT/{id}`/`DELETE/{id}?reassign_to=`); `GET /notes?folder=` filters a directory + its sub-folders (`folder=""` ? unfiled). Capture (`capture_thread`/`capture_file`/`create_manual_note`) and `PUT /notes/{id}` accept `folder` (the note PUT uses `model_fields_set` so an absent key leaves the folder untouched, a present `null` clears it). Frontend `buildKnowledgeTree(notes, folderPaths)` builds the left tree from `folder` (no longer from `vault_path`); folder management dialog + `FolderSelect` in the create/import dialogs and the note detail header.
- **Editable extraction prompts (`knowledge_extract_templates`)** ? the hard-coded distillation prompts are seeded as editable templates on startup (`KnowledgeService.seed_default_templates`, idempotent — `默认 · 对话提炼` / `默认 · 文档提炼`, each `is_default` for its scope). `scope ∈ {thread, document, both}`; capture resolves an explicit `template_id` → the scope default → the built-in constant, and passes it to `llm_extractor.extract_thread_llm_multi` / `extract_document_llm_multi` as `system_prompt` (used verbatim). Router CRUD under `/api/knowledge/extract-templates` (409 on duplicate name). Frontend: extraction-template management dialog + `TemplateSelect` in the batch/file import dialogs. Editing a template is the supported way to fix "提炼太精简" (loosen the prompt to keep more detail) and to add alternate extraction logics.
- **Single source list** ? `extractor._related_section_md` no longer emits a `### Sources` sub-section into the body; the note detail shows sources once via the structured `来源` block (`note.sources`). `## Related` stays pure wikilinks so the frontend extracts it cleanly into the related-docs module.
- **Chinese section headings (display-only)** ? the frontend `localizeHeadings` (wikiUtils) maps `Context/Finding / Decision/Reasoning/Implications/Related/Sources` to 中文 at render time (applied to the body fed to both `extractToc` and the viewer, so anchors stay consistent). No data migration — historical English-heading notes render in Chinese too.
- **Historical-data cleanup** ? `scripts/cleanup_knowledge_related.py` (one-off, idempotent, `--dry-run`/`--note-id`) strips legacy `## Related` structural links (vault scaffold: `index`/`_meta/taxonomy`/`projects/…`, any slash-bearing target) and the old `### Sources` sub-section from existing notes (DB `content_md` + vault files, via `KnowledgeService.update_note`).
- Migration `20260616_02` + idempotent `scripts/sql/column_additions_mysql.sql` cover `knowledge_notes.folder` (+ index); the two new tables are auto-created by `create_all` / `_ensure_orm_columns_sync` on startup. Tests: `tests/test_knowledge.py`.

### Custom Agent Persistence

Custom agent metadata lives in the `agents` ORM table (`deerflow.persistence.agents`). `agents.id` is the stable runtime id and the filesystem directory name under `.deer-flow/agents/{id}/`; `agents.name` is display-only, can contain Chinese characters, and is not unique. `user_id IS NULL` marks built-in agents, user-owned agents are visible only to their owner unless `published = true`, and update/delete operations require ownership. The runtime accepts `agent_id` in context/configurable, and non-default `assistant_id` values are treated as agent ids after visibility checks.

**Listing cache + pagination** ? to keep `GET /api/agents` fast, the gallery's per-card extras (skills + tool_groups) are denormalized into two cache tables instead of reading every agent's `config.yaml` on each request: `agent_skills` (relational, one row per (agent, skill) with `position`) and `agent_extras` (one row per agent: `tool_groups` JSON + `model` + `skills_synced_at`). `config.yaml` remains authoritative for the agent runtime; these tables are a read cache, write-through on create/update and backfilled from disk by `_sync_legacy_agents` (mtime-vs-`skills_synced_at` drift check). The list endpoint enriches via `AgentStore.get_extras_for([ids])` (two batched `IN` queries). Both tables are auto-created by `Base.metadata.create_all` (no migration needed). `GET /api/agents` supports **opt-in server-side pagination**: `scope` (`mine` = owned + favorited / `square` = built-in/published public), `page`, `page_size`, returning `{agents, total, page, page_size}`; omitting `scope`/`page` preserves the legacy "return everything" behavior. Name `search` matches the raw OR desensitized name (the frontend's static term?abbrev map is mirrored server-side in `deerflow/config/agent_name_desensitize.py`); `tag_ids` is an SQL OR-filter; `owner` keeps only agents published by that user id (the card's ??? ? built-in agents never match), powering the gallery's click-???-to-filter. `POST /api/agents/private-square/list` takes the same pagination params (incl. `owner`). Repository entry point: `AgentStore.list_paginated(...)`. Tests: `tests/test_agent_listing_cache.py::test_list_paginated_owner_filter_keeps_one_publisher`.

**Homepage selector (agent picker on the new-chat page)** ? single admin-curated list backed by `agents.featured_order` (nullable BIGINT). NULL = not featured. Only `published = true` agents can be featured (router enforces on writes); any published agent is eligible regardless of ownership. Cleared with `order: null`. The list rendered to end users is `WHERE featured_order IS NOT NULL` + visibility-filter, sorted by `featured_order ASC`. Admins reorder by sending the desired ids to `PUT /api/agents/featured/order` (assigns indices 0..N-1). BIGINT is used rather than INT so the management UI can seed initial values from `Date.now()` (overflows INT on MySQL).

### Publish Squares (????)

Admin-configurable, **access-control-free** categories that published agents/skills are filed under (distinct from the password-gated ????). The square list lives in `system_settings.json` ? `publish_squares` (`PublishSquaresSettings`: ordered `squares: [{id,name,description}]` + `default_square_id`); helpers `list_publish_squares` / `resolve_default_square_id` / `normalize_square_id` always yield a valid square (built-in "????" fallback when none configured). Admin CRUD: `GET/PUT /api/system-settings/publish-squares` (GET readable by any user ? the gallery sub-tabs + publish picker need it; PUT admin-only, auto-slugs missing ids and re-anchors a dangling default).

Each agent/skill carries a single `square_id` column (`AgentRow` / `SkillRow`, `String(64)` NOT NULL default `''`, migration `20260611_01`, idempotent SQL in `column_additions_mysql.sql`). One column = at most one square, so an item can never be "published twice". `''` = unassigned ? resolved to the default square at read/filter time. Writes normalize via `normalize_square_id`: agents on create/update (`square_id` field), skills on publish (`PUT /api/skills/custom/{name}/published` accepts optional `square_id`). Listing filters: `GET /api/agents?square=` and `GET /api/skills?square=` keep only that square's items; the **default** square additionally sweeps in legacy `''` rows (`square_is_default`). Tests: `tests/test_publish_squares.py`. Frontend: `core/system-settings` (`usePublishSquares`/`useUpdatePublishSquares`), admin `PublishSquaresSettingsPage` (embedded in the appearance/settings hub), square picker in the agent edit dialog + per-skill card, and a ??-tab square sub-filter in both galleries.

### Reference-mode Entry Switch

The 参考文献/RAG citation-mode composer entry is system-controlled through
`system_settings.json` → `citation_display.reference_mode_enabled` (default
`false`). `GET /api/system-settings/citation-display` remains readable so the
frontend can hide or show the entry consistently; `PUT` is admin-only and
partially updates `reference_mode_enabled` and/or `cleanup_words`. The normal
workspace input box, iframe chat toolbar, send context, right reference panel,
and appearance settings all use this flag, while already locked citation
threads and selected LLMWiki knowledge bases still render their source
documents in the reference panel. Tests:
`tests/test_prompt_prefix.py` and
`frontend-web/src/pages/iframe-chat-mode-buttons.test.ts`.

### Agent Module Tag Config (zzJC / zzXD 按类型聚合)

Backs the zzJC / zzXD agent-chat left sidebar (「按类型聚合智能体 + 聊天记录」). An admin picks which agent **tags** belong to a module; the module's left sidebar then shows only agents under those tags (empty/unconfigured = show all visible agents). Stored in `system_settings.json` → `agent_modules` (`AgentModuleSettings`: `modules: {module_key → [tag_id, ...]}`); pure helpers `get_module_tag_ids` / `set_module_tag_ids` (de-dupe + strip). Endpoints (in `app/gateway/routers/system_settings.py`): `GET /api/system-settings/agent-modules/{key}` (readable by any user — the sidebar needs it to filter), `PUT /api/system-settings/agent-modules/{key}` (admin only, full replacement of the tag-id list). Module keys are frontend-defined (`AGENT_MODULES` in `components/workspace/agents/agent-module-sidebar.tsx`, keyed by the module's landing agent id: `6bf`→`zzjc`, `10d69…`→`zzxd`). Tests: `tests/test_agent_modules.py`. Frontend: `core/system-settings` (`useAgentModuleConfig`/`useUpdateAgentModuleConfig`), admin `AgentModuleSettingsDialog` (gear button in the agent-chat header).

### Business-mapping authorization

`/api/business-mapping` remains globally readable. Updates to 3Q, 6BF, and 7BF require `system_role == "admin"`; the 8BF public business-chain mapping is intentionally different and can be updated only by the trusted authenticated account whose email prefix is `lqq`. Do not rely on a username supplied in an external login URL. The frontend follows the same email-prefix rule when deciding whether to render the 8BF configuration button, while the router (`app/gateway/routers/business_mapping.py`) remains the enforcement point. Tests: `tests/test_business_mapping.py`.

### Position-collaboration foundation

### Roundtable business-chain enable/disable

`roundtable_chains.enabled` (BOOLEAN NOT NULL DEFAULT 1, migration `20260819_02`)
is the per-chain 启停 switch. New and existing chains default to **enabled**.
Owner or admin can flip it via `PUT /api/roundtable-chains/{id}` `{enabled}`.
Disabled chains stay visible in the management list (with an 启用/停用 filter)
and still show `updated_at`; the frontend hides them from the picker / public
entry rail so they cannot be selected into a roundtable. Tests:
`tests/test_roundtable_chain_stages.py` (`test_create_defaults_enabled_then_toggle`).

### Roundtable business-chain human validation

The existing `/api/roundtable-chains` `seats` JSON also accepts the optional
boolean `human_validation` field on each seat (`ChainSeatSchema`). It is a
frontend Step 2 checkpoint, not a server-side approval task: after the marked
seat completes (or, in a DAG, after its stage completes), the client pauses and
uses the existing resume button or intervention composer to continue. No new
database column, migration, or validation card is required.

The position-collaboration workspace is a separate frontend route and must not
use the original roundtable coordinator or jobs. Its shared configuration
integration point is the existing `/api/roundtable-chains` store: each `seats` JSON item
may carry an optional `position_id` (validated by `ChainSeatSchema`, no schema
migration). The original roundtable only reads its established seat fields and
therefore remains unchanged. The separate `/api/position-roundtable` router
(also mounted at `/api/multi-agent/position-roundtable` for intranet gateway
compatibility)
persists task/intent/frozen-chain snapshots in `position_roundtable_sessions`
and direct-agent node indexes in `position_roundtable_nodes` (migration
`20260721_01`). Message history remains in the established LangGraph thread
store; a node completion unlocks its next chain stage and a later upstream
revision marks downstream records `stale` for refresh.
`POST /api/position-roundtable/sessions/{id}/nodes/{node_key}/reject` uses the
same cascade: its reason is intentionally optional, the selected node becomes
`rejected`, and downstream material becomes `stale` (or `locked` when no
material exists) with `invalidated_by` set. Clients must replace their complete
session snapshot from this response so the flow, delivery list, and active
node state stay consistent. A rejected/stale node returns to `done` only after
a fresh Markdown delivery is recorded.
For direct-agent handoff, the router resolves artifact paths only inside the
completed upstream node's own thread, then includes bounded UTF-8 excerpts for
text outputs; binary artifacts stay as metadata-only references. Because each
seat runs on an isolated LangGraph thread, a downstream `read_file` on the
upstream artifact path would fail with "File not found"; the handoff therefore
also **mirrors** every upstream `outputs/` deliverable into the current seat's
own `workspace/upstream/` directory (`_mirror_upstream_artifacts`, bytes from
the upstream sandbox file with the SQL recovery copy as fallback, idempotent
per turn via size comparison, capped at 8 files / 2 MB each / 8 MB total) and
rewrites each handoff artifact ref's `path` to the mirror (`upstream_path`
keeps the original; `mirrored_to_sandbox: true`), with a matching note in the
hidden instructions telling the agent it may `read_file` the full text. The
mirror deliberately stays outside `outputs/` so it never appears in the seat's
own delivery manifest.
`POST /sessions/{id}/archive` turns the session into a read-only historical
record: task/intent updates, node binding, hidden-context construction, and
turn completion are rejected, while read endpoints remain available.
Built-in summary and action-plan completion also preserves a visible report
answer when the model misses its final `write_file` call: the router writes
that exact answer as the required Markdown artifact in the owning thread's
`outputs/` directory, so users receive both chat text and a downloadable file.
**Per-turn outputs write cap (弱模型防重复产物)**: ordinary position-node chats
and the summary builtin send run-context `position_artifact_cap: 1` (whitelisted
in `services.merge_run_context_overrides`). `PositionArtifactCapMiddleware`
then keeps at most one successful `/mnt/user-data/outputs/` `write_file` **and**
one `present_files` per user turn — truncates excess tool calls in the same
model response (and syncs raw provider `additional_kwargs.tool_calls`), rejects
further writes/presents after an `OK` / `Successfully presented files` or an
earlier sibling in the same batch, ends the turn when only excess calls remain,
and reminds the model to present once (or wrap up if already presented).
**Action-plan does not send the flag** (it must write report `.md` +
`action-plan-subtasks.json`). Tests: `tests/test_position_artifact_cap.py`.

### 岗位 (Position) Subscription System

An admin-managed aggregation layer that decides, per user, which agents / skills
/ scheduled tasks / recommended questions they are subscribed to or can see.
**A user has at most one position** (`users.position_id`, 1:1); unassigned users
inherit the single `is_default` position. Note: the admin-configured prompts
shown to users ARE the **recommended questions** (`recommended_questions` table)
— there is no separate prompt-template store; the per-user `user_custom_prompts`
(personal 常用提示词 bar) is untouched by this system.

**Persistence** (`deerflow/persistence/positions/`, wired as
`app.state.position_store`; tables auto-created by `create_all`, migration
`20260616_01`):
- `positions` (`PositionRow`): `id`, `name` (unique), `description`,
  `is_default` (only one at a time — the store clears the flag on others).
- `position_tag_bindings` (`PositionTagBindingRow`): one row per
  `(position, resource_type, tag)`. **No binding for a resource_type = "use the
  default set".** `POSITION_RESOURCE_TYPES = (agent, skill, scheduled_task,
  recommended_question)` — the same set drives the tag types (`TAG_TYPES` was
  extended with `recommended_question`).

**Sync engine** (`deerflow/services/position_sync.py`, wired as
`app.state.position_sync`). `sync_user(user)` **materializes** three resource
types into the existing "我的" tables, idempotently:
- agents → `agent_favorites` rows with `origin='position'`
- **skills → `skill_favorites` rows with `origin='position'`** (table
  `SkillFavoriteRow`, migration `20260616_04`, auto-created by `create_all`)
- scheduled tasks → `scheduled_task_subscriptions` rows with `origin='position'`

Skills behave **exactly like agents**: a position only *adds* its tagged skills
to the member's 我的技能 — the browse-all 广场 is never restricted. (This replaced
the earlier query-time visibility filter, which wrongly shrank the square to the
bound skill.) `PositionSyncService.resolve_visible(user, type)` returns
`set | None` (None = unrestricted) and is now used **only** for the one remaining
query-time type, `recommended_question`.

Sync only ever adds/removes `origin='position'` rows, so a user's own
favorites/subscriptions are never clobbered. Default sets (no binding for a
type): agents = built-ins (`user_id IS NULL`); skills = empty (square stays
full, nothing added to 我的); scheduled tasks = empty; questions = **untagged
only** (默认岗位 / 无标签绑定 只看到未打标的题目；打标的题目只在绑定了对应标签的
岗位展示 — the router interprets `resolve_visible → None` as "show only untagged",
not "show everything").
`origin`/`position_id` columns were added to `agent_favorites` and
`scheduled_task_subscriptions` — see migration `20260616_01` +
`scripts/sql/column_additions_mysql.sql`. The running app also auto-adds these
via `_ensure_orm_columns_sync` on startup.

**Triggers**: assigning/removing members, changing tag bindings (re-syncs all
members), becoming the default position (`sync_default_members`), and login
(`auth.py::_sync_user_position`, best-effort so a brand-new user inheriting the
default gets their grants materialized).

**API** (`app/gateway/routers/positions.py`, `/api/positions`, **admin only**):
position CRUD; `GET/PUT /{id}/tag-bindings`; `GET/POST/DELETE /{id}/members`
(batch assign/unassign → triggers sync); `POST /{id}/resync`;
`GET /users` (all users + their position); `PUT /users/{user_id}` (set one
user's position). **`GET /me` is the one non-admin route** — it returns the
caller's own effective position (explicit assignment, else the default position
unassigned users inherit, else `null`); powers the top-bar 岗位 label
(`strategy-components/Header.tsx` user cluster, via `useMyPosition`). Related
changes: `GET /api/recommended-questions` is now
filtered by the caller's position (`?all=1` bypasses for admin management) and
admin can tag questions via `/api/tags/assignments/recommended_question/{id}`.
**Default-position rule**: tags are bulk-attached to **all** questions first,
then the visibility filter applies — a position binding recommended_question 标签
sees only those tagged questions; a 默认岗位 / 无绑定 (`resolve_visible → None`)
sees only the **untagged** questions (打标的题目不再泄漏到默认岗位). `?all=1`
(admin management) skips this filter and returns every question with its tags.
The list also accepts a repeatable `tag_ids`
OR-filter — so the 推荐问题配置 page can display per-question 标签 and search by
them (frontend `RecommendedQuestionsPage` + `TagSearchBar`). Tests:
`tests/test_recommended_questions_tags.py`.
`GET /api/skills` no longer restricts by position; each skill carries a
`favorited` flag (true = in the caller's 我的技能, e.g. position-granted) so the
skills page's 我的 tab includes owned ∪ position-granted skills while the 广场
stays complete. Frontend: `core/positions`
+ admin `PositionManagementPage` (侧边栏「综合管理 → 岗位管理」 + 右下角设置菜单).
Recommended questions stay rendered **above** the chat input (their original
position), just filtered by the caller's position. Tests:
`tests/test_positions.py`.

### Portable Column Types (`deerflow/persistence/types.py`)

Two `TypeDecorator`s keep ORM columns correct across the `sqlite` / `postgres` / `mysql` backends. **All ORM timestamp/JSON columns must use these, not the raw SQLAlchemy types.**

- **`PortableJSON`** ? native `JSON` on SQLite/Postgres; `TEXT` (with manual `json.dumps`/`loads`) on MySQL, where older servers lack JSON support.
- **`PortableLongText`** ? a large text column: maps to MySQL `LONGTEXT` (4 GB) and plain `TEXT` on SQLite/Postgres (already unbounded). Use for big JSON/markdown blobs that would overflow MySQL's 64 KB `TEXT` cap ? e.g. `ai_writing_sessions.transcript`.
- **`BeijingDateTime`** ? stores timestamps as Beijing (UTC+8) wall-clock time. The app records time with `datetime.now(UTC)`, but MySQL `DATETIME` / SQLite have no timezone and persist whatever wall clock the driver serializes ? so a raw `DateTime` column would store UTC and read 8 hours behind Beijing. `BeijingDateTime` converts UTC?Beijing on bind and re-attaches `+08:00` on result, so the database holds Beijing time while Python still gets timezone-aware datetimes (instant comparisons stay correct, including in WHERE clauses). PostgreSQL `timestamptz` stores the instant unambiguously, so values pass through unchanged there. Uses a fixed UTC+8 offset (China has no DST), so no `tzdata` dependency.

### Schema Migrations / Manual Column-Add SQL (`scripts/sql/`)

Gateway startup runs Alembic through `app.gateway.app._run_db_migrations` after ORM table creation. The migration graph must always have unique revision ids and exactly one head; `tests/test_migration_graph.py` enforces both. Historical duplicate revision `20260821_01` is repaired by the `20260821_03` / `20260821_04` fixed-question chain and merge revision `20260825_01`. Startup DDL has a bounded timeout and failures are logged without preventing the app from serving.

**Convention (required):** whenever code adds a field to an existing table (a new `mapped_column` on an ORM model), you must, in the same change:
1. Keep the matching alembic version file under `persistence/migrations/versions/` (guarded with `_has_table`/`_has_column`), and
2. Append the idempotent `ADD COLUMN` statement to **`scripts/sql/column_additions_mysql.sql`**, with a definition matching the ORM column exactly (type, length, nullability).

Operators upgrading an existing DB may run the whole file (`USE <db>;` then execute) as a recovery path; it is idempotent (existing columns are skipped via `information_schema.columns`). Index-only changes go in the sibling `scripts/sql/leaderboard_indexes_mysql.sql` (same idempotent pattern) and stay out of startup migrations because online index creation on large metric tables can delay service boot.

Leaderboard scheduled-run exclusion uses correlated `NOT EXISTS` against the
indexed `scheduled_task_runs.agent_run_id`; do not restore `NOT IN`. Legacy
scheduler detection for tool metrics is also a correlated existence check and
must not outer-join the complete `runs` table into metric aggregations. Keep the
created-at + dimension compound indexes in ORM metadata and the idempotent
MySQL index script synchronized.

### Sandbox System (`packages/harness/deerflow/sandbox/`)

**Interface**: Abstract `Sandbox` with `execute_command`, `read_file`, `write_file`, `list_dir`
**Provider Pattern**: `SandboxProvider` with `acquire`, `get`, `release` lifecycle
**Implementations**:
- `LocalSandboxProvider` - Singleton local filesystem execution with path mappings
- `AioSandboxProvider` (`packages/harness/deerflow/community/`) - Docker-based isolation

**Virtual Path System**:
- Agent sees: `/mnt/user-data/{workspace,uploads,outputs}`, `/mnt/skills`
- Physical: `backend/.deer-flow/users/{user_id}/threads/{thread_id}/user-data/...`, `deer-flow/skills/`
- Translation: `replace_virtual_path()` / `replace_virtual_paths_in_command()`
- Detection: `is_local_sandbox()` checks `sandbox_id == "local"`
- **`{user_id}` for path resolution = `resolve_path_user_id(thread_id)`, using the persisted creator for every tracked thread.** The route's regular owner check still authorizes access first; creator resolution only selects the stable physical directory (`threads_meta.user_id`) and avoids a lost request context falling back to `users/default`. This also preserves cross-account task-deeplink conversations (rwfx/3qfx/xdfx) and roundtable system threads. Positive creator mappings are cached process-wide and DB/memory/missing-row failures fall back to the current login. It is wired at the thread-path consumers including `ThreadDataMiddleware.before_agent`, `present_file_tool`, uploads, artifact serving, and embedded `client.get_artifact`. **Rewrite-history recovery is metadata-only and job-owner scoped** (`list_by_session(..., user_id=current_user)`), so its GET/list endpoints intentionally skip redundant thread-owner checks and return an empty history instead of a history-only 404; the start/commit endpoints retain strict thread write authorization. Tests: `tests/test_thread_paths_owner.py`, `tests/test_thread_paths_roundtable.py`.

- **Durable document-rewrite commit:** `document_rewrite_pipeline.atomic_write_text` writes through a deliberately short sibling temporary filename so long Windows artifact paths stay below the path-length limit, and guarantees the target `outputs` parent exists. If the sandbox recreates that directory in the narrow commit window, it repairs the directory and retries; the original file is otherwise only changed through `os.replace`.

- **Deep Research structural regeneration:** `POST /api/deep-research/sessions/{id}/report-variants` creates a distinct completed session from the source report and its selected evidence. It rebases both saved `[来源：source_id]` and legacy streamed `[[source:source_id]]` markers to cloned ids, and removes inherited markers with no selected clone. The frontend uses this endpoint only for “选择结构再生成”, then queues the durable writer with `operation=generate`; generation receives the confirmed outline and complete selected evidence set but never receives the inherited report as text to edit. The new report's completed card can subsequently start a separate in-place `AI 全文改写`. Requirement plans are deterministic and operation-specific, model reasoning is forwarded as live SSE before the first Markdown token, and generation prompts emit display-ready `[来源：id]` citations. Citation validation repairs only a unique near-complete `drs_src_*` prefix (provider-truncated suffix) and still rejects genuinely unknown ids. The report-only stream requests 8192 output tokens (except Codex Responses models, which reject token-limit kwargs) and rejects a malformed trailing citation instead of committing a provider-truncated report. Never restore the former second planning-model call unless its latency and streaming contract are redesigned together.

**Sandbox Tools** (in `packages/harness/deerflow/sandbox/tools.py`):
- `bash` - Execute commands with path translation and error handling
- `ls` - Directory listing (tree format, max 2 levels)
- `read_file` - Read file contents with optional line range
- `write_file` - Write/append to files, creates directories
- `str_replace` - Substring replacement (single or all occurrences); same-path serialization is scoped to `(sandbox.id, path)` so isolated sandboxes do not contend on identical virtual paths inside one process

### Subagent System (`packages/harness/deerflow/subagents/`)

**Built-in Agents**: `general-purpose` (all tools except `task`) and `bash` (command specialist)
**Execution**: Dual thread pool - `_scheduler_pool` (3 workers) + `_execution_pool` (3 workers)
**Concurrency**: `MAX_CONCURRENT_SUBAGENTS = 3` enforced by `SubagentLimitMiddleware` (truncates excess tool calls in `after_model`), 15-minute timeout
**Flow**: `task()` tool ? `SubagentExecutor` ? background thread ? poll 5s ? SSE events ? result
**Events**: `task_started`, `task_running`, `task_completed`/`task_failed`/`task_timed_out`

### Tool System (`packages/harness/deerflow/tools/`)

`get_available_tools(groups, include_mcp, model_name, subagent_enabled)` assembles:
1. **Config-defined tools** - Enabled entries are resolved from `config.yaml` via `resolve_variable()`; entries with `enabled: false` are omitted before import and are not registered with the agent
2. **MCP tools** - From enabled MCP servers (lazy initialized, cached with mtime invalidation)
3. **Built-in tools**:
   - `present_files` - Make output files visible to user (only `/mnt/user-data/outputs`)
   - `ask_clarification` - Request clarification (intercepted by ClarificationMiddleware ? interrupts)
   - `view_image` - Read image as base64 (added only if model supports vision)
4. **Subagent tool** (if enabled):
   - `task` - Delegate to subagent (description, prompt, subagent_type, max_turns)

**Community tools** (`packages/harness/deerflow/community/`):
- `tavily/` - Web search (5 results default) and web fetch (4KB limit)
- `jina_ai/` - Web fetch via Jina reader API with readability extraction
- `firecrawl/` - Web scraping via Firecrawl API
- `configurable_search/` - Config-driven `POST` JSON `web_search` replacement. It deep-copies the configured payload and overwrites only `query`, supports HTTP/HTTPS plus configurable `verify_ssl`, timeout and result limit, normalizes nullable `recUuid`/`content`/`title`/`url` fields, and rewrites a result URL through `result_url_template` only when `recUuid` is non-empty. The required response envelope is `{"results": [...]}`. Tests use `httpx.MockTransport` and never require the deployment endpoint.

**ACP agent tools**:
- `invoke_acp_agent` - Invokes external ACP-compatible agents from `config.yaml`
- ACP launchers must be real ACP adapters. The standard `codex` CLI is not ACP-compatible by itself; configure a wrapper such as `npx -y @zed-industries/codex-acp` or an installed `codex-acp` binary
- Missing ACP executables now return an actionable error message instead of a raw `[Errno 2]`
- Each ACP agent uses a per-thread workspace at `{base_dir}/users/{user_id}/threads/{thread_id}/acp-workspace/`. The workspace is accessible to the lead agent via the virtual path `/mnt/acp-workspace/` (read-only). In docker sandbox mode, the directory is volume-mounted into the container at `/mnt/acp-workspace` (read-only); in local sandbox mode, path translation is handled by `tools.py`
- `image_search/` - Image search via DuckDuckGo

### MCP System (`packages/harness/deerflow/mcp/`)

- Uses `langchain-mcp-adapters` `MultiServerMCPClient` for multi-server management
- **Lazy initialization**: Tools loaded on first use via `get_cached_mcp_tools()`
- **Cache invalidation**: Detects config file changes via mtime comparison
- **Transports**: stdio (command-based), SSE, HTTP
- **OAuth (HTTP/SSE)**: Supports token endpoint flows (`client_credentials`, `refresh_token`) with automatic token refresh + Authorization header injection
- **Runtime updates**: Gateway API saves to extensions_config.json; LangGraph detects via mtime

### Skills System (`packages/harness/deerflow/skills/`)

- **Location**: `deer-flow/skills/{public,custom}/`
- **Format**: Directory with `SKILL.md` (YAML frontmatter: name, description, license, allowed-tools)
- **Loading**: `load_skills()` recursively scans `skills/{public,custom}` for `SKILL.md`, parses metadata, and reads enabled state from extensions_config.json. **A directory containing `SKILL.md` is a self-contained leaf** — discovery does not descend into its subdirectories, so a skill *package* that bundles its own `examples/`/`references/` sub-skills (each with a SKILL.md) surfaces as **one** skill, not a swarm of phantom entries.
- **Discovery↔delete consistency**: a skill is keyed for discovery by its frontmatter `name`, but its on-disk directory name may differ (hand-dropped / mis-extracted package). `LocalSkillStorage._resolve_skill_dir(name, category)` resolves the *discovered* directory (canonical `<category>/<name>` fast-path, else scan-by-frontmatter-name); `custom_skill_exists`/`read_custom_skill`/`delete_custom_skill` all go through it, so **anything that gets listed can also be read and deleted** regardless of directory name (the old code assumed `custom/<name>/` and 404'd "Custom skill 'x' not found." on any non-canonical layout). Delete is path-guarded to never escape the `custom/` root. Tests: `tests/test_skill_storage_delete.py`.
- **Injection**: Enabled skills listed in agent system prompt with container paths
- **Configurable `es_query` routing prompt**: `config.yaml → skills.es_query_routing`
  contains an `enabled` switch and editable `prompt`. When enabled,
  `lead_agent.prompt._build_es_query_routing_section` injects the rule into the
  default lead-agent system prompt. Custom-agent-only prompts receive it only
  when their explicit skill allowlist contains `es_query`; disabling the switch
  or leaving the prompt blank emits no block. Immediately before an ordinary Q&A
  graph is created, `lead_agent.agent._log_es_query_routing_registration` checks
  the exact configured block against the final system-prompt string and logs
  `enabled`, allowlist eligibility, `registered`, character count, and a short
  SHA-256 fingerprint (never the prompt body). Tests:
  `tests/test_es_query_routing_prompt.py`.
- **Installation**: `POST /api/skills/install` extracts .skill ZIP archive to custom/ directory. Direct uploads use the ownership-aware `POST /api/skills/validate?target_category=...` preflight plus `POST /api/skills/install-upload`: a same-owner custom conflict may be uploaded again with `overwrite=true`, while a different owner's conflict is non-overwritable and returns that owner's username (falling back to user id). The install endpoint repeats the ownership check, including for admins targeting `custom`, so callers cannot bypass the preflight. Public-skill overwrite remains admin-only.
- **Installable sentiment analysis package**: `skill-packages/sentiment-analysis.skill` is deliberately separate from the frontend virtual-agent chat page. The chat page goes through Gateway `POST /api/sentiment-agent/stream` (runtime-config URL/Basic Auth in the body, TLS verify off, raw AG-UI SSE passthrough). The skill package's self-contained `config/sentiment-agent.json` owns the external AG-UI address for agent-skill callers; Basic Auth is resolved from `SENTIMENT_AGENT_AUTH_USERNAME` / `SENTIMENT_AGENT_AUTH_PASSWORD` at runtime (not committed into the package). `scripts/sentiment_stream.py` uses only the Python standard library and converts upstream SSE to stable NDJSON events (`thinking_delta`, `answer_delta`, `done`), with a Markdown convenience mode for normal agent calls. Installation/calling guide: `skill-packages/sentiment-analysis/docs/调用与安装说明.md`.
- **Chinese name (`name_zh`)**: a Chinese display name stored in the **`skills` DB table** (`SkillRow.name_zh`, `String(128)`, migration `20260609_01`) ? never written to the on-disk SKILL.md, so it works for built-in skills too. Threaded through `SkillStore` create/ensure_legacy/update/update_admin exactly like `detail`; surfaced on `SkillResponse.name_zh` from the ownership record in `_skill_to_response`. Authorization mirrors `detail`: custom skill ? owner or admin; built-in/public ? admin only (row materialized on demand via `ensure_legacy`). Endpoints (in `app/gateway/routers/skills.py`): `PUT /api/skills/{name}/name-zh` (set/clear), `POST /api/skills/{name}/name-zh/generate` (LLM generates one), `POST /api/skills/name-zh/generate-missing` (????: **admin-only** bulk-generate for every skill that lacks one, bounded-concurrency `asyncio.gather`). **Optional model selection**: both generate endpoints accept an optional JSON body `{model_name}` ? an explicit name wins, else the curator model setting, else the main model (`_resolve_skill_name_zh_model_name`). **Model-error surfacing**: `_generate_skill_name_zh` raises `SkillNameZhModelError` when the model itself fails (build/timeout/API error) ? distinct from the model returning an unusable name (? `None`). The single endpoint maps that to `502 ?????,?????????:?`; the bulk endpoint fails fast on a model-build error (502) and otherwise carries a representative `model_error` string in `SkillNameZhBulkResponse` so the UI can say "N ?????????". The frontend (`skill-settings-page.tsx` list + `skill-detail-dialog.tsx` editor, plus the agent skill pickers/badges in `NewAgentPage.tsx` + `agent-card.tsx`) shows `name_zh || name`, offers per-skill "AI ??" plus an **admin-only** header "???????" button, and a compact model dropdown (`components/workspace/model-name-select.tsx`, default = configured model) beside each action.

#### Skill Address Management (技能地址管理 — URL/IP 批量替换)

Admin-only maintenance tool to find every `http(s)://` URL and IPv4 (`ip[:port]`)
address in the **running** skills (`skills/{public,custom}`, all `.md` + `.py`,
recursing into `scripts/`/`references/`/`templates/`) and rewrite them in bulk —
e.g. swapping a UAT endpoint for production across all skills at once. The same
address commonly lives in both a skill's `SKILL.md` and its `scripts/*.py`, so
results are **aggregated by unique address** with every occurrence (`file:line` +
snippet) and replaced together.

- **Pure engine** — `deerflow/skills/address_scan.py` (harness, no `app.*`):
  `find_spans` / `scan_occurrences` (URL regex with trailing-punctuation strip;
  IPv4 with 0–255 octet check + optional port; an IP inside a URL is *not*
  separately surfaced), `apply_replacements` (**substring-safe**: re-detects exact
  token spans and only rewrites a span whose text equals a requested `from`, so
  `1.2.3.4` is never rewritten inside `11.2.3.40`), `infer_kind`,
  `is_valid_replacement`. Tested in `tests/test_skill_addresses.py`.
- **Router** — `app/gateway/routers/skill_addresses.py` (`/api/skill-addresses`,
  **admin only**). **Availability at scale (150+ skills):** every filesystem
  walk/read/write runs via `asyncio.to_thread` so the event loop is never blocked.
  Routes: `GET /scan` (aggregated, one-shot) + `GET /scan/stream` (per-skill NDJSON
  progress `{type:start|progress|result}`); `POST /preview` (dry-run diff +
  validation warnings); `POST /apply` + `POST /apply/stream` (per-file NDJSON
  progress); `GET /backups`; `POST /rollback/{backup_id}`. **Apply safety:**
  optimistic lock via `expected_total` (409 if files changed since the scan),
  path-guard (`is_relative_to` the skills root), auto disk backup to
  `{skills_root}/.address-edits/{timestamp}/` (the `.`-prefixed dir is excluded
  from scans), then `refresh_skills_system_prompt_cache_async()` so edits take
  effect immediately. Registered in `app.py` after `sensitive_words.router`.
  **Per-edit history:** each apply also writes a sibling manifest
  `{skills_root}/.address-edits/{timestamp}.manifest.json` (best-effort) capturing
  `{created_at, replaced_total, replacements:[{from,to,kind}], changed_files}`; the
  sibling-not-child placement keeps it out of both the restore set and the file
  count. `GET /backups` returns each record enriched with `replaced_total` +
  `replacements` (from the manifest; falls back to dir mtime when absent), and
  `POST /rollback/{backup_id}` restores a single record — so the UI shows a full
  modification history and rolls back any one entry independently.
- **Frontend** — the panel is the reusable `SkillAddressesPanel` (exported from
  `pages/SkillAddressesPage.tsx`; its default export keeps the standalone route
  `/page/strategy/admin/skill-addresses` as a direct-link fallback). It is
  **embedded as a「技能地址管理」tab inside the admin「技能管理」page**
  `pages/AdminSkillsPage.tsx` (Tabs: 技能列表 + 技能地址管理; reached from the
  「技能管理」item in the「设置和更多」dropdown `workspace-nav-menu.tsx` →
  `/page/workspace/admin/skills`) — it is no longer a separate entry in the
  `components/page-sidebar.tsx` settings dropdown. The panel:
  auto-scan on open with a **progress bar** (扫描逐技能 / 应用逐文件 X/N), URL/IP
  type filter, search, a「片段批量套用」helper (e.g. all `uat.4.cn` → `prod.4.cn`),
  per-address「替换为」input with expandable occurrences, inline 预览 panel, 应用,
  **multi-keyword search** (space-separated, all-must-hit), and a **修改记录**
  (modification history) panel — each apply is one record (time / which addresses
  changed from→to / file & replacement counts) with a per-record 回退此次
  (single-record rollback, with inline confirm). Client + NDJSON stream parsing in
  `strategy-components/api/skill-addresses.ts`. Scope is the running skills only;
  the `deploy/minimal/skills/` copy is not touched (sync separately).

#### Skill Compression (技能压缩)

When the skill catalogue grows large, injecting every enabled skill full-text
into the lead agent's system prompt bloats it and wastes tokens. **Skill
compression** is an admin-gated mode that keeps only *常驻(免压缩)* skills in the
prompt body and surfaces the rest on demand via keyword retrieval.

- **Master switch** — `system_settings.json → skill_compression`
  (`SkillCompressionSettings`: `enabled` default **false**, `search_top_n`
  default 5, `keep_index_in_prompt` default true). Read/written by
  `GET`/`PUT /api/system-settings/skill-compression` (GET open to any user — the
  skill detail page reads it; PUT admin-only, refreshes the prompt cache).
- **Per-skill 常驻 flag** — stored in the **DB**: `SkillRow.always_on`
  (`Boolean`, migration `20260616_03`, idempotent SQL in
  `column_additions_mysql.sql`), **not** in `extensions_config.json` (the
  source-tree config file is read-only / unsafe to rewrite in some deployments,
  and built-in skills carry no file entry). Threaded through `SkillStore`
  create/ensure_legacy/update/update_admin + `list_always_on_names()` exactly
  like `name_zh`/`detail`; surfaced on `SkillResponse.always_on` from the
  ownership record in `_skill_to_response`. Toggled via
  `PUT /api/skills/{name}/always-on` (custom skill → owner or admin; built-in /
  public → admin only, materializes a row on demand via `ensure_legacy`). The
  prompt builder stamps `Skill.always_on` from `list_always_on_names()` via a
  sync DB bridge (`_apply_always_on` in `lead_agent/prompt.py`).
- **Prompt split** — `get_skills_prompt_section` (`lead_agent/prompt.py`) reads
  the compression settings (skipped for restricted custom agents — they already
  have a curated allowlist). When **off**, behaviour is unchanged (all enabled
  skills full-text). When **on**, `always_on` skills render as full `<skill>`
  blocks; the rest are dropped from the body, replaced by a `search_skills`
  usage note plus (when `keep_index_in_prompt`) a name-only `<compressed_skills>`
  index. The `_get_cached_skills_prompt_section` cache key includes the
  `always_on` flag + compression flags so changes invalidate correctly.
- **Retrieval tool** — `search_skills` (`tools/builtins/skill_tools.py`,
  `search_skills_impl`) keyword-scores `name + description` over enabled,
  visible, non-archived skills (same visibility filter as `skill_list`) and
  returns top-N `{name, description, category, location}`. It is registered in
  `get_available_tools` **only when compression is enabled** (offline,
  deterministic, zero external deps). The agent then `read_file`s the returned
  `location`.

Tests: `tests/test_skill_compression.py`.

#### Skill Curator (`packages/harness/deerflow/skills/curator.py`)

Background skill lifecycle manager. The Gateway `lifespan` starts a resident
scheduler task (`_curator_scheduler_loop` in `app/gateway/app.py`) that ? after a
5-minute startup delay ? calls `maybe_run_curator()` every hour. A cross-process
file lock (`.curator_scheduler.lock`) elects a single leader worker so the
scheduler runs once per deployment. Per run:

- **Master switch** ? `skill_evolution.enabled` (default **`false`**) gates the
  whole feature; `curator.enabled` (default **`false`**) gates the curator
  specifically. Both must be `true` for the curator to act. `should_run_now()`
  and `_curator_thread_target()` both enforce this ? so manual `POST
  /api/curator/run` also respects the master switch. Config is
  `config.yaml.skill_evolution` deep-merged with per-user
  `users/{user_id}/skill_evolution.json`.
- **Per-user run lock** ? `.curator_state` carries `running` / `started_at`;
  `_try_acquire_run_lock()` prevents the same user's curator from running
  concurrently (scheduler vs. manual trigger, or double manual clicks). A lock
  whose `started_at` is older than `_CURATOR_STALE_LOCK_HOURS` (6h) is treated as
  process-crash residue and can be taken over.
- **Concurrency cap** ? `maybe_run_curator()` collects all due users and hands
  them to `_dispatch_curator_runs()`, which runs them through a
  `ThreadPoolExecutor` capped at `_CURATOR_MAX_CONCURRENCY` (5) ? avoids a
  thundering herd of LLM calls when many users come due at once (e.g. first
  deploy).
- **Failure back-off** ? `_release_run_lock()` always stamps `last_attempt_at`.
  When a first run keeps crashing (`last_run_at` never set), `should_run_now()`
  retries on `_CURATOR_RETRY_HOURS` (6h) instead of every hourly tick.
- **`should_run_now()`** is the real gate: the hourly tick only checks "is the
  per-user `interval_hours` (default 168h) elapsed". Timestamp parsing tolerates
  naive (tz-less) strings ? a corrupt `last_run_at` triggers a re-run rather than
  a permanent deadlock.
- **Published-skill filter** ? `SkillStore.list_published_names()` is a batch
  query; the curator fetches the published-name set once per run instead of one
  DB round-trip per candidate skill.
- **LLM review timeout** ? `_run_llm_review()` runs `agent.invoke()` in a worker
  thread bounded by `curator.review_timeout_seconds` (default 600s). The model's
  per-request `request_timeout` can't bound a multi-turn React agent loop, so on
  timeout the run is abandoned (a warning is written to `curator_log`, visible in
  the Web UI) and the curator finishes cleanly + releases the run lock. The
  worker context is copied via `contextvars.copy_context()` so tool calls still
  resolve the correct `user_id`.
- **Review model** ? `_run_llm_review()` uses `curator.model_name` (a
  `config.yaml` `models[]` entry name) if set, else the primary model; it is
  editable from the Web UI evolution panel. The skill security scanner
  (`scan_skill_content` in `security_scanner.py`) reuses the same
  `skill_evolution.curator.model_name` setting ? review and moderation share one
  model. There is no separate moderation-model config field.

### Model Factory (`packages/harness/deerflow/models/factory.py`)

- `create_chat_model(name, thinking_enabled)` instantiates LLM from config via reflection
- Supports `thinking_enabled` flag with per-model `when_thinking_enabled` overrides
- `force_disable_thinking=True` (kwarg) hard-disables thinking for models that explicitly declare `supports_thinking: true` even when the model config declares no `when_thinking_*`/`thinking` shape — many internal vLLM/Qwen models only set that flag, so `thinking_enabled=False` alone injects nothing and they keep reasoning on. The flag injects the OpenAI-compatible disable shapes (`extra_body.enable_thinking=false`, `extra_body.chat_template_kwargs.enable_thinking=false`, and `extra_body.thinking.type="disabled"`) only for classes that accept `extra_body`. Models with `supports_thinking: false`, especially generic OpenAI-compatible proxies such as LiteLLM, receive no vendor-specific thinking fields because those gateways may reject unknown parameters. Mirrors `TitleMiddleware._force_disable_thinking`. The lead agent reads the runtime flag `thinking_force_disabled` and forwards it (used by `/api/open/chat`)
- Supports vLLM-style thinking toggles via `when_thinking_enabled.extra_body.chat_template_kwargs.enable_thinking` for Qwen reasoning models, while normalizing legacy `thinking` configs for backward compatibility
- Supports `supports_vision` flag for image understanding models
- Config values starting with `$` resolved as environment variables
- Missing provider modules surface actionable install hints from reflection resolvers (for example `uv add langchain-google-genai`)

### Roundtable Model Fault Tolerance (会商模型自动容错 / 换下一个模型)

会商第二步（leader/seat）+ 第三步（summary/report/dashboard + 结构化抽取）单轮执行时，**某模型出问题就自动切到下一个模型继续**，并告知用户「『X』模型异常 → 已切换『Y』」。前台实时（`multi_agent.py`）与后台挂起（`engine.py` + `InProcessRoundtableGateway`）两条路径都覆盖。

- **可检测的模型失败**：`LLMErrorHandlingMiddleware` 在同一模型重试 3 次后**不抛异常**，而是返回兜底 `AIMessage` —— 现在给它打 `additional_kwargs[LLM_ERROR_MARKER="deerflow_llm_error"]=reason`（model_dump 保留，经 checkpointer 序列化仍在）。`is_llm_error_message(msg)` 据标记（回退按已知兜底文案正文）识别，返回 reason。
- **候选模型链**：`app/gateway/roundtable_model_fallback.py::fallback_model_chain(current)` 按 `config.yaml models[]` 顺序给出 `[当前模型, …其余]`（去重保序）。`reason_text(reason)` 翻译成简短中文。总开关 env `ROUNDTABLE_MODEL_FALLBACK`（默认开，`0/false/off` 关 → 单模型旧行为）；`ROUNDTABLE_MODEL_FALLBACK_MAX`（0=试完全部）。
- **触发换模型的情形**：① 单轮 run 抛异常（start_run 失败 / idle 卡死 → `timeout`，其余 `exception`）；② 兜底错误文案（`is_llm_error_message`）；③ 交付无效（仅 seat/leader；leader 还含「既无正文又无派活」）。席位**只认剥掉 `<think>` 后的可见正文**——思考-only / 空串 / 断流都不算已交付（`deerflow.agents.roundtable_orchestrator.delivery`：`visible_seat_text` / `classify_seat_delivery`）。前台 `_special_run` 对此 yield `status:"invalid_delivery"` 且不广播；`GET /last-ai-message` 回 `{delivered, content, length, invalid_reason}`（思考-only 标 `thinking_only`，总控席位清单标 ⚠️）。后台引擎只把可见正文写入 `deliveries`，续轮用 `build_continue_prompt`：有无效席位时强制重派，禁止声称「已完成交付」。Step3 单例（report/summary/dashboard/ingest/结构化）的产物是落盘文件，**不**把「正文为空」当失败，只对真模型错误换模型。
- **后台**：`InProcessRoundtableGateway._run_turn_with_fallback(...)` 包住 `_run_agent_turn`，逐候选模型试；换模型落 `model_switch` 诊断、全失败落 `model_fallback_exhausted`。`run_leader/run_seat/run_report/run_summary/run_dashboard_data` 全走它（report/summary/dashboard 的产物自愈重试保留，并记住已切到的可用模型用于后续续写轮）。各 Turn/Result 带回 `switches: list[ModelSwitch]`，引擎 `_emit_switches` → `JobProgressWriter.model_switch(...)` 往研讨时间线插一条 `role="system"` 系统提示气泡。
- **前台**：`_special_run`/`_leader_run` stream 收尾后检测失败 → yield `{status:"model_switch", role, agent_name, failed_model, next_model, reason}` 帧并用下一个模型重新流式；终态帧带 `model_used`。结构化抽取 `flow_extract.py` 的 round1 `model.astream` 也包了同款换模型循环（emit `event:"model_switch"` 帧）。
- **Step2 单席位隔离**：`engine._run_recommend/_run_chain/_run_dag` 的每个席位执行包 try/except —— 一个席位彻底失败（落 `seat_failed` 诊断 + 置空交付）不拖垮整批/整链/整个作业。
- **前端**：`multi-agent.ts` 解析 `model_switch` 帧 → `onModelSwitch` 回调（清空失败尝试的半截气泡 + 累计的 tool_call delta，phase 退回 connected）；`useStep2Orchestration` 弹 toast + 时间线插「系统提示」气泡；`useStep3Summary/Report/Dashboard` + `flow-extract.ts` 重置对应气泡并在步骤卡留可见提示；后台路径经 `job-live.ts::toStep2Dialogue` 把 `role="system"` 渲染成 amber「系统提示」气泡。
- Tests: `tests/test_roundtable_model_fallback.py`。

### vLLM Provider (`packages/harness/deerflow/models/vllm_provider.py`)

- `VllmChatModel` subclasses `langchain_openai:ChatOpenAI` for vLLM 0.19.0 OpenAI-compatible endpoints
- Preserves vLLM's non-standard assistant `reasoning` field on full responses, streaming deltas, and follow-up tool-call turns
- Designed for configs that enable thinking through `extra_body.chat_template_kwargs.enable_thinking` on vLLM 0.19.0 Qwen reasoning models, while accepting the older `thinking` alias

### IM Channels System (`app/channels/`)

Bridges external messaging platforms (Feishu, Slack, Telegram, DingTalk) to the DeerFlow agent via the LangGraph Server.


**Architecture**: Channels communicate with Gateway through the `langgraph-sdk` HTTP client (same as the frontend), ensuring threads are created and managed server-side. The internal SDK client injects process-local internal auth plus a matching CSRF cookie/header pair so Gateway accepts state-changing thread/run requests from channel workers without relying on browser session cookies.

**Components**:
- `message_bus.py` - Async pub/sub hub (`InboundMessage` ? queue ? dispatcher; `OutboundMessage` ? callbacks ? channels)
- `store.py` - JSON-file persistence mapping `channel_name:chat_id[:topic_id]` ? `thread_id` (keys are `channel:chat` for root conversations and `channel:chat:topic` for threaded conversations)
- `manager.py` - Core dispatcher: creates threads via `client.threads.create()`, routes commands, keeps Slack/Telegram on `client.runs.wait()`, and uses `client.runs.stream(["messages-tuple", "values"])` for Feishu incremental outbound updates
- `base.py` - Abstract `Channel` base class (start/stop/send lifecycle)
- `service.py` - Manages lifecycle of all configured channels from `config.yaml`
- `slack.py` / `feishu.py` / `telegram.py` / `dingtalk.py` - Platform-specific implementations (`feishu.py` tracks the running card `message_id` in memory and patches the same card in place; `dingtalk.py` optionally uses AI Card streaming for in-place updates when `card_template_id` is configured)

**Message Flow**:
1. External platform -> Channel impl -> `MessageBus.publish_inbound()`
2. `ChannelManager._dispatch_loop()` consumes from queue
3. For chat: look up/create thread through Gateway's LangGraph-compatible API
4. Feishu chat: `runs.stream()` ? accumulate AI text ? publish multiple outbound updates (`is_final=False`) ? publish final outbound (`is_final=True`)
5. Slack/Telegram chat: `runs.wait()` ? extract final response ? publish outbound
6. Feishu channel sends one running reply card up front, then patches the same card for each outbound update (card JSON sets `config.update_multi=true` for Feishu's patch API requirement)
7. DingTalk AI Card mode (when `card_template_id` configured): `runs.stream()` ? create card with initial text ? stream updates via `PUT /v1.0/card/streaming` ? finalize on `is_final=True`. Falls back to `sampleMarkdown` if card creation or streaming fails
8. For commands (`/new`, `/status`, `/models`, `/memory`, `/help`): handle locally or query Gateway API
9. Outbound ? channel callbacks ? platform reply

**Configuration** (`config.yaml` -> `channels`):
- `langgraph_url` - LangGraph-compatible Gateway API base URL (default: `http://localhost:8001/api`)
- `gateway_url` - Gateway API URL for auxiliary commands (default: `http://localhost:8001`)
- In Docker Compose, IM channels run inside the `gateway` container, so `localhost` points back to that container. Use `http://gateway:8001/api` for `langgraph_url` and `http://gateway:8001` for `gateway_url`, or set `DEER_FLOW_CHANNELS_LANGGRAPH_URL` / `DEER_FLOW_CHANNELS_GATEWAY_URL`.
- Per-channel configs: `feishu` (app_id, app_secret), `slack` (bot_token, app_token), `telegram` (bot_token), `dingtalk` (client_id, client_secret, optional `card_template_id` for AI Card streaming)


### Memory System ? Memory V2 (`packages/harness/deerflow/agents/memory/`)

Pluggable memory architecture (Hermes-style). See `docs/MEMORY_V2_DESIGN_ZH.md`
and `docs/HINDSIGHT_SETUP_ZH.md` for the full design.

**Components**:
- `provider.py` - `MemoryProvider` abstract base (12 lifecycle/hook methods)
- `manager.py` - `MemoryManager` orchestrates builtin + ?1 external provider; `build_memory_manager()` is the construction entry point
- `providers/builtin.py` - `BuiltinFileProvider`: local USER.md / MEMORY.md, ? delimiter, char quotas, file locking, atomic writes, frozen snapshot
- `providers/hindsight.py` - `HindsightProvider`: HTTP client to a self-hosted Hindsight Docker instance, dual-bank routing
- `security.py` - injection/exfiltration scan + invisible-unicode block for memory content
- `message_processing.py` - `filter_messages_for_memory` (last-turn extraction for Hindsight retain)

**Two stores (binary classification)**:
- `USER.md` at `{base_dir}/users/{user_id}/USER.md` ? user profile, shared across agents
- `MEMORY.md` at `{base_dir}/users/{user_id}/agents/{agent_id}/MEMORY.md` ? agent-private working notes
- `user_id` resolved via `get_effective_user_id()`; `"default"` in no-auth mode

**Write paths**:
- LLM actively calls the `memory` tool (`tools/builtins/memory_tool.py`) ? writes USER.md/MEMORY.md, mirrors to Hindsight
- LLM calls `hindsight_recall`/`reflect`/`retain` tools (`tools/builtins/hindsight_tools.py`) ? only in `tools`/`hybrid` memory_mode
- Hindsight `auto_retain` ? every turn auto-stored to the work bank (via `MemoryMiddleware.after_agent`)

**Injection**:
- builtin frozen snapshot ? injected into the system prompt via `lead_agent/prompt.py` `_get_memory_context`
- Hindsight dynamic recall ? injected per model call via `MemoryMiddleware.wrap_model_call` (transient ? not persisted to state, so it never leaks to the frontend)

**Hindsight dual-bank isolation** (B-scheme):
- global user bank `deerflow-user-{user_id}` ? cross-agent
- agent work bank `deerflow-work-{user_id}-{agent_id}` ? per-agent
- `hindsight-client` is an optional dependency; when missing, `HindsightProvider.is_available()` returns False and the system falls back to builtin-only

**Migration**: `PYTHONPATH=. python scripts/migrate_memory_to_v2.py [--all|--user-id X] [--dry-run]` converts legacy `memory.json` into USER.md/MEMORY.md.

**Configuration** (`config.yaml` ? `memory`, nested): `enabled`, `provider` (builtin/hindsight),
`builtin.{memory_char_limit,user_char_limit,...}`, `hindsight.{mode,api_url,api_key,bank_id_template,work_bank_id_template,memory_mode,...}`,
`injection.{enabled,max_tokens,include_builtin,include_hindsight,context_tag}`, `security.{scan_content,block_invisible_unicode}`.
Per-user runtime overrides (whitelisted fields) live at `{base_dir}/users/{user_id}/memory_config.json` ? see `get_effective_memory_config()`.

**Gateway API** ? see the Memory router below (`/api/memory/v2*`).

### Reflection System (`packages/harness/deerflow/reflection/`)

- `resolve_variable(path)` - Import module and return variable (e.g., `module.path:variable_name`)
- `resolve_class(path, base_class)` - Import and validate class against base class

### Config Schema

**`config.yaml`** key sections:
- `models[]` - LLM configs with `use` class path, `supports_thinking`, `supports_vision`, provider-specific fields
- vLLM reasoning models should use `deerflow.models.vllm_provider:VllmChatModel`; for Qwen-style parsers prefer `when_thinking_enabled.extra_body.chat_template_kwargs.enable_thinking`, and DeerFlow will also normalize the older `thinking` alias
- `tools[]` - Tool configs with `use` variable path and `group`
- `tool_groups[]` - Logical groupings for tools
- `sandbox.use` - Sandbox provider class path
- `skills.path` / `skills.container_path` - Host and container paths to skills directory
- `title` - Auto-title generation (enabled, max_words, max_chars, prompt_template)
- `summarization` - Context summarization (enabled, trigger conditions, keep policy)
- `subagents.enabled` - Master switch for subagent delegation
- `memory` - Memory system (enabled, storage_path, debounce_seconds, model_name, max_facts, fact_confidence_threshold, injection_enabled, max_injection_tokens)
- `auth_login` - Custom login modes layered on the base auth (`deerflow/config/login_config.py`):
  - `username_login.{require_password, password}` - **Shared** password gate for `POST /api/v1/auth/login/username`. When `require_password=true`, the (otherwise passwordless) username login additionally requires `?password=<shared-secret>`; empty `password` falls back to `123ewq`. Gate only ? it does not change which account logs in. **Per-account override (账号级登录口令)**: independent of this shared gate, an admin can assign **one specific account** an individual **plaintext** password via the 白名单管理 page (`users.login_password` column, migration `20260625_04`, nullable). When the target account carries a `login_password`, `/login/username` **requires** `?password=` to match it **exactly** and that personal password **takes precedence over** the shared gate (the shared secret will not unlock that account); accounts with no personal password keep the shared-gate / passwordless behavior. The check happens before auto-register via the lookup-only helper `_lookup_user_by_username` (so a brand-new username is unaffected). Surfaced/managed admin-only: `GET /api/v1/auth/users` returns `UserListItem.login_password` (plaintext — the list is already admin-gated), and `PUT /api/v1/auth/users/{user_id}/login-password` `{password: str|null}` sets (non-empty) / clears (null·empty) it without touching `token_version`. Plaintext storage is intentional (the password travels in the login URL query anyway, so it stays re-copyable). The user logs in via the existing `#/login/<username>?password=xxxx`. Tests: `tests/test_login_personal_password.py`.
  - `token_login.{enabled, token_info_url, service_authorization, timeout_seconds, verify_ssl, ca_cert_path, mock_tokens}` - Token-exchange login (`POST /api/v1/auth/login/token`). The gateway POSTs to `token_info_url` with the caller-supplied token placed directly in the `Authorization: Bearer <token>` header (no cookie; the static `service_authorization` field is deprecated/unused, kept only for backward compat), reads `data.username` from a `{"state":"200",...}` body, then signs the matching local user in (auto-registering if absent). `token_info_url` is HTTPS ? `verify_ssl: false` skips cert verification for self-signed/internal UAT hosts, or set `ca_cert_path` to a PEM bundle (takes precedence, keeps verification on). **`mock_tokens` (offline test mapping `{token: username}`)**: when an incoming token matches a key, `_fetch_username_from_token` returns the mapped username and **skips the `token_info_url` call entirely** — for environments where the upstream `getTokenInfo` host is unreachable (e.g. external networks). Empty in prod (no effect); seeded `{"123ewq": "admin"}` in this deployment so `/login/任意?authToken=123ewq` signs in as admin. **`UsernameLoginResponse` now also returns `username`** (the raw resolved login name, pre email-normalization) on both `/login/token` and `/login/username`, so the frontend can append a faithful `?username=` on jump links (按钮管理 追加登录态) instead of the lossy email local-part. See [docs/AUTH_LOGIN.md](docs/AUTH_LOGIN.md) for the full request/response contract.

Both endpoints are whitelisted in `auth_middleware._PUBLIC_EXACT_PATHS`. The shared `_get_or_create_user_by_username()` helper (in `routers/auth.py`) backs both `/login/username` and `/login/token`.

- `log_system` - Admin-only external log-system entry (`deerflow/config/log_system_config.py`): `{enabled, url, label}`. Surfaced read-only by `GET /api/system-settings/log-system` (admin only); the frontend renders it as a "????" item in the bottom-right settings menu (`workspace-nav-menu.tsx`) that opens `url` in a new tab. Disabled by default.

**`extensions_config.json`**:
- `mcpServers` - Map of server name ? config (enabled, type, command, args, env, url, headers, oauth, description)
- `skills` - Map of skill name ? state (enabled). 常驻(免压缩)/always_on lives in the `skills` DB table, not here.

Both can be modified at runtime via Gateway API endpoints or `DeerFlowClient` methods.

### Conditional WeKnora LLMWiki

Ordinary knowledge-base cards expose an owner/admin-only name/description edit dialog in `WeKnoraKnowledgeWorkspace.tsx`, reusing `useLlmWikiActions().update` and `PATCH /api/llmwiki/knowledge-bases/{id}`. The route trims names, rejects blank or reserved conversation-deposit names, writes to WeKnora before updating local metadata, and retains existing write authorization. A successful remote list is authoritative for every list scope, including `personal`: absent remote IDs are hidden without deleting mappings/references. Upstream failures remain errors. The grid also filters cached `remote_status=missing` rows and no longer renders stale-record cleanup cards. Regression coverage: `tests/test_llmwiki_knowledge_management.py`.

`llmwiki.weknora.api_base_url` is the provider switch: non-empty enables WeKnora, empty preserves the legacy LLMWiki UI. WeKnora is independently deployed and owns PostgreSQL/Redis/object storage/vector storage/parsers/models; DeerFlow config contains only `api_base_url` and optional `web_base_url`. DeerFlow deliberately sends no WeKnora credential, so the WeKnora endpoint must be restricted to a trusted network and allow DeerFlow traffic itself or through its reverse proxy. There is intentionally no `default_embedding_model_id`: `WeKnoraClient` queries `/api/v1/models` and uses WeKnora's active Embedding model when creating a KB; Wiki creation additionally resolves an active KnowledgeQA model inside WeKnora for synthesis. The native BFF is `app/gateway/routers/llmwiki.py`; ownership/publication mappings live in `deerflow.persistence.llmwiki`; the browser only sees DeerFlow mapping IDs. Knowledge-base owners publish immediately (`private` -> `published`) without a review queue, and can unpublish ordinary bases. The conversation-deposit KB is system-owned, always published, shared by all DeerFlow users, excluded from My Knowledge, and immutable through user/admin detail operations; recognition requires both the reserved name and `system` owner. Existing same-name remote mappings are adopted in place as the system deposit, preserving mapping ids and document links. Its user-facing description, generated Markdown source and tags use `cmzs` branding; list/detail reads synchronize stale remote descriptions, and search results, source previews and embedded-detail JSON recursively replace legacy brand text in historical conversation-deposit payloads. Deposit requests must reference a thread accessible to the current account. Personal knowledge-base list/search views are scoped to the current DeerFlow account even for admins; frontend LLMWiki caches include the live account id, and admin write power does not make other users' private KBs appear as owned. Only `system_role === "admin"` sees the WeKnora connection-config tab. Knowledge cards navigate to a dedicated WeKnora-style detail route. Wiki-enabled bases render and default to `Wiki -> documents -> graph`; standard bases omit Wiki and render `documents -> graph`. The detail page supports sequential multi-file upload with an aggregate result for writable ordinary bases, hides it for public read-only/system-deposit bases, and uses a generic loading label while the embed session starts. It also supports URL and published manual-Markdown imports, browses real Wiki pages/content, proxies authenticated original-file previews, pages through `/chunks/{knowledge_id}`, and renders the Wiki page graph from `/knowledgebase/{kb_id}/wiki/graph`; document details use a right sheet rather than the removed KB modal. The credential-free proxy uses one WeKnora service identity, so per-user separation is enforced by DeerFlow mappings while all remote KBs physically share that identity's WeKnora workspace. WeKnora iframe detail pages first call `/api/llmwiki/knowledge-bases/{id}/weknora/session` through the normal DeerFlow API auth path, then load `/platform/knowledge-bases/{weknora_id}` with a signed `deerflow_weknora_embed` cookie for `/platform`, `/assets`, `/locales`, root assets, and API calls. Injected fetch/XHR rewriting sends all iframe `/api/v1/*`, including colliding `/api/v1/auth/*`, through `/api/llmwiki/weknora-embed/api/v1/*`; Gateway auth/CSRF bypass only applies to those signed iframe proxy requests, the proxy still revalidates the mapped KB, and cookie security honors `X-Forwarded-Proto` for HTTPS reverse proxies. `llmwiki_search` is conditionally registered in `tools.py`, is read-only, and rechecks the current DeerFlow user, agent `llmwiki_knowledge_base_ids`, and thread-context selection on every call. Source previews prefer the cleaned Wiki/LLMWiki page text when a retrieved chunk maps to a Wiki page, with raw split chunks as the fallback. Agent bindings use an inline WeKnora-style searchable/grouped multi-select in the standard agent create/detail UI; new chats use the same interaction pattern as a compact composer dropdown, and selections are written to thread metadata/context and restored on reopen. The DeerFlow frontend is native React and the WeKnora frontend is not modified or embedded. Full deployment and behavior notes: `docs/LLMWIKI_WEKNORA_INTEGRATION_ZH.md`. Tests: `tests/test_llmwiki_weknora.py`.

The embedded detail proxy hides WeKnora's outer `.main > .aside_box` application sidebar and expands the route outlet, but deliberately keeps the Wiki page's internal directory/index sidebar. Do not add an HTML `sandbox` attribute to this trusted, permission-filtered detail iframe: a sandboxed ancestor causes Chromium's built-in Blob PDF viewer to render the browser-blocked page. The Gateway follows upstream redirects only for `/knowledge/{id}/preview` and `/knowledge-bases/{id}/files`, keeping PDF bytes and protected Wiki images inside the signed proxy instead of sending the browser to WeKnora object storage.
Native/private and public Wiki readers must rewrite protected Markdown image
sources to their mapping-scoped DeerFlow `/files?file_path=` endpoints and keep
real HTTP URLs on rendered image elements. The embedded guard also rewrites
protected fetch/XHR values and hydrates late `img[data-protected-src]`
placeholders with the signed iframe HTTP proxy URL. Observe both
`data-protected-src` and `src`, because WeKnora may replace the URL after first
paint. Never create/revoke a temporary object URL here: its lightbox reuses the
thumbnail `src` after click. Never expose the service token or relax the
current-KB authorization to make images work.

**Embedded native knowledge-base switching:** The original WeKnora breadcrumb picker remains available inside the detail iframe. A signed 15-minute embed cookie carries only the current DeerFlow user's visible mappings (their own plus published bases); the proxied `/api/v1/knowledge-bases` response is filtered to that allowlist. Choosing another entry performs a full iframe navigation, rechecks the DeerFlow mapping and read/write permission, then reissues the cookie for the selected base. Never expose the upstream service-admin list directly or permit an arbitrary remote knowledge-base ID.

The independent enterprise-research workbench is **temporarily disabled**. Its source remains under `app/gateway/routers/enterprise_research.py` and `deerflow.persistence.enterprise_research`, but Gateway does not mount `/api/enterprise-research`, initialize its stores/executor, or register its ORM rows in `Base.metadata`. This ensures startup does not create or alter `enterprise_research_tasks` / `enterprise_research_report_jobs` on MySQL. It remains separate from `/api/deep-research`; do not re-enable it incidentally while changing the active Deep Research workflow. Regression coverage for its preserved implementation remains in `tests/test_enterprise_research_tasks.py`.

### Embedded Client (`packages/harness/deerflow/client.py`)

`DeerFlowClient` provides direct in-process access to all DeerFlow capabilities without HTTP services. All return types align with the Gateway API response schemas, so consumer code works identically in HTTP and embedded modes.

**Architecture**: Imports the same `deerflow` modules that Gateway API uses. Shares the same config files and data directories. No FastAPI dependency.

**Agent Conversation**:
- `chat(message, thread_id)` ? synchronous, accumulates streaming deltas per message-id and returns the final AI text
- `stream(message, thread_id)` ? subscribes to LangGraph `stream_mode=["values", "messages", "custom"]` and yields `StreamEvent`:
  - `"values"` ? full state snapshot (title, messages, artifacts); AI text already delivered via `messages` mode is **not** re-synthesized here to avoid duplicate deliveries
  - `"messages-tuple"` ? per-chunk update: for AI text this is a **delta** (concat per `id` to rebuild the full message); tool calls and tool results are emitted once each
  - `"custom"` ? forwarded from `StreamWriter`
  - `"end"` ? stream finished (carries cumulative `usage` counted once per message id)
- Agent created lazily via `create_agent()` + `_build_middlewares()`, same as `make_lead_agent`
- Supports `checkpointer` parameter for state persistence across turns
- `reset_agent()` forces agent recreation (e.g. after memory or skill changes)
- See [docs/STREAMING.md](docs/STREAMING.md) for the full design: why Gateway and DeerFlowClient are parallel paths, LangGraph's `stream_mode` semantics, the per-id dedup invariants, and regression testing strategy

**Gateway Equivalent Methods** (replaces Gateway API):

| Category | Methods | Return format |
|----------|---------|---------------|
| Models | `list_models()`, `get_model(name)` | `{"models": [...]}`, `{name, display_name, ...}` |
| MCP | `get_mcp_config()`, `update_mcp_config(servers)` | `{"mcp_servers": {...}}` |
| Skills | `list_skills()`, `get_skill(name)`, `update_skill(name, enabled)`, `install_skill(path)` | `{"skills": [...]}` |
| Memory | `get_memory()`, `reload_memory()`, `get_memory_config()`, `get_memory_status()` | dict |
| Uploads | `upload_files(thread_id, files)`, `list_uploads(thread_id)`, `delete_upload(thread_id, filename)` | `{"success": true, "files": [...]}`, `{"files": [...], "count": N}` |
| Artifacts | `get_artifact(thread_id, path)` ? `(bytes, mime_type)` | tuple |

**Key difference from Gateway**: Upload accepts local `Path` objects instead of HTTP `UploadFile`, rejects directory paths before copying, and reuses a single worker when document conversion must run inside an active event loop. Artifact returns `(bytes, mime_type)` instead of HTTP Response. The new Gateway-only thread cleanup route deletes `.deer-flow/threads/{thread_id}` after LangGraph thread deletion; there is no matching `DeerFlowClient` method yet. `update_mcp_config()` and `update_skill()` automatically invalidate the cached agent.

**Tests**: `tests/test_client.py` (77 unit tests including `TestGatewayConformance`), `tests/test_client_live.py` (live integration tests, requires config.yaml)

**Gateway Conformance Tests** (`TestGatewayConformance`): Validate that every dict-returning client method conforms to the corresponding Gateway Pydantic response model. Each test parses the client output through the Gateway model ? if Gateway adds a required field that the client doesn't provide, Pydantic raises `ValidationError` and CI catches the drift. Covers: `ModelsListResponse`, `ModelResponse`, `SkillsListResponse`, `SkillResponse`, `SkillInstallResponse`, `McpConfigResponse`, `UploadResponse`, `MemoryConfigResponse`, `MemoryStatusResponse`.

## Development Workflow

### Test-Driven Development (TDD) ? MANDATORY

**Every new feature or bug fix MUST be accompanied by unit tests. No exceptions.**

- Write tests in `backend/tests/` following the existing naming convention `test_<feature>.py`
- Run the full suite before and after your change: `make test`
- Tests must pass before a feature is considered complete
- For lightweight config/utility modules, prefer pure unit tests with no external dependencies
- If a module causes circular import issues in tests, add a `sys.modules` mock in `tests/conftest.py` (see existing example for `deerflow.subagents.executor`)

```bash
# Run all tests
make test

# Run a specific test file
PYTHONPATH=. uv run pytest tests/test_<feature>.py -v
```

### Running the Full Application

From the **project root** directory:
```bash
make dev
```

This starts all services and makes the application available at `http://localhost:2026`.

**All startup modes:**

| | **Local Foreground** | **Local Daemon** | **Docker Dev** | **Docker Prod** |
|---|---|---|---|---|
| **Dev** | `./scripts/serve.sh --dev`<br/>`make dev` | `./scripts/serve.sh --dev --daemon`<br/>`make dev-daemon` | `./scripts/docker.sh start`<br/>`make docker-start` | ? |
| **Prod** | `./scripts/serve.sh --prod`<br/>`make start` | `./scripts/serve.sh --prod --daemon`<br/>`make start-daemon` | ? | `./scripts/deploy.sh`<br/>`make up` |

| Action | Local | Docker Dev | Docker Prod |
|---|---|---|---|
| **Stop** | `./scripts/serve.sh --stop`<br/>`make stop` | `./scripts/docker.sh stop`<br/>`make docker-stop` | `./scripts/deploy.sh down`<br/>`make down` |
| **Restart** | `./scripts/serve.sh --restart [flags]` | `./scripts/docker.sh restart` | ? |

**Nginx routing**:
- `/api/langgraph/*` ? Gateway embedded runtime (8001), rewritten to `/api/*`
- `/api/*` (other) ? Gateway API (8001)
- `/` (non-API) ? Frontend (3000)

### Running Backend Services Separately

From the **backend** directory:

```bash
# Gateway API
make gateway
```

Direct access (without nginx):
- Gateway: `http://localhost:8001`

### Browser CORS

The Gateway always permits local Vite origins on ports 5173, 5174, 3000, and
8080 for both `localhost` and `127.0.0.1` with credentials. A deployed
frontend on another origin must be listed exactly in `GATEWAY_CORS_ORIGINS`
(for example `http://47.88.25.99:7010`); explicit origins are merged with the
local development origins. Regression: `tests/test_taskcop_cors.py`.

### Frontend Configuration

The frontend uses environment variables to connect to backend services:
- `NEXT_PUBLIC_LANGGRAPH_BASE_URL` - Defaults to `/api/langgraph` (through nginx)
- `NEXT_PUBLIC_BACKEND_BASE_URL` - Defaults to empty string (through nginx)

When using `make dev` from root, the frontend automatically connects through nginx.

## Key Features

### Administrator-managed custom themes

`system_settings.json` → `appearance` now stores `custom_themes` alongside
the built-in default theme. A row is `CustomThemePreset` (`id`, `name`,
six-digit `primary`, optional primary-button `text_color`, `mode`, `overrides`, `enabled`,
`sort_order`). `overrides` is intentionally a short interaction-only list:
`button_hover`, `navigation_gradient_start`, `navigation_gradient_end`,
`selected_background`, `border`, `muted`, `muted_text`, `destructive`,
`brand_title`, and `primary_foreground`. The navigation gradient endpoints
default from `primary` and are only stored when an administrator overrides them.
All values must be six-digit hex strings; the frontend derives every remaining
semantic CSS/TDesign token so admins do not maintain a full raw token sheet.
Ids must be `custom-theme-*`, names and ids are unique, and the router limits
the list to 12. `GET/PUT /api/system-settings/appearance` is the single
read/write surface: reads are available to all users, writes require admin.
The router re-orders rows deterministically and only permits an enabled custom
theme as `default_theme`; removing or disabling the default safely falls back
to `light`. Regression coverage: `tests/test_appearance_theme.py`.

Compact top-bar shortcuts live in the same `appearance` setting. Their route
parameters support `login`, `theme`, and keyless `path`. A `path` parameter has
no query-key input; the operator selects its value source from the right-hand
dropdown and the frontend appends the resolved, URL-encoded value to the
external URL path (or Hash route) in configured order. Password stays an
ordinary `login` query parameter with the key `password`, so an external login
link can become `#/login/<user>?password=…`. Regression coverage:
`tests/test_appearance_theme.py`.

### File Upload

Multi-file upload with automatic document conversion:
- Endpoint: `POST /api/threads/{thread_id}/uploads`
- Supports: PDF, PPT, Excel, Word documents (converted via `markitdown`)
- Rejects directory inputs before copying so uploads stay all-or-nothing
- Reuses one conversion worker per request when called from an active event loop
- Files stored in thread-isolated directories
- Agent receives uploaded file list via `UploadsMiddleware`

See [docs/FILE_UPLOAD.md](docs/FILE_UPLOAD.md) for details.

### Plan Mode

TodoList middleware for complex multi-step tasks:
- Controlled via runtime config: `config.configurable.is_plan_mode = True`
- Provides `write_todos` tool for task tracking
- One task in_progress at a time, real-time updates

See [docs/plan_mode_usage.md](docs/plan_mode_usage.md) for details.

### Context Summarization

Automatic conversation summarization when approaching token limits:
- **On/off switch is per-request, not the config file.** The chat input box has a
  「上下文压缩」 toggle (below 参考文献, **default off**) → frontend sends
  `context.summarization_enabled` → whitelisted in `_CONTEXT_CONFIGURABLE_KEYS`
  (`app/gateway/services.py`) → `_build_middlewares` reads it from the run config
  and calls `_create_summarization_middleware(force_enabled=summarization_enabled)`.
  `force_enabled=True` mounts the middleware even when `config.summarization.enabled`
  is false; when the flag is absent (channels / scheduled runs) it falls back to
  the config flag. `config.summarization.enabled` now defaults to **false**.
- `config.yaml` `summarization` still supplies the **tuning**: trigger types
  (tokens, messages, or fraction of max input) and the keep policy.
- A compressed request must retain the latest non-empty `HumanMessage`, even
  when it has already been folded into the hidden summary. This keeps strict
  OpenAI-compatible gateways from rejecting a tool-continuation call with
  `No user query found in message`.
- Keeps recent messages while summarizing older ones.

Tests: `tests/test_summarization_toggle.py` (gating + whitelist),
`tests/test_summarization_middleware.py` (cache behavior).
See [docs/summarization.md](docs/summarization.md) for details.

### Vision Support / Image Recognition (??)

Two paths, selected by `config.yaml` ? `vision`:

**Dedicated vision model (delegated recognition)** ? set `vision.model_name` to a model from `models[]`. The `view_image` tool sends the image to that model and returns its **text description** as the tool result, so a text-only main chat model can still "see" images. `view_image` accepts an optional `question` argument for targeted queries; the default prompt is `vision.prompt` (overridable), and `vision.max_tokens` caps the recognition output. Resolution/gating lives in `deerflow/models/vision.py` (`resolve_vision_model_name`, `vision_enabled`, `is_delegated_vision`, `recognize_image`).

**Native vision (fallback)** ? when `vision.model_name` is unset, behavior falls back to the main chat model, which only works if it has `supports_vision: true`:
- `ViewImageMiddleware` injects base64 image data into the conversation before the LLM call
- `view_image` stores the image in `state.viewed_images` for that middleware

Both the `view_image` tool and `ViewImageMiddleware` are registered whenever `vision_enabled(model_name)` is true (dedicated model configured **or** main model supports vision). Config schema: `deerflow/config/vision_config.py` (`VisionConfig`); tests: `tests/test_vision.py`.

### Upload symlink-safety (cross-platform)

`deerflow/uploads/manager.py:open_upload_file_no_symlink` writes uploads without following a pre-existing destination symlink. `O_NOFOLLOW` is applied where available (POSIX) but is **optional** ? Windows lacks `os.O_NOFOLLOW`/`O_NONBLOCK`, so requiring it previously made every Windows upload fail with `UnsafeUploadPathError`. Pre-existing symlinks are still rejected via `lstat`, and the opened fd is re-checked with `fstat`. `O_BINARY` is added on Windows. Regression: `tests/test_uploads_windows_safe.py`.

## Code Style

- Uses `ruff` for linting and formatting
- Line length: 240 characters
- Python 3.12+ with type hints
- Double quotes, space indentation

## Documentation

### WeKnora iframe behind `/deerflow`

The LLMWiki WeKnora detail iframe is publicly served through the Gateway at
`/deerflow`, independently of the frontend deployment prefix (for example
`/magentweb`). Register `llmwiki.proxy_router` with the `/deerflow` prefix
before its unprefixed fallback router, otherwise the unprefixed catch-all
captures `/deerflow/*`. HTML root-relative assets, API strings, and Vite's
`__vite__mapDeps` table are rewritten for `/deerflow`; generic JavaScript
regular-expression literals must not be rewritten. Auth and CSRF bypass only
applies when the short-lived signed WeKnora embed cookie is valid. Keep static
path recognition in `app/gateway/weknora_embed.py` synchronized with the
proxy's static fallback paths in `app/gateway/routers/llmwiki.py`.

### Position-roundtable role directory

`position_roles` is the global shared directory for position-roundtable work
views. It is deliberately separate from organisation/RBAC `positions`.
`app.gateway.deps.langgraph_runtime()` seeds the four stable built-in ids from
`app/gateway/position_role_defaults.py` via `ensure_seed()` (missing rows only;
never overwrite operator edits). The API is `GET/PUT /api/position-roles`;
writes are admin-only. Exactly one enabled `intent` role is required and owns
the initial intent workspace plus the summary/action-plan closing agents. Do
not delete a built-in id because historical chain nodes persist those ids;
custom role ids may be added and removed.

### Position-roundtable task-scoped session history

`position_roundtable_sessions.external_task_id` is optional. When populated from the frontend route `taskId`, new rows set `task_scoped = true`: a task can have multiple sessions and all users opening that task see the same sessions, nodes and persisted conversation snapshots. When it is NULL/empty, keep the original `user_id`-scoped personal workspace behavior. `20260808_01` adds `task_scoped` with default `false` (so all historical records remain personal) plus the task/update-time index; it must not backfill or make `external_task_id` non-null.

### Roundtable artifact submission status

`roundtable_artifact_submissions` keeps task-level submission state for a
generated deliverable and is deliberately separate from task details and any
external task-system delivery. The API is
`GET /api/roundtable/tasks/{taskId}/artifacts?submitted=true` and
`PUT /api/roundtable/tasks/{taskId}/artifacts/submission`. All callers use the
normal Gateway Bearer authentication; records are shared by task while
`created_by`/`submitted_by`/`cancelled_by` are audit-only. The server derives a
SHA-256 idempotency key from `taskId + sourceKey + path + revision`: submitting
the same revision is an upsert, while cancellation only flips `submitted` and
sets `cancelled_at` (it never deletes history). Store only the sandbox file
reference (`threadId`, path, source/node/session fields, name/MIME and revision)
and timestamps; do not persist Markdown content or call a formal delivery
interface. Previews continue to read the DeerFlow artifact by `threadId + path`.

See `docs/` directory for detailed documentation:
- [../README.md](../README.md) - Offline Docker wheelhouse preparation and startup workflow. The pinned versions in `../offline-runtime-requirements.txt` and `../start_backend.sh` must stay synchronized. Startup checks `croniter`, `hindsight-client`, the aiomysql compatibility bundle, and Redis, installing missing or incompatible packages from wheelhouse only. The bundle installs `aiomysql==0.3.2`, `PyMySQL==1.2.0`, and `SQLAlchemy==2.0.51` together; the startup probe also verifies that SQLAlchemy's adapted `ping(reconnect=False)` signature is present, preventing the old-adapter missing-`reconnect` failure. The download target defaults to CPython 3.12 on Linux x86_64 and verifies dependency closure using wheelhouse only.
- [CONFIGURATION.md](docs/CONFIGURATION.md) - Configuration options
- [ARCHITECTURE.md](docs/ARCHITECTURE.md) - Architecture details
- [API.md](docs/API.md) - API reference
- [SETUP.md](docs/SETUP.md) - Setup guide
- [FILE_UPLOAD.md](docs/FILE_UPLOAD.md) - File upload feature
- [PATH_EXAMPLES.md](docs/PATH_EXAMPLES.md) - Path types and usage
- [summarization.md](docs/summarization.md) - Context summarization
- [plan_mode_usage.md](docs/plan_mode_usage.md) - Plan mode with TodoList
- [BACKEND_RUN_STOP_ZH.md](docs/BACKEND_RUN_STOP_ZH.md) - Windows-first backend start/stop and troubleshooting guide

### Deep Research retrieval

`deerflow.agents.deep_research.adapters.material_provider.DeerFlowMaterialProvider`
must remain on the Q&A retrieval path. Enabled `deep-search`, `web-research`,
and knowledge-base skills run through the existing `SubagentExecutor` skill
runtime only when their required Q&A tool is registered; then the provider may
use configured `web_search`. A disabled placeholder endpoint is not a valid
search channel: use the maintained keyless DuckDuckGo provider and a relevant
real-web fallback as read-only sources, keep source provenance, and never
enable or call a placeholder just to make the Deep Research UI look configured. Regression coverage is in
`tests/test_deep_research_material_provider.py`.
When a queued Deep Research job is cancelled, the Gateway must also clear the
owning session's `active_job_id` and mark it `cancelled`; otherwise the stale
session incorrectly consumes that user's concurrency limit.

Report prose is the one Deep Research LLM path that must use
`DeerFlowCompletionBackend.stream_complete()` rather than `complete()`: it
drives `model.astream()`, sends native `report_delta` frames through
`ReportChunkEmitter`, and persists bounded `report_chunk` checkpoints. The
app-layer `DeepResearchLiveHub` fans out those live frames inside one Gateway
worker only; the database event log still owns `?after=<seq>` reconnect replay
and multi-worker recovery. Inject the hub callback into `PersistingEventSink`
and never import `app.*` from `deerflow.*`. Keep structured plan/review calls
on `complete()`. Before the search fan-out, basic mode emits one
`queries_planned` event and detailed/multi-agent modes emit one per subtopic;
they are durable user-visible search terms, not chain-of-thought. On completion, retain
the terminal `lastJobId` in `usage_snapshot` so a refreshed conversation page
can replay its trace from the event log. Detailed mode uses stable targets (`introduction`,
`section:N`, `conclusion`) because section writers are concurrent;
multi-agent revisions must emit `report_reset` before their next report
version. Do not persist raw model reasoning or one persisted event per provider
token. Models that expose thinking inside ordinary `<think>...</think>` text
must be split statefully before a report delta, checkpoint, or Markdown
projection is created; an optional live-only `report_thinking` frame can drive
the message-list thought trace, while the sandbox receives answer Markdown
only. The post-report digest (`summarize_report`) must likewise forward
provider reasoning through live-only `summary_thinking` frames so the
chat timeline can render thought while waiting for the first `summary_delta`.
The Vite page starts Word downloads through
`frontend-web/src/core/scheduled-tasks/markdown-docx.ts`, but the Gateway owns
generation through `POST /api/writing/export/docx`. Keep the standard-library
OOXML builder and its embedded-font validation; do not add a Python document-
conversion dependency merely to export text.
All non-streaming Deep Research control-plane calls (`complete()` for query /
section / plan / review JSON) run through `asyncio.wait_for` using the frozen
`DeepResearchConfig.control_plane_timeout_seconds` (45 seconds by default,
clamped to 120). Do not apply that total-duration cap to `stream_complete()`:
reports may legitimately be long but must continue emitting visible deltas. A
basic query-planning timeout/failure emits `query_planning_fallback` and searches
the original topic; detailed and multi-agent plan creation similarly fall back
to their original topic/section, rather than leaving a durable job stuck in
`planning`. Tests: `tests/test_deep_research_llm_adapter.py` and
`tests/test_deep_research_runners.py`.
The per-user Deep Research concurrency guard counts only session statuses
`running` and `awaiting_input`. A `draft` has no durable job and must never
consume a slot; otherwise saved drafts make `POST /sessions/{id}/jobs` return
429 permanently. Regression: `test_saved_draft_sessions_do_not_consume_research_concurrency`.

`POST /api/deep-research/sessions/{id}/jobs` is retry-safe under MySQL
read/write splitting. Deep Research session/job/event/source/message writes
must serialise their changed row while the write transaction is still open
(`flush` for inserts, select-after-update **before** commit), never through a
post-commit `session.refresh()` or a fresh pooled read. Replica lag otherwise
turns a committed job into a misleading 500, can make dispatcher `claim_job()`
lose its snapshot, or can fail report/follow-up persistence after the model has
already completed. Always nudge the dispatcher for both a newly created job and
an idempotently reused queued job; otherwise a retry after such an interrupted
response returns the old job but leaves it waiting for the periodic scan. The
SSE replay generator must catch temporary event/job-store reads, emit a
recoverable warning, and keep the connection open so the next poll can repair
the cursor instead of surfacing an ASGI `ExceptionGroup`. Regressions:
`test_deep_research_writes_never_depend_on_post_commit_refresh`,
`test_job_create_never_reads_back_after_commit`,
`test_retry_of_existing_queued_job_nudges_dispatcher`, and
`test_stream_stays_open_when_durable_replay_has_a_transient_failure`.

At ordinary report completion, persist and confirm the session's
`report_markdown` projection **before** marking the Job `completed`. Retry a
short transient projection failure; if it cannot be confirmed, let the job
become failed rather than reporting a report that vanishes on refresh.
`test_report_is_persisted_before_job_is_marked_completed` and
`test_report_projection_failure_never_marks_job_completed` cover the ordering.

The chat-collection frontend must keep report-writing progress in the existing
workspace `MessageList` / `MessageGroup` tool-step renderer: inject only
virtual `deep_research_progress` tool calls for the durable phases, keep the
completed phase visible after the job ends, and append the digest as a normal
assistant message. Do not recreate a second deep-research-only conversation
timeline. On `summarizing`, insert that same normal assistant message with its
streaming marker and update it for every SSE `summary_delta`; when the report
completes, replace its content in place with the final completion/digest
message rather than waiting to append a summary. Completed chat-collection
sessions also inject a normal `present_files` card into the transcript, named
from the report Markdown H1 rather than its internal `report.md` path.
Replaying a completed history session must not auto-open its
sandbox; opening that card selects its virtual write-file report instead, and
its file-card download action delegates to the Deep Research Markdown/Word
export dialog. The default in-job concurrency and per-user active-job cap are
both three. Before the user confirms the report configuration, derive its
displayed material count from unique collector tool-result rows because the
durable Deep Research source store is populated when the writing job harvests
that thread. The Deep Research composer model picker must use the installed
TDesign `Select` backed by `/api/models`; its explicit choice is sent as
`model_name` on collector turns and applied to `fast_model`, `smart_model`, and
`strategic_model` when creating a session or starting a report job. Disable it
while collection or report writing is active so one durable run cannot switch
models mid-stream.

Report structures (`/api/report-structures`) may store an optional
`retrieval_directions` list (检索方向) and an optional `retrieval_skills`
list (检索来源 / 指定技能). Empty / unset keeps today's collector
query-planning, default tools, and runner sub-query generation. A non-empty
directions list is frozen into `DeepResearchConfig.retrieval_directions`: the
chat collector's first user message gets a `<retrieval-directions>` block that
asks it to **split concrete retrievable terms along those facets** (never search
the direction labels themselves), and legacy `basic`/`detailed`/`multi_agent`/
`deep` runners pass the same constraint into query-planning instead of using
`课题 + 方向` as the search string. A non-empty skills list is frozen
into `DeepResearchConfig.retrieval_skills`: the chat collector's first user
message gets a `<retrieval-skills>` block, the collector run context receives
`extra_skills` so `skill_view`/`skill_list` can see those names, and legacy
material collection runs those skills instead of the default Q&A research-skill
trio. Helpers live in `deerflow.agents.deep_research.retrieval`.

Completed-report Q&A has two endpoints: retain `POST /sessions/{id}/chat` as the
legacy JSON response for compatibility, and use `POST /sessions/{id}/chat/stream`
for report-grounded Q&A SSE. The workbench must send a strong scoped edit
instruction (for example “修改第一段” / “重新生成一下第一段” / “重写第二节”) to
the separate `POST /sessions/{id}/rewrite/stream` report-section writing pipeline.
That endpoint requires a deterministic current-report range (otherwise 422), so
its output is always a replacement candidate rather than a Q&A answer. The
candidate is streamed in the ordinary message row
while the sandbox preserves the current report. When generation ends, inject a
confirmation card. Only the owner’s explicit
`POST /sessions/{id}/messages/{messageId}/rewrite-proposal` action `apply` may
persist the assembled `report_markdown`; `dismiss` preserves the original. The
proposal is bound to a hash of the report it was generated from, so stale
candidates fail with 409 rather than overwrite a later edit. Ambiguous or
ordinary evidence questions must remain non-mutating, and cancellation or a
stream error must never save a partial report. The rewrite prompt receives only
the selected range plus the selected evidence set, never a new web search.

### DeerFlow-local LLMWiki Wiki index

`llmwiki.local_wiki_index` is an opt-in local-vector subsystem. WeKnora is the
Wiki generation/sync source and the Wiki-only fallback while an entire mapping
has not completed vectorization. The readiness gate requires `state=idle`, a
successful full scan, matching embedding fingerprint, no failed pages,
`ready_page_count == local_page_count`, and vectors for a non-empty library.
Ready mappings read DeerFlow's `llmwiki_wiki_pages` /
`llmwiki_wiki_vectors` / `llmwiki_wiki_sync_states` tables; incomplete mappings
call only WeKnora `/knowledgebase/{id}/wiki/pages?query=...` plus Wiki-page
detail. Never call `/knowledge-search` from internal search, RAG, or the
`llmwiki_search` tool, and never return raw chunks as the fallback. Core code lives under
`deerflow.integrations.weknora.local_index` and
`deerflow.persistence.llmwiki_index`; preserve the harness → app import
firewall. Gateway composition injects the store/search/sync services through
`local_index.runtime`. `enabled` turns on local vector search and `auto_sync`
defaults to the lease-protected background scheduler. The scheduler performs
an immediate incremental scan at startup, then rescans every configured
interval so new WeKnora libraries and asynchronously generated Wiki pages are
picked up without a manual vectorization click. Unchanged content must reuse
existing vectors.

Vectors are normalized little-endian float32 BLOBs. A page mirror and all of
its vectors are replaced in one transaction only after strict batched
embedding succeeds. Never delete the old generation before a remote embedding
call and never fall back to WeKnora raw chunks while the local feature flag is
enabled. NumPy matrices are per-mapping LRU snapshots invalidated by
`index_revision`; large decode/matrix work runs through `asyncio.to_thread`.
`embedding.dimensions` is a local response-size invariant and fingerprint
input, not an outbound API field: fixed-dimension BGE/OpenAI-compatible
endpoints can reject a `dimensions` request parameter with HTTP 400. When the
value is omitted, the embedding client probes and freezes the first response's
dimension before sync lease/rebuild decisions.
Embedding errors must retain the upstream status and bounded response body in
logs, without exposing API keys. SQL vector writes are split into statements
of at most `database_batch_size` rows (default 50) inside the same atomic page
replacement transaction.
During a fingerprint rebuild the mapping uses the Wiki-page API until every
page is successfully rebuilt. `WikiRetrievalService` partitions mixed requests
per mapping, so ready libraries stay local while incomplete libraries use that
Wiki-only fallback.

The frontend must keep Wiki citations visibly distinct from raw-document
citations while hiding provider-brand wording from user-facing labels, status
text, and errors. Any click on a Wiki source card opens the shared right-side
source drawer, which renders the complete Markdown page. When the local index
is disabled, the drawer reads the remote WeKnora Wiki-page endpoint directly
and must not surface the local-index-disabled response. A Wiki target must not
also load or render the legacy original-document preview, even when its citation
still carries a document id. The drawer supports single-page Markdown and Word
downloads, and its **Open in Wiki** link carries both the DeerFlow knowledge-base
mapping id and the exact `wikiSlug`. Wiki search-result cards use this same
drawer instead of navigating immediately. Exact-page navigation uses the native
`knowledge/{mapping_id}/wiki/view?wikiSlug=...` reader, not the WeKnora detail
iframe. Keep legacy `knowledge/{mapping_id}?tab=wiki&wikiSlug=...` links fast by
redirecting them in `LlmWikiKnowledgePage` before its runtime-loading gate.
Drawer-to-reader navigation stays in the current SPA and passes the fetched
Wiki page in router state so the article can render before the background query
refresh completes. The native reader must retain a searchable/collapsible left
Wiki directory, reconstruct hierarchy from `parent_slug`, `wiki_path`,
`category_path`, and slug fallback, expand/highlight the active page, and keep
directory navigation on the same lightweight route rather than mounting the
WeKnora iframe.

Generated HashRouter URLs must retain `window.location.pathname` (for example
the `/web/` reverse-proxy prefix); never emit root-absolute `/#/...` citation,
reader, or public-embed links. Protected Wiki image rendering must use real HTTP
URLs, never `URL.createObjectURL`. Public readers use the published file proxy;
authenticated readers obtain a short-lived signed capability from
`/files/access-url` and load `/api/public/llmwiki/files/{token}`. The signed
payload is restricted to `kind=wiki_file`, mapping id, remote id, validated
protected path, and expiry, and responses provide an inline UTF-8 filename for
new-tab viewing and browser Save As. The proxied WeKnora detail iframe must
likewise leave its scoped HTTP file-proxy URL on every protected image and
restore it if upstream rendering replaces `src`; its click preview reads that
same DOM value, so a fetched-and-revoked object URL is not acceptable.

Third-party pages embed published Wiki content through the chrome-free route
`/#/embed/knowledge/{mapping_id}` (optional exact-page `wikiSlug`). It defaults
to a recognized Wiki index page (title `索引`/`index`, page type `index`, or an
`index`/`home`/`readme` slug) and promotes that page to the first public-directory
entry. When upstream pages exist without an explicit index shell, the frontend
injects a virtual `__index__` page and builds its catalogue from the page list;
an all-empty dynamic index response must also fall back to that page list. It retains
directory navigation and supports the shared public-embed `theme` / `bg` /
`text` appearance inputs. WeKnora dynamically renders index page-type counts,
nested category headings, page summaries, and `data-slug` links even though the
stored index Markdown only contains its introduction; the native public reader
reads the same ordered groups through the published-only
`/api/public/llmwiki/knowledge-bases/{id}/wiki/index` proxy and renders them with
`buildWikiIndexCatalogMarkdown`. The Gateway paginates every WeKnora index type
(`summary`, `entity`, `concept`, `synthesis`, `comparison`) to completion while
preserving upstream group/item order. Index Markdown
may also contain WeKnora shell-only placeholder links;
`resolveWikiIndexPageLinks` matches their visible labels to
the loaded page title, slug, or alias and rewrites them to exact embedded
`wikiSlug` routes. Preserve WeKnora's `wiki-content-link` interaction: primary
brand color, medium weight, dashed underline becoming solid on hover, focus
outline, and in-reader SPA navigation rather than a page reload. `GET
/api/public/llmwiki/knowledge-bases` is the anonymous discovery endpoint. It
validates published ordinary mappings against WeKnora, drops stale mappings,
and returns safe comprehensive metadata from the WeKnora list/detail/page APIs:
document/chunk/processing/share counts, non-secret configuration and timestamps,
Wiki page type/status counts and explicit/generated index entry, plus aggregate
local-vector state/counts/timestamps. Never expose remote WeKnora ids,
tenant/creator identity, storage credentials, API credentials, raw vectors, or
per-page indexing errors. All anonymous reader calls use that
public namespace and the `/knowledge-bases/{id}` Wiki-page children. Those
endpoints must resolve with a synthetic anonymous user and `is_admin=False`,
explicitly require `publication_status=published`, and hard-deny the system
conversation-deposit mapping. Never point this page at the authenticated
`/api/llmwiki` reads or weaken the published-only gate merely because the
mapping id is hard to guess.

Owners manually force a full sync/vectorization through `POST
/api/llmwiki/knowledge-bases/{id}/vectorize`; status is available through `GET
/api/llmwiki/knowledge-bases/{id}/wiki-index-status`. That response includes
aggregate `progress` and per-page `pages` details; current-scan progress is
derived from `active_sync_id` / `last_seen_sync_id`, while each page exposes its
index state, vector count, timestamps, and bounded error. The Wiki management
page must show the progress bar and a status badge beside every article. It
also exports all remote processed Wiki pages via `GET
/api/llmwiki/knowledge-bases/{id}/wiki/export.xlsx`. Directory export must keep
the full path, current directory, parent path, and dynamic per-level columns;
long Markdown is split into Excel-safe continuation columns instead of being
truncated. Workbook creation runs in `asyncio.to_thread`.

Internal search authorization always starts from DeerFlow mapping ids and is
rechecked by the Gateway. External endpoints use the isolated
`/api/external/llmwiki/wiki` namespace and `X-API-Key`; require published +
external-enabled + index-enabled on every request and response, and hard-deny
the `system` / `对话沉淀` mapping. Those API-key endpoints never return remote
WeKnora ids, provenance refs, credentials, or vector bytes.

`POST /api/knowledge/vector-search` is a deliberately separate anonymous API.
It has no token/API-key requirement or rate limit, applies route-scoped
`Access-Control-Allow-Origin: *`, searches every published local Wiki index
regardless of `external_search_enabled`, and still hard-denies the `system` /
`对话沉淀` mapping before and after search. Its contract is section-level (one
result per matched vector, so Wiki-page fields and URLs may repeat), default
`top_k=10`, with optional `include_page_content`. Unlike the protected legacy
external namespace, this endpoint intentionally returns DeerFlow/WeKnora ids
and raw source/chunk provenance, but never credentials or vector bytes. Wiki
links use `llmwiki.local_wiki_index.frontend_base_url` (development default
`http://127.0.0.1:5174`) and point to the native
`/#/page/workspace/knowledge/{mapping_id}/wiki/view?wikiSlug=...` route.
Regression tests:
`tests/test_llmwiki_local_index_repository.py`,
`tests/test_llmwiki_wiki_sync.py`, `tests/test_llmwiki_vector_search.py`,
`tests/test_external_llmwiki_search.py`,
`tests/test_public_knowledge_vector_search.py`, and
`tests/test_llmwiki_wiki_retrieval.py`.

The sandbox whole-report rewrite may carry an optional `reportOutline` Markdown
template from the shared report-structure library. Persist the original template
in the durable input snapshot and append it as an explicit chapter-structure
constraint before requirements planning and prose generation; a template must
never be used as permission to invent facts absent from the current report.

`DocumentRewriteVersionRepository.create` backs the reversible before-image for
both artifact and whole-report rewrites. It must flush and serialize the new row
on the writer before commit, and must never call `session.refresh()` after
commit: a read/write-splitting MySQL endpoint can route that refresh to a
lagging replica, falsely report a version-save failure, and trigger the caller's
intentional restore of the rewritten document.

Version snapshots are an undo enhancement, not a condition for preserving an
already committed rewrite. If snapshot creation fails after the artifact/report
write, log the exception, complete the rewrite with
`versionSnapshotFailed: true`, and do not emit a user-facing failure or restore
the original. The sandbox uses that signal only to enable its ordinary manual
save fallback.

### Offline local-sandbox Office runtime

The DeerFlow-only offline deployment uses `LocalSandboxProvider`, so document
dependencies belong in the backend image rather than a separate sandbox image.
`deploy/linux-docker-offline/Dockerfile.backend` installs python-docx,
LibreOffice Writer/Calc/Impress, Pandoc, Poppler, and CJK fonts in addition to
the MarkItDown Office extras. `file_conversion.py` normalizes legacy
DOC/XLS/PPT and ODT/ODS/ODP through a per-conversion LibreOffice user profile
before MarkItDown extraction, avoiding concurrent profile locks. The image
build and `OFFLINE_INSTALL.sh --office-check` both run
`scripts/office_runtime_selfcheck.py`; that smoke test creates and re-parses
minimal DOCX/XLSX/PPTX files and does not need the application database.

## Skill/assistant knowledge invariants

Skill distillation is implemented in `deerflow.skill_knowledge` and
`deerflow.persistence.skill_knowledge`; native assistant bases are in
`deerflow.assistant_knowledge` and
`deerflow.persistence.assistant_knowledge`. Gateway composition, remote adapters,
the lease dispatcher, and APIs remain in `app.gateway` so the harness-to-app
import firewall is preserved.

Never execute a skill or read/upload/summarize its code and script files.
Knowledge fingerprints exclude those files and Markdown code fences. All
resyncs must stage and validate the knowledge digest before activation; the old
snapshot remains authoritative on failure. A binding may have only one active
item (`active_dedupe_key` plus atomic lease claims). Skill distillation targets
only writable WeKnora Wiki knowledge bases in this product version; reject direct
assistant-base targets, non-Wiki/FAQ/conversation/readonly targets, and direct
Neo4j writes. The create/list/resync/retry/detach flow is available to ordinary
users for their own visible skills, writable knowledge bases, jobs and bindings;
administrator-only auto-sync/review APIs remain privileged. Detach/delete operates on source contributions, not canonical rows,
and shared entities/relations are retired only when their final contribution is
gone.

Assistant knowledge has one global full-library base (`知识梳理总库`) and any
number of admin-created custom bases. Ordinary users have read/search access and
administrators own all mutations. Assistant bases accept local documents independently
of WeKnora via `/api/assistant-knowledge/imports/files`, as well as WeKnora export
packages or imports triggered from a WeKnora mapping. Local documents use the
Gateway `assistant_file_ingest` pipeline: persisted original files, chunked LLM Wiki
extraction, cached body embeddings, and import-job status/retry. Do not add direct skill
imports. Importing into a
custom assistant base must also mirror the package into the global base for
cross-source entity alignment. Admins may rename any assistant base and delete
custom bases. Deletion removes the custom base's Wiki pages, vectors, entities,
relations, import audit rows, and package files; the global base is never
deletable because it owns full-library aggregation.

### Wiki package and local encoder invariants (2026-09)

Ordinary knowledge detail landing is globally configured by `ordinary_knowledge_landing` (`wiki` default / `weknora`), with admin-only PUT `/api/system-settings/ordinary-knowledge-landing`. Explicit document/graph/article links bypass this default. Wiki reading and the embedded WeKnora view provide reciprocal navigation; `?view=weknora` explicitly selects the iframe and must not loop back to the default Wiki route.

- `knowledge_transfer.sync_to_global` is called after successful explicit local
  Wiki vectorization; it skips conversation/system mappings. Do not enable a new
  watcher or automatic remote-write scheduler as a side effect.
- `wiki_packages` exposes file-backed assistant exports and restores into empty
  ordinary Wiki libraries. Validate vectors before any remote mutation; recreate
  folder IDs from category paths using provider APIs (a raw category_path is
  discarded by WeKnora without a matching folder_id). Preserve page metadata and
  rebuild backlinks after restoring all pages.
- `PackageArchive` is a replayable, disk-backed JSONL staging index. Always close
  it in finally. Runtime imports must not use the legacy in-memory read_package.
- Preserve normalized float32 base64 bytes exactly. Same dimension alone is not
  model compatibility. Assistant vector retrieval is fingerprint-gated cosine
  search, and any keyword fallback must be labeled as such.
  Preserve a valid zero section_index. Export/search vectors belonging to each
  current source snapshot (`source.last_import_job_id`), not merely the page's
  current revision: one aligned page may intentionally contain current vectors
  from several sources.
- The `local` embedding provider uses FastEmbed with official BAAI/bge-m3 ONNX,
  1024 dimensions, bounded CPU batches and a per-client async lock. Run inference
  off the event loop. Existing page vectors are reusable; zero-vector nonempty
  pages need backfill, not a forced rebuild of every page.
  FastEmbed is a standard backend dependency. Encoder configuration, directory
  upload and warmup live only in the administrator global settings dialog.
  A runtime `system_settings.json` path override wins over config.yaml; uploaded
  files are streamed into `.deer-flow/models/bge-m3`, never the source tree.
  Encoder warmup is explicitly started from the settings UI/API and must never block application startup
  or turn a model-load failure into a DeerFlow service failure. Local provider
  is offline-only: require local_model_path or DEERFLOW_BGE_M3_MODEL_PATH and
  never let the application download model weights. Keep
  `local_process_isolation=true` by default: inference runs in the bundled JSONL
  worker subprocess, and shutdown must terminate that worker without waiting on
  model teardown.
- Every successful mapping snapshot is mirrored to the global assistant base,
  including empty snapshots so deleted source pages are retired. Re-import is
  source-replacement semantics: absent Wiki/chunk/entity/relation pages must not
  remain visible merely because they existed in an older source snapshot.
- Provider graph nodes bind to their original wiki_slug. They must not create
  duplicate synthetic entity/relation articles. Store original Wiki metadata
  separately from assistant workflow status. Count Wiki pages separately from
  raw document chunks.
- The global base aligns entities by normalized display name and aliases across
  source records, and aligns content Wiki pages by normalized title. Do not use
  source slug as the global canonical identity. Merge unique knowledge blocks
  while retaining source contributions/origins. Uploaded packages must rebind
  the temporary upload source to the stable WeKnora knowledge-base id in the
  package, so re-upload is source-replacement rather than append. Retrieval
  candidates exclude summary/index/directory pages; keyword matching, vector
  snippets, citations and prompts must use knowledge body/sections, never page
  summary as a substitute.
- Migration `20260903_01` adds nullable page metadata and merges the pre-existing
  knowledge/workflow Alembic heads. Regression coverage is in
  `tests/test_wiki_package_roundtrip.py`, `test_llmwiki_wiki_sync.py` and
  `test_assistant_knowledge_repository.py`.

## openGauss / GaussDB persistence invariant

`database.backend=gauss` is an application-ORM backend, parallel to `mysql`.
It normalizes `database.gauss_url` to `opengauss+asyncpg://` and requires the
`gauss` optional dependency (`asyncpg` plus `opengauss-sqlalchemy`). Never route
a Gauss URL through SQLAlchemy's generic PostgreSQL dialect: the wire protocol
may connect, but dialect initialization and schema DDL can fail against the
Gauss server/version behavior. Keep the LangGraph checkpointer on the separately
configured SQLite backend; the PostgreSQL checkpointer schema is not part of the
Gauss compatibility contract. Treat both `postgresql` and `opengauss` dialect
names as timezone-aware PostgreSQL-family engines in portable types and
dialect-specific session settings.