Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9028cb342d | ||
|
|
8d43f4c372 | ||
|
|
e8ba63e087 | ||
|
|
309ed277fe | ||
|
|
2ef31f6115 | ||
|
|
159a9a6a8b | ||
|
|
7ed181e682 | ||
|
|
203f81334a | ||
|
|
5c7f8e15c9 | ||
|
|
f75d9555e0 | ||
|
|
104e6aa396 | ||
|
|
deb84c4069 | ||
|
|
1c50cc5bcb | ||
|
|
406118bba6 | ||
|
|
faac60b1a6 | ||
|
|
dec3cd1236 | ||
|
|
38de3de0a3 | ||
|
|
b3df070481 | ||
|
|
2919229c73 | ||
|
|
4bab921d14 | ||
|
|
cf2aad28ec | ||
|
|
814e75730a | ||
|
|
b3555d2603 | ||
|
|
a4264326bd | ||
|
|
4002f096f4 | ||
|
|
4083dceba3 | ||
|
|
41329943c4 |
@@ -0,0 +1,23 @@
|
||||
# 小白复盘仓库执行约束
|
||||
|
||||
本文件对仓库内所有后续编码任务生效。任何智能体在修改文件前必须完整读取:
|
||||
|
||||
1. `docs/migration/原版保真迁移总纲.md`
|
||||
2. `docs/migration/保真迁移状态.json`
|
||||
3. `docs/migration/next失败冻结记录.md`
|
||||
4. 与本次功能有关的原版源码、页面和测试
|
||||
|
||||
## 不可违反
|
||||
|
||||
- 当前根目录原版是唯一功能、视觉、交互、动画和计算基线。
|
||||
- `next/`是失败冻结实现,禁止部署、继续开发或作为新迁移代码来源。
|
||||
- 后续迁移是原代码保真式整理,不是重写、重新设计或更换技术栈。
|
||||
- 不得根据规格说明书重新实现已经存在的功能;规格书只用于盘点,冲突必须交给用户裁决。
|
||||
- 不得改变用户可观察行为。源码可以移动、拆分和调整引用,但输出必须等价。
|
||||
- 不确定是否有用的代码默认保留。没有引用扫描、运行证据和新旧对比,不得删除。
|
||||
- 每次只处理一个完整纵向功能切片,并同步更新迁移账本和状态文件。
|
||||
- 每个切片必须具有原版基线、新版结果、API/数据库对比、页面与交互对比及Git回档点。
|
||||
- 不以新实现自身测试通过、目录更整齐或代码行数减少证明迁移成功。
|
||||
- 未经用户人工确认,不得宣称视觉等价、完成迁移、切换Docker/NAS或删除原版。
|
||||
|
||||
如果任务要求与以上约束冲突,停止迁移并向用户说明冲突,不自行选择新产品行为。
|
||||
@@ -0,0 +1,19 @@
|
||||
.git
|
||||
.gitignore
|
||||
.codex
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.log
|
||||
data/cache/
|
||||
data/private-mentor-skills/
|
||||
data/*.db
|
||||
data/*.db-shm
|
||||
data/*.db-wal
|
||||
tests/
|
||||
Dockerfile*
|
||||
compose*.yml
|
||||
compose*.yaml
|
||||
DOCKER_DEPLOY.md
|
||||
@@ -0,0 +1,21 @@
|
||||
# Generated automatically when omitted. Back it up together with the database.
|
||||
APP_ENCRYPTION_KEY=
|
||||
|
||||
# Initial shared market-data credential. After first launch it is encrypted into
|
||||
# the system settings; all accounts use the same backend market snapshot.
|
||||
TUSHARE_TOKEN=your_tushare_token_here
|
||||
|
||||
# Optional iFinD HTTP credential. The backend exchanges it for a short-lived
|
||||
# access token and never exposes either token to browsers.
|
||||
IFIND_REFRESH_TOKEN=your_ifind_refresh_token_here
|
||||
|
||||
# Initial platform member models (OpenAI-compatible). After first launch these
|
||||
# are encrypted into system settings and used only by admins and active members.
|
||||
LLM_PRIMARY_BASE_URL=https://api.openai.com/v1
|
||||
LLM_PRIMARY_MODEL=your_primary_model
|
||||
LLM_PRIMARY_API_KEY=your_primary_api_key
|
||||
|
||||
# Optional fallback model. It is used only when the primary model fails.
|
||||
LLM_FALLBACK_BASE_URL=https://api.openai.com/v1
|
||||
LLM_FALLBACK_MODEL=your_fallback_model
|
||||
LLM_FALLBACK_API_KEY=your_fallback_api_key
|
||||
@@ -0,0 +1,25 @@
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
__pycache__/
|
||||
data/cache/
|
||||
data/private-mentor-skills/
|
||||
data/*.db
|
||||
data/*.db-shm
|
||||
data/*.db-wal
|
||||
data/backups/
|
||||
data/*.bak
|
||||
data/*.backup
|
||||
*.log
|
||||
*.pyc
|
||||
.coverage
|
||||
htmlcov/
|
||||
.pytest_cache/
|
||||
test-results/
|
||||
playwright-report/
|
||||
node_modules/
|
||||
next/.venv/
|
||||
next/data/
|
||||
next/frontend/dist/
|
||||
next/frontend/.vite/
|
||||
next/frontend/coverage/
|
||||
@@ -0,0 +1,82 @@
|
||||
# Candidate architecture
|
||||
|
||||
`app/` is the behavior-preserving modular source tree accepted by the user on 2026-08-01.
|
||||
The original `webapp/` runtime remains the deployment rollback baseline until an explicitly
|
||||
approved switch. `next/` is a rejected, frozen implementation and is not a source for this
|
||||
directory.
|
||||
|
||||
The application deliberately remains a modular monolith: one Python process, one SQLite WAL
|
||||
database, and a build-free HTML/CSS/JavaScript client. The migration changed source ownership
|
||||
and imports, not the technology stack or observable product behavior.
|
||||
|
||||
## Runtime path
|
||||
|
||||
```text
|
||||
browser
|
||||
-> frontend/shared/api.js
|
||||
-> backend HTTP transport and feature HTTP mixins
|
||||
-> feature services
|
||||
-> repositories / DataGateway / LLMGateway
|
||||
-> SQLite / market providers / model providers
|
||||
|
||||
background scheduler
|
||||
-> backend/jobs
|
||||
-> the same feature services and repositories
|
||||
```
|
||||
|
||||
## Source ownership
|
||||
|
||||
- `server.py` is the stable command/import facade. Runtime composition lives in
|
||||
`backend/application.py` and `backend/bootstrap/`.
|
||||
- `backend/bootstrap/` owns process configuration, dependency construction, startup, and
|
||||
shared input/display-format contracts. It does not own feature behavior.
|
||||
- `backend/http/` owns common authentication, request IDs, JSON/NDJSON responses, static
|
||||
delivery, streaming connection lifecycle, and error normalization. Feature-specific
|
||||
transport handlers live beside their feature.
|
||||
Exact POST endpoints that only delegate to one of those handlers use the explicit maps in
|
||||
`backend/application.py`; endpoints with path parameters, body handling, or special error
|
||||
semantics remain visible control flow in `RequestHandler`.
|
||||
- `backend/features/<feature>/` owns the mechanically moved service, repository, HTTP, agent,
|
||||
or deterministic calculation code for that product area.
|
||||
- `backend/data/` owns provider construction, source policy, provenance, units, freshness,
|
||||
coverage, display-versus-calculation eligibility, and shared numeric normalization policies.
|
||||
- `backend/database/` owns connection management, ordered migrations, and narrow repository
|
||||
adapters. Root `database.py` remains the legacy schema/composition anchor and combines the
|
||||
feature repository mixins; do not add feature queries to it.
|
||||
- `backend/jobs/` owns job definitions, locks, retries, idempotency, and persisted run state.
|
||||
- `backend/llm/` owns model selection, membership/quota checks, fallback, provider transport,
|
||||
streaming rules, and call audit. Feature agents only prepare messages and interpret
|
||||
feature-specific results.
|
||||
- `frontend/shared/` is the only browser API/state/Shell/component boundary.
|
||||
- `frontend/pages/` owns page-local behavior. The original runtime was split mechanically;
|
||||
source markers and preservation tests prove that the pieces reassemble to the audited
|
||||
original, apart from explicitly registered trial retirements.
|
||||
- `frontend/styles/`, `frontend/shared/tokens.css`, and the Wentian page stylesheet preserve
|
||||
the approved cascade and light/dark/mobile behavior.
|
||||
- `config/` is the versioned registry for pages, features, APIs, datasets, quality rules,
|
||||
jobs, and the generated candidate architecture inventory.
|
||||
|
||||
Root modules such as `screener.py`, `tushare_client.py`, and `mentor_agent.py` are compatibility
|
||||
aliases to canonical modules. They contain no second implementation and remain only because
|
||||
the original public import surface is part of the preservation contract. Canonical backend
|
||||
modules must import other canonical modules directly rather than routing through these aliases.
|
||||
The remaining `api_access` import in `backend/application.py` and preserved lazy
|
||||
`sentiment_engine` import in the screener repository are registered transition boundaries;
|
||||
the root `database.py` remains the documented schema/composition anchor.
|
||||
|
||||
## Non-negotiable maintenance rules
|
||||
|
||||
1. Preserve account ownership in every user-private query and test it with two accounts.
|
||||
2. Browser requests go through `frontend/shared/api.js`; provider calls go through the data
|
||||
boundary; model calls go through `backend/llm/`.
|
||||
3. Calculation datasets fail closed when required source, date, unit, freshness, or coverage
|
||||
evidence is missing. Display fallbacks do not silently enter calculations.
|
||||
4. Do not implement logic in both a root compatibility module and a canonical module.
|
||||
5. Do not remove compatibility or uncertain code without reference scanning, old/new
|
||||
differential evidence, browser checks, and manual acceptance.
|
||||
6. Run `python tools/verify_baseline.py` for every change and add `--e2e` when runtime or
|
||||
frontend behavior can be affected.
|
||||
|
||||
The authoritative migration constraints and handoff procedure are in
|
||||
`../docs/migration/原版保真迁移总纲.md` and
|
||||
`../docs/migration/人工维护与本地切换指南.md`.
|
||||
@@ -0,0 +1,257 @@
|
||||
# 小白复盘局域网 Docker 部署
|
||||
|
||||
本文以 Linux 服务器为目标,容器内外均使用 `8765` 端口,宿主机监听
|
||||
`0.0.0.0:8765`。局域网用户通过 `http://服务器局域网IP:8765` 访问。
|
||||
|
||||
## 1. 部署结构
|
||||
|
||||
```text
|
||||
局域网浏览器
|
||||
|
|
||||
v
|
||||
服务器 0.0.0.0:8765
|
||||
|
|
||||
v
|
||||
xiaobai-review 容器 :8765
|
||||
|-- /app 只读应用代码
|
||||
`-- /app/data 宿主机 ./data 持久化挂载
|
||||
```
|
||||
|
||||
账号、加密后的公共数据 Token、平台模型 API Key、生辰资料、行情快照和复盘数据均在
|
||||
`data/review.db`。解密密钥来自 `.env` 中的 `APP_ENCRYPTION_KEY`。数据库与
|
||||
密钥必须成对备份,任意一个丢失都无法恢复账号内的加密资料。
|
||||
|
||||
管理员私有的问师 Skill 保存在宿主机 `data/private-mentor-skills/`。该目录随 `data`
|
||||
挂载进入容器,但被 Git 与 Docker 构建上下文排除,不会进入 Gitea 或镜像。私有 Skill
|
||||
只对管理员账号返回和开放调用,也会随本指南的 `data` 备份一起保存。
|
||||
|
||||
首个注册账号自动成为管理员。管理员在“系统管理”中配置全站共享行情、后台刷新、平台会员模型及手动会员;普通用户的“账号设置”用于个人资料、会员状态、修改密码和切换账号。后台行情更新不会主动刷新任何浏览器页面。
|
||||
|
||||
## 2. 服务器要求
|
||||
|
||||
- 64 位 Linux 服务器;
|
||||
- Docker Engine 24 或更新版本;
|
||||
- Docker Compose v2,命令形式为 `docker compose`;
|
||||
- 服务器可以访问 Tushare、已配置的 LLM 和实时聚合数据源;
|
||||
- 局域网内没有其他服务占用 TCP `8765`。
|
||||
|
||||
验证 Docker:
|
||||
|
||||
```bash
|
||||
docker --version
|
||||
docker compose version
|
||||
```
|
||||
|
||||
## 3. 迁移现有数据
|
||||
|
||||
迁移前先停止当前 Windows 上的 `8765` 服务,避免复制过程中 SQLite 继续写入。
|
||||
然后在 `webapp` 目录执行一次 WAL 检查点:
|
||||
|
||||
```powershell
|
||||
python -c "import sqlite3; c=sqlite3.connect('data/review.db'); print(c.execute('PRAGMA wal_checkpoint(TRUNCATE)').fetchone()); c.close()"
|
||||
```
|
||||
|
||||
结果第一项应为 `0`。必须迁移以下内容:
|
||||
|
||||
```text
|
||||
webapp/data/
|
||||
webapp/.env
|
||||
webapp/Dockerfile
|
||||
webapp/compose.yaml
|
||||
webapp/其余程序文件
|
||||
```
|
||||
|
||||
不要重新生成 `APP_ENCRYPTION_KEY`。部署已有数据库时,目标服务器 `.env` 中的
|
||||
值必须与原服务器完全一致。
|
||||
|
||||
可以在项目目录生成迁移包:
|
||||
|
||||
```powershell
|
||||
tar --exclude='__pycache__' --exclude='*.log' --exclude='data/cache' -czf xiaobai-review.tar.gz -C webapp .
|
||||
scp .\xiaobai-review.tar.gz 用户名@服务器IP:/tmp/
|
||||
```
|
||||
|
||||
迁移包包含数据库和密钥,传输完成后应及时删除两端的压缩包。
|
||||
|
||||
## 4. 首次启动
|
||||
|
||||
在 Linux 服务器执行:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /opt/xiaobai-review
|
||||
sudo chown "$USER":"$USER" /opt/xiaobai-review
|
||||
tar -xzf /tmp/xiaobai-review.tar.gz -C /opt/xiaobai-review
|
||||
cd /opt/xiaobai-review
|
||||
chmod 600 .env
|
||||
sudo chown -R 10001:10001 data
|
||||
docker compose config
|
||||
docker compose build --pull
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
镜像使用 UID/GID `10001` 的非 root 用户运行,因此宿主机 `data` 目录必须允许
|
||||
该用户写入。不要把整个应用目录设为可写。
|
||||
|
||||
检查运行状态:
|
||||
|
||||
```bash
|
||||
docker compose ps
|
||||
docker compose logs --tail=100 xiaobai-review
|
||||
curl http://127.0.0.1:8765/api/health
|
||||
docker inspect --format '{{.State.Health.Status}}' xiaobai-review
|
||||
```
|
||||
|
||||
健康接口应返回类似内容:
|
||||
|
||||
```json
|
||||
{"ok": true, "storage": "sqlite", "account_required": true}
|
||||
```
|
||||
|
||||
随后在局域网电脑访问:
|
||||
|
||||
```text
|
||||
http://服务器局域网IP:8765
|
||||
```
|
||||
|
||||
## 5. 防火墙
|
||||
|
||||
Compose 已明确绑定 `0.0.0.0:8765`。服务器防火墙建议只允许实际局域网网段,
|
||||
不要在路由器上把该端口映射到公网。
|
||||
|
||||
Ubuntu/UFW 示例,假设局域网为 `192.168.1.0/24`:
|
||||
|
||||
```bash
|
||||
sudo ufw allow from 192.168.1.0/24 to any port 8765 proto tcp
|
||||
sudo ufw status
|
||||
```
|
||||
|
||||
如果服务器位于其他网段,应替换为实际 CIDR。访问失败时同时检查云服务器安全组、
|
||||
虚拟化平台防火墙和宿主机防火墙。
|
||||
|
||||
## 6. 日常管理
|
||||
|
||||
查看日志:
|
||||
|
||||
```bash
|
||||
cd /opt/xiaobai-review
|
||||
docker compose logs -f --tail=100 xiaobai-review
|
||||
```
|
||||
|
||||
重启:
|
||||
|
||||
```bash
|
||||
docker compose restart xiaobai-review
|
||||
```
|
||||
|
||||
停止:
|
||||
|
||||
```bash
|
||||
docker compose down
|
||||
```
|
||||
|
||||
### 使用 Gitea 更新程序(推荐)
|
||||
|
||||
代码仓库为:
|
||||
|
||||
```text
|
||||
http://192.168.200.36:3200/leefer/xiaobaifupan.git
|
||||
```
|
||||
|
||||
首次在服务器部署代码时,可以直接克隆到目标目录:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /opt/xiaobai-review
|
||||
sudo chown "$USER":"$USER" /opt/xiaobai-review
|
||||
git clone http://192.168.200.36:3200/leefer/xiaobaifupan.git /opt/xiaobai-review
|
||||
cd /opt/xiaobai-review
|
||||
```
|
||||
|
||||
私有仓库会提示输入 Gitea 用户名和密码或访问令牌。不要把密码写入仓库 URL、
|
||||
`compose.yaml` 或脚本。然后把原 `.env` 与 `data/` 放回该目录;这两项已被 Git
|
||||
忽略,后续拉取代码不会覆盖数据库与密钥。
|
||||
|
||||
如需部署管理员私有问师,通过 NAS 文件管理器将本地
|
||||
`data/private-mentor-skills/` 复制到服务器项目的同名 `data` 目录,并保持目录仅由
|
||||
部署账号和容器运行用户读取。该内容不会通过 Gitea 同步。
|
||||
|
||||
每次更新前先创建 SQLite 一致性备份,再拉取并重建容器:
|
||||
|
||||
```bash
|
||||
cd /opt/xiaobai-review
|
||||
docker compose exec -T xiaobai-review python -c "import sqlite3; s=sqlite3.connect('/app/data/review.db'); d=sqlite3.connect('/app/data/review-before-update.db'); s.backup(d); d.close(); s.close()"
|
||||
git pull --ff-only origin main
|
||||
docker compose up -d --build
|
||||
docker compose ps
|
||||
curl --fail http://127.0.0.1:8765/api/health
|
||||
```
|
||||
|
||||
`docker compose up -d --build` 会原地替换应用容器,不删除宿主机的 `data` 目录。
|
||||
数据库迁移会在新容器启动时自动执行。若 `git pull --ff-only` 提示本地代码有修改,
|
||||
先用 `git status` 查明原因,不要用强制重置覆盖 `.env` 或 `data`。
|
||||
|
||||
### 不使用 Git 时更新
|
||||
|
||||
重新上传代码后执行:
|
||||
|
||||
```bash
|
||||
docker compose down
|
||||
docker compose build --pull
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
`docker compose down` 不会删除宿主机的 `data` 目录。不要使用带有手工删除
|
||||
`data` 目录的清理命令。
|
||||
|
||||
## 7. 备份与恢复
|
||||
|
||||
最稳妥的备份方式是短暂停服后同时备份数据库目录和密钥:
|
||||
|
||||
```bash
|
||||
cd /opt/xiaobai-review
|
||||
docker compose stop xiaobai-review
|
||||
tar -czf "xiaobai-backup-$(date +%Y%m%d-%H%M%S).tar.gz" data .env
|
||||
docker compose start xiaobai-review
|
||||
```
|
||||
|
||||
恢复时先停止容器,再恢复 `data` 和与其配套的 `.env`,修复权限后启动:
|
||||
|
||||
```bash
|
||||
docker compose down
|
||||
sudo chown -R 10001:10001 data
|
||||
chmod 600 .env
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
## 8. 常见问题
|
||||
|
||||
### 容器反复重启
|
||||
|
||||
```bash
|
||||
docker compose logs --tail=200 xiaobai-review
|
||||
```
|
||||
|
||||
优先检查 `.env` 是否存在、`APP_ENCRYPTION_KEY` 是否为空,以及 `data` 是否可写。
|
||||
|
||||
### 提示账号加密数据无法解密
|
||||
|
||||
目标服务器使用了错误的 `APP_ENCRYPTION_KEY`。停止容器并恢复与数据库配套的
|
||||
原始 `.env`,不要通过重置密钥绕过该错误。
|
||||
|
||||
### SQLite 显示只读或无法打开
|
||||
|
||||
```bash
|
||||
sudo chown -R 10001:10001 /opt/xiaobai-review/data
|
||||
sudo chmod -R u+rwX /opt/xiaobai-review/data
|
||||
docker compose restart xiaobai-review
|
||||
```
|
||||
|
||||
### 本机健康检查正常但其他电脑无法访问
|
||||
|
||||
确认 `docker compose ps` 显示 `0.0.0.0:8765->8765/tcp`,然后检查服务器防火墙和
|
||||
客户端到服务器的网络路由。
|
||||
|
||||
## 9. 安全边界
|
||||
|
||||
当前部署使用局域网 HTTP,账号密码和会话只适合可信内网使用。不要直接将
|
||||
`8765` 暴露到互联网。以后需要公网访问时,应在容器前增加 Caddy 或 Nginx,
|
||||
启用 HTTPS,并限制可信来源。
|
||||
@@ -0,0 +1,36 @@
|
||||
FROM python:3.12-slim-bookworm
|
||||
|
||||
ARG APP_UID=10001
|
||||
ARG APP_GID=10001
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
PYTHONUTF8=1 \
|
||||
PIP_DISABLE_PIP_VERSION_CHECK=1 \
|
||||
TZ=Asia/Shanghai
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
RUN apt-get update \
|
||||
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
|
||||
ca-certificates \
|
||||
tzdata \
|
||||
&& groupadd --gid "${APP_GID}" xiaobai \
|
||||
&& useradd --uid "${APP_UID}" --gid "${APP_GID}" --create-home --shell /usr/sbin/nologin xiaobai \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY requirements.txt ./
|
||||
RUN python -m pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
COPY --chown=xiaobai:xiaobai . .
|
||||
RUN mkdir -p /app/data && chown -R xiaobai:xiaobai /app/data
|
||||
|
||||
USER xiaobai
|
||||
|
||||
EXPOSE 8765
|
||||
STOPSIGNAL SIGINT
|
||||
|
||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
|
||||
CMD ["python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8765/api/health', timeout=4).read()"]
|
||||
|
||||
CMD ["python", "-u", "server.py", "--host", "0.0.0.0", "--port", "8765"]
|
||||
@@ -0,0 +1,70 @@
|
||||
# 小白复盘 Web
|
||||
|
||||
一个面向 A 股盘后复盘的本地 Web 工作台。后端使用 Python 访问 Tushare Pro,前端不依赖构建工具。
|
||||
|
||||
本目录是从原版源码逐项移动、机械拆分并完成差分验证与用户人工验收的模块化正式源码,
|
||||
不是依据规格书重新开发的第二套产品。正式部署切换前,`webapp/`根目录继续作为当前部署与
|
||||
回档基线;冻结的`next/`不得用于部署或后续开发。目录职责见[ARCHITECTURE.md](ARCHITECTURE.md)。
|
||||
|
||||
当前包含集合竞价、涨停池、炸板池、跌停板、昨日涨停、涨停表现、市场天梯、板块轮动、题材库、人气热榜、龙虎榜和个人复盘工作区。交易日快照与同步记录保存在本地 SQLite 数据库 `data/review.db`。
|
||||
|
||||
集合竞价中心采用盘前生命周期:9:15 前显示预告,9:15–9:25 明确等待最终竞价,9:25–9:30 自动读取并重试最终竞价筛选,9:30 后停止更新并冻结为复盘归档。当前 Tushare 只提供 9:25 最终竞价快照,不将其表述为动态虚拟撮合行情。
|
||||
|
||||
第三阶段加入了机构席位、席位别名、个股复权日 K、资金流、自选股、涨停原因修订、个股笔记、每日复盘和历史数据回补。
|
||||
|
||||
股票代码在桌面端悬停后会显示分时与日 K 快速预览,默认优先展示日 K;移动端点击代码后从底部打开预览面板。股票详情以及板块、题材、指数详情均可在日 K 与最新分时之间切换。日 K 复用个股详情缓存;分时优先使用 iFinD,东方财富仅作隔离的展示兜底,并使用短时内存缓存。图表数据不写入主行情、不参与情绪、选股或问天计算;不可用时明确显示“分时不可用”,不会用日 K 模拟分时走势。
|
||||
|
||||
智能选股包含六阶段盘后候选、29 套精选策略、自定义公式 DSL、自然语言公式编译、候选排名和滚动回测。阶段与精选策略在当日行情更新后由后台确定性计算;自定义选股由用户手动执行,LLM 只负责编译自然语言条件,不参与候选筛选。竞价、估值、财务、资金、人气和席位等字段按已登记的数据可用性进入因子库,缺失时明确显示覆盖问题。
|
||||
|
||||
候选只有经用户手动加入后才进入五交易日持续跟踪,展示 T+1 开盘/收盘、T+3、T+5、最大涨幅与最大回撤。提醒中心支持手工日期提醒,并在策略首日反馈和五日跟踪完成时生成账号私有的站内提醒。
|
||||
|
||||
问师模块会读取当前复盘、近十日市场情绪、涨跌停、昨日反馈、板块轮动、市场阶段、龙虎榜和指定个股数据,再按选中的游资思维 Skill 进行单师对话。对话记录按账号、老师和交易日期保存在服务端;主模型不可用时自动切换辅助模型。
|
||||
|
||||
新增公开问师角色时,在 `游资skills` 下增加一个包含 `SKILL.md` 的独立目录,并在 `游资skills/mentor_catalog.json` 中登记素材等级与结构质检。管理员私有角色放在 `data/private-mentor-skills`,该目录不进入 Git 或 Docker 镜像,且只会出现在管理员的问师列表中。系统会从 Skill 的 frontmatter、一级标题、核心模型和引用语中自动生成角色信息,无需修改注册代码。
|
||||
|
||||
问天模块包含三个相互独立的部分:观势以市场数据生成三才六爻,用于观察“势”,行情缺失或自动取象明显偏差时可显式手动校准六爻,人工结果与自动来源严格区分;观气依据干支、精确节气、五运六气及客主加临关系观察“运”,行业五行仅作传统取象归类;观心先准备1秒,再完成5轮“吸3秒、顿2秒、呼4秒”,随后以六次三枚铜钱起卦、察念和解卦完成一次不输入问题的问心仪式。卦象、干支、节气与气机关系均由本地确定性程序计算,LLM只负责解释,不参与起卦或改动结果。
|
||||
|
||||
问天模块使用项目本地的 `lunar-python` 计算历法,并使用 `data/iching_zh.json` 中的固定六十四卦、卦辞和爻辞。第三方授权见 `THIRD_PARTY_NOTICES.md`。
|
||||
|
||||
“我的复盘”包含结构化手工交易日志,可记录方向、价格、数量、仓位、盈亏、逻辑、执行、情绪和标签,不接券商也不自动下单。顶部“复盘助手”以流式方式读取市场统计、策略跟踪、提醒、个人复盘和交易日志;对话按账号保存,只提供分析和条件化计划。
|
||||
|
||||
## 启动
|
||||
|
||||
```powershell
|
||||
cd webapp\app
|
||||
python -m pip install -r requirements.txt
|
||||
python server.py
|
||||
```
|
||||
|
||||
浏览器打开 `http://127.0.0.1:8765`,首次使用先注册账号。首个账号自动成为管理员,后续账号默认为普通用户。主行情不再回退演示数据:盘前、非交易日或临时取数失败时沿用最近真实收盘快照;没有任何真实快照时提示等待管理员完成首次同步。
|
||||
|
||||
局域网 Docker 部署使用 `Dockerfile` 与 `compose.yaml`,完整的迁移、持久化、
|
||||
防火墙、备份和恢复步骤见 [DOCKER_DEPLOY.md](DOCKER_DEPLOY.md)。
|
||||
|
||||
账号密码使用 scrypt 哈希;公共 Tushare Token、平台模型密钥以及原始生辰资料均使用 `APP_ENCRYPTION_KEY` 加密后保存在 SQLite。公共数据和平台模型归系统所有,生辰资料仍按账号隔离。普通用户不配置 LLM,只有管理员授权的有效会员可以使用平台模型。请将 `.env` 与数据库一起备份,丢失加密密钥后无法恢复这些资料。
|
||||
|
||||
## 系统与账号配置
|
||||
|
||||
管理员通过页面右上角“系统管理”保存公共 Tushare Token、平台主/辅助模型、会员每日额度和后台刷新开关。所有用户读取同一份 SQLite 行情快照,不再分别配置行情 Token。已有个人凭据中的 Tushare Token 会在升级时迁移到系统配置并从个人凭据移除。
|
||||
|
||||
```text
|
||||
TUSHARE_TOKEN=你的Token
|
||||
```
|
||||
|
||||
`.env` 中的 Tushare 和平台 LLM 配置只用于初始化系统配置,密钥不会返回到浏览器。后台刷新只在交易时段更新 SQLite 快照,不会主动刷新或重绘用户页面;用户点击页面“刷新”时读取最新快照。管理员也可点“后台刷新”立即启动一次后台同步,当前页面仍保持不变。
|
||||
|
||||
普通用户在“账号设置”中维护个人资料、查看会员状态和修改密码,不配置个人 LLM。有效会员自动使用平台模型;管理员可在“系统管理”中手动开通、续期、停用会员。平台模型受管理员设置的每日调用次数限制,管理员账号始终可用。
|
||||
|
||||
Tushare 各接口有独立积分权限。程序优先使用 `limit_list_d` 获取涨跌停明细;该接口不可用时,会尝试通过日线和每日涨跌停价格推算。
|
||||
|
||||
## 隔离实时聚合验证
|
||||
|
||||
`realtime_aggregator.py` 用于验证东方财富、同花顺和选股宝网页数据源。它不写入 SQLite 主行情快照,也不参与情绪评分或智能选股;当 Tushare 实时指数权限不可用时,观势会使用东方财富三大指数和板块外显,并继续使用 Tushare 的板块成分内核与个股数据。
|
||||
|
||||
登录后可调用:
|
||||
|
||||
```text
|
||||
GET /api/realtime-aggregate/health?sector=元器件
|
||||
```
|
||||
|
||||
返回内容包括东方财富三大指数及板块快照、指数时间差、同花顺和选股宝可用性、每个来源的耗时与错误。盘中指数时间差不超过15秒,收盘后不超过120秒。`ready=true` 仅表示本次验证满足聚合层约束,不代表这些网页内部接口具有长期稳定性或商业使用授权。
|
||||
@@ -0,0 +1,79 @@
|
||||
# Third-Party Notices
|
||||
|
||||
## lunar-python
|
||||
|
||||
Source: https://github.com/6tail/lunar-python
|
||||
Copyright (c) 2020 6tail
|
||||
|
||||
MIT License
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
||||
## Lucide
|
||||
|
||||
The local browser icon bundle at `static/vendor/lucide.min.js` is Lucide
|
||||
version 0.468.0.
|
||||
|
||||
Source: https://github.com/lucide-icons/lucide
|
||||
|
||||
ISC License
|
||||
|
||||
Copyright (c) for portions of Lucide are held by Cole Bemis 2013-2022 as part
|
||||
of Feather (MIT). All other copyright (c) for Lucide are held by Lucide
|
||||
Contributors 2022.
|
||||
|
||||
Permission to use, copy, modify, and/or distribute this software for any
|
||||
purpose with or without fee is hereby granted, provided that the above
|
||||
copyright notice and this permission notice appear in all copies.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
|
||||
REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
|
||||
AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
|
||||
INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
|
||||
LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR
|
||||
OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
|
||||
PERFORMANCE OF THIS SOFTWARE.
|
||||
|
||||
## ichingpy classic text data
|
||||
|
||||
The fixed Chinese hexagram, judgement and line text data in
|
||||
`data/iching_zh.json` is derived from the MIT-licensed ichingpy project.
|
||||
|
||||
Source: https://github.com/JinyangWang27/ichingpy
|
||||
Copyright (c) 2024 Jinyang Wang
|
||||
|
||||
MIT License
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,7 @@
|
||||
"""Compatibility alias for the canonical curated strategy library."""
|
||||
|
||||
import sys
|
||||
|
||||
from backend.features.screener import strategies as _implementation
|
||||
|
||||
sys.modules[__name__] = _implementation
|
||||
@@ -0,0 +1,3 @@
|
||||
from backend.features.alerts.service import AlertService
|
||||
|
||||
__all__ = ["AlertService"]
|
||||
@@ -0,0 +1,15 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from backend.http import AccessRole, ApiRouteRegistry
|
||||
|
||||
|
||||
ROUTES = ApiRouteRegistry.load()
|
||||
|
||||
|
||||
def required_role(method: str, path: str) -> AccessRole:
|
||||
"""Compatibility access lookup backed by the authoritative route registry."""
|
||||
route = ROUTES.resolve(method, path)
|
||||
return route.access if route else "authenticated"
|
||||
|
||||
|
||||
__all__ = ["ROUTES", "AccessRole", "required_role"]
|
||||
@@ -0,0 +1,3 @@
|
||||
"""Compatibility imports for code that still uses the original configuration module."""
|
||||
|
||||
from backend.bootstrap.config import * # noqa: F401,F403
|
||||
@@ -0,0 +1,7 @@
|
||||
"""Compatibility alias for the canonical review-assistant implementation."""
|
||||
|
||||
import sys
|
||||
|
||||
from backend.features.review import agent as _implementation
|
||||
|
||||
sys.modules[__name__] = _implementation
|
||||
@@ -0,0 +1 @@
|
||||
"""Application packages introduced by architecture governance."""
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,23 @@
|
||||
__all__ = [
|
||||
"ApplicationContainer",
|
||||
"RuntimeSettings",
|
||||
"build_application_container",
|
||||
"load_runtime_settings",
|
||||
"main",
|
||||
]
|
||||
|
||||
|
||||
def __getattr__(name: str):
|
||||
if name in {"ApplicationContainer", "build_application_container"}:
|
||||
from . import container
|
||||
|
||||
return getattr(container, name)
|
||||
if name in {"RuntimeSettings", "load_runtime_settings"}:
|
||||
from . import settings
|
||||
|
||||
return getattr(settings, name)
|
||||
if name == "main":
|
||||
from .runtime import main
|
||||
|
||||
return main
|
||||
raise AttributeError(name)
|
||||
@@ -0,0 +1,133 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import calendar
|
||||
import os
|
||||
import re
|
||||
from datetime import date, datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
APP_DIR = Path(__file__).resolve().parents[2]
|
||||
STATIC_DIR = APP_DIR / "frontend"
|
||||
DATA_DIR = APP_DIR / "data"
|
||||
ENV_FILE = APP_DIR / ".env"
|
||||
MENTOR_SKILLS_DIR = APP_DIR / "游资skills"
|
||||
PRIVATE_MENTOR_SKILLS_DIR = DATA_DIR / "private-mentor-skills"
|
||||
TOKEN_PATTERN = re.compile(r"^[A-Za-z0-9_-]{20,128}$")
|
||||
USERNAME_PATTERN = re.compile(r"^[A-Za-z0-9_\-\u4e00-\u9fff]{3,30}$")
|
||||
SESSION_COOKIE = "xiaobai_session"
|
||||
SESSION_MAX_AGE = 30 * 24 * 60 * 60
|
||||
|
||||
|
||||
def load_local_env() -> None:
|
||||
if not ENV_FILE.exists():
|
||||
return
|
||||
for raw_line in ENV_FILE.read_text(encoding="utf-8").splitlines():
|
||||
line = raw_line.strip()
|
||||
if not line or line.startswith("#") or "=" not in line:
|
||||
continue
|
||||
key, value = line.split("=", 1)
|
||||
os.environ.setdefault(key.strip(), value.strip().strip('"').strip("'"))
|
||||
|
||||
|
||||
def save_local_env(updates: dict[str, str]) -> None:
|
||||
values: dict[str, str] = {}
|
||||
if ENV_FILE.exists():
|
||||
for raw_line in ENV_FILE.read_text(encoding="utf-8").splitlines():
|
||||
if "=" in raw_line and not raw_line.lstrip().startswith("#"):
|
||||
key, value = raw_line.split("=", 1)
|
||||
values[key.strip()] = value.strip().strip('"').strip("'")
|
||||
values.update(updates)
|
||||
ENV_FILE.write_text(
|
||||
"".join(f"{key}={value}\n" for key, value in values.items()),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
|
||||
def remove_local_env(keys: set[str]) -> None:
|
||||
if not ENV_FILE.exists():
|
||||
return
|
||||
kept = []
|
||||
for raw_line in ENV_FILE.read_text(encoding="utf-8").splitlines():
|
||||
if "=" in raw_line and not raw_line.lstrip().startswith("#"):
|
||||
key = raw_line.split("=", 1)[0].strip()
|
||||
if key in keys:
|
||||
continue
|
||||
kept.append(raw_line)
|
||||
ENV_FILE.write_text("".join(f"{line}\n" for line in kept), encoding="utf-8")
|
||||
for key in keys:
|
||||
os.environ.pop(key, None)
|
||||
|
||||
|
||||
def normalize_date(value: str) -> str:
|
||||
compact = value.replace("-", "").strip()
|
||||
try:
|
||||
parsed = datetime.strptime(compact, "%Y%m%d")
|
||||
except ValueError as exc:
|
||||
raise ValueError("日期格式应为 YYYY-MM-DD。") from exc
|
||||
if parsed.date() > date.today():
|
||||
raise ValueError("不能查询未来日期。")
|
||||
return parsed.strftime("%Y%m%d")
|
||||
|
||||
|
||||
def display_compact_date(value: str) -> str:
|
||||
return f"{value[:4]}-{value[4:6]}-{value[6:8]}" if len(value) == 8 else value
|
||||
|
||||
|
||||
def validate_stock_code(value: str) -> str:
|
||||
code = value.strip()
|
||||
if not re.fullmatch(r"\d{6}", code):
|
||||
raise ValueError("股票代码应为 6 位数字。")
|
||||
return code
|
||||
|
||||
|
||||
def tushare_code(code: str) -> str:
|
||||
if code.startswith(("4", "8", "9")):
|
||||
suffix = "BJ"
|
||||
elif code.startswith("6"):
|
||||
suffix = "SH"
|
||||
else:
|
||||
suffix = "SZ"
|
||||
return f"{code}.{suffix}"
|
||||
|
||||
|
||||
def validate_text(value: Any, label: str, maximum: int, required: bool = False) -> str:
|
||||
text = str(value or "").strip()
|
||||
if required and not text:
|
||||
raise ValueError(f"{label}不能为空。")
|
||||
if len(text) > maximum:
|
||||
raise ValueError(f"{label}不能超过 {maximum} 个字符。")
|
||||
return text
|
||||
|
||||
|
||||
def parse_iso_datetime(value: Any) -> datetime | None:
|
||||
text = str(value or "").strip()
|
||||
if not text:
|
||||
return None
|
||||
try:
|
||||
parsed = datetime.fromisoformat(text)
|
||||
except ValueError:
|
||||
return None
|
||||
return parsed.replace(tzinfo=timezone.utc) if parsed.tzinfo is None else parsed.astimezone(timezone.utc)
|
||||
|
||||
|
||||
def membership_boundary(value: Any, end: bool) -> str | None:
|
||||
text = str(value or "").strip()
|
||||
if not text:
|
||||
return None
|
||||
try:
|
||||
day = datetime.strptime(text, "%Y-%m-%d").replace(tzinfo=timezone.utc)
|
||||
except ValueError as exc:
|
||||
raise ValueError("会员日期格式应为 YYYY-MM-DD。") from exc
|
||||
if end:
|
||||
day += timedelta(days=1)
|
||||
return day.isoformat(timespec="seconds")
|
||||
|
||||
|
||||
def add_months(value: datetime, months: int) -> datetime:
|
||||
month_index = value.year * 12 + value.month - 1 + months
|
||||
year, zero_based_month = divmod(month_index, 12)
|
||||
month = zero_based_month + 1
|
||||
day = min(value.day, calendar.monthrange(year, month)[1])
|
||||
return value.replace(year=year, month=month, day=day)
|
||||
@@ -0,0 +1,60 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from collections.abc import Callable
|
||||
|
||||
from backend.data import DataGateway, build_data_gateway
|
||||
from backend.database.repositories import RepositoryBundle, build_repository_bundle
|
||||
from backend.features.alerts import AlertService
|
||||
from backend.features.mentor.agent import MentorSkillRegistry
|
||||
from backend.features.review import TradeJournalService
|
||||
from backend.features.screener.engine import ScreenerEngine
|
||||
from backend.features.screener.tracking import StrategyTrackingService
|
||||
from backend.jobs import InProcessJobRunner, JobRegistry, SQLiteJobRunRepository
|
||||
from database import ReviewDatabase
|
||||
from backend.data.providers.ifind_client import IfindHttpClient
|
||||
from backend.data.realtime import WebRealtimeAggregator
|
||||
from backend.features.market.charts import MarketChartClient
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ApplicationContainer:
|
||||
database: ReviewDatabase
|
||||
repositories: RepositoryBundle
|
||||
data_gateway: DataGateway
|
||||
ifind: IfindHttpClient
|
||||
screener: ScreenerEngine
|
||||
strategy_tracking: StrategyTrackingService
|
||||
alert_service: AlertService
|
||||
trade_journal: TradeJournalService
|
||||
mentor_skills: MentorSkillRegistry
|
||||
realtime_aggregator: WebRealtimeAggregator
|
||||
chart_data: MarketChartClient
|
||||
jobs: InProcessJobRunner
|
||||
|
||||
|
||||
def build_application_container(
|
||||
database: ReviewDatabase,
|
||||
credentials: dict[str, object],
|
||||
mentor_skills_dir: Path,
|
||||
private_mentor_skills_dir: Path,
|
||||
tushare_token_supplier: Callable[[], str] | None = None,
|
||||
) -> ApplicationContainer:
|
||||
data_gateway = build_data_gateway(credentials, tushare_token_supplier)
|
||||
repositories = build_repository_bundle(database)
|
||||
jobs = InProcessJobRunner(JobRegistry.load(), SQLiteJobRunRepository(database))
|
||||
return ApplicationContainer(
|
||||
database=database,
|
||||
repositories=repositories,
|
||||
data_gateway=data_gateway,
|
||||
ifind=data_gateway.ifind,
|
||||
screener=ScreenerEngine(database),
|
||||
strategy_tracking=StrategyTrackingService(repositories.strategy_tracking),
|
||||
alert_service=AlertService(repositories.alerts),
|
||||
trade_journal=TradeJournalService(repositories.trades),
|
||||
mentor_skills=MentorSkillRegistry(mentor_skills_dir, private_mentor_skills_dir),
|
||||
realtime_aggregator=data_gateway.realtime_observer,
|
||||
chart_data=data_gateway.chart_data,
|
||||
jobs=jobs,
|
||||
)
|
||||
@@ -0,0 +1,27 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from http.server import ThreadingHTTPServer
|
||||
from typing import Any
|
||||
|
||||
|
||||
def main(handler_class: type[Any] | None = None, service: Any | None = None) -> None:
|
||||
if handler_class is None or service is None:
|
||||
from backend.application import RequestHandler, SERVICE
|
||||
|
||||
handler_class = handler_class or RequestHandler
|
||||
service = service or SERVICE
|
||||
parser = argparse.ArgumentParser(description="Xiaobai stock review web application")
|
||||
parser.add_argument("--host", default="127.0.0.1")
|
||||
parser.add_argument("--port", type=int, default=8765)
|
||||
args = parser.parse_args()
|
||||
server = ThreadingHTTPServer((args.host, args.port), handler_class)
|
||||
print(f"Xiaobai Review Web is running at http://{args.host}:{args.port}")
|
||||
print("Press Ctrl+C to stop.")
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
finally:
|
||||
service._background_stop.set()
|
||||
server.server_close()
|
||||
@@ -0,0 +1,55 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from typing import Mapping
|
||||
|
||||
from backend.bootstrap.config import load_local_env, save_local_env
|
||||
from backend.features.accounts.security import SecretVault
|
||||
|
||||
|
||||
def environment_credentials(environment: Mapping[str, str]) -> dict[str, str]:
|
||||
return {
|
||||
"tushare_token": str(environment.get("TUSHARE_TOKEN") or "").strip(),
|
||||
"ifind_refresh_token": str(environment.get("IFIND_REFRESH_TOKEN") or "").strip(),
|
||||
"ifind_access_token": str(environment.get("IFIND_ACCESS_TOKEN") or "").strip(),
|
||||
"platform_llm_primary_api_key": str(
|
||||
environment.get("LLM_PRIMARY_API_KEY") or environment.get("LLM_API_KEY") or ""
|
||||
).strip(),
|
||||
"platform_llm_primary_base_url": str(
|
||||
environment.get("LLM_PRIMARY_BASE_URL")
|
||||
or environment.get("LLM_BASE_URL")
|
||||
or "https://api.openai.com/v1"
|
||||
).strip(),
|
||||
"platform_llm_primary_model": str(
|
||||
environment.get("LLM_PRIMARY_MODEL") or environment.get("LLM_MODEL") or ""
|
||||
).strip(),
|
||||
"platform_llm_fallback_api_key": str(
|
||||
environment.get("LLM_FALLBACK_API_KEY") or ""
|
||||
).strip(),
|
||||
"platform_llm_fallback_base_url": str(
|
||||
environment.get("LLM_FALLBACK_BASE_URL") or ""
|
||||
).strip(),
|
||||
"platform_llm_fallback_model": str(
|
||||
environment.get("LLM_FALLBACK_MODEL") or ""
|
||||
).strip(),
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RuntimeSettings:
|
||||
encryption_key: str
|
||||
initial_credentials: dict[str, str]
|
||||
|
||||
|
||||
def load_runtime_settings() -> RuntimeSettings:
|
||||
load_local_env()
|
||||
encryption_key = os.environ.get("APP_ENCRYPTION_KEY", "").strip()
|
||||
if not encryption_key:
|
||||
encryption_key = SecretVault.generate_key()
|
||||
save_local_env({"APP_ENCRYPTION_KEY": encryption_key})
|
||||
os.environ["APP_ENCRYPTION_KEY"] = encryption_key
|
||||
return RuntimeSettings(
|
||||
encryption_key=encryption_key,
|
||||
initial_credentials=environment_credentials(os.environ),
|
||||
)
|
||||
@@ -0,0 +1,21 @@
|
||||
from .policy import DataPolicyError, DataSourcePolicy
|
||||
from .quality import DataQualityError, DataQualityGate, QualityEvidence, QualityReport
|
||||
|
||||
__all__ = [
|
||||
"DataGateway",
|
||||
"DataPolicyError",
|
||||
"DataQualityError",
|
||||
"DataQualityGate",
|
||||
"DataSourcePolicy",
|
||||
"QualityEvidence",
|
||||
"QualityReport",
|
||||
"build_data_gateway",
|
||||
]
|
||||
|
||||
|
||||
def __getattr__(name: str):
|
||||
if name in {"DataGateway", "build_data_gateway"}:
|
||||
from .gateway import DataGateway, build_data_gateway
|
||||
|
||||
return {"DataGateway": DataGateway, "build_data_gateway": build_data_gateway}[name]
|
||||
raise AttributeError(name)
|
||||
@@ -0,0 +1,29 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Literal
|
||||
|
||||
|
||||
DataUsage = Literal["display", "calculation"]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ProviderContract:
|
||||
id: str
|
||||
provider_class: str
|
||||
calculation_allowed: bool
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DatasetContract:
|
||||
id: str
|
||||
entity: str
|
||||
frequency: str
|
||||
primary: str
|
||||
fallbacks: tuple[str, ...]
|
||||
usage: str
|
||||
fields: tuple[str, ...]
|
||||
|
||||
@property
|
||||
def providers(self) -> tuple[str, ...]:
|
||||
return (self.primary, *self.fallbacks)
|
||||
@@ -0,0 +1,83 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
|
||||
from backend.data.contracts import DataUsage
|
||||
from backend.data.policy import DataSourcePolicy
|
||||
from backend.data.providers import IfindProvider, TushareProvider
|
||||
from backend.data.quality import DataQualityGate, QualityEvidence, QualityReport
|
||||
from backend.data.providers.ifind_client import IfindHttpClient
|
||||
from backend.data.providers.tushare_client import TushareClient
|
||||
from backend.data.realtime import WebRealtimeAggregator
|
||||
from backend.features.market.charts import EastmoneyChartClient, MarketChartClient
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DataGateway:
|
||||
policy: DataSourcePolicy
|
||||
quality: DataQualityGate
|
||||
tushare_provider: TushareProvider
|
||||
ifind_provider: IfindProvider
|
||||
chart_data: MarketChartClient
|
||||
realtime_observer: WebRealtimeAggregator
|
||||
|
||||
@property
|
||||
def ifind(self) -> IfindHttpClient:
|
||||
return self.ifind_provider.client
|
||||
|
||||
def tushare(
|
||||
self,
|
||||
dataset_id: str = "",
|
||||
usage: DataUsage = "calculation",
|
||||
) -> TushareClient:
|
||||
if dataset_id:
|
||||
self.policy.assert_allowed(dataset_id, "tushare", usage)
|
||||
return self.tushare_provider.client()
|
||||
|
||||
def assert_source(self, dataset_id: str, provider_id: str, usage: DataUsage) -> None:
|
||||
self.policy.assert_allowed(dataset_id, provider_id, usage)
|
||||
|
||||
def provider_chain(self, dataset_id: str, usage: DataUsage) -> tuple[str, ...]:
|
||||
dataset = self.policy.dataset(dataset_id)
|
||||
allowed = []
|
||||
for provider_id in dataset.providers:
|
||||
try:
|
||||
self.policy.assert_allowed(dataset_id, provider_id, usage)
|
||||
except Exception:
|
||||
continue
|
||||
allowed.append(provider_id)
|
||||
if not allowed:
|
||||
raise RuntimeError(f"No permitted provider for {dataset_id} ({usage})")
|
||||
return tuple(allowed)
|
||||
|
||||
def require_quality(
|
||||
self,
|
||||
evidence: QualityEvidence,
|
||||
usage: DataUsage,
|
||||
as_of: str | datetime | None = None,
|
||||
) -> QualityReport:
|
||||
return self.quality.require(evidence, usage, as_of)
|
||||
|
||||
|
||||
def build_data_gateway(
|
||||
credentials: dict[str, object],
|
||||
tushare_token_supplier: Callable[[], str] | None = None,
|
||||
) -> DataGateway:
|
||||
ifind = IfindHttpClient(
|
||||
str(credentials.get("ifind_refresh_token") or ""),
|
||||
str(credentials.get("ifind_access_token") or ""),
|
||||
)
|
||||
token_supplier = tushare_token_supplier or (
|
||||
lambda: str(credentials.get("tushare_token") or "")
|
||||
)
|
||||
policy = DataSourcePolicy.load()
|
||||
return DataGateway(
|
||||
policy=policy,
|
||||
quality=DataQualityGate.load(policy),
|
||||
tushare_provider=TushareProvider(token_supplier),
|
||||
ifind_provider=IfindProvider(ifind),
|
||||
chart_data=MarketChartClient(ifind, EastmoneyChartClient()),
|
||||
realtime_observer=WebRealtimeAggregator(),
|
||||
)
|
||||
@@ -0,0 +1,20 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from typing import Any
|
||||
|
||||
|
||||
def finite_number(value: Any, default: float = 0.0) -> float:
|
||||
try:
|
||||
number = float(value)
|
||||
return number if math.isfinite(number) else default
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
|
||||
|
||||
def non_nan_number(value: Any, default: float = 0.0) -> float:
|
||||
try:
|
||||
number = float(value)
|
||||
return number if number == number else default
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
@@ -0,0 +1,77 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from backend.bootstrap.config import APP_DIR
|
||||
from backend.data.contracts import DataUsage, DatasetContract, ProviderContract
|
||||
|
||||
|
||||
class DataPolicyError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class DataSourcePolicy:
|
||||
def __init__(
|
||||
self,
|
||||
providers: dict[str, ProviderContract],
|
||||
datasets: dict[str, DatasetContract],
|
||||
) -> None:
|
||||
self.providers = dict(providers)
|
||||
self.datasets = dict(datasets)
|
||||
|
||||
@classmethod
|
||||
def load(cls, path: Path | None = None) -> "DataSourcePolicy":
|
||||
config_path = path or APP_DIR / "config" / "data-fields.config.json"
|
||||
payload = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
providers = {
|
||||
provider_id: ProviderContract(
|
||||
id=provider_id,
|
||||
provider_class=str(item["class"]),
|
||||
calculation_allowed=bool(item["calculation_allowed"]),
|
||||
)
|
||||
for provider_id, item in payload["providers"].items()
|
||||
}
|
||||
datasets = {
|
||||
item["id"]: DatasetContract(
|
||||
id=str(item["id"]),
|
||||
entity=str(item["entity"]),
|
||||
frequency=str(item["frequency"]),
|
||||
primary=str(item["primary"]),
|
||||
fallbacks=tuple(str(value) for value in item.get("fallbacks", [])),
|
||||
usage=str(item["usage"]),
|
||||
fields=tuple(str(value) for value in item.get("fields", [])),
|
||||
)
|
||||
for item in payload["datasets"]
|
||||
}
|
||||
return cls(providers, datasets)
|
||||
|
||||
def dataset(self, dataset_id: str) -> DatasetContract:
|
||||
try:
|
||||
return self.datasets[dataset_id]
|
||||
except KeyError as exc:
|
||||
raise DataPolicyError(f"Unregistered dataset: {dataset_id}") from exc
|
||||
|
||||
def assert_allowed(
|
||||
self,
|
||||
dataset_id: str,
|
||||
provider_id: str,
|
||||
usage: DataUsage,
|
||||
) -> DatasetContract:
|
||||
dataset = self.dataset(dataset_id)
|
||||
if dataset.usage == "blocked":
|
||||
raise DataPolicyError(f"Dataset is blocked: {dataset_id}")
|
||||
if provider_id not in dataset.providers:
|
||||
raise DataPolicyError(
|
||||
f"Provider {provider_id} is not registered for dataset {dataset_id}"
|
||||
)
|
||||
try:
|
||||
provider = self.providers[provider_id]
|
||||
except KeyError as exc:
|
||||
raise DataPolicyError(f"Unregistered provider: {provider_id}") from exc
|
||||
if usage == "calculation":
|
||||
if dataset.usage != "calculation" or not provider.calculation_allowed:
|
||||
raise DataPolicyError(
|
||||
f"Provider {provider_id} cannot calculate dataset {dataset_id}"
|
||||
)
|
||||
return dataset
|
||||
@@ -0,0 +1,4 @@
|
||||
from .ifind import IfindProvider
|
||||
from .tushare import TushareProvider
|
||||
|
||||
__all__ = ["IfindProvider", "TushareProvider"]
|
||||
@@ -0,0 +1,11 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from backend.data.providers.ifind_client import IfindHttpClient
|
||||
|
||||
|
||||
class IfindProvider:
|
||||
def __init__(self, client: IfindHttpClient) -> None:
|
||||
self.client = client
|
||||
|
||||
def set_credentials(self, refresh_token: str, access_token: str = "") -> None:
|
||||
self.client.set_credentials(refresh_token, access_token)
|
||||
@@ -0,0 +1,385 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import json
|
||||
import threading
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Any
|
||||
|
||||
|
||||
class IfindError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class IfindHttpClient:
|
||||
BASE_URL = "https://quantapi.51ifind.com/api/v1"
|
||||
AUTH_ENDPOINT = "get_access_token"
|
||||
AUTH_ERROR_CODES = {-1302, -1303, -1304, -4302, -4303}
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
refresh_token: str = "",
|
||||
access_token: str = "",
|
||||
timeout: int = 15,
|
||||
) -> None:
|
||||
self.timeout = max(3, int(timeout))
|
||||
self._refresh_token = str(refresh_token or "").strip()
|
||||
self._access_token = str(access_token or "").strip()
|
||||
self._access_expires_at: datetime | None = None
|
||||
self._token_lock = threading.Lock()
|
||||
self._cache_lock = threading.Lock()
|
||||
self._cache: dict[str, dict[str, Any]] = {}
|
||||
|
||||
@property
|
||||
def configured(self) -> bool:
|
||||
return bool(self._refresh_token or self._access_token)
|
||||
|
||||
def set_credentials(self, refresh_token: str, access_token: str = "") -> None:
|
||||
refresh_token = str(refresh_token or "").strip()
|
||||
access_token = str(access_token or "").strip()
|
||||
with self._token_lock:
|
||||
refresh_changed = refresh_token != self._refresh_token
|
||||
self._refresh_token = refresh_token
|
||||
if access_token or refresh_changed:
|
||||
self._access_token = access_token
|
||||
self._access_expires_at = None
|
||||
if refresh_changed:
|
||||
with self._cache_lock:
|
||||
self._cache.clear()
|
||||
|
||||
def status(self) -> dict[str, Any]:
|
||||
return {
|
||||
"configured": self.configured,
|
||||
"access_ready": bool(self._access_token),
|
||||
"access_expires_at": (
|
||||
self._access_expires_at.isoformat(timespec="seconds")
|
||||
if self._access_expires_at
|
||||
else ""
|
||||
),
|
||||
}
|
||||
|
||||
def test_connection(self) -> dict[str, Any]:
|
||||
payload = self.real_time(
|
||||
"000001.SH",
|
||||
["open", "high", "low", "latest", "preClose"],
|
||||
cache_ttl=0,
|
||||
)
|
||||
return {
|
||||
"ok": bool(payload),
|
||||
"sample_time": str(payload[0].get("time") or "") if payload else "",
|
||||
}
|
||||
|
||||
def real_time(
|
||||
self,
|
||||
codes: str | list[str],
|
||||
indicators: list[str],
|
||||
cache_ttl: int = 10,
|
||||
) -> list[dict[str, Any]]:
|
||||
code_text = self._codes(codes)
|
||||
payload = self._request(
|
||||
"real_time_quotation",
|
||||
{"codes": code_text, "indicators": ",".join(indicators)},
|
||||
cache_key=f"rq:{code_text}:{','.join(indicators)}",
|
||||
cache_ttl=cache_ttl,
|
||||
)
|
||||
return self._table_rows(payload)
|
||||
|
||||
def history(
|
||||
self,
|
||||
codes: str | list[str],
|
||||
indicators: list[str],
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
cache_ttl: int = 300,
|
||||
) -> list[dict[str, Any]]:
|
||||
code_text = self._codes(codes)
|
||||
payload = self._request(
|
||||
"cmd_history_quotation",
|
||||
{
|
||||
"codes": code_text,
|
||||
"indicators": ",".join(indicators),
|
||||
"startdate": self._display_date(start_date),
|
||||
"enddate": self._display_date(end_date),
|
||||
"functionpara": {"CPS": "forward1", "Fill": "Omit"},
|
||||
},
|
||||
cache_key=f"hq:{code_text}:{start_date}:{end_date}:{','.join(indicators)}",
|
||||
cache_ttl=cache_ttl,
|
||||
)
|
||||
return self._table_rows(payload)
|
||||
|
||||
def intraday(
|
||||
self,
|
||||
code: str,
|
||||
start_time: str,
|
||||
end_time: str,
|
||||
cache_ttl: int = 20,
|
||||
) -> list[dict[str, Any]]:
|
||||
indicators = ["open", "high", "low", "close", "volume", "amount", "avgPrice"]
|
||||
payload = self._request(
|
||||
"high_frequency",
|
||||
{
|
||||
"codes": self._codes(code),
|
||||
"indicators": ",".join(indicators),
|
||||
"starttime": start_time,
|
||||
"endtime": end_time,
|
||||
"functionpara": {
|
||||
"CPS": "forward1",
|
||||
"Fill": "Previous",
|
||||
"Timeformat": "LocalTime",
|
||||
"Interval": "1",
|
||||
"Limitstart": "09:30:00",
|
||||
"Limitend": "15:00:00",
|
||||
},
|
||||
},
|
||||
cache_key=f"hf:{code}:{start_time}:{end_time}",
|
||||
cache_ttl=cache_ttl,
|
||||
)
|
||||
return self._table_rows(payload)
|
||||
|
||||
def snapshots(
|
||||
self,
|
||||
codes: str | list[str],
|
||||
indicators: list[str],
|
||||
start_time: str,
|
||||
end_time: str,
|
||||
cache_ttl: int = 8,
|
||||
) -> list[dict[str, Any]]:
|
||||
code_text = self._codes(codes)
|
||||
payload = self._request(
|
||||
"snap_shot",
|
||||
{
|
||||
"codes": code_text,
|
||||
"indicators": ",".join(indicators),
|
||||
"starttime": start_time,
|
||||
"endtime": end_time,
|
||||
},
|
||||
cache_key=f"ss:{code_text}:{start_time}:{end_time}:{','.join(indicators)}",
|
||||
cache_ttl=cache_ttl,
|
||||
)
|
||||
return self._table_rows(payload)
|
||||
|
||||
def wencai(self, query: str, search_type: str = "stock", cache_ttl: int = 300) -> list[dict[str, Any]]:
|
||||
normalized = " ".join(str(query or "").split())
|
||||
if not normalized:
|
||||
raise IfindError("问财查询不能为空。")
|
||||
payload = self._request(
|
||||
"smart_stock_picking",
|
||||
{"searchstring": normalized, "searchtype": search_type},
|
||||
cache_key=f"wc:{search_type}:{normalized}",
|
||||
cache_ttl=cache_ttl,
|
||||
)
|
||||
return self._table_rows(payload)
|
||||
|
||||
def report_query(
|
||||
self,
|
||||
codes: str | list[str],
|
||||
begin_date: str,
|
||||
end_date: str,
|
||||
cache_ttl: int = 300,
|
||||
) -> list[dict[str, Any]]:
|
||||
code_text = self._codes(codes)
|
||||
payload = self._request(
|
||||
"report_query",
|
||||
{
|
||||
"codes": code_text,
|
||||
"beginrDate": self._display_date(begin_date),
|
||||
"endrDate": self._display_date(end_date),
|
||||
"outputpara": (
|
||||
"reportDate:Y,thscode:Y,secName:Y,ctime:Y,"
|
||||
"reportTitle:Y,pdfURL:Y,seq:Y"
|
||||
),
|
||||
},
|
||||
cache_key=f"report:{code_text}:{begin_date}:{end_date}",
|
||||
cache_ttl=cache_ttl,
|
||||
)
|
||||
return self._table_rows(payload)
|
||||
|
||||
def _request(
|
||||
self,
|
||||
endpoint: str,
|
||||
body: dict[str, Any],
|
||||
cache_key: str = "",
|
||||
cache_ttl: int = 0,
|
||||
) -> dict[str, Any]:
|
||||
if not self.configured:
|
||||
raise IfindError("iFinD 尚未配置。")
|
||||
if cache_key and cache_ttl > 0:
|
||||
cached = self._cached(cache_key, cache_ttl)
|
||||
if cached is not None:
|
||||
return cached
|
||||
|
||||
payload = self._post(endpoint, body, self._ensure_access_token())
|
||||
if self._is_auth_error(payload) and self._refresh_token:
|
||||
self._invalidate_access_token()
|
||||
payload = self._post(endpoint, body, self._ensure_access_token(force=True))
|
||||
self._validate_payload(payload)
|
||||
if cache_key and cache_ttl > 0:
|
||||
with self._cache_lock:
|
||||
self._cache[cache_key] = {
|
||||
"created_at": time.time(),
|
||||
"payload": copy.deepcopy(payload),
|
||||
}
|
||||
return payload
|
||||
|
||||
def _ensure_access_token(self, force: bool = False) -> str:
|
||||
with self._token_lock:
|
||||
now = datetime.now().astimezone().replace(tzinfo=None)
|
||||
token_valid = bool(self._access_token) and (
|
||||
self._access_expires_at is None
|
||||
or self._access_expires_at > now + timedelta(minutes=2)
|
||||
)
|
||||
if token_valid and not force:
|
||||
return self._access_token
|
||||
if not self._refresh_token:
|
||||
if self._access_token:
|
||||
return self._access_token
|
||||
raise IfindError("iFinD Refresh Token 尚未配置。")
|
||||
payload = self._post(self.AUTH_ENDPOINT, {}, "", self._refresh_token)
|
||||
self._validate_payload(payload)
|
||||
data = payload.get("data") or {}
|
||||
token = str(data.get("access_token") or "").strip()
|
||||
if not token:
|
||||
raise IfindError("iFinD 未返回 Access Token。")
|
||||
expires_at = self._parse_datetime(data.get("expired_time"))
|
||||
self._access_token = token
|
||||
self._access_expires_at = expires_at
|
||||
return token
|
||||
|
||||
def _post(
|
||||
self,
|
||||
endpoint: str,
|
||||
body: dict[str, Any],
|
||||
access_token: str,
|
||||
refresh_token: str = "",
|
||||
) -> dict[str, Any]:
|
||||
headers = {
|
||||
"Accept": "application/json",
|
||||
"Content-Type": "application/json",
|
||||
"User-Agent": "XiaobaiReviewWeb/1.0",
|
||||
"ifindlang": "cn",
|
||||
}
|
||||
if access_token:
|
||||
headers["access_token"] = access_token
|
||||
if refresh_token:
|
||||
headers["refresh_token"] = refresh_token
|
||||
request = urllib.request.Request(
|
||||
f"{self.BASE_URL}/{endpoint}",
|
||||
data=json.dumps(body, ensure_ascii=False, separators=(",", ":")).encode("utf-8"),
|
||||
headers=headers,
|
||||
method="POST",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=self.timeout) as response:
|
||||
payload = json.loads(response.read().decode("utf-8"))
|
||||
except urllib.error.HTTPError as exc:
|
||||
detail = ""
|
||||
try:
|
||||
detail_payload = json.loads(exc.read().decode("utf-8", errors="replace"))
|
||||
detail = str(detail_payload.get("errmsg") or detail_payload.get("message") or "")
|
||||
except (json.JSONDecodeError, OSError):
|
||||
pass
|
||||
raise IfindError(f"iFinD HTTP {exc.code}{f':{detail[:160]}' if detail else ''}") from exc
|
||||
except (urllib.error.URLError, TimeoutError, OSError, json.JSONDecodeError) as exc:
|
||||
raise IfindError("iFinD 数据请求失败。") from exc
|
||||
if not isinstance(payload, dict):
|
||||
raise IfindError("iFinD 返回格式不正确。")
|
||||
return payload
|
||||
|
||||
def _cached(self, key: str, ttl: int) -> dict[str, Any] | None:
|
||||
with self._cache_lock:
|
||||
cached = self._cache.get(key)
|
||||
if not cached:
|
||||
return None
|
||||
if time.time() - float(cached.get("created_at") or 0) > ttl:
|
||||
self._cache.pop(key, None)
|
||||
return None
|
||||
return copy.deepcopy(cached["payload"])
|
||||
|
||||
def _invalidate_access_token(self) -> None:
|
||||
with self._token_lock:
|
||||
self._access_token = ""
|
||||
self._access_expires_at = None
|
||||
|
||||
@classmethod
|
||||
def _validate_payload(cls, payload: dict[str, Any]) -> None:
|
||||
try:
|
||||
error_code = int(payload.get("errorcode") or 0)
|
||||
except (TypeError, ValueError):
|
||||
error_code = -1
|
||||
if error_code != 0:
|
||||
message = str(payload.get("errmsg") or "未知错误")
|
||||
raise IfindError(f"iFinD 返回错误:{message[:200]}")
|
||||
|
||||
@classmethod
|
||||
def _is_auth_error(cls, payload: dict[str, Any]) -> bool:
|
||||
try:
|
||||
error_code = int(payload.get("errorcode") or 0)
|
||||
except (TypeError, ValueError):
|
||||
error_code = 0
|
||||
message = str(payload.get("errmsg") or "").casefold()
|
||||
return error_code in cls.AUTH_ERROR_CODES or "token" in message or "鉴权" in message
|
||||
|
||||
@staticmethod
|
||||
def _table_rows(payload: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
tables = payload.get("tables") or []
|
||||
if isinstance(tables, dict):
|
||||
tables = [tables]
|
||||
rows: list[dict[str, Any]] = []
|
||||
for block in tables if isinstance(tables, list) else []:
|
||||
if not isinstance(block, dict):
|
||||
continue
|
||||
table = block.get("table") or {}
|
||||
if not isinstance(table, dict):
|
||||
continue
|
||||
times = block.get("time") or []
|
||||
codes = block.get("thscode") or block.get("thscodes") or []
|
||||
if isinstance(codes, str):
|
||||
codes = [codes]
|
||||
lengths = [len(value) for value in table.values() if isinstance(value, list)]
|
||||
row_count = max(lengths or [len(times) if isinstance(times, list) else 0, 1 if table else 0])
|
||||
for index in range(row_count):
|
||||
row: dict[str, Any] = {}
|
||||
if isinstance(times, list) and index < len(times):
|
||||
row["time"] = times[index]
|
||||
if codes:
|
||||
row["thscode"] = codes[index] if index < len(codes) else codes[0]
|
||||
for field, values in table.items():
|
||||
if isinstance(values, list):
|
||||
row[field] = values[index] if index < len(values) else None
|
||||
elif index == 0:
|
||||
row[field] = values
|
||||
rows.append(row)
|
||||
return rows
|
||||
|
||||
@staticmethod
|
||||
def _codes(codes: str | list[str]) -> str:
|
||||
if isinstance(codes, list):
|
||||
values = [str(code or "").strip().upper() for code in codes]
|
||||
else:
|
||||
values = [part.strip().upper() for part in str(codes or "").split(",")]
|
||||
values = [value for value in values if value]
|
||||
if not values:
|
||||
raise IfindError("iFinD 证券代码不能为空。")
|
||||
if len(values) > 100:
|
||||
raise IfindError("iFinD 单次证券代码过多。")
|
||||
return ",".join(values)
|
||||
|
||||
@staticmethod
|
||||
def _display_date(value: str) -> str:
|
||||
compact = str(value or "").replace("-", "")
|
||||
if len(compact) != 8 or not compact.isdigit():
|
||||
raise IfindError("iFinD 日期格式不正确。")
|
||||
return f"{compact[:4]}-{compact[4:6]}-{compact[6:]}"
|
||||
|
||||
@staticmethod
|
||||
def _parse_datetime(value: Any) -> datetime | None:
|
||||
text = str(value or "").strip()
|
||||
if not text:
|
||||
return None
|
||||
try:
|
||||
return datetime.fromisoformat(text)
|
||||
except ValueError:
|
||||
return None
|
||||
@@ -0,0 +1,18 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
|
||||
from backend.data.providers.tushare_client import TushareClient
|
||||
|
||||
|
||||
class TushareProvider:
|
||||
def __init__(
|
||||
self,
|
||||
token_supplier: Callable[[], str],
|
||||
client_factory: Callable[[str], TushareClient] = TushareClient,
|
||||
) -> None:
|
||||
self._token_supplier = token_supplier
|
||||
self._client_factory = client_factory
|
||||
|
||||
def client(self) -> TushareClient:
|
||||
return self._client_factory(str(self._token_supplier() or "").strip())
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,202 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from datetime import date, datetime, time, timedelta, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
|
||||
|
||||
from backend.bootstrap.config import APP_DIR
|
||||
from backend.data.contracts import DataUsage
|
||||
from backend.data.policy import DataPolicyError, DataSourcePolicy
|
||||
|
||||
|
||||
class DataQualityError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
def market_timezone(name: str = "Asia/Shanghai"):
|
||||
try:
|
||||
return ZoneInfo(name)
|
||||
except ZoneInfoNotFoundError:
|
||||
if name != "Asia/Shanghai":
|
||||
raise
|
||||
return timezone(timedelta(hours=8), name)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class QualityEvidence:
|
||||
dataset_id: str
|
||||
provider_id: str
|
||||
data_time: str | datetime
|
||||
observed_at: str | datetime
|
||||
actual_count: int | None = None
|
||||
expected_count: int | None = None
|
||||
units: dict[str, str] | None = None
|
||||
adjustment: str = ""
|
||||
available_at: str | datetime | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class QualityReport:
|
||||
accepted: bool
|
||||
dataset_id: str
|
||||
provider_id: str
|
||||
usage: DataUsage
|
||||
coverage_ratio: float | None
|
||||
age_seconds: float
|
||||
issues: tuple[str, ...]
|
||||
|
||||
def as_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"accepted": self.accepted,
|
||||
"dataset_id": self.dataset_id,
|
||||
"provider_id": self.provider_id,
|
||||
"usage": self.usage,
|
||||
"coverage_ratio": self.coverage_ratio,
|
||||
"age_seconds": round(self.age_seconds, 3),
|
||||
"issues": list(self.issues),
|
||||
}
|
||||
|
||||
|
||||
class DataQualityGate:
|
||||
def __init__(
|
||||
self,
|
||||
source_policy: DataSourcePolicy,
|
||||
payload: dict[str, Any],
|
||||
) -> None:
|
||||
self.source_policy = source_policy
|
||||
self.timezone = market_timezone(
|
||||
str(payload.get("timezone") or "Asia/Shanghai")
|
||||
)
|
||||
self.defaults = dict(payload.get("defaults") or {})
|
||||
self.unit_profiles = dict(payload.get("unit_profiles") or {})
|
||||
self.rules = dict(payload.get("datasets") or {})
|
||||
|
||||
@classmethod
|
||||
def load(
|
||||
cls,
|
||||
source_policy: DataSourcePolicy,
|
||||
path: Path | None = None,
|
||||
) -> "DataQualityGate":
|
||||
config_path = path or APP_DIR / "config" / "data-quality.config.json"
|
||||
payload = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
return cls(source_policy, payload)
|
||||
|
||||
def evaluate(
|
||||
self,
|
||||
evidence: QualityEvidence,
|
||||
usage: DataUsage,
|
||||
as_of: str | datetime | None = None,
|
||||
) -> QualityReport:
|
||||
issues: list[str] = []
|
||||
try:
|
||||
self.source_policy.assert_allowed(
|
||||
evidence.dataset_id, evidence.provider_id, usage
|
||||
)
|
||||
except DataPolicyError as exc:
|
||||
issues.append(str(exc))
|
||||
|
||||
rule = self.rules.get(evidence.dataset_id)
|
||||
if rule is None:
|
||||
issues.append(f"Missing quality rule: {evidence.dataset_id}")
|
||||
rule = {}
|
||||
if rule.get("blocked"):
|
||||
issues.append(f"Dataset quality is blocked: {evidence.dataset_id}")
|
||||
|
||||
reference = self._datetime(as_of or datetime.now(self.timezone))
|
||||
data_time = self._datetime(evidence.data_time)
|
||||
observed_at = self._datetime(evidence.observed_at)
|
||||
tolerance = float(
|
||||
(self.defaults.get(usage) or {}).get("future_tolerance_seconds") or 0
|
||||
)
|
||||
if data_time > reference + timedelta(seconds=tolerance):
|
||||
issues.append("Data time is later than the evaluation time")
|
||||
if observed_at > reference + timedelta(seconds=tolerance):
|
||||
issues.append("Observation time is later than the evaluation time")
|
||||
if observed_at < data_time:
|
||||
issues.append("Observation time precedes data time")
|
||||
|
||||
age_seconds = max(0.0, (reference - data_time).total_seconds())
|
||||
freshness = rule.get("freshness_seconds")
|
||||
if freshness is not None and age_seconds > float(freshness):
|
||||
issues.append(
|
||||
f"Data is stale: {age_seconds:.1f}s exceeds {float(freshness):.1f}s"
|
||||
)
|
||||
|
||||
coverage_ratio: float | None = None
|
||||
if evidence.expected_count is not None:
|
||||
if evidence.expected_count <= 0:
|
||||
issues.append("Expected count must be positive")
|
||||
elif evidence.actual_count is None or evidence.actual_count < 0:
|
||||
issues.append("Actual count is missing or invalid")
|
||||
else:
|
||||
coverage_ratio = min(1.0, evidence.actual_count / evidence.expected_count)
|
||||
minimum = float(rule.get("min_coverage_ratio") or 0)
|
||||
if coverage_ratio < minimum:
|
||||
issues.append(
|
||||
f"Coverage {coverage_ratio:.3f} is below {minimum:.3f}"
|
||||
)
|
||||
|
||||
required_adjustment = str(rule.get("adjustment") or "")
|
||||
if required_adjustment and evidence.adjustment != required_adjustment:
|
||||
issues.append(
|
||||
f"Adjustment {evidence.adjustment or 'missing'} does not match {required_adjustment}"
|
||||
)
|
||||
|
||||
profile_id = str(rule.get("unit_profile") or "none")
|
||||
required_units = dict(self.unit_profiles.get(profile_id) or {})
|
||||
supplied_units = evidence.units or {}
|
||||
for field, expected_unit in required_units.items():
|
||||
actual_unit = supplied_units.get(field)
|
||||
if actual_unit != expected_unit:
|
||||
issues.append(
|
||||
f"Unit for {field} is {actual_unit or 'missing'}, expected {expected_unit}"
|
||||
)
|
||||
|
||||
if rule.get("point_in_time") == "announcement_date" and usage == "calculation":
|
||||
if evidence.available_at is None:
|
||||
issues.append("Point-in-time availability is missing")
|
||||
elif self._datetime(evidence.available_at) > reference:
|
||||
issues.append("Point-in-time data was not available at evaluation time")
|
||||
|
||||
return QualityReport(
|
||||
accepted=not issues,
|
||||
dataset_id=evidence.dataset_id,
|
||||
provider_id=evidence.provider_id,
|
||||
usage=usage,
|
||||
coverage_ratio=coverage_ratio,
|
||||
age_seconds=age_seconds,
|
||||
issues=tuple(issues),
|
||||
)
|
||||
|
||||
def require(
|
||||
self,
|
||||
evidence: QualityEvidence,
|
||||
usage: DataUsage,
|
||||
as_of: str | datetime | None = None,
|
||||
) -> QualityReport:
|
||||
report = self.evaluate(evidence, usage, as_of)
|
||||
if not report.accepted:
|
||||
raise DataQualityError("; ".join(report.issues))
|
||||
return report
|
||||
|
||||
def _datetime(self, value: str | datetime) -> datetime:
|
||||
if isinstance(value, datetime):
|
||||
parsed = value
|
||||
else:
|
||||
text = str(value or "").strip()
|
||||
if not text:
|
||||
raise DataQualityError("Quality evidence timestamp is missing")
|
||||
try:
|
||||
parsed = datetime.fromisoformat(text)
|
||||
except ValueError:
|
||||
try:
|
||||
day = date.fromisoformat(text)
|
||||
except ValueError as exc:
|
||||
raise DataQualityError(f"Invalid quality timestamp: {text}") from exc
|
||||
parsed = datetime.combine(day, time.min)
|
||||
if parsed.tzinfo is None:
|
||||
return parsed.replace(tzinfo=self.timezone)
|
||||
return parsed.astimezone(self.timezone)
|
||||
@@ -0,0 +1,426 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import http.client
|
||||
import json
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from threading import Lock
|
||||
from typing import Any, ClassVar
|
||||
|
||||
|
||||
class RealtimeAggregateError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
EASTMONEY_INDEX_URL = "https://push2.eastmoney.com/api/qt/ulist.np/get"
|
||||
EASTMONEY_SECTOR_URL = "https://push2.eastmoney.com/api/qt/clist/get"
|
||||
TENCENT_INDEX_URL = "https://qt.gtimg.cn/q=sh000001,sz399001,sz399006"
|
||||
THS_LIMIT_URL = "https://data.10jqka.com.cn/dataapi/limit_up/limit_up_pool"
|
||||
XGB_POOL_URL = "https://flash-api.xuangubao.cn/api/pool/detail"
|
||||
BROWSER_USER_AGENT = (
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
|
||||
"AppleWebKit/537.36 (KHTML, like Gecko) "
|
||||
"Chrome/138.0.0.0 Safari/537.36"
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class WebRealtimeAggregator:
|
||||
timeout: int = 8
|
||||
retry_attempts: int = 3
|
||||
retry_delay_seconds: float = 0.2
|
||||
response_cache_ttl_seconds: int = 90
|
||||
_sector_cache: ClassVar[dict[str, Any]] = {}
|
||||
_sector_cache_lock: ClassVar[Lock] = Lock()
|
||||
_response_cache: ClassVar[dict[str, dict[str, Any]]] = {}
|
||||
_response_cache_lock: ClassVar[Lock] = Lock()
|
||||
|
||||
def health_snapshot(self, sector: str = "") -> dict[str, Any]:
|
||||
started = time.perf_counter()
|
||||
sources: dict[str, dict[str, Any]] = {}
|
||||
indices: list[dict[str, Any]] = []
|
||||
sector_payload: dict[str, Any] | None = None
|
||||
|
||||
indices, sources["eastmoney_indices"] = self._capture(self.eastmoney_indices)
|
||||
if sector.strip():
|
||||
sector_payload, sources["eastmoney_sector"] = self._capture(
|
||||
lambda: self.eastmoney_sector(sector)
|
||||
)
|
||||
ths_observation, sources["ths_limit_pool"] = self._capture(self.ths_limit_pool)
|
||||
xgb_observation, sources["xgb_limit_pool"] = self._capture(self.xgb_limit_pool)
|
||||
|
||||
index_times = [int(item.get("quote_time_epoch") or 0) for item in indices or []]
|
||||
now = datetime.now().astimezone()
|
||||
max_skew = 120 if now.hour >= 15 else 15
|
||||
index_consistent = bool(index_times) and max(index_times) - min(index_times) <= max_skew
|
||||
ready = (
|
||||
bool(indices)
|
||||
and len(indices) == 3
|
||||
and index_consistent
|
||||
and (not sector.strip() or bool(sector_payload))
|
||||
)
|
||||
return {
|
||||
"ready": ready,
|
||||
"isolated": True,
|
||||
"generated_at": datetime.now().astimezone().isoformat(timespec="seconds"),
|
||||
"elapsed_ms": round((time.perf_counter() - started) * 1000),
|
||||
"indices": indices or [],
|
||||
"index_consistent": index_consistent,
|
||||
"sector": sector_payload,
|
||||
"sources": sources,
|
||||
"observations": {
|
||||
"ths_limit_pool": ths_observation,
|
||||
"xgb_limit_pool": xgb_observation,
|
||||
},
|
||||
"policy": {
|
||||
"integration": "heaven_realtime_fallback",
|
||||
"max_index_time_skew_seconds": max_skew,
|
||||
"notice": "聚合源仅作为盘中观势的实时指数与板块外显,主行情快照仍由Tushare维护。",
|
||||
},
|
||||
}
|
||||
|
||||
def eastmoney_indices(self) -> list[dict[str, Any]]:
|
||||
try:
|
||||
payload = self._get_json(
|
||||
EASTMONEY_INDEX_URL,
|
||||
{
|
||||
"secids": "1.000001,0.399001,0.399006",
|
||||
"fltt": "2",
|
||||
"invt": "2",
|
||||
"fields": "f12,f14,f2,f3,f4,f15,f16,f17,f18,f6,f124",
|
||||
},
|
||||
referer="https://quote.eastmoney.com/",
|
||||
)
|
||||
except RealtimeAggregateError:
|
||||
return self.tencent_indices()
|
||||
cache_meta = payload.get("_aggregate_cache") or {}
|
||||
rows = list((payload.get("data") or {}).get("diff") or [])
|
||||
result = []
|
||||
for row in rows:
|
||||
code = str(row.get("f12") or "")
|
||||
if code not in {"000001", "399001", "399006"}:
|
||||
continue
|
||||
epoch = int(_number(row.get("f124")))
|
||||
result.append(
|
||||
{
|
||||
"code": code,
|
||||
"name": row.get("f14") or code,
|
||||
"price": _number(row.get("f2")),
|
||||
"change": _number(row.get("f3")),
|
||||
"change_amount": _number(row.get("f4")),
|
||||
"open": _number(row.get("f17")),
|
||||
"high": _number(row.get("f15")),
|
||||
"low": _number(row.get("f16")),
|
||||
"previous_close": _number(row.get("f18")),
|
||||
"amount_billion": round(_number(row.get("f6")) / 100000000, 2),
|
||||
"quote_time_epoch": epoch,
|
||||
"quote_time": (
|
||||
datetime.fromtimestamp(epoch).astimezone().isoformat(timespec="seconds")
|
||||
if epoch else ""
|
||||
),
|
||||
"source": (
|
||||
"eastmoney_push2_cache" if cache_meta else "eastmoney_push2"
|
||||
),
|
||||
"cache_age_seconds": cache_meta.get("age_seconds", 0),
|
||||
}
|
||||
)
|
||||
if len(result) != 3:
|
||||
raise RealtimeAggregateError(f"Eastmoney returned {len(result)}/3 indices")
|
||||
return result
|
||||
|
||||
def tencent_indices(self) -> list[dict[str, Any]]:
|
||||
raw, cache_age = self._get_text(
|
||||
TENCENT_INDEX_URL,
|
||||
referer="https://gu.qq.com/",
|
||||
encoding="gb18030",
|
||||
)
|
||||
result = []
|
||||
for line in raw.splitlines():
|
||||
if '="' not in line:
|
||||
continue
|
||||
fields = line.split('="', 1)[1].rsplit('";', 1)[0].split("~")
|
||||
if len(fields) < 38:
|
||||
continue
|
||||
code = fields[2]
|
||||
if code not in {"000001", "399001", "399006"}:
|
||||
continue
|
||||
try:
|
||||
quote_time = datetime.strptime(fields[30], "%Y%m%d%H%M%S").astimezone()
|
||||
except ValueError as exc:
|
||||
raise RealtimeAggregateError(
|
||||
f"Tencent returned invalid quote time for {code}"
|
||||
) from exc
|
||||
result.append(
|
||||
{
|
||||
"code": code,
|
||||
"name": fields[1] or code,
|
||||
"price": _number(fields[3]),
|
||||
"change": _number(fields[32]),
|
||||
"change_amount": _number(fields[31]),
|
||||
"open": _number(fields[5]),
|
||||
"high": _number(fields[33]),
|
||||
"low": _number(fields[34]),
|
||||
"previous_close": _number(fields[4]),
|
||||
"amount_billion": round(_number(fields[37]) / 10000, 2),
|
||||
"quote_time_epoch": int(quote_time.timestamp()),
|
||||
"quote_time": quote_time.isoformat(timespec="seconds"),
|
||||
"source": "tencent_qt_cache" if cache_age else "tencent_qt",
|
||||
"cache_age_seconds": cache_age,
|
||||
}
|
||||
)
|
||||
if len(result) != 3:
|
||||
raise RealtimeAggregateError(f"Tencent returned {len(result)}/3 indices")
|
||||
return result
|
||||
|
||||
def eastmoney_sector(self, query: str) -> dict[str, Any]:
|
||||
target = _normalize_sector(query)
|
||||
candidates = self._eastmoney_sector_catalog()
|
||||
matched = _match_sector(candidates, target)
|
||||
if not matched:
|
||||
raise RealtimeAggregateError(f"Eastmoney sector not found: {query}")
|
||||
epoch = int(_number(matched.get("f124")))
|
||||
return {
|
||||
"code": matched.get("f12") or "",
|
||||
"name": matched.get("f14") or query,
|
||||
"price": _number(matched.get("f2")),
|
||||
"change": _number(matched.get("f3")),
|
||||
"change_amount": _number(matched.get("f4")),
|
||||
"turnover_rate": _number(matched.get("f8")),
|
||||
"up_count": int(_number(matched.get("f104"))),
|
||||
"down_count": int(_number(matched.get("f105"))),
|
||||
"leader": matched.get("f128") or "--",
|
||||
"leader_code": matched.get("f140") or "",
|
||||
"leading_pct": _number(matched.get("f136")),
|
||||
"quote_time_epoch": epoch,
|
||||
"quote_time": (
|
||||
datetime.fromtimestamp(epoch).astimezone().isoformat(timespec="seconds")
|
||||
if epoch else ""
|
||||
),
|
||||
"source": "eastmoney_push2",
|
||||
"match_query": query,
|
||||
}
|
||||
|
||||
def _eastmoney_sector_catalog(self) -> list[dict[str, Any]]:
|
||||
now = time.time()
|
||||
with self._sector_cache_lock:
|
||||
cached = self._sector_cache.get("eastmoney")
|
||||
if cached and now - float(cached.get("created_at") or 0) < 600:
|
||||
return list(cached.get("rows") or [])
|
||||
|
||||
def load_page(page: int) -> list[dict[str, Any]]:
|
||||
payload = self._get_json(
|
||||
EASTMONEY_SECTOR_URL,
|
||||
{
|
||||
"pn": str(page),
|
||||
"pz": "100",
|
||||
"po": "1",
|
||||
"np": "1",
|
||||
"fltt": "2",
|
||||
"invt": "2",
|
||||
"fid": "f3",
|
||||
"fs": "m:90+t:2",
|
||||
"fields": "f12,f14,f2,f3,f4,f8,f104,f105,f128,f136,f140,f124",
|
||||
},
|
||||
referer="https://quote.eastmoney.com/center/boardlist.html",
|
||||
)
|
||||
return list((payload.get("data") or {}).get("diff") or [])
|
||||
|
||||
with ThreadPoolExecutor(max_workers=5) as executor:
|
||||
pages = list(executor.map(load_page, range(1, 6)))
|
||||
rows = [row for page in pages for row in page]
|
||||
if not rows:
|
||||
raise RealtimeAggregateError("Eastmoney sector catalog is empty")
|
||||
with self._sector_cache_lock:
|
||||
self._sector_cache["eastmoney"] = {"created_at": now, "rows": rows}
|
||||
return rows
|
||||
|
||||
def ths_limit_pool(self) -> dict[str, Any]:
|
||||
payload = self._get_json(
|
||||
THS_LIMIT_URL,
|
||||
{"page": "1", "limit": "3", "field": "199112"},
|
||||
referer="https://data.10jqka.com.cn/limit_up/",
|
||||
)
|
||||
data = payload.get("data") or payload
|
||||
return {
|
||||
"available": True,
|
||||
"keys": sorted(str(key) for key in data.keys()) if isinstance(data, dict) else [],
|
||||
"source": "ths_web_dataapi",
|
||||
}
|
||||
|
||||
def xgb_limit_pool(self) -> dict[str, Any]:
|
||||
payload = self._get_json(
|
||||
XGB_POOL_URL,
|
||||
{"pool_name": "limit_up"},
|
||||
referer="https://xuangubao.cn/",
|
||||
)
|
||||
data = payload.get("data") or {}
|
||||
rows = data if isinstance(data, list) else data.get("pool") or data.get("list") or []
|
||||
return {
|
||||
"available": True,
|
||||
"count": len(rows) if isinstance(rows, list) else 0,
|
||||
"source": "xuangubao_web_api",
|
||||
}
|
||||
|
||||
def _capture(self, operation):
|
||||
started = time.perf_counter()
|
||||
try:
|
||||
value = operation()
|
||||
return value, {
|
||||
"ok": True,
|
||||
"elapsed_ms": round((time.perf_counter() - started) * 1000),
|
||||
"error": "",
|
||||
}
|
||||
except Exception as exc:
|
||||
return None, {
|
||||
"ok": False,
|
||||
"elapsed_ms": round((time.perf_counter() - started) * 1000),
|
||||
"error": str(exc)[:500],
|
||||
}
|
||||
|
||||
def _get_json(
|
||||
self,
|
||||
url: str,
|
||||
params: dict[str, str],
|
||||
referer: str,
|
||||
) -> dict[str, Any]:
|
||||
request_url = f"{url}?{urllib.parse.urlencode(params)}"
|
||||
last_error: Exception | None = None
|
||||
attempts = max(1, int(self.retry_attempts))
|
||||
for attempt in range(attempts):
|
||||
request = urllib.request.Request(
|
||||
request_url,
|
||||
headers={
|
||||
"Accept": "application/json,text/plain,*/*",
|
||||
"Connection": "close",
|
||||
"Referer": referer,
|
||||
"User-Agent": BROWSER_USER_AGENT,
|
||||
},
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=self.timeout) as response:
|
||||
content_type = response.headers.get("Content-Type", "")
|
||||
raw = response.read().decode("utf-8", errors="replace")
|
||||
if "json" not in content_type.lower() and not raw.lstrip().startswith(("{", "[")):
|
||||
raise RealtimeAggregateError(
|
||||
f"non-JSON response: {raw[:120].strip()}"
|
||||
)
|
||||
payload = json.loads(raw)
|
||||
if not isinstance(payload, dict):
|
||||
raise RealtimeAggregateError("unexpected response shape")
|
||||
if payload.get("rc") not in (None, 0):
|
||||
raise RealtimeAggregateError(f"provider rc={payload.get('rc')}")
|
||||
with self._response_cache_lock:
|
||||
self._response_cache[request_url] = {
|
||||
"created_at": time.time(),
|
||||
"payload": copy.deepcopy(payload),
|
||||
}
|
||||
return payload
|
||||
except (
|
||||
urllib.error.URLError,
|
||||
TimeoutError,
|
||||
ConnectionError,
|
||||
OSError,
|
||||
http.client.HTTPException,
|
||||
json.JSONDecodeError,
|
||||
RealtimeAggregateError,
|
||||
) as exc:
|
||||
last_error = exc
|
||||
if attempt + 1 < attempts and self.retry_delay_seconds > 0:
|
||||
time.sleep(self.retry_delay_seconds * (attempt + 1))
|
||||
|
||||
now = time.time()
|
||||
with self._response_cache_lock:
|
||||
cached = self._response_cache.get(request_url)
|
||||
cache_age = now - float((cached or {}).get("created_at") or 0)
|
||||
if cached and cache_age <= self.response_cache_ttl_seconds:
|
||||
payload = copy.deepcopy(cached.get("payload") or {})
|
||||
payload["_aggregate_cache"] = {"age_seconds": round(cache_age, 1)}
|
||||
return payload
|
||||
raise RealtimeAggregateError(f"request failed after {attempts} attempts: {last_error}") from last_error
|
||||
|
||||
def _get_text(
|
||||
self,
|
||||
request_url: str,
|
||||
referer: str,
|
||||
encoding: str = "utf-8",
|
||||
) -> tuple[str, float]:
|
||||
cache_key = f"text:{request_url}"
|
||||
last_error: Exception | None = None
|
||||
attempts = max(1, int(self.retry_attempts))
|
||||
for attempt in range(attempts):
|
||||
request = urllib.request.Request(
|
||||
request_url,
|
||||
headers={
|
||||
"Accept": "text/plain,*/*",
|
||||
"Connection": "close",
|
||||
"Referer": referer,
|
||||
"User-Agent": BROWSER_USER_AGENT,
|
||||
},
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=self.timeout) as response:
|
||||
raw = response.read().decode(encoding, errors="replace")
|
||||
if not raw.strip():
|
||||
raise RealtimeAggregateError("empty text response")
|
||||
with self._response_cache_lock:
|
||||
self._response_cache[cache_key] = {
|
||||
"created_at": time.time(),
|
||||
"payload": raw,
|
||||
}
|
||||
return raw, 0
|
||||
except (
|
||||
urllib.error.URLError,
|
||||
TimeoutError,
|
||||
ConnectionError,
|
||||
OSError,
|
||||
http.client.HTTPException,
|
||||
RealtimeAggregateError,
|
||||
) as exc:
|
||||
last_error = exc
|
||||
if attempt + 1 < attempts and self.retry_delay_seconds > 0:
|
||||
time.sleep(self.retry_delay_seconds * (attempt + 1))
|
||||
|
||||
now = time.time()
|
||||
with self._response_cache_lock:
|
||||
cached = self._response_cache.get(cache_key)
|
||||
cache_age = now - float((cached or {}).get("created_at") or 0)
|
||||
if cached and cache_age <= self.response_cache_ttl_seconds:
|
||||
return str(cached.get("payload") or ""), round(cache_age, 1)
|
||||
raise RealtimeAggregateError(
|
||||
f"text request failed after {attempts} attempts: {last_error}"
|
||||
) from last_error
|
||||
|
||||
|
||||
def _normalize_sector(value: Any) -> str:
|
||||
text = str(value or "").strip().replace(" ", "")
|
||||
for suffix in ("板块", "概念", "行业", "Ⅱ", "Ⅲ", "(A股)", "(A股)"):
|
||||
text = text.replace(suffix, "")
|
||||
aliases = {"元器件": "元件", "电子元器件": "元件"}
|
||||
return aliases.get(text, text)
|
||||
|
||||
|
||||
def _match_sector(rows: list[dict[str, Any]], target: str) -> dict[str, Any] | None:
|
||||
exact = [row for row in rows if _normalize_sector(row.get("f14")) == target]
|
||||
if exact:
|
||||
return min(exact, key=lambda row: len(str(row.get("f14") or "")))
|
||||
fuzzy = [
|
||||
row for row in rows
|
||||
if target and (
|
||||
target in _normalize_sector(row.get("f14"))
|
||||
or _normalize_sector(row.get("f14")) in target
|
||||
)
|
||||
]
|
||||
return min(fuzzy, key=lambda row: len(_normalize_sector(row.get("f14")))) if fuzzy else None
|
||||
|
||||
|
||||
def _number(value: Any, default: float = 0.0) -> float:
|
||||
try:
|
||||
return float(value)
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
@@ -0,0 +1,11 @@
|
||||
from .connection import ManagedConnection, SQLiteConnectionFactory
|
||||
from .migrations import MIGRATIONS, Migration, MigrationError, MigrationRunner
|
||||
|
||||
__all__ = [
|
||||
"MIGRATIONS",
|
||||
"ManagedConnection",
|
||||
"Migration",
|
||||
"MigrationError",
|
||||
"MigrationRunner",
|
||||
"SQLiteConnectionFactory",
|
||||
]
|
||||
@@ -0,0 +1,33 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlite3
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
class ManagedConnection(sqlite3.Connection):
|
||||
"""Commit or roll back, then release the SQLite handle on context exit."""
|
||||
|
||||
def __exit__(self, exc_type, exc_value, traceback):
|
||||
try:
|
||||
return super().__exit__(exc_type, exc_value, traceback)
|
||||
finally:
|
||||
self.close()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SQLiteConnectionFactory:
|
||||
path: Path
|
||||
timeout_seconds: float = 20
|
||||
|
||||
def connect(self) -> sqlite3.Connection:
|
||||
connection = sqlite3.connect(
|
||||
self.path,
|
||||
timeout=self.timeout_seconds,
|
||||
factory=ManagedConnection,
|
||||
)
|
||||
connection.row_factory = sqlite3.Row
|
||||
connection.execute("PRAGMA journal_mode=WAL")
|
||||
connection.execute("PRAGMA foreign_keys=ON")
|
||||
connection.execute("PRAGMA busy_timeout=20000")
|
||||
return connection
|
||||
@@ -0,0 +1,8 @@
|
||||
from .m0001_adopt_legacy import MIGRATION as M0001_ADOPT_LEGACY
|
||||
from .m0002_job_runs import MIGRATION as M0002_JOB_RUNS
|
||||
from .m0003_llm_audit import MIGRATION as M0003_LLM_AUDIT
|
||||
from .runner import Migration, MigrationError, MigrationRunner
|
||||
|
||||
MIGRATIONS = (M0001_ADOPT_LEGACY, M0002_JOB_RUNS, M0003_LLM_AUDIT)
|
||||
|
||||
__all__ = ["MIGRATIONS", "Migration", "MigrationError", "MigrationRunner"]
|
||||
@@ -0,0 +1,42 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlite3
|
||||
|
||||
from backend.database.migrations.runner import Migration, MigrationError
|
||||
|
||||
|
||||
REQUIRED_TABLES = frozenset(
|
||||
{
|
||||
"users",
|
||||
"user_sessions",
|
||||
"dashboard_snapshots",
|
||||
"watchlist",
|
||||
"review_notes",
|
||||
"stock_master",
|
||||
"daily_bars",
|
||||
"screener_runs",
|
||||
"mentor_messages",
|
||||
"trade_entries",
|
||||
"heaven_readings",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def adopt_legacy_schema(connection: sqlite3.Connection) -> None:
|
||||
tables = {
|
||||
str(row["name"])
|
||||
for row in connection.execute(
|
||||
"SELECT name FROM sqlite_master WHERE type = 'table'"
|
||||
)
|
||||
}
|
||||
missing = sorted(REQUIRED_TABLES - tables)
|
||||
if missing:
|
||||
raise MigrationError(f"Legacy schema is incomplete: {', '.join(missing)}")
|
||||
|
||||
|
||||
MIGRATION = Migration(
|
||||
version="0001",
|
||||
name="adopt_legacy_schema",
|
||||
action=adopt_legacy_schema,
|
||||
signature="required-tables:v1:" + ",".join(sorted(REQUIRED_TABLES)),
|
||||
)
|
||||
@@ -0,0 +1,47 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlite3
|
||||
|
||||
from backend.database.migrations.runner import Migration
|
||||
|
||||
|
||||
def create_job_runs(connection: sqlite3.Connection) -> None:
|
||||
connection.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS job_runs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
job_id TEXT NOT NULL,
|
||||
idempotency_key TEXT NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
attempt INTEGER NOT NULL DEFAULT 1,
|
||||
started_at TEXT NOT NULL,
|
||||
finished_at TEXT,
|
||||
elapsed_ms INTEGER NOT NULL DEFAULT 0,
|
||||
error_code TEXT NOT NULL DEFAULT '',
|
||||
message TEXT NOT NULL DEFAULT '',
|
||||
output_version TEXT NOT NULL DEFAULT '',
|
||||
metadata TEXT NOT NULL DEFAULT '{}',
|
||||
UNIQUE(job_id, idempotency_key, attempt)
|
||||
)
|
||||
"""
|
||||
)
|
||||
connection.execute(
|
||||
"""
|
||||
CREATE INDEX IF NOT EXISTS idx_job_runs_job_started
|
||||
ON job_runs(job_id, started_at DESC, id DESC)
|
||||
"""
|
||||
)
|
||||
connection.execute(
|
||||
"""
|
||||
CREATE INDEX IF NOT EXISTS idx_job_runs_status
|
||||
ON job_runs(status, started_at DESC, id DESC)
|
||||
"""
|
||||
)
|
||||
|
||||
|
||||
MIGRATION = Migration(
|
||||
version="0002",
|
||||
name="create_job_runs",
|
||||
action=create_job_runs,
|
||||
signature="job-runs:v1:id,job,key,status,attempt,times,elapsed,error,output,metadata",
|
||||
)
|
||||
@@ -0,0 +1,32 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlite3
|
||||
|
||||
from backend.database.migrations.runner import Migration
|
||||
|
||||
|
||||
def extend_llm_audit(connection: sqlite3.Connection) -> None:
|
||||
columns = {
|
||||
str(row["name"])
|
||||
for row in connection.execute("PRAGMA table_info(llm_usage)")
|
||||
}
|
||||
additions = (
|
||||
("role", "TEXT NOT NULL DEFAULT ''"),
|
||||
("prompt_version", "TEXT NOT NULL DEFAULT ''"),
|
||||
("error_code", "TEXT NOT NULL DEFAULT ''"),
|
||||
("input_tokens", "INTEGER NOT NULL DEFAULT 0"),
|
||||
("output_tokens", "INTEGER NOT NULL DEFAULT 0"),
|
||||
)
|
||||
for name, declaration in additions:
|
||||
if name not in columns:
|
||||
connection.execute(
|
||||
f"ALTER TABLE llm_usage ADD COLUMN {name} {declaration}"
|
||||
)
|
||||
|
||||
|
||||
MIGRATION = Migration(
|
||||
version="0003",
|
||||
name="extend_llm_audit",
|
||||
action=extend_llm_audit,
|
||||
signature="llm-audit:v1:role,prompt-version,error-code,input-tokens,output-tokens",
|
||||
)
|
||||
@@ -0,0 +1,98 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import sqlite3
|
||||
from collections.abc import Callable, Iterable
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
MigrationAction = Callable[[sqlite3.Connection], None]
|
||||
|
||||
|
||||
class MigrationError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Migration:
|
||||
version: str
|
||||
name: str
|
||||
action: MigrationAction
|
||||
signature: str
|
||||
|
||||
@property
|
||||
def checksum(self) -> str:
|
||||
return hashlib.sha256(self.signature.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
class MigrationRunner:
|
||||
def apply(
|
||||
self,
|
||||
connection: sqlite3.Connection,
|
||||
migrations: Iterable[Migration],
|
||||
) -> tuple[str, ...]:
|
||||
ordered = sorted(migrations, key=lambda item: item.version)
|
||||
versions = [item.version for item in ordered]
|
||||
if versions != sorted(set(versions)):
|
||||
raise MigrationError("Migration versions must be unique and ordered")
|
||||
self._ensure_ledger(connection)
|
||||
applied = {
|
||||
str(row["version"]): str(row["checksum"])
|
||||
for row in connection.execute(
|
||||
"SELECT version, checksum FROM schema_migrations ORDER BY version"
|
||||
)
|
||||
}
|
||||
known = set(versions)
|
||||
unknown = sorted(set(applied) - known)
|
||||
if unknown:
|
||||
raise MigrationError(f"Database contains unknown migrations: {', '.join(unknown)}")
|
||||
|
||||
completed: list[str] = []
|
||||
for migration in ordered:
|
||||
existing = applied.get(migration.version)
|
||||
if existing:
|
||||
if existing != migration.checksum:
|
||||
raise MigrationError(
|
||||
f"Migration checksum changed: {migration.version} {migration.name}"
|
||||
)
|
||||
continue
|
||||
savepoint = f"migration_{migration.version.replace('-', '_')}"
|
||||
connection.execute(f"SAVEPOINT {savepoint}")
|
||||
try:
|
||||
migration.action(connection)
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO schema_migrations
|
||||
(version, name, checksum, applied_at)
|
||||
VALUES (?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
migration.version,
|
||||
migration.name,
|
||||
migration.checksum,
|
||||
datetime.now(timezone.utc).isoformat(),
|
||||
),
|
||||
)
|
||||
connection.execute(f"RELEASE SAVEPOINT {savepoint}")
|
||||
except Exception as exc:
|
||||
connection.execute(f"ROLLBACK TO SAVEPOINT {savepoint}")
|
||||
connection.execute(f"RELEASE SAVEPOINT {savepoint}")
|
||||
raise MigrationError(
|
||||
f"Migration failed: {migration.version} {migration.name}"
|
||||
) from exc
|
||||
completed.append(migration.version)
|
||||
return tuple(completed)
|
||||
|
||||
@staticmethod
|
||||
def _ensure_ledger(connection: sqlite3.Connection) -> None:
|
||||
connection.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
version TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
checksum TEXT NOT NULL,
|
||||
applied_at TEXT NOT NULL
|
||||
)
|
||||
"""
|
||||
)
|
||||
@@ -0,0 +1,21 @@
|
||||
from .ports import AlertRepository, StrategyTrackingRepository, TradeJournalRepository
|
||||
from .sqlite import (
|
||||
RepositoryBundle,
|
||||
SQLiteAlertRepository,
|
||||
SQLiteStrategyTrackingRepository,
|
||||
SQLiteTradeJournalRepository,
|
||||
build_repository_bundle,
|
||||
require_user_id,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"AlertRepository",
|
||||
"RepositoryBundle",
|
||||
"SQLiteAlertRepository",
|
||||
"SQLiteStrategyTrackingRepository",
|
||||
"SQLiteTradeJournalRepository",
|
||||
"StrategyTrackingRepository",
|
||||
"TradeJournalRepository",
|
||||
"build_repository_bundle",
|
||||
"require_user_id",
|
||||
]
|
||||
@@ -0,0 +1,52 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Protocol
|
||||
|
||||
|
||||
class AlertRepository(Protocol):
|
||||
def save_alert(
|
||||
self, user_id: int, kind: str, title: str, content: str,
|
||||
available_date: str, code: str, dedupe_key: str,
|
||||
) -> int: ...
|
||||
|
||||
def list_alerts(
|
||||
self, user_id: int, as_of: str, unread_only: bool = False, limit: int = 100,
|
||||
) -> list[dict[str, Any]]: ...
|
||||
|
||||
def count_unread_alerts(self, user_id: int, as_of: str) -> int: ...
|
||||
|
||||
def mark_alert_read(self, user_id: int, alert_id: int) -> bool: ...
|
||||
|
||||
def mark_all_alerts_read(self, user_id: int, as_of: str) -> int: ...
|
||||
|
||||
def delete_alert(self, user_id: int, alert_id: int) -> bool: ...
|
||||
|
||||
|
||||
class TradeJournalRepository(Protocol):
|
||||
def save_trade_entry(self, *args: Any, **kwargs: Any) -> int: ...
|
||||
|
||||
def list_trade_entries(
|
||||
self, user_id: int, start_date: str = "", end_date: str = "",
|
||||
code: str = "", limit: int = 300,
|
||||
) -> list[dict[str, Any]]: ...
|
||||
|
||||
def delete_trade_entry(self, user_id: int, trade_id: int) -> bool: ...
|
||||
|
||||
|
||||
class StrategyTrackingRepository(Protocol):
|
||||
def save_strategy_tracks(
|
||||
self, user_id: int, run_id: int, selection_date: str,
|
||||
strategy_name: str, candidates: list[dict[str, Any]],
|
||||
) -> int: ...
|
||||
|
||||
def get_screener_run(self, user_id: int, run_id: int) -> dict[str, Any] | None: ...
|
||||
|
||||
def delete_strategy_track(self, user_id: int, track_id: int) -> bool: ...
|
||||
|
||||
def list_strategy_tracks(
|
||||
self, user_id: int, limit_batches: int = 12,
|
||||
) -> list[dict[str, Any]]: ...
|
||||
|
||||
def load_tracking_bars(
|
||||
self, targets: list[tuple[str, str]], limit: int = 5,
|
||||
) -> dict[tuple[str, str], list[dict[str, Any]]]: ...
|
||||
@@ -0,0 +1,108 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from database import ReviewDatabase
|
||||
|
||||
|
||||
def require_user_id(value: int) -> int:
|
||||
user_id = int(value)
|
||||
if user_id <= 0:
|
||||
raise ValueError("A positive account owner is required")
|
||||
return user_id
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SQLiteAlertRepository:
|
||||
database: ReviewDatabase
|
||||
|
||||
def save_alert(self, user_id: int, *args: Any, **kwargs: Any) -> int:
|
||||
return self.database.save_alert(require_user_id(user_id), *args, **kwargs)
|
||||
|
||||
def list_alerts(
|
||||
self, user_id: int, as_of: str, unread_only: bool = False, limit: int = 100,
|
||||
) -> list[dict[str, Any]]:
|
||||
return self.database.list_alerts(
|
||||
require_user_id(user_id), as_of, unread_only, limit
|
||||
)
|
||||
|
||||
def count_unread_alerts(self, user_id: int, as_of: str) -> int:
|
||||
return self.database.count_unread_alerts(require_user_id(user_id), as_of)
|
||||
|
||||
def mark_alert_read(self, user_id: int, alert_id: int) -> bool:
|
||||
return self.database.mark_alert_read(require_user_id(user_id), alert_id)
|
||||
|
||||
def mark_all_alerts_read(self, user_id: int, as_of: str) -> int:
|
||||
return self.database.mark_all_alerts_read(require_user_id(user_id), as_of)
|
||||
|
||||
def delete_alert(self, user_id: int, alert_id: int) -> bool:
|
||||
return self.database.delete_alert(require_user_id(user_id), alert_id)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SQLiteTradeJournalRepository:
|
||||
database: ReviewDatabase
|
||||
|
||||
def save_trade_entry(self, user_id: int, *args: Any, **kwargs: Any) -> int:
|
||||
return self.database.save_trade_entry(require_user_id(user_id), *args, **kwargs)
|
||||
|
||||
def list_trade_entries(
|
||||
self, user_id: int, start_date: str = "", end_date: str = "",
|
||||
code: str = "", limit: int = 300,
|
||||
) -> list[dict[str, Any]]:
|
||||
return self.database.list_trade_entries(
|
||||
require_user_id(user_id), start_date, end_date, code, limit
|
||||
)
|
||||
|
||||
def delete_trade_entry(self, user_id: int, trade_id: int) -> bool:
|
||||
return self.database.delete_trade_entry(require_user_id(user_id), trade_id)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SQLiteStrategyTrackingRepository:
|
||||
database: ReviewDatabase
|
||||
|
||||
def save_strategy_tracks(
|
||||
self, user_id: int, run_id: int, selection_date: str,
|
||||
strategy_name: str, candidates: list[dict[str, Any]],
|
||||
) -> int:
|
||||
return self.database.save_strategy_tracks(
|
||||
require_user_id(user_id), run_id, selection_date, strategy_name, candidates
|
||||
)
|
||||
|
||||
def get_screener_run(self, user_id: int, run_id: int) -> dict[str, Any] | None:
|
||||
owner_id = int(user_id)
|
||||
if owner_id < 0:
|
||||
raise ValueError("Account owner cannot be negative")
|
||||
return self.database.get_screener_run(owner_id, run_id)
|
||||
|
||||
def delete_strategy_track(self, user_id: int, track_id: int) -> bool:
|
||||
return self.database.delete_strategy_track(require_user_id(user_id), track_id)
|
||||
|
||||
def list_strategy_tracks(
|
||||
self, user_id: int, limit_batches: int = 12,
|
||||
) -> list[dict[str, Any]]:
|
||||
return self.database.list_strategy_tracks(
|
||||
require_user_id(user_id), limit_batches
|
||||
)
|
||||
|
||||
def load_tracking_bars(
|
||||
self, targets: list[tuple[str, str]], limit: int = 5,
|
||||
) -> dict[tuple[str, str], list[dict[str, Any]]]:
|
||||
return self.database.load_tracking_bars(targets, limit)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RepositoryBundle:
|
||||
alerts: SQLiteAlertRepository
|
||||
trades: SQLiteTradeJournalRepository
|
||||
strategy_tracking: SQLiteStrategyTrackingRepository
|
||||
|
||||
|
||||
def build_repository_bundle(database: ReviewDatabase) -> RepositoryBundle:
|
||||
return RepositoryBundle(
|
||||
alerts=SQLiteAlertRepository(database),
|
||||
trades=SQLiteTradeJournalRepository(database),
|
||||
strategy_tracking=SQLiteStrategyTrackingRepository(database),
|
||||
)
|
||||
@@ -0,0 +1 @@
|
||||
"""Feature-owned application services."""
|
||||
@@ -0,0 +1,24 @@
|
||||
__all__ = [
|
||||
"AccountHttpMixin",
|
||||
"AccountService",
|
||||
"SecretVault",
|
||||
"hash_password",
|
||||
"token_hash",
|
||||
"verify_password",
|
||||
]
|
||||
|
||||
|
||||
def __getattr__(name: str):
|
||||
if name == "AccountHttpMixin":
|
||||
from .http import AccountHttpMixin
|
||||
|
||||
return AccountHttpMixin
|
||||
if name == "AccountService":
|
||||
from .service import AccountService
|
||||
|
||||
return AccountService
|
||||
if name in {"SecretVault", "hash_password", "token_hash", "verify_password"}:
|
||||
from . import security
|
||||
|
||||
return getattr(security, name)
|
||||
raise AttributeError(name)
|
||||
@@ -0,0 +1,110 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from http import HTTPStatus
|
||||
|
||||
|
||||
class AccountHttpMixin:
|
||||
def auth_register(self) -> None:
|
||||
try:
|
||||
body = self.read_json_body()
|
||||
result = self.application_service.register_account(
|
||||
str(body.get("username") or ""),
|
||||
str(body.get("password") or ""),
|
||||
)
|
||||
self.send_json(
|
||||
{
|
||||
"ok": True,
|
||||
"authenticated": True,
|
||||
"user": result["user"],
|
||||
"csrf_token": result["csrf_token"],
|
||||
},
|
||||
HTTPStatus.CREATED,
|
||||
{"Set-Cookie": self.session_cookie(result["session_token"])},
|
||||
)
|
||||
except (ValueError, json.JSONDecodeError) as exc:
|
||||
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
|
||||
|
||||
def auth_login(self) -> None:
|
||||
try:
|
||||
body = self.read_json_body()
|
||||
result = self.application_service.login_account(
|
||||
str(body.get("username") or ""),
|
||||
str(body.get("password") or ""),
|
||||
)
|
||||
self.send_json(
|
||||
{
|
||||
"ok": True,
|
||||
"authenticated": True,
|
||||
"user": result["user"],
|
||||
"csrf_token": result["csrf_token"],
|
||||
},
|
||||
headers={"Set-Cookie": self.session_cookie(result["session_token"])},
|
||||
)
|
||||
except (ValueError, json.JSONDecodeError) as exc:
|
||||
self.send_json({"error": str(exc)}, HTTPStatus.UNAUTHORIZED)
|
||||
|
||||
def auth_me(self) -> None:
|
||||
service = self.application_service
|
||||
if not self.require_auth(send_error=False):
|
||||
self.send_json(
|
||||
{
|
||||
"ok": True,
|
||||
"authenticated": False,
|
||||
"registration_required": service.database.count_users() == 0,
|
||||
}
|
||||
)
|
||||
return
|
||||
self.send_json(
|
||||
{
|
||||
"ok": True,
|
||||
"authenticated": True,
|
||||
"user": {
|
||||
"id": int(self.auth_user["id"]),
|
||||
"username": str(self.auth_user["username"]),
|
||||
"role": str(self.auth_user.get("role") or "user"),
|
||||
"membership": service.membership(),
|
||||
},
|
||||
"csrf_token": str(self.auth_user["csrf_token"]),
|
||||
}
|
||||
)
|
||||
|
||||
def auth_logout(self) -> None:
|
||||
raw_token = self.session_token()
|
||||
if raw_token:
|
||||
from backend.features.accounts.security import token_hash
|
||||
|
||||
self.application_service.database.delete_session(token_hash(raw_token))
|
||||
self.send_json(
|
||||
{"ok": True},
|
||||
headers={"Set-Cookie": self.session_cookie("", clear=True)},
|
||||
)
|
||||
|
||||
def save_birth_profile(self) -> None:
|
||||
try:
|
||||
body = self.read_json_body()
|
||||
personal = self.application_service.save_birth_profile(body)
|
||||
self.send_json({"ok": True, "personal": personal})
|
||||
except (ValueError, json.JSONDecodeError) as exc:
|
||||
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
|
||||
|
||||
def change_password(self) -> None:
|
||||
try:
|
||||
body = self.read_json_body()
|
||||
current = str(body.get("current_password") or "")
|
||||
new = str(body.get("new_password") or "")
|
||||
confirmation = str(body.get("confirm_password") or "")
|
||||
if new != confirmation:
|
||||
raise ValueError("两次输入的新密码不一致。")
|
||||
self.application_service.change_password(current, new)
|
||||
self.send_json({"ok": True})
|
||||
except (ValueError, json.JSONDecodeError) as exc:
|
||||
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
|
||||
|
||||
def save_membership(self) -> None:
|
||||
try:
|
||||
service = self.application_service
|
||||
service.update_membership(self.read_json_body())
|
||||
self.send_json({"ok": True, "users": service.admin_users()})
|
||||
except (ValueError, json.JSONDecodeError) as exc:
|
||||
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
|
||||
@@ -0,0 +1,236 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlite3
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any
|
||||
|
||||
|
||||
class AccountRepositoryMixin:
|
||||
"""Original SQLite account persistence methods, moved without query changes."""
|
||||
|
||||
def count_users(self) -> int:
|
||||
with self.connect() as connection:
|
||||
row = connection.execute("SELECT COUNT(*) AS total FROM users").fetchone()
|
||||
return int(row["total"] if row else 0)
|
||||
|
||||
def first_user_id(self) -> int:
|
||||
with self.connect() as connection:
|
||||
row = connection.execute("SELECT MIN(id) AS id FROM users").fetchone()
|
||||
return int(row["id"] or 0) if row else 0
|
||||
|
||||
def create_user(
|
||||
self,
|
||||
username: str,
|
||||
password_salt: str,
|
||||
password_hash: str,
|
||||
) -> dict[str, Any]:
|
||||
now = datetime.now(timezone.utc).isoformat(timespec="seconds")
|
||||
try:
|
||||
with self.connect() as connection:
|
||||
role = "admin" if int(connection.execute("SELECT COUNT(*) FROM users").fetchone()[0]) == 0 else "user"
|
||||
cursor = connection.execute(
|
||||
"""
|
||||
INSERT INTO users
|
||||
(username, password_salt, password_hash, role, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(username, password_salt, password_hash, role, now, now),
|
||||
)
|
||||
user_id = int(cursor.lastrowid)
|
||||
except sqlite3.IntegrityError as exc:
|
||||
raise ValueError("该账号名已被使用。") from exc
|
||||
return {"id": user_id, "username": username, "role": role, "created_at": now}
|
||||
|
||||
def user_by_username(self, username: str) -> dict[str, Any] | None:
|
||||
with self.connect() as connection:
|
||||
row = connection.execute(
|
||||
"""
|
||||
SELECT id, username, password_salt, password_hash, role, llm_mode,
|
||||
membership_status, membership_plan, membership_starts_at,
|
||||
membership_expires_at, created_at
|
||||
FROM users WHERE username = ? COLLATE NOCASE
|
||||
""",
|
||||
(username,),
|
||||
).fetchone()
|
||||
return dict(row) if row else None
|
||||
|
||||
def user_password(self, user_id: int) -> dict[str, str] | None:
|
||||
with self.connect() as connection:
|
||||
row = connection.execute(
|
||||
"SELECT password_salt, password_hash FROM users WHERE id = ?",
|
||||
(user_id,),
|
||||
).fetchone()
|
||||
return dict(row) if row else None
|
||||
|
||||
def update_user_password(self, user_id: int, password_salt: str, password_hash: str) -> bool:
|
||||
now = datetime.now(timezone.utc).isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"UPDATE users SET password_salt = ?, password_hash = ?, updated_at = ? WHERE id = ?",
|
||||
(password_salt, password_hash, now, user_id),
|
||||
)
|
||||
return cursor.rowcount > 0
|
||||
|
||||
def delete_user(self, user_id: int) -> bool:
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute("DELETE FROM users WHERE id = ?", (user_id,))
|
||||
return cursor.rowcount > 0
|
||||
|
||||
def create_session(
|
||||
self,
|
||||
session_hash: str,
|
||||
user_id: int,
|
||||
csrf_token: str,
|
||||
expires_at: str,
|
||||
) -> None:
|
||||
now = datetime.now(timezone.utc).isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
connection.execute("DELETE FROM user_sessions WHERE expires_at <= ?", (now,))
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO user_sessions
|
||||
(token_hash, user_id, csrf_token, expires_at, created_at, last_seen_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(session_hash, user_id, csrf_token, expires_at, now, now),
|
||||
)
|
||||
|
||||
def session_user(self, session_hash: str) -> dict[str, Any] | None:
|
||||
now = datetime.now(timezone.utc).isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
row = connection.execute(
|
||||
"""
|
||||
SELECT u.id, u.username, u.role, u.llm_mode, u.membership_status,
|
||||
u.membership_plan, u.membership_starts_at, u.membership_expires_at,
|
||||
u.created_at, s.csrf_token, s.expires_at
|
||||
FROM user_sessions AS s
|
||||
JOIN users AS u ON u.id = s.user_id
|
||||
WHERE s.token_hash = ? AND s.expires_at > ?
|
||||
""",
|
||||
(session_hash, now),
|
||||
).fetchone()
|
||||
if row:
|
||||
connection.execute(
|
||||
"UPDATE user_sessions SET last_seen_at = ? WHERE token_hash = ?",
|
||||
(now, session_hash),
|
||||
)
|
||||
return dict(row) if row else None
|
||||
|
||||
def delete_session(self, session_hash: str) -> bool:
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"DELETE FROM user_sessions WHERE token_hash = ?",
|
||||
(session_hash,),
|
||||
)
|
||||
return cursor.rowcount > 0
|
||||
|
||||
def get_user_credentials(self, user_id: int) -> str:
|
||||
with self.connect() as connection:
|
||||
row = connection.execute(
|
||||
"SELECT encrypted_payload FROM user_credentials WHERE user_id = ?",
|
||||
(user_id,),
|
||||
).fetchone()
|
||||
return str(row["encrypted_payload"]) if row else ""
|
||||
|
||||
def save_user_credentials(self, user_id: int, encrypted_payload: str) -> None:
|
||||
now = datetime.now(timezone.utc).isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO user_credentials (user_id, encrypted_payload, updated_at)
|
||||
VALUES (?, ?, ?)
|
||||
ON CONFLICT(user_id) DO UPDATE SET
|
||||
encrypted_payload = excluded.encrypted_payload,
|
||||
updated_at = excluded.updated_at
|
||||
""",
|
||||
(user_id, encrypted_payload, now),
|
||||
)
|
||||
|
||||
def list_user_credentials(self) -> list[dict[str, Any]]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"SELECT user_id, encrypted_payload FROM user_credentials ORDER BY user_id"
|
||||
).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
def user_access(self, user_id: int) -> dict[str, Any] | None:
|
||||
with self.connect() as connection:
|
||||
row = connection.execute(
|
||||
"""
|
||||
SELECT id, username, role, llm_mode, membership_status, membership_plan,
|
||||
membership_starts_at, membership_expires_at, created_at
|
||||
FROM users WHERE id = ?
|
||||
""",
|
||||
(user_id,),
|
||||
).fetchone()
|
||||
return dict(row) if row else None
|
||||
|
||||
def list_users(self) -> list[dict[str, Any]]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"""
|
||||
SELECT id, username, role, llm_mode, membership_status, membership_plan,
|
||||
membership_starts_at, membership_expires_at, created_at
|
||||
FROM users ORDER BY id
|
||||
"""
|
||||
).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
def update_user_llm_mode(self, user_id: int, mode: str) -> None:
|
||||
now = datetime.now(timezone.utc).isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
connection.execute(
|
||||
"UPDATE users SET llm_mode = ?, updated_at = ? WHERE id = ?",
|
||||
(mode, now, user_id),
|
||||
)
|
||||
|
||||
def update_membership(
|
||||
self,
|
||||
user_id: int,
|
||||
status: str,
|
||||
plan: str,
|
||||
starts_at: str | None,
|
||||
expires_at: str | None,
|
||||
) -> bool:
|
||||
now = datetime.now(timezone.utc).isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"""
|
||||
UPDATE users
|
||||
SET membership_status = ?, membership_plan = ?,
|
||||
membership_starts_at = ?, membership_expires_at = ?, updated_at = ?
|
||||
WHERE id = ?
|
||||
""",
|
||||
(status, plan, starts_at, expires_at, now, user_id),
|
||||
)
|
||||
return cursor.rowcount > 0
|
||||
|
||||
def get_user_birth_profile(self, user_id: int) -> str:
|
||||
with self.connect() as connection:
|
||||
row = connection.execute(
|
||||
"SELECT encrypted_payload FROM user_birth_profiles WHERE user_id = ?",
|
||||
(user_id,),
|
||||
).fetchone()
|
||||
return str(row["encrypted_payload"]) if row else ""
|
||||
|
||||
def save_user_birth_profile(self, user_id: int, encrypted_payload: str) -> None:
|
||||
now = datetime.now(timezone.utc).isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO user_birth_profiles (user_id, encrypted_payload, updated_at)
|
||||
VALUES (?, ?, ?)
|
||||
ON CONFLICT(user_id) DO UPDATE SET
|
||||
encrypted_payload = excluded.encrypted_payload,
|
||||
updated_at = excluded.updated_at
|
||||
""",
|
||||
(user_id, encrypted_payload, now),
|
||||
)
|
||||
|
||||
def delete_user_birth_profile(self, user_id: int) -> bool:
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"DELETE FROM user_birth_profiles WHERE user_id = ?",
|
||||
(user_id,),
|
||||
)
|
||||
return cursor.rowcount > 0
|
||||
@@ -0,0 +1,71 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import os
|
||||
from typing import Any
|
||||
|
||||
from cryptography.fernet import Fernet, InvalidToken
|
||||
|
||||
|
||||
PASSWORD_SCRYPT_N = 2**14
|
||||
PASSWORD_SCRYPT_R = 8
|
||||
PASSWORD_SCRYPT_P = 1
|
||||
|
||||
|
||||
class SecretVault:
|
||||
def __init__(self, key: str) -> None:
|
||||
try:
|
||||
self._fernet = Fernet(key.encode("ascii"))
|
||||
except (ValueError, TypeError) as exc:
|
||||
raise ValueError("APP_ENCRYPTION_KEY 格式无效。") from exc
|
||||
|
||||
@staticmethod
|
||||
def generate_key() -> str:
|
||||
return Fernet.generate_key().decode("ascii")
|
||||
|
||||
def encrypt_json(self, payload: dict[str, Any]) -> str:
|
||||
raw = json.dumps(payload, ensure_ascii=False, separators=(",", ":")).encode("utf-8")
|
||||
return self._fernet.encrypt(raw).decode("ascii")
|
||||
|
||||
def decrypt_json(self, token: str) -> dict[str, Any]:
|
||||
if not token:
|
||||
return {}
|
||||
try:
|
||||
payload = json.loads(self._fernet.decrypt(token.encode("ascii")).decode("utf-8"))
|
||||
except (InvalidToken, UnicodeDecodeError, json.JSONDecodeError) as exc:
|
||||
raise ValueError("账号加密数据无法解密,请检查 APP_ENCRYPTION_KEY。") from exc
|
||||
if not isinstance(payload, dict):
|
||||
raise ValueError("账号加密数据格式无效。")
|
||||
return payload
|
||||
|
||||
|
||||
def hash_password(password: str, salt: bytes | None = None) -> tuple[str, str]:
|
||||
raw_salt = salt or os.urandom(16)
|
||||
digest = hashlib.scrypt(
|
||||
password.encode("utf-8"),
|
||||
salt=raw_salt,
|
||||
n=PASSWORD_SCRYPT_N,
|
||||
r=PASSWORD_SCRYPT_R,
|
||||
p=PASSWORD_SCRYPT_P,
|
||||
dklen=32,
|
||||
)
|
||||
return (
|
||||
base64.urlsafe_b64encode(raw_salt).decode("ascii"),
|
||||
base64.urlsafe_b64encode(digest).decode("ascii"),
|
||||
)
|
||||
|
||||
|
||||
def verify_password(password: str, salt_text: str, expected_hash: str) -> bool:
|
||||
try:
|
||||
salt = base64.urlsafe_b64decode(salt_text.encode("ascii"))
|
||||
_, actual_hash = hash_password(password, salt)
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
return hmac.compare_digest(actual_hash, expected_hash)
|
||||
|
||||
|
||||
def token_hash(token: str) -> str:
|
||||
return hashlib.sha256(token.encode("utf-8")).hexdigest()
|
||||
@@ -0,0 +1,256 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import secrets
|
||||
import threading
|
||||
from collections.abc import Callable
|
||||
from datetime import date, datetime, timedelta, timezone
|
||||
from typing import Any
|
||||
|
||||
from backend.bootstrap.config import (
|
||||
SESSION_MAX_AGE,
|
||||
USERNAME_PATTERN,
|
||||
add_months,
|
||||
normalize_date,
|
||||
parse_iso_datetime,
|
||||
)
|
||||
from backend.features.accounts.security import (
|
||||
SecretVault,
|
||||
hash_password,
|
||||
token_hash,
|
||||
verify_password,
|
||||
)
|
||||
|
||||
|
||||
class AccountService:
|
||||
"""Preserved account, session, membership and birth-profile behavior."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
database: Any,
|
||||
vault: SecretVault,
|
||||
current_user_supplier: Callable[[], int],
|
||||
access_supplier: Callable[[], dict[str, Any]],
|
||||
bind_user: Callable[[int], None],
|
||||
personal_field_builder: Callable[..., dict[str, Any]],
|
||||
auth_lock: threading.Lock,
|
||||
) -> None:
|
||||
self.database = database
|
||||
self.vault = vault
|
||||
self.current_user_supplier = current_user_supplier
|
||||
self.access_supplier = access_supplier
|
||||
self.bind_user = bind_user
|
||||
self.personal_field_builder = personal_field_builder
|
||||
self.auth_lock = auth_lock
|
||||
|
||||
@property
|
||||
def current_user_id(self) -> int:
|
||||
return int(self.current_user_supplier())
|
||||
|
||||
@staticmethod
|
||||
def membership_for_access(access: dict[str, Any]) -> dict[str, Any]:
|
||||
now = datetime.now(timezone.utc)
|
||||
starts = parse_iso_datetime(access.get("membership_starts_at"))
|
||||
expires = parse_iso_datetime(access.get("membership_expires_at"))
|
||||
subscribed = (
|
||||
access.get("membership_status") == "active"
|
||||
and (not starts or starts <= now)
|
||||
and (not expires or expires > now)
|
||||
)
|
||||
is_admin = str(access.get("role")) == "admin"
|
||||
active = is_admin or subscribed
|
||||
remaining_seconds = None
|
||||
if expires:
|
||||
remaining_seconds = max(0, int((expires - now).total_seconds()))
|
||||
return {
|
||||
"active": active,
|
||||
"subscribed": subscribed,
|
||||
"status": "active" if subscribed else str(access.get("membership_status") or "inactive"),
|
||||
"plan": str(access.get("membership_plan") or ""),
|
||||
"starts_at": str(access.get("membership_starts_at") or ""),
|
||||
"expires_at": str(access.get("membership_expires_at") or ""),
|
||||
"is_admin": is_admin,
|
||||
"remaining_seconds": remaining_seconds,
|
||||
"remaining_days": None if remaining_seconds is None else (remaining_seconds + 86399) // 86400,
|
||||
}
|
||||
|
||||
def membership(self) -> dict[str, Any]:
|
||||
access = self.access_supplier() or self.database.user_access(self.current_user_id) or {}
|
||||
return self.membership_for_access(access)
|
||||
|
||||
def register(self, username: str, password: str) -> dict[str, Any]:
|
||||
username = username.strip()
|
||||
self.validate_input(username, password)
|
||||
with self.auth_lock:
|
||||
salt, password_digest = hash_password(password)
|
||||
user = self.database.create_user(username, salt, password_digest)
|
||||
return self.create_session(user)
|
||||
|
||||
def login(self, username: str, password: str) -> dict[str, Any]:
|
||||
username = username.strip()
|
||||
if not username or not password:
|
||||
raise ValueError("账号名和密码不能为空。")
|
||||
user = self.database.user_by_username(username)
|
||||
if not user or not verify_password(
|
||||
password,
|
||||
str(user.get("password_salt") or ""),
|
||||
str(user.get("password_hash") or ""),
|
||||
):
|
||||
raise ValueError("账号名或密码不正确。")
|
||||
return self.create_session(user)
|
||||
|
||||
def change_password(self, current_password: str, new_password: str) -> None:
|
||||
current_password = str(current_password or "")
|
||||
access = self.database.user_access(self.current_user_id)
|
||||
self.validate_input(str(access["username"]), new_password)
|
||||
credentials = self.database.user_password(self.current_user_id)
|
||||
if not credentials or not verify_password(
|
||||
current_password,
|
||||
str(credentials.get("password_salt") or ""),
|
||||
str(credentials.get("password_hash") or ""),
|
||||
):
|
||||
raise ValueError("当前密码不正确。")
|
||||
salt, digest = hash_password(new_password)
|
||||
if not self.database.update_user_password(self.current_user_id, salt, digest):
|
||||
raise ValueError("账号不存在。")
|
||||
|
||||
def create_session(self, user: dict[str, Any]) -> dict[str, Any]:
|
||||
session_token = secrets.token_urlsafe(32)
|
||||
csrf_token = secrets.token_urlsafe(24)
|
||||
expires = datetime.now(timezone.utc) + timedelta(seconds=SESSION_MAX_AGE)
|
||||
self.database.create_session(
|
||||
token_hash(session_token),
|
||||
int(user["id"]),
|
||||
csrf_token,
|
||||
expires.isoformat(timespec="seconds"),
|
||||
)
|
||||
self.bind_user(int(user["id"]))
|
||||
access = self.database.user_access(int(user["id"])) or {}
|
||||
return {
|
||||
"user": {
|
||||
"id": int(user["id"]),
|
||||
"username": str(user["username"]),
|
||||
"role": str(access.get("role") or "user"),
|
||||
"membership": self.membership(),
|
||||
},
|
||||
"session_token": session_token,
|
||||
"csrf_token": csrf_token,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def validate_input(username: str, password: str) -> None:
|
||||
if not USERNAME_PATTERN.fullmatch(username):
|
||||
raise ValueError("账号名应为 3 至 30 位中文、字母、数字、下划线或连字符。")
|
||||
if len(password) < 8 or len(password) > 128:
|
||||
raise ValueError("密码长度应为 8 至 128 位。")
|
||||
if password.isalpha() or password.isdigit():
|
||||
raise ValueError("密码应同时包含字母、数字或符号中的至少两类。")
|
||||
|
||||
def save_birth_profile(self, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
birth_datetime = str(payload.get("birth_datetime") or "").strip()
|
||||
gender = str(payload.get("gender") or "unspecified").strip()
|
||||
current_date = normalize_date(str(payload.get("trade_date") or date.today().isoformat()))
|
||||
personal = self.personal_field_builder(birth_datetime, gender, current_date)
|
||||
encrypted = self.vault.encrypt_json(
|
||||
{"birth_datetime": birth_datetime, "gender": gender}
|
||||
)
|
||||
self.database.save_user_birth_profile(self.current_user_id, encrypted)
|
||||
return self.public_personal_profile(personal)
|
||||
|
||||
def stored_birth_profile(self) -> dict[str, str] | None:
|
||||
encrypted = self.database.get_user_birth_profile(self.current_user_id)
|
||||
if not encrypted:
|
||||
return None
|
||||
payload = self.vault.decrypt_json(encrypted)
|
||||
birth_datetime = str(payload.get("birth_datetime") or "").strip()
|
||||
if not birth_datetime:
|
||||
return None
|
||||
return {
|
||||
"birth_datetime": birth_datetime,
|
||||
"gender": str(payload.get("gender") or "unspecified"),
|
||||
}
|
||||
|
||||
def personal_field(
|
||||
self,
|
||||
current_date: str,
|
||||
current_field: dict[str, Any],
|
||||
public: bool = False,
|
||||
) -> dict[str, Any] | None:
|
||||
stored = self.stored_birth_profile()
|
||||
if not stored:
|
||||
return None
|
||||
personal = self.personal_field_builder(
|
||||
stored["birth_datetime"],
|
||||
stored["gender"],
|
||||
current_date,
|
||||
current_field,
|
||||
)
|
||||
if public:
|
||||
return self.public_personal_profile(personal)
|
||||
personal.pop("birth", None)
|
||||
return personal
|
||||
|
||||
@staticmethod
|
||||
def public_personal_profile(personal: dict[str, Any]) -> dict[str, Any]:
|
||||
allowed = {
|
||||
"day_master",
|
||||
"ten_god_tendency",
|
||||
"element_balance",
|
||||
"balance_tendency",
|
||||
"current",
|
||||
"notice",
|
||||
}
|
||||
return {key: value for key, value in personal.items() if key in allowed}
|
||||
|
||||
def update_membership(self, payload: dict[str, Any]) -> None:
|
||||
try:
|
||||
user_id = int(payload.get("user_id"))
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError("会员账号不正确。") from exc
|
||||
status = str(payload.get("status") or "inactive")
|
||||
if status not in {"active", "inactive", "suspended"}:
|
||||
raise ValueError("会员状态不正确。")
|
||||
access = self.database.user_access(user_id)
|
||||
if not access:
|
||||
raise ValueError("用户不存在。")
|
||||
starts_at = None
|
||||
expires_at = None
|
||||
plan = ""
|
||||
if status == "active":
|
||||
duration = str(payload.get("duration") or "").strip()
|
||||
durations = {
|
||||
"1_month": (1, "1个月"),
|
||||
"3_months": (3, "3个月"),
|
||||
"12_months": (12, "12个月"),
|
||||
"3_years": (36, "3年"),
|
||||
"permanent": (0, "永久"),
|
||||
}
|
||||
if duration not in durations:
|
||||
raise ValueError("请选择会员开通时长。")
|
||||
now = datetime.now(timezone.utc)
|
||||
existing_start = parse_iso_datetime(access.get("membership_starts_at"))
|
||||
existing_expiry = parse_iso_datetime(access.get("membership_expires_at"))
|
||||
starts = existing_start if existing_start and existing_start <= now else now
|
||||
months, plan = durations[duration]
|
||||
starts_at = starts.isoformat(timespec="seconds")
|
||||
if months:
|
||||
renewal_base = existing_expiry if existing_expiry and existing_expiry > now else now
|
||||
expires_at = add_months(renewal_base, months).isoformat(timespec="seconds")
|
||||
if not self.database.update_membership(
|
||||
user_id, status, plan, starts_at, expires_at
|
||||
):
|
||||
raise ValueError("用户不存在。")
|
||||
|
||||
def admin_users(
|
||||
self, usage_supplier: Callable[[int], int]
|
||||
) -> list[dict[str, Any]]:
|
||||
rows = []
|
||||
for user in self.database.list_users():
|
||||
membership = self.membership_for_access(user)
|
||||
used = usage_supplier(int(user["id"])) if membership["active"] else 0
|
||||
rows.append({
|
||||
**user,
|
||||
"membership_active": membership["active"],
|
||||
"membership_subscribed": membership["subscribed"],
|
||||
"used_today": used,
|
||||
})
|
||||
return rows
|
||||
@@ -0,0 +1,18 @@
|
||||
from .facade import AlertServiceMixin
|
||||
from .http import AlertHttpMixin
|
||||
from .repository import AlertRepositoryMixin
|
||||
|
||||
__all__ = [
|
||||
"AlertHttpMixin",
|
||||
"AlertRepositoryMixin",
|
||||
"AlertService",
|
||||
"AlertServiceMixin",
|
||||
]
|
||||
|
||||
|
||||
def __getattr__(name: str):
|
||||
if name == "AlertService":
|
||||
from .service import AlertService
|
||||
|
||||
return AlertService
|
||||
raise AttributeError(name)
|
||||
@@ -0,0 +1,30 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date
|
||||
from typing import Any
|
||||
|
||||
|
||||
class AlertServiceMixin:
|
||||
def alert_center(self, status: str = "all", as_of: str = "") -> dict[str, Any]:
|
||||
tracking = self.strategy_tracking.list_tracking(self.current_user_id, 12)
|
||||
self.alert_service.sync_strategy_tracking(self.current_user_id, tracking)
|
||||
return self.alert_service.list_alerts(
|
||||
self.current_user_id, status, as_of
|
||||
)
|
||||
|
||||
def create_alert(self, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
alert_id = self.alert_service.create_manual(self.current_user_id, payload)
|
||||
return {"id": alert_id, **self.alert_center()}
|
||||
|
||||
def mark_alert_read(self, alert_id: int) -> dict[str, Any]:
|
||||
self.alert_service.mark_read(self.current_user_id, alert_id)
|
||||
return self.alert_center()
|
||||
|
||||
def mark_all_alerts_read(self, as_of: str = "") -> dict[str, Any]:
|
||||
compact_date = self.alert_service.calendar_date(as_of or date.today().isoformat())
|
||||
self.alert_service.mark_all_read(self.current_user_id, compact_date)
|
||||
return self.alert_center(as_of=compact_date)
|
||||
|
||||
def delete_alert(self, alert_id: int) -> dict[str, Any]:
|
||||
deleted = self.alert_service.delete(self.current_user_id, alert_id)
|
||||
return {"deleted": deleted, **self.alert_center()}
|
||||
@@ -0,0 +1,16 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from http import HTTPStatus
|
||||
|
||||
|
||||
class AlertHttpMixin:
|
||||
def save_alert(self) -> None:
|
||||
try:
|
||||
body = self.read_json_body()
|
||||
self.send_json(
|
||||
{"ok": True, **self.application_service.create_alert(body)},
|
||||
HTTPStatus.CREATED,
|
||||
)
|
||||
except (ValueError, json.JSONDecodeError) as exc:
|
||||
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
|
||||
@@ -0,0 +1,110 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
|
||||
class AlertRepositoryMixin:
|
||||
def save_alert(
|
||||
self,
|
||||
user_id: int,
|
||||
kind: str,
|
||||
title: str,
|
||||
content: str,
|
||||
available_date: str,
|
||||
code: str,
|
||||
dedupe_key: str,
|
||||
) -> int:
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO alerts
|
||||
(user_id, kind, title, content, available_date, code, dedupe_key,
|
||||
is_read, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, 0, ?, ?)
|
||||
ON CONFLICT(user_id, dedupe_key) DO UPDATE SET
|
||||
title=excluded.title, content=excluded.content,
|
||||
available_date=excluded.available_date, updated_at=excluded.updated_at
|
||||
""",
|
||||
(
|
||||
int(user_id), kind, title, content, available_date, code,
|
||||
dedupe_key, now, now,
|
||||
),
|
||||
)
|
||||
row = connection.execute(
|
||||
"SELECT id FROM alerts WHERE user_id = ? AND dedupe_key = ?",
|
||||
(int(user_id), dedupe_key),
|
||||
).fetchone()
|
||||
return int(row["id"])
|
||||
|
||||
def list_alerts(
|
||||
self, user_id: int, as_of: str, unread_only: bool = False, limit: int = 100
|
||||
) -> list[dict[str, Any]]:
|
||||
with self.connect() as connection:
|
||||
if unread_only:
|
||||
rows = connection.execute(
|
||||
"""
|
||||
SELECT id, kind, title, content, available_date, code, is_read,
|
||||
created_at, updated_at, read_at
|
||||
FROM alerts
|
||||
WHERE user_id = ? AND available_date <= ? AND is_read = 0
|
||||
ORDER BY available_date DESC, id DESC LIMIT ?
|
||||
""",
|
||||
(int(user_id), as_of, max(1, min(300, int(limit)))),
|
||||
).fetchall()
|
||||
else:
|
||||
rows = connection.execute(
|
||||
"""
|
||||
SELECT id, kind, title, content, available_date, code, is_read,
|
||||
created_at, updated_at, read_at
|
||||
FROM alerts WHERE user_id = ?
|
||||
ORDER BY CASE WHEN available_date > ? THEN 0 ELSE 1 END,
|
||||
is_read, available_date, id DESC LIMIT ?
|
||||
""",
|
||||
(int(user_id), as_of, max(1, min(300, int(limit)))),
|
||||
).fetchall()
|
||||
return [{**dict(row), "is_read": bool(row["is_read"])} for row in rows]
|
||||
|
||||
def count_unread_alerts(self, user_id: int, as_of: str) -> int:
|
||||
with self.connect() as connection:
|
||||
row = connection.execute(
|
||||
"""
|
||||
SELECT COUNT(*) AS total FROM alerts
|
||||
WHERE user_id = ? AND available_date <= ? AND is_read = 0
|
||||
""",
|
||||
(int(user_id), as_of),
|
||||
).fetchone()
|
||||
return int(row["total"] if row else 0)
|
||||
|
||||
def mark_alert_read(self, user_id: int, alert_id: int) -> bool:
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"""
|
||||
UPDATE alerts SET is_read = 1, read_at = ?, updated_at = ?
|
||||
WHERE id = ? AND user_id = ?
|
||||
""",
|
||||
(now, now, int(alert_id), int(user_id)),
|
||||
)
|
||||
return cursor.rowcount > 0
|
||||
|
||||
def mark_all_alerts_read(self, user_id: int, as_of: str) -> int:
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"""
|
||||
UPDATE alerts SET is_read = 1, read_at = ?, updated_at = ?
|
||||
WHERE user_id = ? AND available_date <= ? AND is_read = 0
|
||||
""",
|
||||
(now, now, int(user_id), as_of),
|
||||
)
|
||||
return int(cursor.rowcount)
|
||||
|
||||
def delete_alert(self, user_id: int, alert_id: int) -> bool:
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"DELETE FROM alerts WHERE id = ? AND user_id = ?",
|
||||
(int(alert_id), int(user_id)),
|
||||
)
|
||||
return cursor.rowcount > 0
|
||||
@@ -0,0 +1,95 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import secrets
|
||||
from datetime import date, datetime
|
||||
from typing import Any
|
||||
|
||||
from backend.bootstrap.config import validate_text
|
||||
from backend.database.repositories import AlertRepository
|
||||
|
||||
|
||||
class AlertService:
|
||||
def __init__(self, repository: AlertRepository) -> None:
|
||||
self.repository = repository
|
||||
|
||||
def create_manual(self, user_id: int, payload: dict[str, Any]) -> int:
|
||||
title = validate_text(payload.get("title"), "提醒标题", 80, required=True)
|
||||
content = validate_text(payload.get("content"), "提醒内容", 500)
|
||||
code = validate_text(payload.get("code"), "股票代码", 12)
|
||||
available_date = self.calendar_date(
|
||||
str(payload.get("remind_date") or date.today().isoformat())
|
||||
)
|
||||
return self.repository.save_alert(
|
||||
user_id=user_id,
|
||||
kind="manual",
|
||||
title=title,
|
||||
content=content,
|
||||
available_date=available_date,
|
||||
code=code,
|
||||
dedupe_key=f"manual:{secrets.token_hex(12)}",
|
||||
)
|
||||
|
||||
def sync_strategy_tracking(self, user_id: int, tracking: dict[str, Any]) -> int:
|
||||
synced = 0
|
||||
today = date.today().strftime("%Y%m%d")
|
||||
for batch in tracking.get("batches") or []:
|
||||
items = batch.get("items") or []
|
||||
summary = batch.get("summary") or {}
|
||||
if not items:
|
||||
continue
|
||||
run_id = int(batch.get("run_id") or 0)
|
||||
strategy_name = str(batch.get("strategy_name") or "选股策略")
|
||||
observed = int(summary.get("observed") or 0)
|
||||
completed = int(summary.get("completed") or 0)
|
||||
if observed:
|
||||
win_rate = summary.get("t1_win_rate")
|
||||
suffix = f",当前红盘率 {win_rate:.1f}%" if win_rate is not None else ""
|
||||
self.repository.save_alert(
|
||||
user_id, "strategy_t1", f"{strategy_name} 已有 T+1 反馈",
|
||||
f"{observed}/{len(items)} 只标的已有首日表现{suffix}。",
|
||||
today, "", f"strategy:{run_id}:t1",
|
||||
)
|
||||
synced += 1
|
||||
if completed == len(items):
|
||||
average = summary.get("average_t5")
|
||||
suffix = f",平均收益 {average:+.2f}%" if average is not None else ""
|
||||
self.repository.save_alert(
|
||||
user_id, "strategy_t5", f"{strategy_name} 五日跟踪完成",
|
||||
f"本批 {len(items)} 只标的已完成 T+5 跟踪{suffix}。",
|
||||
today, "", f"strategy:{run_id}:t5",
|
||||
)
|
||||
synced += 1
|
||||
return synced
|
||||
|
||||
def list_alerts(
|
||||
self, user_id: int, status: str = "all", as_of: str = ""
|
||||
) -> dict[str, Any]:
|
||||
if status not in {"all", "unread"}:
|
||||
raise ValueError("提醒筛选不支持。")
|
||||
compact_date = self.calendar_date(as_of or date.today().isoformat())
|
||||
items = self.repository.list_alerts(user_id, compact_date, status == "unread")
|
||||
for item in items:
|
||||
item["due"] = str(item.get("available_date") or "") <= compact_date
|
||||
return {
|
||||
"items": items,
|
||||
"unread_count": self.repository.count_unread_alerts(user_id, compact_date),
|
||||
"as_of": compact_date,
|
||||
}
|
||||
|
||||
def mark_read(self, user_id: int, alert_id: int) -> bool:
|
||||
return self.repository.mark_alert_read(user_id, alert_id)
|
||||
|
||||
def mark_all_read(self, user_id: int, as_of: str) -> int:
|
||||
return self.repository.mark_all_alerts_read(user_id, as_of)
|
||||
|
||||
def delete(self, user_id: int, alert_id: int) -> bool:
|
||||
return self.repository.delete_alert(user_id, alert_id)
|
||||
|
||||
@staticmethod
|
||||
def calendar_date(value: str) -> str:
|
||||
compact = value.replace("-", "").strip()
|
||||
try:
|
||||
parsed = datetime.strptime(compact, "%Y%m%d")
|
||||
except ValueError as exc:
|
||||
raise ValueError("提醒日期格式应为 YYYY-MM-DD。") from exc
|
||||
return parsed.strftime("%Y%m%d")
|
||||
@@ -0,0 +1,4 @@
|
||||
from .repository import AuctionRepositoryMixin
|
||||
from .service import AuctionServiceMixin
|
||||
|
||||
__all__ = ["AuctionRepositoryMixin", "AuctionServiceMixin"]
|
||||
@@ -0,0 +1,63 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
|
||||
class AuctionRepositoryMixin:
|
||||
def upsert_auction_factors(self, rows: list[dict[str, Any]]) -> int:
|
||||
values = []
|
||||
for row in rows:
|
||||
trade_date = str(row.get("trade_date") or "")
|
||||
ts_code = str(row.get("ts_code") or "")
|
||||
price = float(row.get("price") or 0)
|
||||
pre_close = float(row.get("pre_close") or 0)
|
||||
if not trade_date or not ts_code or price <= 0 or pre_close <= 0:
|
||||
continue
|
||||
values.append(
|
||||
(
|
||||
trade_date,
|
||||
ts_code,
|
||||
price,
|
||||
pre_close,
|
||||
(price / pre_close - 1) * 100,
|
||||
float(row.get("vol") or 0),
|
||||
float(row.get("amount") or 0),
|
||||
float(row.get("turnover_rate") or 0),
|
||||
float(row.get("volume_ratio") or 0),
|
||||
)
|
||||
)
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO auction_factors
|
||||
(trade_date, ts_code, price, pre_close, change, vol, amount,
|
||||
turnover_rate, volume_ratio)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(trade_date, ts_code) DO UPDATE SET
|
||||
price=excluded.price, pre_close=excluded.pre_close,
|
||||
change=excluded.change, vol=excluded.vol, amount=excluded.amount,
|
||||
turnover_rate=excluded.turnover_rate,
|
||||
volume_ratio=excluded.volume_ratio
|
||||
""",
|
||||
values,
|
||||
)
|
||||
return len(values)
|
||||
|
||||
def auction_factor_dates(self, end_date: str = "", limit: int = 80) -> list[str]:
|
||||
where = "WHERE trade_date <= ?" if end_date else ""
|
||||
parameters: tuple[Any, ...] = (end_date, limit) if end_date else (limit,)
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
f"SELECT DISTINCT trade_date FROM auction_factors {where} "
|
||||
"ORDER BY trade_date DESC LIMIT ?",
|
||||
parameters,
|
||||
).fetchall()
|
||||
return [row["trade_date"] for row in reversed(rows)]
|
||||
|
||||
def auction_factors_for_date(self, trade_date: str) -> list[dict[str, Any]]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"SELECT * FROM auction_factors WHERE trade_date = ? ORDER BY ts_code",
|
||||
(trade_date,),
|
||||
).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
@@ -0,0 +1,13 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from backend.bootstrap.config import normalize_date
|
||||
from backend.features.market.insights import MarketInsightsService
|
||||
|
||||
|
||||
class AuctionServiceMixin:
|
||||
def auction_center(self, trade_date: str, force: bool = False) -> dict[str, Any]:
|
||||
return self._market_insights().auction_center(
|
||||
normalize_date(trade_date), force, self.current_user_id
|
||||
)
|
||||
@@ -0,0 +1,4 @@
|
||||
from .repository import DragonTigerRepositoryMixin
|
||||
from .service import DragonTigerServiceMixin
|
||||
|
||||
__all__ = ["DragonTigerRepositoryMixin", "DragonTigerServiceMixin"]
|
||||
@@ -0,0 +1,61 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
|
||||
class DragonTigerRepositoryMixin:
|
||||
def list_seat_aliases(self) -> dict[str, str]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute("SELECT seat_name, alias FROM seat_aliases").fetchall()
|
||||
return {row["seat_name"]: row["alias"] for row in rows}
|
||||
|
||||
def save_seat_alias(self, seat_name: str, alias: str) -> None:
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO seat_aliases (seat_name, alias, updated_at)
|
||||
VALUES (?, ?, ?)
|
||||
ON CONFLICT(seat_name) DO UPDATE SET
|
||||
alias = excluded.alias,
|
||||
updated_at = excluded.updated_at
|
||||
""",
|
||||
(seat_name, alias, now),
|
||||
)
|
||||
|
||||
def upsert_lhb_institutions(self, rows: list[dict[str, Any]]) -> int:
|
||||
grouped: dict[tuple[str, str], dict[str, float | int]] = {}
|
||||
for row in rows:
|
||||
trade_date = str(row.get("trade_date") or "")
|
||||
ts_code = str(row.get("ts_code") or "")
|
||||
seat_name = str(row.get("exalter") or row.get("seat_name") or "")
|
||||
if not trade_date or not ts_code or "机构专用" not in seat_name:
|
||||
continue
|
||||
group = grouped.setdefault(
|
||||
(trade_date, ts_code),
|
||||
{"net": 0.0, "buy": 0.0, "sell": 0.0, "seats": 0},
|
||||
)
|
||||
group["net"] = float(group["net"]) + float(row.get("net_buy") or row.get("net_amount") or 0)
|
||||
group["buy"] = float(group["buy"]) + float(row.get("buy") or row.get("buy_amount") or 0)
|
||||
group["sell"] = float(group["sell"]) + float(row.get("sell") or row.get("sell_amount") or 0)
|
||||
group["seats"] = int(group["seats"]) + 1
|
||||
values = [
|
||||
(trade_date, ts_code, item["net"], item["buy"], item["sell"], item["seats"])
|
||||
for (trade_date, ts_code), item in grouped.items()
|
||||
]
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO lhb_institution_daily
|
||||
(trade_date, ts_code, net_buy_amount, buy_amount, sell_amount, seat_count)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(trade_date, ts_code) DO UPDATE SET
|
||||
net_buy_amount=excluded.net_buy_amount,
|
||||
buy_amount=excluded.buy_amount,
|
||||
sell_amount=excluded.sell_amount,
|
||||
seat_count=excluded.seat_count
|
||||
""",
|
||||
values,
|
||||
)
|
||||
return len(values)
|
||||
@@ -0,0 +1,288 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from backend.bootstrap.config import normalize_date
|
||||
from backend.data.providers.tushare_client import TushareError
|
||||
|
||||
|
||||
class DragonTigerServiceMixin:
|
||||
def get_hot_money_profiles(self, force: bool = False) -> dict[str, Any]:
|
||||
cache_kind = "hot_money_profiles_v1"
|
||||
cache_key = "directory"
|
||||
cached = self.database.get_data_snapshot(cache_kind, cache_key)
|
||||
if cached and not force:
|
||||
cached["meta"] = {**cached.get("meta", {}), "cached": True}
|
||||
return cached
|
||||
if self.configured:
|
||||
try:
|
||||
payload = self._tushare_client().hot_money_profiles()
|
||||
except TushareError:
|
||||
if cached:
|
||||
cached["meta"] = {
|
||||
**cached.get("meta", {}),
|
||||
"cached": True,
|
||||
"stale": True,
|
||||
"notice": "名录暂未完成更新,当前展示最近一次收录结果。",
|
||||
}
|
||||
return cached
|
||||
return {
|
||||
"meta": {
|
||||
"source": "unavailable",
|
||||
"status": "unavailable",
|
||||
"schema_version": 1,
|
||||
"cached": False,
|
||||
"updated_at": datetime.now().astimezone().isoformat(timespec="seconds"),
|
||||
"notice": "游资名录暂不可用,请稍后重试。",
|
||||
},
|
||||
"summary": {
|
||||
"profile_count": 0,
|
||||
"described_count": 0,
|
||||
"organization_count": 0,
|
||||
},
|
||||
"profiles": [],
|
||||
}
|
||||
payload["meta"]["cached"] = False
|
||||
if payload.get("meta", {}).get("status") == "success":
|
||||
self.database.save_data_snapshot(cache_kind, cache_key, "tushare", payload)
|
||||
return payload
|
||||
if cached:
|
||||
cached["meta"] = {**cached.get("meta", {}), "cached": True}
|
||||
return cached
|
||||
return {
|
||||
"meta": {
|
||||
"source": "unavailable",
|
||||
"status": "unavailable",
|
||||
"schema_version": 1,
|
||||
"cached": False,
|
||||
"updated_at": datetime.now().astimezone().isoformat(timespec="seconds"),
|
||||
"notice": "游资名录暂不可用,请联系管理员检查行情配置。",
|
||||
},
|
||||
"summary": {
|
||||
"profile_count": 0,
|
||||
"described_count": 0,
|
||||
"organization_count": 0,
|
||||
},
|
||||
"profiles": [],
|
||||
}
|
||||
|
||||
def get_dragon_tiger(self, trade_date: str, force: bool = False) -> dict[str, Any]:
|
||||
normalized_date = normalize_date(trade_date)
|
||||
cache_kind = "hot_money_detail_v3"
|
||||
if not force:
|
||||
cached = self.database.get_data_snapshot(cache_kind, normalized_date)
|
||||
if (
|
||||
cached
|
||||
and cached.get("meta", {}).get("source") == "tushare"
|
||||
and cached.get("meta", {}).get("status") == "success"
|
||||
and int(cached.get("meta", {}).get("schema_version") or 0) == 3
|
||||
):
|
||||
cached["meta"] = {**cached.get("meta", {}), "cached": True}
|
||||
return cached
|
||||
if self.configured:
|
||||
try:
|
||||
payload = self._tushare_client().dragon_tiger(normalized_date)
|
||||
except TushareError as exc:
|
||||
return {
|
||||
"meta": {
|
||||
"requested_date": f"{normalized_date[:4]}-{normalized_date[4:6]}-{normalized_date[6:8]}",
|
||||
"trade_date": f"{normalized_date[:4]}-{normalized_date[4:6]}-{normalized_date[6:8]}",
|
||||
"source": "tushare_error",
|
||||
"status": "error",
|
||||
"schema_version": 3,
|
||||
"cached": False,
|
||||
"updated_at": datetime.now().astimezone().isoformat(timespec="seconds"),
|
||||
"notice": "龙虎榜数据暂不可用,请稍后重试。",
|
||||
},
|
||||
"summary": {
|
||||
"trader_count": 0,
|
||||
"identity_count": 0,
|
||||
"operation_count": 0,
|
||||
"active_stock_count": 0,
|
||||
"seat_net_buy_million": 0,
|
||||
"unclassified_count": 0,
|
||||
"directory_count": 0,
|
||||
},
|
||||
"traders": [],
|
||||
"unclassified_seats": [],
|
||||
"rows": [],
|
||||
}
|
||||
payload["meta"]["cached"] = False
|
||||
if payload.get("meta", {}).get("status") == "success":
|
||||
self.database.save_data_snapshot(cache_kind, normalized_date, "tushare", payload)
|
||||
return payload
|
||||
|
||||
return {
|
||||
"meta": {
|
||||
"requested_date": f"{normalized_date[:4]}-{normalized_date[4:6]}-{normalized_date[6:8]}",
|
||||
"trade_date": f"{normalized_date[:4]}-{normalized_date[4:6]}-{normalized_date[6:8]}",
|
||||
"source": "unavailable",
|
||||
"status": "unavailable",
|
||||
"schema_version": 3,
|
||||
"cached": False,
|
||||
"notice": "龙虎榜数据暂不可用,请联系管理员检查行情配置。",
|
||||
},
|
||||
"summary": {
|
||||
"trader_count": 0,
|
||||
"identity_count": 0,
|
||||
"operation_count": 0,
|
||||
"active_stock_count": 0,
|
||||
"seat_net_buy_million": 0,
|
||||
"unclassified_count": 0,
|
||||
"directory_count": 0,
|
||||
},
|
||||
"traders": [],
|
||||
"unclassified_seats": [],
|
||||
"rows": [],
|
||||
}
|
||||
|
||||
def _apply_seat_aliases(self, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
aliases = self.database.list_seat_aliases()
|
||||
result = dict(payload)
|
||||
rows = payload.get("rows") or []
|
||||
for row in rows:
|
||||
for institution in row.get("institutions") or []:
|
||||
institution["alias"] = aliases.get(institution.get("seat_name", ""), "")
|
||||
traders: dict[tuple[str, str], dict[str, Any]] = {}
|
||||
unclassified: dict[str, dict[str, Any]] = {}
|
||||
seen_operations: set[tuple[Any, ...]] = set()
|
||||
builtin_aliases = {
|
||||
"国泰海通证券股份有限公司南京太平南路证券营业部": "作手新一",
|
||||
}
|
||||
|
||||
for row in rows:
|
||||
for institution in row.get("institutions") or []:
|
||||
seat_name = str(institution.get("seat_name") or "未知席位").strip()
|
||||
saved_alias = str(institution.get("alias") or "").strip()
|
||||
builtin_alias = builtin_aliases.get(seat_name, "")
|
||||
if saved_alias or builtin_alias:
|
||||
identity_name = saved_alias or builtin_alias
|
||||
identity_type = "trader"
|
||||
recognized = True
|
||||
identity_source = "manual" if saved_alias else "builtin"
|
||||
elif "机构专用" in seat_name:
|
||||
identity_name = "机构专用"
|
||||
identity_type = "institution"
|
||||
recognized = True
|
||||
identity_source = "system"
|
||||
elif "沪股通专用" in seat_name or "深股通专用" in seat_name:
|
||||
identity_name = "北向资金"
|
||||
identity_type = "channel"
|
||||
recognized = True
|
||||
identity_source = "system"
|
||||
else:
|
||||
identity_name = seat_name
|
||||
identity_type = "unclassified"
|
||||
recognized = False
|
||||
identity_source = "raw"
|
||||
|
||||
buy = round(float(institution.get("buy_million") or 0), 2)
|
||||
sell = round(float(institution.get("sell_million") or 0), 2)
|
||||
net_buy = round(float(institution.get("net_buy_million") or 0), 2)
|
||||
operation_key = (row.get("code"), seat_name, buy, sell, net_buy)
|
||||
if operation_key in seen_operations:
|
||||
continue
|
||||
seen_operations.add(operation_key)
|
||||
|
||||
group_key = (identity_type, identity_name)
|
||||
group = traders.setdefault(
|
||||
group_key,
|
||||
{
|
||||
"name": identity_name,
|
||||
"identity_type": identity_type,
|
||||
"identity_source": identity_source,
|
||||
"recognized": recognized,
|
||||
"buy_million": 0.0,
|
||||
"sell_million": 0.0,
|
||||
"net_buy_million": 0.0,
|
||||
"seat_names": set(),
|
||||
"stock_codes": set(),
|
||||
"operations": [],
|
||||
},
|
||||
)
|
||||
group["buy_million"] += buy
|
||||
group["sell_million"] += sell
|
||||
group["net_buy_million"] += net_buy
|
||||
group["seat_names"].add(seat_name)
|
||||
group["stock_codes"].add(str(row.get("code") or ""))
|
||||
group["operations"].append(
|
||||
{
|
||||
"code": row.get("code") or "",
|
||||
"name": row.get("name") or "--",
|
||||
"change": row.get("change") or 0,
|
||||
"direction": "买入" if net_buy > 0 else "卖出" if net_buy < 0 else "持平",
|
||||
"buy_million": buy,
|
||||
"sell_million": sell,
|
||||
"net_buy_million": net_buy,
|
||||
"reason": row.get("reason") or "--",
|
||||
"seat_name": seat_name,
|
||||
"seat_alias": identity_name if recognized else "",
|
||||
}
|
||||
)
|
||||
|
||||
if not recognized:
|
||||
pending = unclassified.setdefault(
|
||||
seat_name,
|
||||
{
|
||||
"seat_name": seat_name,
|
||||
"stock_codes": set(),
|
||||
"operation_count": 0,
|
||||
"buy_million": 0.0,
|
||||
"sell_million": 0.0,
|
||||
"net_buy_million": 0.0,
|
||||
},
|
||||
)
|
||||
pending["stock_codes"].add(str(row.get("code") or ""))
|
||||
pending["operation_count"] += 1
|
||||
pending["buy_million"] += buy
|
||||
pending["sell_million"] += sell
|
||||
pending["net_buy_million"] += net_buy
|
||||
|
||||
type_order = {"trader": 0, "institution": 1, "channel": 2, "unclassified": 3}
|
||||
aggregated = list(traders.values())
|
||||
aggregated.sort(
|
||||
key=lambda item: (
|
||||
type_order.get(item["identity_type"], 9),
|
||||
-abs(item["net_buy_million"]),
|
||||
item["name"],
|
||||
)
|
||||
)
|
||||
for index, group in enumerate(aggregated, start=1):
|
||||
group["id"] = f"identity-{index}"
|
||||
group["buy_million"] = round(group["buy_million"], 2)
|
||||
group["sell_million"] = round(group["sell_million"], 2)
|
||||
group["net_buy_million"] = round(group["net_buy_million"], 2)
|
||||
group["seat_count"] = len(group.pop("seat_names"))
|
||||
group["stock_count"] = len(group.pop("stock_codes"))
|
||||
group["operation_count"] = len(group["operations"])
|
||||
group["operations"].sort(
|
||||
key=lambda item: abs(float(item.get("net_buy_million") or 0)), reverse=True
|
||||
)
|
||||
|
||||
pending_seats = list(unclassified.values())
|
||||
for pending in pending_seats:
|
||||
pending["stock_count"] = len(pending.pop("stock_codes"))
|
||||
pending["buy_million"] = round(pending["buy_million"], 2)
|
||||
pending["sell_million"] = round(pending["sell_million"], 2)
|
||||
pending["net_buy_million"] = round(pending["net_buy_million"], 2)
|
||||
pending_seats.sort(key=lambda item: abs(item["net_buy_million"]), reverse=True)
|
||||
|
||||
operation_count = sum(item["operation_count"] for item in aggregated)
|
||||
active_stocks = {
|
||||
operation["code"] for item in aggregated for operation in item["operations"]
|
||||
}
|
||||
seat_net_buy = round(sum(item["net_buy_million"] for item in aggregated), 2)
|
||||
result["rows"] = rows
|
||||
result["traders"] = aggregated
|
||||
result["unclassified_seats"] = pending_seats
|
||||
result["summary"] = {
|
||||
**(payload.get("summary") or {}),
|
||||
"trader_count": sum(item["identity_type"] == "trader" for item in aggregated),
|
||||
"identity_count": len(aggregated),
|
||||
"operation_count": operation_count,
|
||||
"active_stock_count": len(active_stocks),
|
||||
"seat_net_buy_million": seat_net_buy,
|
||||
"unclassified_count": len(pending_seats),
|
||||
}
|
||||
return result
|
||||
@@ -0,0 +1,24 @@
|
||||
from .agent import HeavenAgentError, interpret_heaven
|
||||
from .engine import (
|
||||
build_five_phase_field,
|
||||
build_market_hexagram,
|
||||
build_manual_market_hexagram,
|
||||
build_personal_field,
|
||||
hexagram_from_lines,
|
||||
)
|
||||
from .http import HeavenHttpMixin
|
||||
from .repository import HeavenRepositoryMixin
|
||||
from .service import HeavenServiceMixin
|
||||
|
||||
__all__ = [
|
||||
"HeavenAgentError",
|
||||
"HeavenHttpMixin",
|
||||
"HeavenRepositoryMixin",
|
||||
"HeavenServiceMixin",
|
||||
"build_five_phase_field",
|
||||
"build_manual_market_hexagram",
|
||||
"build_market_hexagram",
|
||||
"build_personal_field",
|
||||
"hexagram_from_lines",
|
||||
"interpret_heaven",
|
||||
]
|
||||
@@ -0,0 +1,88 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
from backend.llm import transport as llm_transport
|
||||
|
||||
|
||||
class HeavenAgentError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
def interpret_heaven(
|
||||
mode: str,
|
||||
context: dict[str, Any],
|
||||
api_key: str,
|
||||
base_url: str,
|
||||
model: str,
|
||||
timeout: int = 90,
|
||||
) -> dict[str, Any]:
|
||||
if mode not in {"trend", "fortune", "heart"}:
|
||||
raise HeavenAgentError("不支持的问天解读模式。")
|
||||
if not api_key or not model:
|
||||
raise HeavenAgentError("LLM API Key 或模型尚未配置。")
|
||||
system_prompt = _system_prompt(mode)
|
||||
messages = [
|
||||
{"role": "system", "content": system_prompt},
|
||||
{
|
||||
"role": "user",
|
||||
"content": json.dumps(context, ensure_ascii=False, separators=(",", ":")),
|
||||
},
|
||||
]
|
||||
try:
|
||||
result = llm_transport.chat_completion(
|
||||
api_key=api_key,
|
||||
base_url=base_url,
|
||||
model=model,
|
||||
messages=messages,
|
||||
timeout=timeout,
|
||||
user_agent="XiaobaiReviewWeb/0.7",
|
||||
)
|
||||
answer = str(result.content).strip()
|
||||
if not answer:
|
||||
raise KeyError("empty response")
|
||||
except llm_transport.OpenAIHTTPError as exc:
|
||||
raise HeavenAgentError(exc.describe("问天模型调用失败")) from exc
|
||||
except (llm_transport.OpenAITransportError, KeyError) as exc:
|
||||
raise HeavenAgentError(f"问天模型调用失败:{exc}") from exc
|
||||
return {
|
||||
"answer": answer,
|
||||
"model": model,
|
||||
"latency_ms": result.latency_ms,
|
||||
}
|
||||
|
||||
|
||||
def _system_prompt(mode: str) -> str:
|
||||
common = """
|
||||
你是“小白复盘”的问天解读器。所有历法、卦象、爻位和市场指标已经由确定性程序计算,你只能解释提供的数据,不得改卦、改爻、改干支或编造行情。
|
||||
问天属于传统文化与娱乐化观察,不是预测模型,不承诺应验,不输出无条件买卖指令,不用神秘话术制造确定性。
|
||||
使用中文,先给核心判断,再解释结构。引用市场数字时标明数据日期。输出纯文本,可使用简短标题。
|
||||
""".strip()
|
||||
if mode == "trend":
|
||||
return common + """
|
||||
|
||||
当前任务是“观势·解势”。六爻从初爻到上爻依次是个股内核、个股外显、板块内核、板块外显、指数内核、指数外显;初二为地、三四为人、五上为天。
|
||||
行情数据只负责生成六爻,本次解势必须以卦象本身为主,不得根据指数涨跌、板块强弱、涨停家数、成交量或个股表现直接推演方向。context中不会提供这些数字,也不会提供爻位对应的市场角色。
|
||||
先解释本卦卦名的核心义、上下卦组合及大象;再只解释实际动爻所代表的转折,并说明本卦如何走向之卦;最后可把这一组卦势翻译成克制的市场语言。
|
||||
重点是“本卦为当下之势,动爻为变化关节,之卦为所趋之势”。不要说明某一动爻对应指数、板块或个股,也不要输出“一看指数、二看涨停家数”一类行情观察条件。
|
||||
全文控制在300至450个中文字符,最多四小段。卦理约占九成,市场翻译最多一句,只能落到节制、等待、守信、辨伪等行为态度,不得据此预测市场下一阶段、涨跌方向或动能变化。不直接荐股,不使用Markdown表格。
|
||||
不要使用“必然、确定、必涨、必跌、后续将、进入某阶段”等断语;天机只点出势的性质与变化关系,不替用户宣布结果。
|
||||
""".strip()
|
||||
if mode == "fortune":
|
||||
return common + """
|
||||
|
||||
当前任务是“观气·解运”。严格区分五运、六气、节气、月令和日干,不把丙午简单解释为火年。
|
||||
严格服从five_phase_field.framework提供的确定性结构,不自行重新计算五行:年纲由中运与司天在泉构成;岁半以前司天为主、在泉为辅,岁半以后在泉为主、司天为辅;当前六气层以客气加临主气为核心;日辰只负责触发。节气只用于定位当前六气阶段,不得再次叠加为独立力量。
|
||||
重点解释framework.relations中的客主同气、客生主、主生客、客克主或主克客,以及客胜为从、主胜为逆、司天在泉同位、天符岁会等已经判定的关系。不得把司天、在泉、主气、客气视为彼此独立的证据重复计权,也不得自行增删传统格局。
|
||||
首要解释当日气场容易放大参与者的哪些情绪、判断偏差和操作冲动,例如急躁、恐惧、迟疑、追涨、过早止损或路径依赖;再给出一至两个调节动作。
|
||||
如有personal_profile,结合其日主、十神、五行平衡倾向说明当日对该用户主观状态的影响,但不得把简化平衡倾向说成唯一喜用神,也不得复述或猜测出生日期。
|
||||
不得引用市场上涨下跌家数、涨跌停数量、成交额、板块强度或个股表现来证明气场。industry_affinity只是五行行业取象示例,不是行情旁证;行业契合度最多在末尾用一句话说明,不得写“当日共振”或暗示相关行业必然涨跌。
|
||||
全文控制在420至600个中文字符,按“三层气机、人的状态、操作偏向、个人影响(如有)、制衡动作”组织,标题必须写“三层气机”。明确这些是传统历法框架下的观察语言,不宣称气候或五行直接导致股价。
|
||||
""".strip()
|
||||
return common + """
|
||||
|
||||
当前任务是“观心·解卦”。用户的问题始终只在心中,没有输入给你,因此你不能猜测问题内容,也不能替用户作具体决定。
|
||||
全文控制在180至350个中文字符。只写一句卦意;一小段动爻与之卦;最后三句极短的问心句。
|
||||
不要重述六条爻辞,不猜用户未说出口的问题,不以吉凶二字替代思考,不给出股票涨跌预测。语气安静、克制,越短越有余味。
|
||||
""".strip()
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,30 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from http import HTTPStatus
|
||||
|
||||
|
||||
class HeavenHttpMixin:
|
||||
def heaven_hexagram(self) -> None:
|
||||
try:
|
||||
body = self.read_json_body()
|
||||
result = self.application_service.heaven_hexagram(body.get("lines"))
|
||||
self.send_json({"ok": True, "hexagram": result})
|
||||
except (ValueError, json.JSONDecodeError) as exc:
|
||||
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
|
||||
|
||||
def heaven_personal(self) -> None:
|
||||
try:
|
||||
body = self.read_json_body()
|
||||
result = self.application_service.heaven_personal(body)
|
||||
self.send_json({"ok": True, "personal": result})
|
||||
except (ValueError, json.JSONDecodeError) as exc:
|
||||
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
|
||||
|
||||
def heaven_interpret(self) -> None:
|
||||
try:
|
||||
body = self.read_json_body()
|
||||
result = self.application_service.heaven_interpret(body)
|
||||
self.send_json({"ok": True, **result})
|
||||
except (ValueError, json.JSONDecodeError) as exc:
|
||||
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
|
||||
@@ -0,0 +1,111 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
|
||||
class HeavenRepositoryMixin:
|
||||
@staticmethod
|
||||
def _heaven_reading_dict(row: sqlite3.Row | None) -> dict[str, Any] | None:
|
||||
if not row:
|
||||
return None
|
||||
return {
|
||||
"id": int(row["id"]),
|
||||
"mode": str(row["mode"]),
|
||||
"context_date": str(row["context_date"]),
|
||||
"subject": str(row["subject"]),
|
||||
"subject_detail": str(row["subject_detail"]),
|
||||
"answer": str(row["answer"]),
|
||||
"created_at": str(row["created_at"]),
|
||||
}
|
||||
|
||||
def save_heaven_reading(
|
||||
self,
|
||||
user_id: int,
|
||||
mode: str,
|
||||
context_date: str,
|
||||
subject: str,
|
||||
subject_detail: str,
|
||||
answer: str,
|
||||
context_snapshot: dict[str, Any],
|
||||
dedupe_key: str,
|
||||
) -> dict[str, Any]:
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
snapshot_json = json.dumps(
|
||||
context_snapshot, ensure_ascii=False, separators=(",", ":")
|
||||
)
|
||||
with self.connect() as connection:
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO heaven_readings
|
||||
(user_id, mode, context_date, subject, subject_detail, answer,
|
||||
context_snapshot, dedupe_key, created_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(user_id, dedupe_key) DO NOTHING
|
||||
""",
|
||||
(
|
||||
int(user_id), mode, context_date, subject, subject_detail,
|
||||
answer, snapshot_json, dedupe_key, now,
|
||||
),
|
||||
)
|
||||
row = connection.execute(
|
||||
"""
|
||||
SELECT id, mode, context_date, subject, subject_detail, answer, created_at
|
||||
FROM heaven_readings WHERE user_id = ? AND dedupe_key = ?
|
||||
""",
|
||||
(int(user_id), dedupe_key),
|
||||
).fetchone()
|
||||
connection.execute(
|
||||
"""
|
||||
DELETE FROM heaven_readings
|
||||
WHERE user_id = ? AND mode = ? AND id NOT IN (
|
||||
SELECT id FROM heaven_readings
|
||||
WHERE user_id = ? AND mode = ? ORDER BY id DESC LIMIT 100
|
||||
)
|
||||
""",
|
||||
(int(user_id), mode, int(user_id), mode),
|
||||
)
|
||||
result = self._heaven_reading_dict(row)
|
||||
if not result:
|
||||
raise ValueError("解读记录保存失败。")
|
||||
return result
|
||||
|
||||
def list_heaven_readings(
|
||||
self,
|
||||
user_id: int,
|
||||
mode: str,
|
||||
context_date: str = "",
|
||||
limit: int = 100,
|
||||
) -> list[dict[str, Any]]:
|
||||
clauses = ["user_id = ?", "mode = ?"]
|
||||
parameters: list[Any] = [int(user_id), mode]
|
||||
if context_date:
|
||||
clauses.append("context_date = ?")
|
||||
parameters.append(context_date)
|
||||
parameters.append(max(1, min(100, int(limit))))
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
f"""
|
||||
SELECT id, mode, context_date, subject, subject_detail, answer, created_at
|
||||
FROM heaven_readings WHERE {' AND '.join(clauses)}
|
||||
ORDER BY context_date DESC, id DESC LIMIT ?
|
||||
""",
|
||||
parameters,
|
||||
).fetchall()
|
||||
return [self._heaven_reading_dict(row) for row in rows if row]
|
||||
|
||||
def latest_heaven_reading(
|
||||
self, user_id: int, mode: str, context_date: str = ""
|
||||
) -> dict[str, Any] | None:
|
||||
items = self.list_heaven_readings(user_id, mode, context_date, 1)
|
||||
return items[0] if items else None
|
||||
|
||||
def delete_heaven_reading(self, user_id: int, reading_id: int) -> bool:
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"DELETE FROM heaven_readings WHERE id = ? AND user_id = ?",
|
||||
(int(reading_id), int(user_id)),
|
||||
)
|
||||
return cursor.rowcount > 0
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,13 @@
|
||||
"""Public market data, search, detail and chart feature."""
|
||||
|
||||
from .charts import ChartDataError, EastmoneyChartClient, MarketChartClient
|
||||
from .repository import MarketRepositoryMixin
|
||||
from .service import MarketServiceMixin
|
||||
|
||||
__all__ = [
|
||||
"ChartDataError",
|
||||
"EastmoneyChartClient",
|
||||
"MarketChartClient",
|
||||
"MarketRepositoryMixin",
|
||||
"MarketServiceMixin",
|
||||
]
|
||||
@@ -0,0 +1,488 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import http.client
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, time as dt_time, timedelta
|
||||
from threading import Lock
|
||||
from typing import Any, ClassVar
|
||||
|
||||
from backend.bootstrap.config import tushare_code as _stock_market_code
|
||||
from backend.data.providers.ifind_client import IfindError, IfindHttpClient
|
||||
|
||||
|
||||
class ChartDataError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
TRENDS_URL = "https://push2delay.eastmoney.com/api/qt/stock/trends2/get"
|
||||
BOARD_LIST_URL = "https://push2delay.eastmoney.com/api/qt/clist/get"
|
||||
BROWSER_USER_AGENT = (
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
|
||||
"AppleWebKit/537.36 (KHTML, like Gecko) "
|
||||
"Chrome/138.0.0.0 Safari/537.36"
|
||||
)
|
||||
INDEX_SECIDS = {
|
||||
"000001.SH": "1.000001",
|
||||
"399001.SZ": "0.399001",
|
||||
"399006.SZ": "0.399006",
|
||||
}
|
||||
|
||||
|
||||
class MarketChartClient:
|
||||
"""Prefer iFinD for display charts and retain Eastmoney as a last resort."""
|
||||
|
||||
def __init__(self, ifind: IfindHttpClient, fallback: "EastmoneyChartClient") -> None:
|
||||
self.ifind = ifind
|
||||
self.fallback = fallback
|
||||
|
||||
def stock_intraday(self, code: str) -> dict[str, Any]:
|
||||
normalized = str(code or "").strip()
|
||||
if not re.fullmatch(r"\d{6}", normalized):
|
||||
raise ChartDataError("Invalid stock code")
|
||||
ifind_code = _stock_market_code(normalized)
|
||||
try:
|
||||
return self._ifind_intraday(ifind_code, "stock", normalized)
|
||||
except (IfindError, ChartDataError):
|
||||
return self.fallback.stock_intraday(normalized)
|
||||
|
||||
def stock_daily(self, code: str, end_date: str, limit: int = 90) -> list[dict[str, Any]]:
|
||||
normalized = str(code or "").strip()
|
||||
if not re.fullmatch(r"\d{6}", normalized):
|
||||
raise ChartDataError("Invalid stock code")
|
||||
return self._ifind_daily(_stock_market_code(normalized), end_date, limit)
|
||||
|
||||
def index_daily(self, identifier: str, end_date: str, limit: int = 90) -> list[dict[str, Any]]:
|
||||
normalized = str(identifier or "").strip().upper()
|
||||
if normalized not in INDEX_SECIDS:
|
||||
raise ChartDataError("Unsupported index")
|
||||
return self._ifind_daily(normalized, end_date, limit)
|
||||
|
||||
def board_daily(self, identifier: str, end_date: str, limit: int = 90) -> list[dict[str, Any]]:
|
||||
normalized = str(identifier or "").strip().upper()
|
||||
if not normalized:
|
||||
raise ChartDataError("Invalid board code")
|
||||
return self._ifind_daily(normalized, end_date, limit)
|
||||
|
||||
def index_intraday(self, identifier: str) -> dict[str, Any]:
|
||||
normalized = str(identifier or "").strip().upper()
|
||||
if normalized not in INDEX_SECIDS:
|
||||
raise ChartDataError("Unsupported index")
|
||||
try:
|
||||
return self._ifind_intraday(normalized, "index", normalized)
|
||||
except (IfindError, ChartDataError):
|
||||
return self.fallback.index_intraday(normalized)
|
||||
|
||||
def board_intraday(self, identifier: str, name: str = "") -> dict[str, Any]:
|
||||
normalized = str(identifier or "").strip().upper()
|
||||
try:
|
||||
return self._ifind_intraday(normalized, "board", normalized, name)
|
||||
except (IfindError, ChartDataError):
|
||||
return self.fallback.board_intraday(normalized, name)
|
||||
|
||||
def _ifind_intraday(
|
||||
self,
|
||||
ifind_code: str,
|
||||
entity_type: str,
|
||||
identifier: str,
|
||||
name: str = "",
|
||||
) -> dict[str, Any]:
|
||||
if not self.ifind.configured:
|
||||
raise ChartDataError("iFinD is not configured")
|
||||
now = datetime.now().astimezone()
|
||||
rows: list[dict[str, Any]] = []
|
||||
for offset in range(0, 8):
|
||||
candidate = now.date() - timedelta(days=offset)
|
||||
if candidate.weekday() >= 5:
|
||||
continue
|
||||
display_date = candidate.isoformat()
|
||||
rows = self.ifind.intraday(
|
||||
ifind_code,
|
||||
f"{display_date} 09:30:00",
|
||||
f"{display_date} 15:00:00",
|
||||
cache_ttl=20 if offset == 0 else 6 * 60 * 60,
|
||||
)
|
||||
if rows:
|
||||
break
|
||||
points = [point for row in rows if (point := _ifind_point(row))]
|
||||
if not points:
|
||||
raise ChartDataError("No iFinD intraday chart data returned")
|
||||
latest_date = points[-1]["date"]
|
||||
points = [point for point in points if point["date"] == latest_date]
|
||||
previous_close = self._previous_close(ifind_code, latest_date, points[0]["open"])
|
||||
return {
|
||||
"entity_type": entity_type,
|
||||
"identifier": identifier,
|
||||
"name": name,
|
||||
"code": identifier,
|
||||
"trade_date": latest_date,
|
||||
"previous_close": previous_close,
|
||||
"points": points,
|
||||
"source": "ifind",
|
||||
}
|
||||
|
||||
def _ifind_daily(
|
||||
self, ifind_code: str, end_date: str, limit: int
|
||||
) -> list[dict[str, Any]]:
|
||||
if not self.ifind.configured:
|
||||
raise ChartDataError("iFinD is not configured")
|
||||
compact_end = str(end_date or "").replace("-", "")
|
||||
if not re.fullmatch(r"\d{8}", compact_end):
|
||||
raise ChartDataError("Invalid chart end date")
|
||||
end = datetime.strptime(compact_end, "%Y%m%d")
|
||||
start = (end - timedelta(days=max(190, limit * 3))).strftime("%Y%m%d")
|
||||
try:
|
||||
rows = self.ifind.history(
|
||||
ifind_code,
|
||||
["open", "high", "low", "close", "volume", "amount"],
|
||||
start,
|
||||
compact_end,
|
||||
cache_ttl=300,
|
||||
)
|
||||
except IfindError as exc:
|
||||
raise ChartDataError("No iFinD daily chart data returned") from exc
|
||||
normalized = []
|
||||
for row in rows:
|
||||
stamp = str(row.get("time") or "").strip()
|
||||
trade_date = stamp[:10]
|
||||
close = _number(row.get("close"))
|
||||
if not re.fullmatch(r"\d{4}-\d{2}-\d{2}", trade_date) or close <= 0:
|
||||
continue
|
||||
normalized.append(
|
||||
{
|
||||
"trade_date": trade_date,
|
||||
"open": _number(row.get("open")),
|
||||
"high": _number(row.get("high")),
|
||||
"low": _number(row.get("low")),
|
||||
"close": close,
|
||||
"volume": _number(row.get("volume")),
|
||||
"amount_billion": _number(row.get("amount")) / 100_000_000,
|
||||
}
|
||||
)
|
||||
normalized.sort(key=lambda row: row["trade_date"])
|
||||
for index, row in enumerate(normalized):
|
||||
previous = normalized[index - 1]["close"] if index > 0 else 0
|
||||
row["change"] = round((row["close"] / previous - 1) * 100, 4) if previous else 0.0
|
||||
|
||||
market_now = datetime.now().astimezone()
|
||||
today = market_now.strftime("%Y%m%d")
|
||||
market_open = (
|
||||
market_now.weekday() < 5
|
||||
and market_now.time().replace(tzinfo=None) >= dt_time(9, 30)
|
||||
)
|
||||
today_display = market_now.date().isoformat()
|
||||
if normalized and normalized[-1]["trade_date"] == today_display:
|
||||
current_bar = normalized[-1]
|
||||
current_bar_is_valid = (
|
||||
current_bar["open"] > 0
|
||||
and current_bar["high"] >= max(current_bar["open"], current_bar["close"])
|
||||
and 0 < current_bar["low"] <= min(current_bar["open"], current_bar["close"])
|
||||
and (current_bar["volume"] > 0 or current_bar["amount_billion"] > 0)
|
||||
)
|
||||
if not market_open or not current_bar_is_valid:
|
||||
normalized.pop()
|
||||
if compact_end == today and market_open:
|
||||
try:
|
||||
quote_rows = self.ifind.real_time(
|
||||
ifind_code,
|
||||
["open", "high", "low", "latest", "preClose", "volume", "amount"],
|
||||
cache_ttl=10,
|
||||
)
|
||||
quote = quote_rows[0] if quote_rows else {}
|
||||
latest = _number(quote.get("latest"))
|
||||
previous = _number(quote.get("preClose"))
|
||||
open_price = _number(quote.get("open"))
|
||||
high = _number(quote.get("high"))
|
||||
low = _number(quote.get("low"))
|
||||
volume = _number(quote.get("volume"))
|
||||
amount = _number(quote.get("amount"))
|
||||
quote_date = str(quote.get("time") or "")[:10].replace("-", "")
|
||||
quote_is_current = not quote_date or quote_date == today
|
||||
has_market_activity = volume > 0 or amount > 0
|
||||
if (
|
||||
latest > 0
|
||||
and open_price > 0
|
||||
and high >= max(open_price, latest)
|
||||
and 0 < low <= min(open_price, latest)
|
||||
and has_market_activity
|
||||
and quote_is_current
|
||||
):
|
||||
realtime = {
|
||||
"trade_date": end.strftime("%Y-%m-%d"),
|
||||
"open": open_price,
|
||||
"high": high,
|
||||
"low": low,
|
||||
"close": latest,
|
||||
"change": round((latest / previous - 1) * 100, 4) if previous else 0.0,
|
||||
"volume": volume,
|
||||
"amount_billion": amount / 100_000_000,
|
||||
"realtime": True,
|
||||
}
|
||||
if normalized and normalized[-1]["trade_date"] == realtime["trade_date"]:
|
||||
normalized[-1] = realtime
|
||||
else:
|
||||
normalized.append(realtime)
|
||||
except IfindError:
|
||||
pass
|
||||
if not normalized:
|
||||
raise ChartDataError("No iFinD daily chart data returned")
|
||||
return normalized[-max(20, min(180, int(limit))):]
|
||||
|
||||
def _previous_close(self, code: str, trade_date: str, fallback: float) -> float:
|
||||
today = datetime.now().astimezone().date().isoformat()
|
||||
if trade_date == today:
|
||||
try:
|
||||
quote = self.ifind.real_time(code, ["preClose"], cache_ttl=20)
|
||||
value = _number((quote[0] if quote else {}).get("preClose"))
|
||||
if value > 0:
|
||||
return value
|
||||
except IfindError:
|
||||
pass
|
||||
end = datetime.strptime(trade_date, "%Y-%m-%d")
|
||||
try:
|
||||
rows = self.ifind.history(
|
||||
code,
|
||||
["close"],
|
||||
(end - timedelta(days=12)).strftime("%Y%m%d"),
|
||||
end.strftime("%Y%m%d"),
|
||||
cache_ttl=6 * 60 * 60,
|
||||
)
|
||||
closes = [_number(row.get("close")) for row in rows if _number(row.get("close")) > 0]
|
||||
if len(closes) >= 2:
|
||||
return closes[-2]
|
||||
except IfindError:
|
||||
pass
|
||||
return fallback
|
||||
|
||||
|
||||
@dataclass
|
||||
class EastmoneyChartClient:
|
||||
"""Isolated display-only minute chart source.
|
||||
|
||||
The returned data must not be used by market snapshots, scoring, screening,
|
||||
or divination. Its only consumer is a chart-rendering endpoint.
|
||||
"""
|
||||
|
||||
timeout: int = 6
|
||||
cache_ttl_seconds: int = 20
|
||||
retry_attempts: int = 2
|
||||
_cache: ClassVar[dict[str, dict[str, Any]]] = {}
|
||||
_cache_lock: ClassVar[Lock] = Lock()
|
||||
_board_catalog: ClassVar[dict[str, dict[str, str]]] = {}
|
||||
_board_catalog_at: ClassVar[float] = 0.0
|
||||
_board_catalog_lock: ClassVar[Lock] = Lock()
|
||||
|
||||
def stock_intraday(self, code: str) -> dict[str, Any]:
|
||||
normalized = str(code or "").strip()
|
||||
if not re.fullmatch(r"\d{6}", normalized):
|
||||
raise ChartDataError("Invalid stock code")
|
||||
market = "1" if normalized.startswith(("5", "6", "9")) else "0"
|
||||
return self._intraday(f"{market}.{normalized}", "stock", normalized)
|
||||
|
||||
def index_intraday(self, identifier: str) -> dict[str, Any]:
|
||||
normalized = str(identifier or "").strip().upper()
|
||||
secid = INDEX_SECIDS.get(normalized)
|
||||
if not secid:
|
||||
raise ChartDataError("Unsupported index")
|
||||
return self._intraday(secid, "index", normalized)
|
||||
|
||||
def board_intraday(self, identifier: str, name: str = "") -> dict[str, Any]:
|
||||
normalized = str(identifier or "").strip().upper()
|
||||
if re.fullmatch(r"BK\d{4}", normalized):
|
||||
board_code = normalized
|
||||
else:
|
||||
board_code = self._resolve_board_code(name or identifier)
|
||||
return self._intraday(f"90.{board_code}", "board", board_code)
|
||||
|
||||
def _intraday(self, secid: str, entity_type: str, identifier: str) -> dict[str, Any]:
|
||||
cache_key = f"{entity_type}:{identifier}"
|
||||
cached = self._get_cached(cache_key)
|
||||
if cached is not None:
|
||||
return cached
|
||||
|
||||
payload = self._request_json(
|
||||
TRENDS_URL,
|
||||
{
|
||||
"secid": secid,
|
||||
"fields1": "f1,f2,f3,f4,f5,f6,f7,f8,f9,f10,f11,f12,f13",
|
||||
"fields2": "f51,f52,f53,f54,f55,f56,f57,f58",
|
||||
"iscr": "0",
|
||||
"ndays": "1",
|
||||
},
|
||||
"https://quote.eastmoney.com/",
|
||||
)
|
||||
data = payload.get("data") or {}
|
||||
points = [point for raw in data.get("trends") or [] if (point := _parse_trend(raw))]
|
||||
if not points:
|
||||
raise ChartDataError("No intraday chart data returned")
|
||||
|
||||
result = {
|
||||
"entity_type": entity_type,
|
||||
"identifier": identifier,
|
||||
"name": str(data.get("name") or ""),
|
||||
"code": str(data.get("code") or identifier),
|
||||
"trade_date": points[-1]["date"],
|
||||
"previous_close": _number(data.get("preClose")),
|
||||
"points": points,
|
||||
}
|
||||
with self._cache_lock:
|
||||
self._cache[cache_key] = {"created_at": time.time(), "payload": result}
|
||||
return result
|
||||
|
||||
def _get_cached(self, cache_key: str) -> dict[str, Any] | None:
|
||||
with self._cache_lock:
|
||||
cached = self._cache.get(cache_key)
|
||||
if not cached:
|
||||
return None
|
||||
if time.time() - float(cached.get("created_at") or 0) > self.cache_ttl_seconds:
|
||||
with self._cache_lock:
|
||||
self._cache.pop(cache_key, None)
|
||||
return None
|
||||
return dict(cached["payload"])
|
||||
|
||||
def _resolve_board_code(self, name: str) -> str:
|
||||
normalized = _normalize_name(name)
|
||||
if not normalized:
|
||||
raise ChartDataError("Board name is required")
|
||||
catalog = self._load_board_catalog()
|
||||
item = catalog.get(normalized)
|
||||
if not item:
|
||||
raise ChartDataError("No matching chart board")
|
||||
return item["code"]
|
||||
|
||||
def _load_board_catalog(self) -> dict[str, dict[str, str]]:
|
||||
now = time.time()
|
||||
with self._board_catalog_lock:
|
||||
if self._board_catalog and now - self._board_catalog_at < 6 * 60 * 60:
|
||||
return dict(self._board_catalog)
|
||||
|
||||
rows: list[dict[str, Any]] = []
|
||||
for board_type in ("1", "2", "3"):
|
||||
for page in range(1, 6):
|
||||
payload = self._request_json(
|
||||
BOARD_LIST_URL,
|
||||
{
|
||||
"pn": str(page),
|
||||
"pz": "100",
|
||||
"po": "1",
|
||||
"np": "1",
|
||||
"fltt": "2",
|
||||
"invt": "2",
|
||||
"fid": "f3",
|
||||
"fs": f"m:90+t:{board_type}",
|
||||
"fields": "f12,f14",
|
||||
},
|
||||
"https://quote.eastmoney.com/center/boardlist.html",
|
||||
)
|
||||
page_rows = (payload.get("data") or {}).get("diff") or []
|
||||
rows.extend(page_rows)
|
||||
if len(page_rows) < 100:
|
||||
break
|
||||
|
||||
catalog: dict[str, dict[str, str]] = {}
|
||||
for row in rows:
|
||||
code = str(row.get("f12") or "").strip().upper()
|
||||
board_name = str(row.get("f14") or "").strip()
|
||||
if re.fullmatch(r"BK\d{4}", code) and board_name:
|
||||
catalog.setdefault(_normalize_name(board_name), {"code": code, "name": board_name})
|
||||
if not catalog:
|
||||
raise ChartDataError("Board chart directory is unavailable")
|
||||
with self._board_catalog_lock:
|
||||
type(self)._board_catalog = catalog
|
||||
type(self)._board_catalog_at = now
|
||||
return dict(catalog)
|
||||
|
||||
def _request_json(
|
||||
self, url: str, params: dict[str, str], referer: str
|
||||
) -> dict[str, Any]:
|
||||
request_url = f"{url}?{urllib.parse.urlencode(params)}"
|
||||
last_error: Exception | None = None
|
||||
for attempt in range(max(1, int(self.retry_attempts))):
|
||||
request = urllib.request.Request(
|
||||
request_url,
|
||||
headers={
|
||||
"Accept": "application/json,text/plain,*/*",
|
||||
"Connection": "close",
|
||||
"Referer": referer,
|
||||
"User-Agent": BROWSER_USER_AGENT,
|
||||
},
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=self.timeout) as response:
|
||||
payload = json.loads(response.read().decode("utf-8"))
|
||||
if not isinstance(payload, dict):
|
||||
raise ChartDataError("Invalid intraday chart response")
|
||||
return payload
|
||||
except (
|
||||
urllib.error.URLError,
|
||||
TimeoutError,
|
||||
ConnectionError,
|
||||
OSError,
|
||||
http.client.HTTPException,
|
||||
json.JSONDecodeError,
|
||||
ChartDataError,
|
||||
) as exc:
|
||||
last_error = exc
|
||||
if attempt + 1 < self.retry_attempts:
|
||||
time.sleep(0.12)
|
||||
raise ChartDataError("Intraday chart request failed") from last_error
|
||||
|
||||
|
||||
def _parse_trend(raw: Any) -> dict[str, Any] | None:
|
||||
fields = str(raw or "").split(",")
|
||||
if len(fields) < 8 or " " not in fields[0]:
|
||||
return None
|
||||
stamp = fields[0].strip()
|
||||
trade_date, trade_time = stamp.split(" ", 1)
|
||||
close = _number(fields[2])
|
||||
if close <= 0:
|
||||
return None
|
||||
return {
|
||||
"date": trade_date,
|
||||
"time": trade_time[:5],
|
||||
"open": _number(fields[1]),
|
||||
"close": close,
|
||||
"high": _number(fields[3]),
|
||||
"low": _number(fields[4]),
|
||||
"volume": _number(fields[5]),
|
||||
"amount": _number(fields[6]),
|
||||
"average": _number(fields[7]),
|
||||
}
|
||||
|
||||
|
||||
def _ifind_point(row: dict[str, Any]) -> dict[str, Any] | None:
|
||||
stamp = str(row.get("time") or "").strip()
|
||||
if " " not in stamp:
|
||||
return None
|
||||
trade_date, trade_time = stamp.split(" ", 1)
|
||||
close = _number(row.get("close"))
|
||||
if close <= 0:
|
||||
return None
|
||||
return {
|
||||
"date": trade_date,
|
||||
"time": trade_time[:5],
|
||||
"open": _number(row.get("open")),
|
||||
"close": close,
|
||||
"high": _number(row.get("high")),
|
||||
"low": _number(row.get("low")),
|
||||
"volume": _number(row.get("volume")),
|
||||
"amount": _number(row.get("amount")),
|
||||
"average": _number(row.get("avgPrice")),
|
||||
}
|
||||
|
||||
|
||||
def _number(value: Any) -> float:
|
||||
try:
|
||||
return float(value or 0)
|
||||
except (TypeError, ValueError):
|
||||
return 0.0
|
||||
|
||||
|
||||
def _normalize_name(value: Any) -> str:
|
||||
normalized = re.sub(r"[\s·・()()\-_/]", "", str(value or "")).casefold()
|
||||
return re.sub(r"(?:概念|行业|[ⅠⅡⅢ])$", "", normalized)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,290 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
|
||||
class MarketRepositoryMixin:
|
||||
def upsert_stock_master(self, rows: list[dict[str, Any]]) -> int:
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
values = [
|
||||
(
|
||||
row.get("ts_code", ""),
|
||||
str(row.get("ts_code", "")).split(".")[0],
|
||||
row.get("name") or "--",
|
||||
row.get("industry") or "",
|
||||
row.get("market") or "",
|
||||
str(row.get("list_date") or ""),
|
||||
now,
|
||||
)
|
||||
for row in rows if row.get("ts_code")
|
||||
]
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO stock_master
|
||||
(ts_code, code, name, industry, market, list_date, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(ts_code) DO UPDATE SET
|
||||
code=excluded.code, name=excluded.name, industry=excluded.industry,
|
||||
market=excluded.market, list_date=excluded.list_date, updated_at=excluded.updated_at
|
||||
""",
|
||||
values,
|
||||
)
|
||||
return len(values)
|
||||
|
||||
def list_stock_master(self) -> list[dict[str, Any]]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"SELECT ts_code, code, name, industry, market, list_date FROM stock_master"
|
||||
).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
def upsert_daily_bars(self, rows: list[dict[str, Any]]) -> int:
|
||||
values = [
|
||||
(
|
||||
str(row.get("trade_date") or ""), row.get("ts_code", ""),
|
||||
float(row.get("open") or 0), float(row.get("high") or 0),
|
||||
float(row.get("low") or 0), float(row.get("close") or 0),
|
||||
float(row.get("pct_chg") or 0), float(row.get("vol") or 0),
|
||||
float(row.get("amount") or 0),
|
||||
)
|
||||
for row in rows if row.get("trade_date") and row.get("ts_code")
|
||||
]
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO daily_bars
|
||||
(trade_date, ts_code, open, high, low, close, pct_chg, vol, amount)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(trade_date, ts_code) DO UPDATE SET
|
||||
open=excluded.open, high=excluded.high, low=excluded.low,
|
||||
close=excluded.close, pct_chg=excluded.pct_chg,
|
||||
vol=excluded.vol, amount=excluded.amount
|
||||
""",
|
||||
values,
|
||||
)
|
||||
return len(values)
|
||||
|
||||
def daily_bars_for_date(self, trade_date: str) -> list[dict[str, Any]]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"SELECT * FROM daily_bars WHERE trade_date = ? ORDER BY ts_code",
|
||||
(trade_date,),
|
||||
).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
def get_snapshot(self, trade_date: str) -> dict[str, Any] | None:
|
||||
with self.connect() as connection:
|
||||
row = connection.execute(
|
||||
"SELECT payload FROM dashboard_snapshots WHERE trade_date = ?",
|
||||
(trade_date,),
|
||||
).fetchone()
|
||||
if not row:
|
||||
return None
|
||||
try:
|
||||
return json.loads(row["payload"])
|
||||
except json.JSONDecodeError:
|
||||
return None
|
||||
|
||||
def get_latest_real_snapshot(
|
||||
self, trade_date: str, strictly_before: bool = False
|
||||
) -> dict[str, Any] | None:
|
||||
operator = "<" if strictly_before else "<="
|
||||
with self.connect() as connection:
|
||||
row = connection.execute(
|
||||
f"""
|
||||
SELECT payload FROM dashboard_snapshots
|
||||
WHERE trade_date {operator} ? AND source != 'demo'
|
||||
ORDER BY trade_date DESC LIMIT 1
|
||||
""",
|
||||
(trade_date,),
|
||||
).fetchone()
|
||||
if not row:
|
||||
return None
|
||||
try:
|
||||
return json.loads(row["payload"])
|
||||
except json.JSONDecodeError:
|
||||
return None
|
||||
|
||||
def save_snapshot(self, trade_date: str, source: str, payload: dict[str, Any]) -> None:
|
||||
updated_at = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
record_count = sum(
|
||||
len(payload.get(key) or [])
|
||||
for key in ("limits", "broken", "down_limits", "yesterday_limits")
|
||||
)
|
||||
content = json.dumps(payload, ensure_ascii=False, separators=(",", ":"))
|
||||
with self.connect() as connection:
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO dashboard_snapshots
|
||||
(trade_date, source, payload, record_count, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
ON CONFLICT(trade_date) DO UPDATE SET
|
||||
source = excluded.source,
|
||||
payload = excluded.payload,
|
||||
record_count = excluded.record_count,
|
||||
updated_at = excluded.updated_at
|
||||
""",
|
||||
(trade_date, source, content, record_count, updated_at),
|
||||
)
|
||||
|
||||
def get_data_snapshot(self, kind: str, cache_key: str) -> dict[str, Any] | None:
|
||||
with self.connect() as connection:
|
||||
row = connection.execute(
|
||||
"SELECT payload FROM data_snapshots WHERE kind = ? AND cache_key = ?",
|
||||
(kind, cache_key),
|
||||
).fetchone()
|
||||
if not row:
|
||||
return None
|
||||
try:
|
||||
return json.loads(row["payload"])
|
||||
except json.JSONDecodeError:
|
||||
return None
|
||||
|
||||
def get_latest_data_snapshot(
|
||||
self,
|
||||
kind: str,
|
||||
cache_key_prefix: str,
|
||||
maximum_cache_key: str,
|
||||
exclude_source: str = "",
|
||||
) -> dict[str, Any] | None:
|
||||
source_clause = " AND source != ?" if exclude_source else ""
|
||||
parameters: list[Any] = [kind, f"{cache_key_prefix}%", maximum_cache_key]
|
||||
if exclude_source:
|
||||
parameters.append(exclude_source)
|
||||
with self.connect() as connection:
|
||||
row = connection.execute(
|
||||
f"""
|
||||
SELECT payload FROM data_snapshots
|
||||
WHERE kind = ? AND cache_key LIKE ? AND cache_key <= ?{source_clause}
|
||||
ORDER BY cache_key DESC LIMIT 1
|
||||
""",
|
||||
parameters,
|
||||
).fetchone()
|
||||
if not row:
|
||||
return None
|
||||
try:
|
||||
return json.loads(row["payload"])
|
||||
except json.JSONDecodeError:
|
||||
return None
|
||||
|
||||
def save_data_snapshot(
|
||||
self, kind: str, cache_key: str, source: str, payload: dict[str, Any]
|
||||
) -> None:
|
||||
updated_at = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
content = json.dumps(payload, ensure_ascii=False, separators=(",", ":"))
|
||||
with self.connect() as connection:
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO data_snapshots (kind, cache_key, source, payload, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
ON CONFLICT(kind, cache_key) DO UPDATE SET
|
||||
source = excluded.source,
|
||||
payload = excluded.payload,
|
||||
updated_at = excluded.updated_at
|
||||
""",
|
||||
(kind, cache_key, source, content, updated_at),
|
||||
)
|
||||
|
||||
def search_stock_master(self, query: str, limit: int = 12) -> list[dict[str, Any]]:
|
||||
text = str(query or "").strip()
|
||||
if not text:
|
||||
return []
|
||||
escaped = text.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"""
|
||||
SELECT ts_code, code, name, industry, market, list_date
|
||||
FROM stock_master
|
||||
WHERE code = ? OR name = ? OR name LIKE ? ESCAPE '\\'
|
||||
ORDER BY
|
||||
CASE WHEN code = ? THEN 0 WHEN name = ? THEN 1 ELSE 2 END,
|
||||
list_date DESC,
|
||||
code
|
||||
LIMIT ?
|
||||
""",
|
||||
(text, text, f"%{escaped}%", text, text, max(1, min(30, int(limit)))),
|
||||
).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
def list_snapshot_payloads(self, end_date: str, limit: int = 260) -> list[dict[str, Any]]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"""
|
||||
SELECT trade_date, payload FROM dashboard_snapshots
|
||||
WHERE trade_date <= ? ORDER BY trade_date DESC LIMIT ?
|
||||
""",
|
||||
(end_date, limit),
|
||||
).fetchall()
|
||||
result: list[dict[str, Any]] = []
|
||||
for row in reversed(rows):
|
||||
try:
|
||||
payload = json.loads(row["payload"])
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
payload["_snapshot_date"] = row["trade_date"]
|
||||
result.append(payload)
|
||||
return result
|
||||
|
||||
def start_sync(self, trade_date: str, source: str) -> int:
|
||||
started_at = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"""
|
||||
INSERT INTO sync_runs (trade_date, source, status, started_at)
|
||||
VALUES (?, ?, 'running', ?)
|
||||
""",
|
||||
(trade_date, source, started_at),
|
||||
)
|
||||
return int(cursor.lastrowid)
|
||||
|
||||
def finish_sync(
|
||||
self,
|
||||
sync_id: int,
|
||||
status: str,
|
||||
record_count: int = 0,
|
||||
message: str = "",
|
||||
source: str | None = None,
|
||||
) -> None:
|
||||
finished_at = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
connection.execute(
|
||||
"""
|
||||
UPDATE sync_runs
|
||||
SET status = ?, finished_at = ?, record_count = ?, message = ?,
|
||||
source = COALESCE(?, source)
|
||||
WHERE id = ?
|
||||
""",
|
||||
(status, finished_at, record_count, message[:1000], source, sync_id),
|
||||
)
|
||||
|
||||
def status(self) -> dict[str, Any]:
|
||||
with self.connect() as connection:
|
||||
last_sync = connection.execute(
|
||||
"""
|
||||
SELECT id, trade_date, source, status, started_at, finished_at,
|
||||
record_count, message
|
||||
FROM sync_runs ORDER BY id DESC LIMIT 1
|
||||
"""
|
||||
).fetchone()
|
||||
snapshot_stats = connection.execute(
|
||||
"""
|
||||
SELECT COUNT(*) AS dates, COALESCE(SUM(record_count), 0) AS records,
|
||||
MAX(updated_at) AS updated_at
|
||||
FROM dashboard_snapshots
|
||||
"""
|
||||
).fetchone()
|
||||
watchlist_count = connection.execute("SELECT COUNT(*) FROM watchlist").fetchone()[0]
|
||||
note_count = connection.execute("SELECT COUNT(*) FROM review_notes").fetchone()[0]
|
||||
|
||||
return {
|
||||
"database": str(self.path.name),
|
||||
"snapshot_dates": int(snapshot_stats["dates"]),
|
||||
"snapshot_records": int(snapshot_stats["records"]),
|
||||
"updated_at": snapshot_stats["updated_at"],
|
||||
"last_sync": dict(last_sync) if last_sync else None,
|
||||
"watchlist_count": int(watchlist_count),
|
||||
"note_count": int(note_count),
|
||||
}
|
||||
|
||||
@@ -0,0 +1,958 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import re
|
||||
from datetime import date, datetime, time as dt_time, timedelta
|
||||
from typing import Any
|
||||
|
||||
from backend.bootstrap.config import (
|
||||
normalize_date,
|
||||
tushare_code,
|
||||
validate_stock_code,
|
||||
validate_text,
|
||||
)
|
||||
from backend.data.providers.ifind_client import IfindError
|
||||
from backend.data.providers.tushare_client import TushareClient, TushareError
|
||||
from backend.features.market.charts import ChartDataError
|
||||
from backend.features.market.insights import MarketInsightsService
|
||||
from backend.features.sentiment.engine import SENTIMENT_ENGINE_VERSION
|
||||
|
||||
|
||||
SEARCH_INDEXES = (
|
||||
{"id": "000001.SH", "code": "000001.SH", "name": "上证指数", "type": "index", "subtitle": "沪市综合指数"},
|
||||
{"id": "399001.SZ", "code": "399001.SZ", "name": "深证成指", "type": "index", "subtitle": "深市成份指数"},
|
||||
{"id": "399006.SZ", "code": "399006.SZ", "name": "创业板指", "type": "index", "subtitle": "创业板核心指数"},
|
||||
)
|
||||
SEARCH_TYPE_LABELS = {
|
||||
"stock": "股票",
|
||||
"sector": "板块",
|
||||
"theme": "题材",
|
||||
"index": "指数",
|
||||
}
|
||||
THS_SEARCH_TYPES = {
|
||||
"I": ("sector", "行业板块"),
|
||||
"R": ("sector", "地域板块"),
|
||||
"N": ("theme", "概念题材"),
|
||||
}
|
||||
|
||||
|
||||
class MarketServiceMixin:
|
||||
def _market_insights(self) -> MarketInsightsService:
|
||||
if not self.configured:
|
||||
raise ValueError("行情数据尚未配置。")
|
||||
return MarketInsightsService(
|
||||
self.database,
|
||||
self._tushare_client(),
|
||||
ifind=self.ifind,
|
||||
)
|
||||
def _tushare_client(self) -> TushareClient:
|
||||
gateway = getattr(self, "data_gateway", None)
|
||||
if gateway is not None:
|
||||
return gateway.tushare()
|
||||
# Compatibility for isolated legacy unit-test service stubs.
|
||||
return TushareClient(self.token)
|
||||
|
||||
def get_dashboard(self, trade_date: str, force: bool = False) -> dict[str, Any]:
|
||||
normalized_date = normalize_date(trade_date)
|
||||
now = datetime.now().astimezone()
|
||||
if (
|
||||
normalized_date == now.strftime("%Y%m%d")
|
||||
and now.time().replace(tzinfo=None) < datetime.strptime("09:15", "%H:%M").time()
|
||||
):
|
||||
previous = self.database.get_latest_real_snapshot(normalized_date, strictly_before=True)
|
||||
if previous:
|
||||
carried = self._carry_dashboard(previous, normalized_date, "盘前沿用最近交易日收盘行情")
|
||||
return self._apply_reason_overrides(self._with_storage(carried, cached=True))
|
||||
if not force:
|
||||
snapshot = self.database.get_snapshot(normalized_date)
|
||||
if snapshot and str((snapshot.get("meta") or {}).get("source") or "") != "demo":
|
||||
snapshot = copy.deepcopy(snapshot)
|
||||
if normalized_date != now.strftime("%Y%m%d"):
|
||||
snapshot.setdefault("meta", {}).update(
|
||||
{"realtime": False, "market_status": "closed"}
|
||||
)
|
||||
if not self._dashboard_sentiment_ready(snapshot):
|
||||
snapshot = self._enrich_dashboard_sentiment(snapshot, normalized_date)
|
||||
self.database.save_snapshot(
|
||||
normalized_date,
|
||||
str((snapshot.get("meta") or {}).get("source") or "tushare"),
|
||||
snapshot,
|
||||
)
|
||||
snapshot.setdefault("meta", {})["requested_date"] = self._display_compact_date(normalized_date)
|
||||
return self._apply_reason_overrides(self._with_storage(snapshot, cached=True))
|
||||
resolved = self.database.get_data_snapshot(
|
||||
"dashboard_request_v1", normalized_date
|
||||
)
|
||||
if resolved and str((resolved.get("meta") or {}).get("source") or "") != "demo":
|
||||
resolved = copy.deepcopy(resolved)
|
||||
resolved.setdefault("meta", {})["requested_date"] = self._display_compact_date(
|
||||
normalized_date
|
||||
)
|
||||
return self._apply_reason_overrides(
|
||||
self._with_storage(resolved, cached=True)
|
||||
)
|
||||
if datetime.strptime(normalized_date, "%Y%m%d").weekday() >= 5:
|
||||
previous = self.database.get_latest_real_snapshot(normalized_date)
|
||||
if previous:
|
||||
carried = self._carry_dashboard(
|
||||
previous,
|
||||
normalized_date,
|
||||
"非交易日沿用最近交易日收盘行情",
|
||||
)
|
||||
self.database.save_data_snapshot(
|
||||
"dashboard_request_v1", normalized_date, "sqlite", carried
|
||||
)
|
||||
return self._apply_reason_overrides(
|
||||
self._with_storage(carried, cached=True)
|
||||
)
|
||||
return self.sync_dashboard(normalized_date)
|
||||
|
||||
@staticmethod
|
||||
def _dashboard_sentiment_ready(dashboard: dict[str, Any]) -> bool:
|
||||
overview = dashboard.get("overview") or {}
|
||||
return int(overview.get("sentiment_engine_version") or 0) == SENTIMENT_ENGINE_VERSION and all(
|
||||
key in overview
|
||||
for key in (
|
||||
"sentiment_score",
|
||||
"sentiment_label",
|
||||
"sentiment_phase",
|
||||
"sentiment_direction",
|
||||
"sentiment_components",
|
||||
)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _display_compact_date(compact: str) -> str:
|
||||
return f"{compact[:4]}-{compact[4:6]}-{compact[6:8]}"
|
||||
|
||||
def _carry_dashboard(
|
||||
self, snapshot: dict[str, Any], requested_date: str, reason: str
|
||||
) -> dict[str, Any]:
|
||||
carried = copy.deepcopy(snapshot)
|
||||
meta = carried.setdefault("meta", {})
|
||||
meta.update(
|
||||
{
|
||||
"requested_date": self._display_compact_date(requested_date),
|
||||
"carried_forward": True,
|
||||
"realtime": False,
|
||||
"market_status": "closed",
|
||||
"notice": reason,
|
||||
}
|
||||
)
|
||||
return carried
|
||||
|
||||
def _realtime_snapshot_due(
|
||||
self,
|
||||
normalized_date: str,
|
||||
snapshot: dict[str, Any],
|
||||
) -> bool:
|
||||
if not self.configured or normalized_date != date.today().strftime("%Y%m%d"):
|
||||
return False
|
||||
now = datetime.now().astimezone()
|
||||
local_time = now.time().replace(tzinfo=None)
|
||||
realtime_start = datetime.strptime("09:15", "%H:%M").time()
|
||||
morning_end = datetime.strptime("11:35", "%H:%M").time()
|
||||
afternoon_start = datetime.strptime("12:55", "%H:%M").time()
|
||||
realtime_end = datetime.strptime("15:05", "%H:%M").time()
|
||||
in_session = (
|
||||
realtime_start <= local_time < morning_end
|
||||
or afternoon_start <= local_time < realtime_end
|
||||
)
|
||||
if not in_session:
|
||||
return False
|
||||
meta = snapshot.get("meta") or {}
|
||||
snapshot_trade_date = str(meta.get("trade_date") or "").replace("-", "")
|
||||
if snapshot_trade_date and snapshot_trade_date != normalized_date:
|
||||
return False
|
||||
if not meta.get("realtime"):
|
||||
return True
|
||||
try:
|
||||
updated_at = datetime.fromisoformat(str(meta.get("updated_at") or ""))
|
||||
if updated_at.tzinfo is None:
|
||||
updated_at = updated_at.replace(tzinfo=now.tzinfo)
|
||||
except ValueError:
|
||||
return True
|
||||
age_seconds = (now - updated_at.astimezone(now.tzinfo)).total_seconds()
|
||||
return age_seconds >= 8
|
||||
|
||||
def sync_dashboard(self, trade_date: str) -> dict[str, Any]:
|
||||
normalized_date = normalize_date(trade_date)
|
||||
source = "tushare"
|
||||
with self.sync_lock:
|
||||
sync_id = self.database.start_sync(normalized_date, source)
|
||||
try:
|
||||
if not self.configured:
|
||||
raise TushareError("公共行情尚未配置")
|
||||
dashboard = self._tushare_client().dashboard(normalized_date)
|
||||
|
||||
dashboard["meta"]["source"] = source
|
||||
dashboard["meta"]["requested_date"] = self._display_compact_date(normalized_date)
|
||||
dashboard = self._enrich_dashboard_sentiment(dashboard, normalized_date)
|
||||
record_count = self._record_count(dashboard)
|
||||
actual_date = normalize_date(
|
||||
str(dashboard.get("meta", {}).get("trade_date") or normalized_date)
|
||||
)
|
||||
self.database.save_snapshot(actual_date, source, dashboard)
|
||||
if actual_date != normalized_date:
|
||||
dashboard.setdefault("meta", {}).update(
|
||||
{
|
||||
"carried_forward": True,
|
||||
"realtime": False,
|
||||
"market_status": "closed",
|
||||
}
|
||||
)
|
||||
self.database.save_data_snapshot(
|
||||
"dashboard_request_v1", normalized_date, source, dashboard
|
||||
)
|
||||
self.database.finish_sync(
|
||||
sync_id,
|
||||
"success",
|
||||
record_count,
|
||||
dashboard.get("meta", {}).get("notice", ""),
|
||||
source,
|
||||
)
|
||||
return self._apply_reason_overrides(self._with_storage(dashboard, cached=False))
|
||||
except TushareError as exc:
|
||||
fallback = self.database.get_latest_real_snapshot(normalized_date)
|
||||
if fallback:
|
||||
carried = self._carry_dashboard(
|
||||
fallback, normalized_date, f"最新行情暂不可用,沿用最近收盘快照:{exc}"
|
||||
)
|
||||
self.database.finish_sync(
|
||||
sync_id, "fallback", self._record_count(carried), str(exc), "tushare"
|
||||
)
|
||||
return self._apply_reason_overrides(self._with_storage(carried, cached=True))
|
||||
self.database.finish_sync(sync_id, "failed", message=str(exc))
|
||||
raise ValueError("暂无可用的真实行情快照,请等待后台完成首次同步。") from exc
|
||||
except Exception as exc:
|
||||
self.database.finish_sync(sync_id, "failed", message=str(exc))
|
||||
raise
|
||||
|
||||
def realtime_aggregate_health(self, sector: str = "") -> dict[str, Any]:
|
||||
sector = validate_text(sector, "板块名称", 50)
|
||||
return self.realtime_aggregator.health_snapshot(sector)
|
||||
|
||||
def _search_market_directory(self) -> list[dict[str, Any]]:
|
||||
cached = self.database.get_data_snapshot("search_directory", "ths") or {}
|
||||
cached_items = list(cached.get("items") or [])
|
||||
if cached_items and int(cached.get("schema_version") or 0) >= 2:
|
||||
return cached_items
|
||||
if not self.configured:
|
||||
return cached_items
|
||||
|
||||
try:
|
||||
rows = self._tushare_client().query(
|
||||
"ths_index",
|
||||
{},
|
||||
"ts_code,name,count,exchange,list_date,type",
|
||||
)
|
||||
except TushareError:
|
||||
return cached_items
|
||||
|
||||
items = []
|
||||
for row in rows:
|
||||
mapping = THS_SEARCH_TYPES.get(str(row.get("type") or "").upper())
|
||||
code = str(row.get("ts_code") or "").strip().upper()
|
||||
name = str(row.get("name") or "").strip()
|
||||
if not mapping or not code or not name or str(row.get("exchange") or "").upper() != "A":
|
||||
continue
|
||||
entity_type, subtitle = mapping
|
||||
items.append(
|
||||
{
|
||||
"id": code,
|
||||
"code": code,
|
||||
"name": name,
|
||||
"type": entity_type,
|
||||
"subtitle": subtitle,
|
||||
"member_count": int(float(row.get("count") or 0)),
|
||||
}
|
||||
)
|
||||
if items:
|
||||
self.database.save_data_snapshot(
|
||||
"search_directory", "ths", "tushare", {"schema_version": 2, "items": items}
|
||||
)
|
||||
return items
|
||||
|
||||
@staticmethod
|
||||
def _search_match_score(item: dict[str, Any], query: str) -> tuple[int, int, str]:
|
||||
name = str(item.get("name") or "").casefold()
|
||||
code = str(item.get("code") or item.get("id") or "").casefold()
|
||||
needle = query.casefold()
|
||||
if code == needle:
|
||||
rank = 0
|
||||
elif name == needle:
|
||||
rank = 1
|
||||
elif code.startswith(needle):
|
||||
rank = 2
|
||||
elif name.startswith(needle):
|
||||
rank = 3
|
||||
else:
|
||||
rank = 4
|
||||
return rank, len(name), code
|
||||
|
||||
def search_entities(self, query: str, trade_date: str) -> dict[str, Any]:
|
||||
needle = str(query or "").strip()
|
||||
normalized_date = normalize_date(trade_date)
|
||||
groups: dict[str, list[dict[str, Any]]] = {
|
||||
"stocks": [],
|
||||
"sectors": [],
|
||||
"themes": [],
|
||||
"indices": [],
|
||||
}
|
||||
if not needle:
|
||||
return {"query": "", "trade_date": normalized_date, "groups": groups}
|
||||
|
||||
stocks = []
|
||||
for row in self.database.search_stock_master(needle, 12):
|
||||
stocks.append(
|
||||
{
|
||||
"id": str(row.get("code") or ""),
|
||||
"code": str(row.get("code") or ""),
|
||||
"name": str(row.get("name") or "--"),
|
||||
"type": "stock",
|
||||
"type_label": SEARCH_TYPE_LABELS["stock"],
|
||||
"industry": str(row.get("industry") or "其他"),
|
||||
"market": str(row.get("market") or ""),
|
||||
"subtitle": " · ".join(
|
||||
part for part in (str(row.get("industry") or ""), str(row.get("market") or "")) if part
|
||||
) or "A股",
|
||||
}
|
||||
)
|
||||
groups["stocks"] = stocks[:8]
|
||||
|
||||
market_items = list(self._search_market_directory()) + [dict(item) for item in SEARCH_INDEXES]
|
||||
matched = [
|
||||
item for item in market_items
|
||||
if needle.casefold() in str(item.get("name") or "").casefold()
|
||||
or needle.casefold() in str(item.get("code") or "").casefold()
|
||||
]
|
||||
matched.sort(key=lambda item: self._search_match_score(item, needle))
|
||||
group_keys = {"sector": "sectors", "theme": "themes", "index": "indices"}
|
||||
for item in matched:
|
||||
group_key = group_keys.get(str(item.get("type") or ""))
|
||||
if not group_key or len(groups[group_key]) >= 8:
|
||||
continue
|
||||
groups[group_key].append(
|
||||
{
|
||||
**item,
|
||||
"type_label": SEARCH_TYPE_LABELS[str(item["type"])],
|
||||
}
|
||||
)
|
||||
return {"query": needle, "trade_date": normalized_date, "groups": groups}
|
||||
|
||||
def get_search_detail(
|
||||
self, entity_type: str, identifier: str, trade_date: str
|
||||
) -> dict[str, Any]:
|
||||
entity_type = str(entity_type or "").strip().lower()
|
||||
identifier = str(identifier or "").strip().upper()
|
||||
normalized_date = normalize_date(trade_date)
|
||||
if entity_type not in {"sector", "theme", "index"}:
|
||||
raise ValueError("搜索详情类型不支持。")
|
||||
if not re.fullmatch(r"[A-Z0-9.]{3,24}", identifier):
|
||||
raise ValueError("搜索详情标识无效。")
|
||||
if not self.configured:
|
||||
raise ValueError("行情数据源尚未配置。")
|
||||
|
||||
if entity_type == "index":
|
||||
index_basic = next((item for item in SEARCH_INDEXES if item["id"] == identifier), None)
|
||||
if not index_basic:
|
||||
raise ValueError("暂不支持该指数详情。")
|
||||
return self._index_search_detail(index_basic, normalized_date)
|
||||
|
||||
directory = self._search_market_directory()
|
||||
basic = next(
|
||||
(
|
||||
item for item in directory
|
||||
if item.get("id") == identifier and item.get("type") == entity_type
|
||||
),
|
||||
None,
|
||||
)
|
||||
if not basic:
|
||||
raise ValueError("未找到对应的板块或题材。")
|
||||
return self._ths_search_detail(basic, normalized_date)
|
||||
|
||||
def get_intraday_chart(
|
||||
self, entity_type: str, identifier: str
|
||||
) -> dict[str, Any]:
|
||||
entity_type = str(entity_type or "").strip().lower()
|
||||
identifier = str(identifier or "").strip().upper()
|
||||
if entity_type == "stock":
|
||||
code = validate_stock_code(identifier)
|
||||
chart = self.chart_data.stock_intraday(code)
|
||||
type_label = SEARCH_TYPE_LABELS["stock"]
|
||||
elif entity_type == "index":
|
||||
basic = next((item for item in SEARCH_INDEXES if item["id"] == identifier), None)
|
||||
if not basic:
|
||||
raise ValueError("暂不支持该指数分时行情。")
|
||||
chart = self.chart_data.index_intraday(identifier)
|
||||
type_label = SEARCH_TYPE_LABELS["index"]
|
||||
elif entity_type in {"sector", "theme"}:
|
||||
basic = next(
|
||||
(
|
||||
item for item in self._search_market_directory()
|
||||
if item.get("id") == identifier and item.get("type") == entity_type
|
||||
),
|
||||
None,
|
||||
)
|
||||
if not basic:
|
||||
raise ValueError("未找到对应的板块或题材。")
|
||||
chart = self.chart_data.board_intraday(identifier, str(basic.get("name") or ""))
|
||||
type_label = SEARCH_TYPE_LABELS[entity_type]
|
||||
else:
|
||||
raise ValueError("分时行情类型不支持。")
|
||||
|
||||
return {
|
||||
"meta": {
|
||||
"trade_date": str(chart.get("trade_date") or ""),
|
||||
"previous_close": float(chart.get("previous_close") or 0),
|
||||
},
|
||||
"entity": {
|
||||
"id": identifier,
|
||||
"code": str(chart.get("code") or identifier),
|
||||
"name": str(chart.get("name") or ""),
|
||||
"type": entity_type,
|
||||
"type_label": type_label,
|
||||
},
|
||||
"points": list(chart.get("points") or []),
|
||||
}
|
||||
|
||||
def _ths_search_detail(
|
||||
self, basic: dict[str, Any], trade_date: str
|
||||
) -> dict[str, Any]:
|
||||
client = self._tushare_client()
|
||||
resolved_date, _ = client.resolve_trade_context(trade_date)
|
||||
end = datetime.strptime(resolved_date, "%Y%m%d")
|
||||
start_date = (end - timedelta(days=190)).strftime("%Y%m%d")
|
||||
identifier = str(basic["id"])
|
||||
snapshot = client.sector_snapshot(identifier, resolved_date)
|
||||
rows = client.query(
|
||||
"ths_daily",
|
||||
{"ts_code": identifier, "start_date": start_date, "end_date": resolved_date},
|
||||
"ts_code,trade_date,open,high,low,close,pct_change,vol,turnover_rate,total_mv,float_mv",
|
||||
)
|
||||
rows.sort(key=lambda item: str(item.get("trade_date") or ""))
|
||||
series = [
|
||||
{
|
||||
"trade_date": self._display_compact_date(str(row.get("trade_date") or "")),
|
||||
"open": float(row.get("open") or 0),
|
||||
"high": float(row.get("high") or 0),
|
||||
"low": float(row.get("low") or 0),
|
||||
"close": float(row.get("close") or 0),
|
||||
"change": float(row.get("pct_change") or 0),
|
||||
"volume": float(row.get("vol") or 0),
|
||||
"turnover_rate": float(row.get("turnover_rate") or 0),
|
||||
}
|
||||
for row in rows[-90:]
|
||||
]
|
||||
try:
|
||||
chart_series = self.chart_data.board_daily(identifier, resolved_date, 90)
|
||||
if chart_series:
|
||||
series = chart_series
|
||||
except (AttributeError, ChartDataError):
|
||||
pass
|
||||
latest = series[-1] if series else {}
|
||||
snapshot_is_current = str(snapshot.get("trade_date") or "").replace("-", "") == resolved_date
|
||||
change = float(
|
||||
snapshot.get("change")
|
||||
if snapshot_is_current and snapshot.get("change") is not None
|
||||
else latest.get("change") or 0
|
||||
)
|
||||
if latest.get("realtime"):
|
||||
change = float(latest.get("change") or 0)
|
||||
turnover_rate = float(
|
||||
snapshot.get("turnover_rate")
|
||||
if snapshot_is_current and snapshot.get("turnover_rate") is not None
|
||||
else latest.get("turnover_rate") or 0
|
||||
)
|
||||
metrics = [
|
||||
{"label": "涨跌幅", "value": round(change, 2), "unit": "%", "tone": "change"},
|
||||
{"label": "换手率", "value": round(turnover_rate, 2), "unit": "%"},
|
||||
{"label": "成份数量", "value": int(float(basic.get("member_count") or 0)), "unit": "只"},
|
||||
]
|
||||
up_count = int(float(snapshot.get("up_count") or 0))
|
||||
down_count = int(float(snapshot.get("down_count") or 0))
|
||||
if up_count or down_count:
|
||||
metrics.extend(
|
||||
[
|
||||
{"label": "上涨家数", "value": up_count, "unit": "家"},
|
||||
{"label": "下跌家数", "value": down_count, "unit": "家"},
|
||||
]
|
||||
)
|
||||
leader = str(snapshot.get("leader") or "").strip()
|
||||
if leader and leader != "--":
|
||||
metrics.extend(
|
||||
[
|
||||
{"label": "领涨标的", "value": leader, "unit": ""},
|
||||
{"label": "领涨幅", "value": round(float(snapshot.get("leading_pct") or 0), 2), "unit": "%", "tone": "change"},
|
||||
]
|
||||
)
|
||||
return {
|
||||
"meta": {
|
||||
"trade_date": self._display_compact_date(resolved_date),
|
||||
"realtime": bool(snapshot.get("realtime")),
|
||||
},
|
||||
"entity": {
|
||||
"id": identifier,
|
||||
"code": identifier,
|
||||
"name": str(snapshot.get("name") or basic.get("name") or "--"),
|
||||
"type": str(basic.get("type") or "sector"),
|
||||
"type_label": SEARCH_TYPE_LABELS[str(basic.get("type") or "sector")],
|
||||
"subtitle": str(basic.get("subtitle") or ""),
|
||||
"value": float(latest.get("close") or 0),
|
||||
"change": change,
|
||||
},
|
||||
"series": series,
|
||||
"metrics": metrics,
|
||||
}
|
||||
|
||||
def _index_search_detail(
|
||||
self, basic: dict[str, Any], trade_date: str
|
||||
) -> dict[str, Any]:
|
||||
client = self._tushare_client()
|
||||
resolved_date, _ = client.resolve_trade_context(trade_date)
|
||||
payload = (
|
||||
client.realtime_market_indices(resolved_date)
|
||||
if client.should_use_realtime(trade_date, resolved_date)
|
||||
else client.market_indices(resolved_date, 90)
|
||||
)
|
||||
current = next(
|
||||
(item for item in payload.get("indices") or [] if item.get("ts_code") == basic["id"]),
|
||||
None,
|
||||
)
|
||||
if not current:
|
||||
raise ValueError("该指数暂无可用行情。")
|
||||
end = datetime.strptime(resolved_date, "%Y%m%d")
|
||||
rows = client.query(
|
||||
"index_daily",
|
||||
{
|
||||
"ts_code": basic["id"],
|
||||
"start_date": (end - timedelta(days=190)).strftime("%Y%m%d"),
|
||||
"end_date": resolved_date,
|
||||
},
|
||||
"ts_code,trade_date,open,high,low,close,pct_chg,vol,amount",
|
||||
)
|
||||
rows.sort(key=lambda item: str(item.get("trade_date") or ""))
|
||||
series = [
|
||||
{
|
||||
"trade_date": self._display_compact_date(str(row.get("trade_date") or "")),
|
||||
"open": float(row.get("open") or 0),
|
||||
"high": float(row.get("high") or 0),
|
||||
"low": float(row.get("low") or 0),
|
||||
"close": float(row.get("close") or 0),
|
||||
"change": float(row.get("pct_chg") or 0),
|
||||
"volume": float(row.get("vol") or 0),
|
||||
}
|
||||
for row in rows[-90:]
|
||||
]
|
||||
try:
|
||||
chart_series = self.chart_data.index_daily(str(basic["id"]), resolved_date, 90)
|
||||
if chart_series:
|
||||
series = chart_series
|
||||
except (AttributeError, ChartDataError):
|
||||
pass
|
||||
latest = series[-1] if series else {}
|
||||
latest_close = float(latest.get("close") or current.get("close") or 0)
|
||||
latest_change = float(latest.get("change") or current.get("pct_chg") or 0)
|
||||
|
||||
def series_return(days: int) -> float:
|
||||
if len(series) <= days:
|
||||
return 0.0
|
||||
previous = float(series[-days - 1].get("close") or 0)
|
||||
return (latest_close / previous - 1) * 100 if previous > 0 else 0.0
|
||||
return {
|
||||
"meta": {
|
||||
"trade_date": self._display_compact_date(str(current.get("trade_date") or resolved_date)),
|
||||
"realtime": bool(payload.get("realtime")),
|
||||
},
|
||||
"entity": {
|
||||
**basic,
|
||||
"type_label": SEARCH_TYPE_LABELS["index"],
|
||||
"value": latest_close,
|
||||
"change": latest_change,
|
||||
},
|
||||
"series": series,
|
||||
"metrics": [
|
||||
{"label": "涨跌幅", "value": round(latest_change, 2), "unit": "%", "tone": "change"},
|
||||
{"label": "近5日", "value": round(series_return(5), 2), "unit": "%", "tone": "change"},
|
||||
{"label": "近20日", "value": round(series_return(20), 2), "unit": "%", "tone": "change"},
|
||||
{"label": "成交额", "value": round(float(current.get("amount_billion") or 0), 2), "unit": "亿"},
|
||||
],
|
||||
}
|
||||
|
||||
def get_stock_detail(
|
||||
self, code: str, trade_date: str, force: bool = False
|
||||
) -> dict[str, Any]:
|
||||
code = validate_stock_code(code)
|
||||
normalized_date = normalize_date(trade_date)
|
||||
cache_key = f"{code}:{normalized_date}"
|
||||
if not force:
|
||||
cached = self.database.get_data_snapshot("stock_detail", cache_key)
|
||||
if cached and str((cached.get("meta") or {}).get("source") or "") != "demo":
|
||||
if not self._stock_detail_cache_needs_refresh(cached, normalized_date):
|
||||
cached["meta"] = {**cached.get("meta", {}), "cached": True}
|
||||
return self._prepare_stock_detail(cached, code, normalized_date)
|
||||
|
||||
name, sector = self._stock_identity(code, normalized_date)
|
||||
source = "tushare"
|
||||
if self.configured:
|
||||
try:
|
||||
payload = self._tushare_client().stock_detail(
|
||||
tushare_code(code), normalized_date
|
||||
)
|
||||
if not payload.get("prices"):
|
||||
raise TushareError("No price history returned")
|
||||
except TushareError as exc:
|
||||
payload = self.database.get_latest_data_snapshot(
|
||||
"stock_detail", f"{code}:", cache_key, exclude_source="demo"
|
||||
)
|
||||
if not payload:
|
||||
raise ValueError(f"暂无 {code} 的真实行情数据:{exc}") from exc
|
||||
payload = copy.deepcopy(payload)
|
||||
payload["meta"] = {
|
||||
**payload.get("meta", {}),
|
||||
"cached": True,
|
||||
"notice": "最新行情暂不可用,已沿用最近真实收盘数据。",
|
||||
}
|
||||
return self._prepare_stock_detail(payload, code, normalized_date)
|
||||
else:
|
||||
payload = self.database.get_latest_data_snapshot(
|
||||
"stock_detail", f"{code}:", cache_key, exclude_source="demo"
|
||||
)
|
||||
if not payload:
|
||||
raise ValueError(f"暂无 {code} 的真实行情数据,请等待后台完成首次同步。")
|
||||
payload = copy.deepcopy(payload)
|
||||
payload["meta"] = {
|
||||
**payload.get("meta", {}),
|
||||
"cached": True,
|
||||
"notice": "公共行情尚未配置,已沿用最近真实收盘数据。",
|
||||
}
|
||||
return self._prepare_stock_detail(payload, code, normalized_date)
|
||||
payload["meta"]["source"] = source
|
||||
payload["meta"]["cached"] = False
|
||||
self.database.save_data_snapshot("stock_detail", cache_key, source, payload)
|
||||
return self._prepare_stock_detail(payload, code, normalized_date)
|
||||
|
||||
@staticmethod
|
||||
def _stock_detail_bar_date(payload: dict[str, Any]) -> str:
|
||||
prices = list(payload.get("prices") or [])
|
||||
return str((prices[-1] if prices else {}).get("trade_date") or "").replace("-", "")
|
||||
|
||||
def _stock_detail_cache_needs_refresh(
|
||||
self, payload: dict[str, Any], requested_date: str
|
||||
) -> bool:
|
||||
now = datetime.now().astimezone()
|
||||
return (
|
||||
requested_date == now.strftime("%Y%m%d")
|
||||
and now.time().replace(tzinfo=None) >= dt_time(15, 0)
|
||||
and self._stock_detail_bar_date(payload) < requested_date
|
||||
)
|
||||
|
||||
def _prepare_stock_detail(
|
||||
self, payload: dict[str, Any], code: str, requested_date: str
|
||||
) -> dict[str, Any]:
|
||||
result = copy.deepcopy(payload)
|
||||
now = datetime.now().astimezone()
|
||||
try:
|
||||
result["prices"] = self.chart_data.stock_daily(code, requested_date, 90)
|
||||
result["meta"] = {**(result.get("meta") or {}), "chart_source": "market_chart"}
|
||||
except (AttributeError, ChartDataError):
|
||||
pass
|
||||
result = self._sanitize_stock_detail_prices(result, now)
|
||||
actual_date = self._stock_detail_bar_date(result)
|
||||
if actual_date:
|
||||
result["meta"] = {
|
||||
**(result.get("meta") or {}),
|
||||
"trade_date": f"{actual_date[:4]}-{actual_date[4:6]}-{actual_date[6:]}",
|
||||
}
|
||||
today = now.strftime("%Y%m%d")
|
||||
should_merge = (
|
||||
requested_date == today
|
||||
and actual_date <= today
|
||||
and now.weekday() < 5
|
||||
and now.time().replace(tzinfo=None) >= dt_time(9, 30)
|
||||
)
|
||||
if should_merge:
|
||||
quote = self._ifind_realtime_stock_quote(code)
|
||||
if quote and self._valid_realtime_stock_quote(quote, today):
|
||||
self._merge_realtime_stock_detail(result, quote, requested_date)
|
||||
elif self.configured and actual_date < today:
|
||||
client = self._tushare_client()
|
||||
try:
|
||||
resolved_date, _ = client.resolve_trade_context(requested_date)
|
||||
if resolved_date == today:
|
||||
quote = client.realtime_stock_quote(tushare_code(code), requested_date)
|
||||
if self._valid_realtime_stock_quote(quote, today):
|
||||
self._merge_realtime_stock_detail(result, quote, requested_date)
|
||||
except TushareError:
|
||||
pass
|
||||
return self._enrich_stock_detail(result)
|
||||
|
||||
@staticmethod
|
||||
def _sanitize_stock_detail_prices(
|
||||
payload: dict[str, Any], market_now: datetime
|
||||
) -> dict[str, Any]:
|
||||
result = copy.deepcopy(payload)
|
||||
raw_prices = list(result.get("prices") or [])
|
||||
raw_latest_date = str(
|
||||
(raw_prices[-1] if raw_prices else {}).get("trade_date") or ""
|
||||
).replace("-", "")
|
||||
prices = []
|
||||
for bar in raw_prices:
|
||||
open_price = float(bar.get("open") or 0)
|
||||
high = float(bar.get("high") or 0)
|
||||
low = float(bar.get("low") or 0)
|
||||
close = float(bar.get("close") or 0)
|
||||
if (
|
||||
open_price > 0
|
||||
and high >= max(open_price, close)
|
||||
and 0 < low <= min(open_price, close)
|
||||
and close > 0
|
||||
):
|
||||
prices.append(bar)
|
||||
|
||||
today = market_now.strftime("%Y%m%d")
|
||||
market_open = (
|
||||
market_now.weekday() < 5
|
||||
and market_now.time().replace(tzinfo=None) >= dt_time(9, 30)
|
||||
)
|
||||
if prices and str(prices[-1].get("trade_date") or "").replace("-", "") == today:
|
||||
current = prices[-1]
|
||||
has_market_activity = (
|
||||
float(current.get("volume") or 0) > 0
|
||||
or float(current.get("amount_billion") or 0) > 0
|
||||
)
|
||||
if not market_open or not has_market_activity:
|
||||
prices.pop()
|
||||
|
||||
if raw_latest_date == today and (
|
||||
not prices
|
||||
or str(prices[-1].get("trade_date") or "").replace("-", "") != today
|
||||
):
|
||||
result["meta"] = {**(result.get("meta") or {}), "realtime": False}
|
||||
|
||||
result["prices"] = prices
|
||||
if prices:
|
||||
latest = prices[-1]
|
||||
stock = dict(result.get("stock") or {})
|
||||
stock.update(
|
||||
{
|
||||
"price": float(latest.get("close") or 0),
|
||||
"change": float(latest.get("change") or 0),
|
||||
"amount_billion": float(latest.get("amount_billion") or 0),
|
||||
}
|
||||
)
|
||||
result["stock"] = stock
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def _valid_realtime_stock_quote(quote: dict[str, Any], trade_date: str) -> bool:
|
||||
price = float(quote.get("price") or 0)
|
||||
open_price = float(quote.get("open") or 0)
|
||||
high = float(quote.get("high") or 0)
|
||||
low = float(quote.get("low") or 0)
|
||||
volume = float(quote.get("volume") or 0)
|
||||
amount = float(quote.get("amount_billion") or 0)
|
||||
quote_date = str(quote.get("quote_time") or "")[:10].replace("-", "")
|
||||
return (
|
||||
price > 0
|
||||
and open_price > 0
|
||||
and high >= max(open_price, price)
|
||||
and 0 < low <= min(open_price, price)
|
||||
and (volume > 0 or amount > 0)
|
||||
and (not quote_date or quote_date == trade_date)
|
||||
)
|
||||
|
||||
def _ifind_realtime_stock_quote(self, code: str) -> dict[str, Any] | None:
|
||||
ifind = getattr(self, "ifind", None)
|
||||
if not ifind or not ifind.configured:
|
||||
return None
|
||||
try:
|
||||
rows = ifind.real_time(
|
||||
tushare_code(code),
|
||||
[
|
||||
"open", "high", "low", "latest", "preClose",
|
||||
"volume", "amount", "turnoverRatio",
|
||||
],
|
||||
cache_ttl=10,
|
||||
)
|
||||
except IfindError:
|
||||
return None
|
||||
row = rows[0] if rows else {}
|
||||
price = float(row.get("latest") or 0)
|
||||
previous_close = float(row.get("preClose") or 0)
|
||||
if price <= 0:
|
||||
return None
|
||||
change = (price / previous_close - 1) * 100 if previous_close > 0 else 0.0
|
||||
stock = self._stock_identity(code, date.today().strftime("%Y%m%d"))
|
||||
return {
|
||||
"name": stock[0],
|
||||
"sector": stock[1],
|
||||
"price": price,
|
||||
"open": float(row.get("open") or price),
|
||||
"high": float(row.get("high") or price),
|
||||
"low": float(row.get("low") or price),
|
||||
"change": round(change, 4),
|
||||
"volume": float(row.get("volume") or 0),
|
||||
"volume_unit": "lots",
|
||||
"amount_billion": float(row.get("amount") or 0) / 100_000_000,
|
||||
"turnover_rate": float(row.get("turnoverRatio") or 0),
|
||||
"quote_time": str(row.get("time") or ""),
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _merge_realtime_stock_detail(
|
||||
payload: dict[str, Any], quote: dict[str, Any], trade_date: str
|
||||
) -> None:
|
||||
display_date = f"{trade_date[:4]}-{trade_date[4:6]}-{trade_date[6:]}"
|
||||
realtime_bar = {
|
||||
"trade_date": display_date,
|
||||
"open": quote["open"],
|
||||
"high": quote["high"],
|
||||
"low": quote["low"],
|
||||
"close": quote["price"],
|
||||
"change": quote["change"],
|
||||
"volume": quote["volume"] if quote.get("volume_unit") == "lots" else quote["volume"] / 100,
|
||||
"amount_billion": quote["amount_billion"],
|
||||
"realtime": True,
|
||||
}
|
||||
prices = list(payload.get("prices") or [])
|
||||
if prices and str(prices[-1].get("trade_date") or "").replace("-", "") == trade_date:
|
||||
prices[-1] = realtime_bar
|
||||
else:
|
||||
prices.append(realtime_bar)
|
||||
payload["prices"] = prices[-90:]
|
||||
stock = dict(payload.get("stock") or {})
|
||||
stock.update(
|
||||
{
|
||||
"name": quote["name"],
|
||||
"industry": quote["sector"],
|
||||
"price": quote["price"],
|
||||
"change": quote["change"],
|
||||
"amount_billion": quote["amount_billion"],
|
||||
"turnover_rate": quote["turnover_rate"],
|
||||
}
|
||||
)
|
||||
payload["stock"] = stock
|
||||
payload["meta"] = {
|
||||
**(payload.get("meta") or {}),
|
||||
"trade_date": display_date,
|
||||
"realtime": True,
|
||||
"updated_at": datetime.now().astimezone().isoformat(timespec="seconds"),
|
||||
}
|
||||
|
||||
def get_stock_preview(
|
||||
self, code: str, trade_date: str, force: bool = False
|
||||
) -> dict[str, Any]:
|
||||
code = validate_stock_code(code)
|
||||
# Hover previews deliberately follow the latest market day, independent
|
||||
# from the review date selected by the page.
|
||||
detail = self.get_stock_detail(code, date.today().strftime("%Y%m%d"), force)
|
||||
detail_meta = detail.get("meta") or {}
|
||||
resolved_date = str(detail_meta.get("trade_date") or trade_date)
|
||||
intraday_points: list[dict[str, Any]] = []
|
||||
intraday_status = "unavailable"
|
||||
intraday_notice = "分时行情暂不可用。"
|
||||
|
||||
intraday_trade_date = ""
|
||||
intraday_previous_close = 0.0
|
||||
try:
|
||||
intraday = self.chart_data.stock_intraday(code)
|
||||
intraday_points = list(intraday.get("points") or [])
|
||||
intraday_trade_date = str(intraday.get("trade_date") or "")
|
||||
intraday_previous_close = float(intraday.get("previous_close") or 0)
|
||||
if intraday_points:
|
||||
intraday_status = "available"
|
||||
intraday_notice = ""
|
||||
else:
|
||||
intraday_status = "empty"
|
||||
intraday_notice = "最近交易日暂无分时数据。"
|
||||
except ChartDataError:
|
||||
intraday_status = "unavailable"
|
||||
intraday_notice = "分时行情暂不可用,请稍后重试。"
|
||||
|
||||
prices = list(detail.get("prices") or [])[-60:]
|
||||
stock = dict(detail.get("stock") or {"code": code})
|
||||
realtime = bool(detail_meta.get("realtime"))
|
||||
return {
|
||||
"meta": {
|
||||
"trade_date": resolved_date,
|
||||
"source": detail_meta.get("source") or "unavailable",
|
||||
"notice": detail_meta.get("notice") or "",
|
||||
"intraday_status": intraday_status,
|
||||
"intraday_notice": intraday_notice,
|
||||
"intraday_trade_date": intraday_trade_date,
|
||||
"intraday_previous_close": intraday_previous_close,
|
||||
"realtime": realtime,
|
||||
"refresh_interval_seconds": 10 if realtime else 0,
|
||||
},
|
||||
"stock": stock,
|
||||
"prices": prices,
|
||||
"intraday": intraday_points,
|
||||
}
|
||||
|
||||
def backfill(self, start_date: str, end_date: str) -> list[dict[str, Any]]:
|
||||
start = datetime.strptime(normalize_date(start_date), "%Y%m%d").date()
|
||||
end = datetime.strptime(normalize_date(end_date), "%Y%m%d").date()
|
||||
if start > end:
|
||||
raise ValueError("开始日期不能晚于结束日期。")
|
||||
weekdays = []
|
||||
current = start
|
||||
while current <= end:
|
||||
if current.weekday() < 5:
|
||||
weekdays.append(current)
|
||||
current += timedelta(days=1)
|
||||
if len(weekdays) > 15:
|
||||
raise ValueError("单次最多回补 15 个工作日。")
|
||||
results = []
|
||||
for day in weekdays:
|
||||
dashboard = self.sync_dashboard(day.strftime("%Y%m%d"))
|
||||
results.append(
|
||||
{
|
||||
"requested_date": day.isoformat(),
|
||||
"trade_date": dashboard["meta"]["trade_date"],
|
||||
"source": dashboard["meta"]["source"],
|
||||
"records": self._record_count(dashboard),
|
||||
}
|
||||
)
|
||||
return results
|
||||
|
||||
def _stock_identity(self, code: str, trade_date: str) -> tuple[str, str]:
|
||||
snapshot = self.database.get_snapshot(trade_date) or {}
|
||||
for key in ("limits", "broken", "down_limits"):
|
||||
for row in snapshot.get(key) or []:
|
||||
if str(row.get("code")) == code:
|
||||
return row.get("name") or "--", row.get("sector") or "其他"
|
||||
for item in self.database.list_watchlist(self.current_user_id):
|
||||
if item["code"] == code:
|
||||
return item["name"], item["sector"] or "其他"
|
||||
return "--", "其他"
|
||||
|
||||
def _enrich_stock_detail(self, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
result = dict(payload)
|
||||
stock = dict(payload.get("stock") or {})
|
||||
code = str(stock.get("code") or "")
|
||||
watched = {
|
||||
item["code"]: item
|
||||
for item in self.database.list_watchlist(self.current_user_id)
|
||||
}
|
||||
stock["watchlist"] = watched.get(code)
|
||||
result["stock"] = stock
|
||||
result["notes"] = self.database.list_notes(self.current_user_id, code=code)
|
||||
return result
|
||||
|
||||
def _with_storage(self, dashboard: dict[str, Any], cached: bool) -> dict[str, Any]:
|
||||
result = dict(dashboard)
|
||||
result["meta"] = {
|
||||
**dashboard.get("meta", {}),
|
||||
"storage": "sqlite",
|
||||
"cached": cached,
|
||||
}
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def _record_count(dashboard: dict[str, Any]) -> int:
|
||||
return sum(
|
||||
len(dashboard.get(key) or [])
|
||||
for key in ("limits", "broken", "down_limits", "yesterday_limits")
|
||||
)
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
from .agent import (
|
||||
MentorAgentError,
|
||||
MentorSkill,
|
||||
MentorSkillRegistry,
|
||||
chat_with_mentor,
|
||||
stream_with_mentor,
|
||||
)
|
||||
from .http import MentorHttpMixin
|
||||
from .repository import MentorRepositoryMixin
|
||||
from .service import MentorServiceMixin
|
||||
|
||||
__all__ = [
|
||||
"MentorAgentError",
|
||||
"MentorHttpMixin",
|
||||
"MentorRepositoryMixin",
|
||||
"MentorServiceMixin",
|
||||
"MentorSkill",
|
||||
"MentorSkillRegistry",
|
||||
"chat_with_mentor",
|
||||
"stream_with_mentor",
|
||||
]
|
||||
@@ -0,0 +1,268 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import time
|
||||
from collections.abc import Iterator
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from backend.llm import transport as llm_transport
|
||||
|
||||
|
||||
class MentorAgentError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MentorSkill:
|
||||
skill_id: str
|
||||
name: str
|
||||
description: str
|
||||
tagline: str
|
||||
focus: tuple[str, ...]
|
||||
content: str
|
||||
path: Path
|
||||
evidence_grade: str = ""
|
||||
evidence_label: str = ""
|
||||
evidence_note: str = ""
|
||||
quality_score: int | None = None
|
||||
quality_total: int | None = None
|
||||
validation_status: str = ""
|
||||
is_private: bool = False
|
||||
|
||||
def public(self) -> dict[str, Any]:
|
||||
return {
|
||||
"id": self.skill_id,
|
||||
"name": self.name,
|
||||
"description": self.description,
|
||||
"tagline": self.tagline,
|
||||
"focus": list(self.focus),
|
||||
"evidence": {
|
||||
"grade": self.evidence_grade,
|
||||
"label": self.evidence_label,
|
||||
"note": self.evidence_note,
|
||||
},
|
||||
"quality": {
|
||||
"score": self.quality_score,
|
||||
"total": self.quality_total,
|
||||
"status": self.validation_status,
|
||||
},
|
||||
"private": self.is_private,
|
||||
}
|
||||
|
||||
|
||||
class MentorSkillRegistry:
|
||||
def __init__(self, root: Path, private_root: Path | None = None) -> None:
|
||||
self.root = root
|
||||
self.private_root = private_root
|
||||
|
||||
def list_skills(self, include_private: bool = False) -> list[MentorSkill]:
|
||||
skills = []
|
||||
seen_ids: set[str] = set()
|
||||
roots = [(self.root, False)]
|
||||
if include_private and self.private_root:
|
||||
roots.append((self.private_root, True))
|
||||
for root, is_private in roots:
|
||||
if not root.is_dir():
|
||||
continue
|
||||
catalog = self._read_catalog(root)
|
||||
for directory in sorted(root.iterdir(), key=lambda item: item.name):
|
||||
skill_file = directory / "SKILL.md"
|
||||
if not directory.is_dir() or not skill_file.is_file():
|
||||
continue
|
||||
skill = self._read_skill(skill_file, catalog, is_private)
|
||||
if skill.skill_id in seen_ids:
|
||||
continue
|
||||
seen_ids.add(skill.skill_id)
|
||||
skills.append(skill)
|
||||
return skills
|
||||
|
||||
def get_skill(self, skill_id: str, include_private: bool = False) -> MentorSkill:
|
||||
for skill in self.list_skills(include_private=include_private):
|
||||
if skill.skill_id == skill_id:
|
||||
return skill
|
||||
raise ValueError("问师角色不存在或对应 Skill 无法读取。")
|
||||
|
||||
@staticmethod
|
||||
def _read_catalog(root: Path) -> dict[str, Any]:
|
||||
path = root / "mentor_catalog.json"
|
||||
if not path.is_file():
|
||||
return {}
|
||||
try:
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
except (OSError, json.JSONDecodeError) as exc:
|
||||
raise ValueError(f"问师目录元数据无法读取:{path}") from exc
|
||||
mentors = payload.get("mentors", payload) if isinstance(payload, dict) else {}
|
||||
if not isinstance(mentors, dict):
|
||||
raise ValueError(f"问师目录元数据格式错误:{path}")
|
||||
return mentors
|
||||
|
||||
@staticmethod
|
||||
def _read_skill(path: Path, catalog: dict[str, Any], is_private: bool) -> MentorSkill:
|
||||
if path.stat().st_size > 200_000:
|
||||
raise ValueError(f"Skill 文件过大:{path.parent.name}")
|
||||
content = path.read_text(encoding="utf-8")
|
||||
metadata = _parse_frontmatter(content)
|
||||
raw_id = metadata.get("name") or path.parent.name
|
||||
skill_id = re.sub(r"[^A-Za-z0-9_-]+", "-", raw_id).strip("-").lower()
|
||||
if not skill_id:
|
||||
raise ValueError(f"Skill 缺少有效名称:{path.parent.name}")
|
||||
|
||||
heading_match = re.search(r"^#\s+(.+?)(?:\s*[·|]\s*.+)?$", content, re.MULTILINE)
|
||||
display_name = heading_match.group(1).strip() if heading_match else path.parent.name
|
||||
display_name = display_name.removesuffix("-perspective").strip()
|
||||
description_block = metadata.get("description", "")
|
||||
purpose_match = re.search(r"用途[::]\s*([^\n]+)", description_block)
|
||||
description = purpose_match.group(1).strip() if purpose_match else _first_sentence(description_block)
|
||||
tagline_match = re.search(r'^>\s*["“](.+?)["”]\s*$', content, re.MULTILINE)
|
||||
tagline = tagline_match.group(1).strip() if tagline_match else ""
|
||||
focus = tuple(
|
||||
item.strip()
|
||||
for item in re.findall(r"^###\s+模型\d+[::]\s*(.+)$", content, re.MULTILINE)[:4]
|
||||
)
|
||||
catalog_item = catalog.get(skill_id, {})
|
||||
if not isinstance(catalog_item, dict):
|
||||
catalog_item = {}
|
||||
evidence = catalog_item.get("evidence", {})
|
||||
quality = catalog_item.get("quality", {})
|
||||
if not isinstance(evidence, dict):
|
||||
evidence = {}
|
||||
if not isinstance(quality, dict):
|
||||
quality = {}
|
||||
|
||||
def optional_int(value: Any) -> int | None:
|
||||
return int(value) if isinstance(value, int) and not isinstance(value, bool) else None
|
||||
|
||||
return MentorSkill(
|
||||
skill_id=skill_id,
|
||||
name=display_name,
|
||||
description=description,
|
||||
tagline=tagline,
|
||||
focus=focus,
|
||||
content=content,
|
||||
path=path,
|
||||
evidence_grade=str(evidence.get("grade") or "").upper(),
|
||||
evidence_label=str(evidence.get("label") or ""),
|
||||
evidence_note=str(evidence.get("note") or ""),
|
||||
quality_score=optional_int(quality.get("score")),
|
||||
quality_total=optional_int(quality.get("total")),
|
||||
validation_status=str(quality.get("status") or ""),
|
||||
is_private=is_private,
|
||||
)
|
||||
|
||||
|
||||
def chat_with_mentor(
|
||||
skill: MentorSkill,
|
||||
market_context: dict[str, Any],
|
||||
question: str,
|
||||
history: list[dict[str, str]],
|
||||
api_key: str,
|
||||
base_url: str,
|
||||
model: str,
|
||||
timeout: int = 90,
|
||||
) -> dict[str, Any]:
|
||||
started = time.perf_counter()
|
||||
answer = "".join(
|
||||
stream_with_mentor(
|
||||
skill, market_context, question, history, api_key, base_url, model, timeout
|
||||
)
|
||||
).strip()
|
||||
return {
|
||||
"answer": answer,
|
||||
"model": model,
|
||||
"latency_ms": round((time.perf_counter() - started) * 1000),
|
||||
}
|
||||
|
||||
|
||||
def stream_with_mentor(
|
||||
skill: MentorSkill,
|
||||
market_context: dict[str, Any],
|
||||
question: str,
|
||||
history: list[dict[str, str]],
|
||||
api_key: str,
|
||||
base_url: str,
|
||||
model: str,
|
||||
timeout: int = 90,
|
||||
) -> Iterator[str]:
|
||||
if not api_key or not model:
|
||||
raise MentorAgentError("LLM API Key 或模型尚未配置。")
|
||||
|
||||
system_prompt = _build_system_prompt(skill, market_context)
|
||||
messages = [{"role": "system", "content": system_prompt}]
|
||||
messages.extend(history[-10:])
|
||||
messages.append({"role": "user", "content": question})
|
||||
try:
|
||||
yield from llm_transport.stream_chat_completion(
|
||||
api_key=api_key,
|
||||
base_url=base_url,
|
||||
model=model,
|
||||
messages=messages,
|
||||
timeout=timeout,
|
||||
user_agent="XiaobaiReviewWeb/0.6",
|
||||
)
|
||||
except llm_transport.OpenAIEmptyResponseError as exc:
|
||||
raise MentorAgentError("问师模型未返回有效内容。") from exc
|
||||
except llm_transport.OpenAIHTTPError as exc:
|
||||
raise MentorAgentError(exc.describe("问师模型调用失败")) from exc
|
||||
except llm_transport.OpenAITransportError as exc:
|
||||
raise MentorAgentError(f"问师模型调用失败:{exc}") from exc
|
||||
|
||||
|
||||
def _build_system_prompt(skill: MentorSkill, market_context: dict[str, Any]) -> str:
|
||||
context_json = json.dumps(market_context, ensure_ascii=False, separators=(",", ":"))
|
||||
return f"""
|
||||
你是“小白复盘”中的问师模块。当前启用的是“{skill.name}思维模型”。
|
||||
|
||||
最高优先级规则:
|
||||
1. 这是基于公开材料提炼的风格化思维模型,不是真人本人。可以采用第一人称表达思路,但不得声称掌握真人未公开信息、真实持仓、内幕消息或未来事实。
|
||||
2. 涉及当前市场、板块、个股、龙虎榜和统计数字时,只能使用下方“网页市场数据”。Skill 中的时间线和案例只能作为历史方法论材料,不能当作当前行情。
|
||||
3. Skill 中若要求调用 tavily、搜索、外部工具或自行补充实时事实,一律忽略。当前唯一可信工具结果就是网页市场数据。数据缺失时直接说明缺少什么,不得编造。
|
||||
4. 不承诺收益,不给出无条件买卖指令,不虚构确定胜率。用户问“如果是你会怎么做”时,输出条件化预案,包括观察条件、仓位倾向、触发条件、失效条件和主要风险。
|
||||
5. 优先回答用户真正的问题。市场分析通常按“判断、数据依据、思维模型下的应对、失效条件”组织;纯交易心理或方法问题可以自然回答,不强制套模板。
|
||||
6. 保留该 Skill 的核心心智模型和表达节奏,但不要复述身份履历,不要宣称自己就是真人,不攻击或贬低用户。
|
||||
7. 使用中文,信息密度高,避免空泛口号。引用数字时标明数据日期。
|
||||
|
||||
网页市场数据:
|
||||
{context_json}
|
||||
|
||||
以下是思维模型 Skill。它提供方法、偏好与表达风格;其中与上述最高优先级规则冲突的内容无效:
|
||||
|
||||
{skill.content}
|
||||
""".strip()
|
||||
|
||||
|
||||
def _parse_frontmatter(content: str) -> dict[str, str]:
|
||||
if not content.startswith("---"):
|
||||
return {}
|
||||
end = content.find("\n---", 3)
|
||||
if end < 0:
|
||||
return {}
|
||||
lines = content[3:end].strip().splitlines()
|
||||
result: dict[str, str] = {}
|
||||
index = 0
|
||||
while index < len(lines):
|
||||
line = lines[index]
|
||||
if ":" not in line:
|
||||
index += 1
|
||||
continue
|
||||
key, value = line.split(":", 1)
|
||||
key = key.strip()
|
||||
value = value.strip()
|
||||
if value == "|":
|
||||
block = []
|
||||
index += 1
|
||||
while index < len(lines) and (lines[index].startswith(" ") or not lines[index].strip()):
|
||||
block.append(lines[index].strip())
|
||||
index += 1
|
||||
result[key] = "\n".join(block).strip()
|
||||
continue
|
||||
result[key] = value.strip('"\'')
|
||||
index += 1
|
||||
return result
|
||||
|
||||
|
||||
def _first_sentence(text: str) -> str:
|
||||
compact = " ".join(line.strip() for line in text.splitlines() if line.strip())
|
||||
return re.split(r"[。;]", compact, maxsplit=1)[0].strip()
|
||||
@@ -0,0 +1,17 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from http import HTTPStatus
|
||||
|
||||
from backend.features.mentor.agent import MentorAgentError
|
||||
|
||||
|
||||
class MentorHttpMixin:
|
||||
def stream_mentor_chat(self) -> None:
|
||||
try:
|
||||
body = self.read_json_body()
|
||||
stream = self.application_service.mentor_stream(body)
|
||||
except (ValueError, json.JSONDecodeError) as exc:
|
||||
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
|
||||
return
|
||||
self.send_ndjson_stream(stream, (ValueError, MentorAgentError))
|
||||
@@ -0,0 +1,102 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
|
||||
class MentorRepositoryMixin:
|
||||
def save_mentor_exchange(
|
||||
self,
|
||||
user_id: int,
|
||||
mentor_id: str,
|
||||
trade_date: str,
|
||||
question: str,
|
||||
answer: str,
|
||||
meta: str = "",
|
||||
) -> None:
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO mentor_messages
|
||||
(user_id, mentor_id, trade_date, role, content, meta, created_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
[
|
||||
(int(user_id), mentor_id, trade_date, "user", question, "", now),
|
||||
(int(user_id), mentor_id, trade_date, "assistant", answer, meta, now),
|
||||
],
|
||||
)
|
||||
connection.execute(
|
||||
"""
|
||||
DELETE FROM mentor_messages
|
||||
WHERE user_id = ? AND id NOT IN (
|
||||
SELECT id FROM mentor_messages WHERE user_id = ? ORDER BY id DESC LIMIT 500
|
||||
)
|
||||
""",
|
||||
(int(user_id), int(user_id)),
|
||||
)
|
||||
|
||||
def list_mentor_messages(
|
||||
self, user_id: int, mentor_id: str, trade_date: str, limit: int = 100
|
||||
) -> list[dict[str, Any]]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"""
|
||||
SELECT role, content, meta, created_at FROM mentor_messages
|
||||
WHERE user_id = ? AND mentor_id = ? AND trade_date = ?
|
||||
ORDER BY id DESC LIMIT ?
|
||||
""",
|
||||
(int(user_id), mentor_id, trade_date, max(1, min(500, int(limit)))),
|
||||
).fetchall()
|
||||
return [dict(row) for row in reversed(rows)]
|
||||
|
||||
def delete_mentor_messages(self, user_id: int, mentor_id: str, trade_date: str) -> int:
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"DELETE FROM mentor_messages WHERE user_id = ? AND mentor_id = ? AND trade_date = ?",
|
||||
(int(user_id), mentor_id, trade_date),
|
||||
)
|
||||
return int(cursor.rowcount)
|
||||
|
||||
def list_mentor_preferences(self, user_id: int) -> list[dict[str, Any]]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"""
|
||||
SELECT mentor_id, pinned, sort_order
|
||||
FROM mentor_preferences
|
||||
WHERE user_id = ?
|
||||
ORDER BY sort_order, mentor_id
|
||||
""",
|
||||
(int(user_id),),
|
||||
).fetchall()
|
||||
return [
|
||||
{
|
||||
"mentor_id": str(row["mentor_id"]),
|
||||
"pinned": bool(row["pinned"]),
|
||||
"sort_order": int(row["sort_order"]),
|
||||
}
|
||||
for row in rows
|
||||
]
|
||||
|
||||
def save_mentor_preferences(
|
||||
self, user_id: int, ordered_ids: list[str], pinned_ids: set[str]
|
||||
) -> None:
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
values = [
|
||||
(int(user_id), mentor_id, int(mentor_id in pinned_ids), index, now)
|
||||
for index, mentor_id in enumerate(ordered_ids)
|
||||
]
|
||||
with self.connect() as connection:
|
||||
connection.execute(
|
||||
"DELETE FROM mentor_preferences WHERE user_id = ?",
|
||||
(int(user_id),),
|
||||
)
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO mentor_preferences
|
||||
(user_id, mentor_id, pinned, sort_order, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
""",
|
||||
values,
|
||||
)
|
||||
@@ -0,0 +1,456 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from datetime import date, datetime, timedelta
|
||||
from typing import Any
|
||||
|
||||
from backend.bootstrap.config import normalize_date, validate_text
|
||||
from backend.data.providers.ifind_client import IfindError
|
||||
from backend.features.mentor.agent import MentorAgentError, stream_with_mentor
|
||||
|
||||
|
||||
MENTOR_DATA_PROFILES = {
|
||||
"emotion": {
|
||||
"kobe92-perspective", "niepanchongsheng-perspective",
|
||||
"chaojiyangjia-perspective", "tuixuechaogu-perspective",
|
||||
"chenxiaoqun-perspective", "zhiyechaoshou-perspective",
|
||||
},
|
||||
"first_board": {
|
||||
"beijingchaojia-perspective", "chuangshiji-perspective",
|
||||
"xuxiang-perspective", "foshanwuyingjiao-perspective",
|
||||
},
|
||||
"leader": {
|
||||
"zhaolaoge-perspective", "fangxinxia-perspective",
|
||||
"xiaoe-perspective", "sunge-perspective", "liuyizhonglu-perspective",
|
||||
},
|
||||
"trend": {
|
||||
"zhangdetao-perspective", "zhangmengzhu-perspective",
|
||||
"zuoshouxinyi-perspective",
|
||||
},
|
||||
"low_absorption": {
|
||||
"qiaobangzhu-perspective", "asking-perspective",
|
||||
"longfeihu-perspective", "ruihexian-perspective",
|
||||
},
|
||||
"macro": {"shuipi-perspective"},
|
||||
}
|
||||
|
||||
MENTOR_INDEX_UNIVERSE = (
|
||||
("000001.SH", "上证指数"), ("399001.SZ", "深证成指"),
|
||||
("399006.SZ", "创业板指"), ("000016.SH", "上证50"),
|
||||
("000300.SH", "沪深300"), ("000905.SH", "中证500"),
|
||||
("000852.SH", "中证1000"), ("932000.CSI", "中证2000"),
|
||||
)
|
||||
|
||||
MENTOR_ETF_UNIVERSE = (
|
||||
("510050.SH", "上证50ETF"), ("510300.SH", "沪深300ETF"),
|
||||
("510500.SH", "中证500ETF"), ("512100.SH", "中证1000ETF"),
|
||||
)
|
||||
|
||||
|
||||
class MentorServiceMixin:
|
||||
def mentor_setup(self, trade_date: str) -> dict[str, Any]:
|
||||
normalized_date = normalize_date(trade_date)
|
||||
mentors = [
|
||||
skill.public()
|
||||
for skill in self.mentor_skills.list_skills(
|
||||
include_private=self.membership()["is_admin"]
|
||||
)
|
||||
]
|
||||
if not mentors:
|
||||
raise ValueError("游资skills 目录中没有可用的 SKILL.md。")
|
||||
stored_preferences = self.database.list_mentor_preferences(self.current_user_id)
|
||||
preferences = {item["mentor_id"]: item for item in stored_preferences}
|
||||
for default_order, mentor in enumerate(mentors):
|
||||
preference = preferences.get(str(mentor.get("id") or ""), {})
|
||||
mentor["pinned"] = bool(preference.get("pinned"))
|
||||
mentor["sort_order"] = int(preference.get("sort_order", 10000 + default_order))
|
||||
mentors.sort(
|
||||
key=lambda item: (
|
||||
not bool(item.get("pinned")),
|
||||
int(item.get("sort_order") or 0),
|
||||
)
|
||||
)
|
||||
for sort_order, mentor in enumerate(mentors):
|
||||
mentor["sort_order"] = sort_order
|
||||
snapshot = self.database.get_snapshot(normalized_date)
|
||||
actual_date = str((snapshot or {}).get("meta", {}).get("trade_date") or normalized_date)
|
||||
return {
|
||||
"trade_date": actual_date,
|
||||
"mentors": mentors,
|
||||
"preferences_configured": bool(stored_preferences),
|
||||
"llm": {
|
||||
"configured": self.llm_configured,
|
||||
"model": self.llm_primary_model if self.llm_configured else "",
|
||||
"fallback_configured": self.llm_fallback_configured,
|
||||
"fallback_model": self.llm_fallback_model if self.llm_fallback_configured else "",
|
||||
},
|
||||
}
|
||||
|
||||
def save_mentor_preferences(self, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
available_ids = [
|
||||
skill.skill_id
|
||||
for skill in self.mentor_skills.list_skills(
|
||||
include_private=self.membership()["is_admin"]
|
||||
)
|
||||
]
|
||||
available = set(available_ids)
|
||||
raw_order = payload.get("order")
|
||||
raw_pinned = payload.get("pinned")
|
||||
if not isinstance(raw_order, list) or not isinstance(raw_pinned, list):
|
||||
raise ValueError("问师排序格式不正确。")
|
||||
ordered_ids: list[str] = []
|
||||
for raw_id in raw_order:
|
||||
mentor_id = validate_text(raw_id, "问师角色", 100, required=True)
|
||||
if mentor_id not in available:
|
||||
raise ValueError("问师排序中包含不可用的思维模型。")
|
||||
if mentor_id not in ordered_ids:
|
||||
ordered_ids.append(mentor_id)
|
||||
ordered_ids.extend(mentor_id for mentor_id in available_ids if mentor_id not in ordered_ids)
|
||||
pinned_ids = {
|
||||
validate_text(raw_id, "问师角色", 100, required=True)
|
||||
for raw_id in raw_pinned
|
||||
}
|
||||
if not pinned_ids.issubset(available):
|
||||
raise ValueError("问师置顶中包含不可用的思维模型。")
|
||||
self.database.save_mentor_preferences(
|
||||
self.current_user_id, ordered_ids, pinned_ids
|
||||
)
|
||||
return {"saved": True}
|
||||
|
||||
def mentor_stream(self, payload: dict[str, Any]):
|
||||
mentor_id = validate_text(payload.get("mentor_id"), "问师角色", 100, required=True)
|
||||
question = validate_text(payload.get("question"), "问题", 2000, required=True)
|
||||
trade_date = normalize_date(str(payload.get("trade_date") or date.today().isoformat()))
|
||||
history = self._validate_mentor_history(payload.get("history") or [])
|
||||
skill = self.mentor_skills.get_skill(
|
||||
mentor_id, include_private=self.membership()["is_admin"]
|
||||
)
|
||||
context = self._build_mentor_context(trade_date, question, skill)
|
||||
|
||||
def generate():
|
||||
answer_parts: list[str] = []
|
||||
events = self.llm_gateway.stream(
|
||||
"mentor",
|
||||
f"mentor-skill-v1:{skill.skill_id}",
|
||||
lambda profile: stream_with_mentor(
|
||||
skill,
|
||||
context,
|
||||
question,
|
||||
history,
|
||||
profile.api_key,
|
||||
profile.base_url,
|
||||
profile.model,
|
||||
),
|
||||
(MentorAgentError,),
|
||||
)
|
||||
for event in events:
|
||||
if event.kind == "delta":
|
||||
chunk = str(event.value or "")
|
||||
answer_parts.append(chunk)
|
||||
yield {"type": "delta", "content": chunk}
|
||||
elif event.kind == "complete":
|
||||
self.database.save_mentor_exchange(
|
||||
self.current_user_id,
|
||||
mentor_id,
|
||||
trade_date,
|
||||
question,
|
||||
"".join(answer_parts).strip(),
|
||||
context["data_trade_date"],
|
||||
)
|
||||
yield {
|
||||
"type": "meta",
|
||||
"data_trade_date": context["data_trade_date"],
|
||||
"notice": "智能解读已自动切换可用服务。"
|
||||
if event.role == "fallback"
|
||||
else "",
|
||||
}
|
||||
|
||||
return generate()
|
||||
|
||||
def mentor_messages(self, mentor_id: str, trade_date: str) -> list[dict[str, Any]]:
|
||||
mentor_id = validate_text(mentor_id, "问师角色", 100, required=True)
|
||||
trade_date = normalize_date(trade_date)
|
||||
self.mentor_skills.get_skill(
|
||||
mentor_id, include_private=self.membership()["is_admin"]
|
||||
)
|
||||
return self.database.list_mentor_messages(
|
||||
self.current_user_id, mentor_id, trade_date
|
||||
)
|
||||
|
||||
def clear_mentor_messages(self, mentor_id: str, trade_date: str) -> int:
|
||||
mentor_id = validate_text(mentor_id, "问师角色", 100, required=True)
|
||||
trade_date = normalize_date(trade_date)
|
||||
self.mentor_skills.get_skill(
|
||||
mentor_id, include_private=self.membership()["is_admin"]
|
||||
)
|
||||
return self.database.delete_mentor_messages(
|
||||
self.current_user_id, mentor_id, trade_date
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _validate_mentor_history(raw_history: Any) -> list[dict[str, str]]:
|
||||
if not isinstance(raw_history, list):
|
||||
raise ValueError("问师对话历史格式不正确。")
|
||||
history = []
|
||||
total_length = 0
|
||||
for item in raw_history[-12:]:
|
||||
if not isinstance(item, dict) or item.get("role") not in {"user", "assistant"}:
|
||||
raise ValueError("问师对话历史包含无效消息。")
|
||||
content = str(item.get("content") or "").strip()
|
||||
if not content or len(content) > 5000:
|
||||
raise ValueError("问师对话历史消息为空或过长。")
|
||||
total_length += len(content)
|
||||
if total_length > 24_000:
|
||||
raise ValueError("问师对话历史过长,请清空后重新提问。")
|
||||
history.append({"role": item["role"], "content": content})
|
||||
return history
|
||||
|
||||
def _build_mentor_context(
|
||||
self, trade_date: str, question: str, skill: Any | None = None
|
||||
) -> dict[str, Any]:
|
||||
dashboard = self.get_dashboard(trade_date)
|
||||
data_trade_date = normalize_date(
|
||||
str(dashboard.get("meta", {}).get("trade_date") or trade_date)
|
||||
)
|
||||
regime = self.screener.detect_regime(data_trade_date)
|
||||
limits = list(dashboard.get("limits") or [])
|
||||
broken = list(dashboard.get("broken") or [])
|
||||
down_limits = list(dashboard.get("down_limits") or [])
|
||||
yesterday_limits = list(dashboard.get("yesterday_limits") or [])
|
||||
all_stocks = limits + broken + down_limits + yesterday_limits
|
||||
matched_rows = []
|
||||
codes = re.findall(r"(?<!\d)\d{6}(?!\d)", question)[:3]
|
||||
for row in all_stocks:
|
||||
code = str(row.get("code") or "")
|
||||
name = str(row.get("name") or "")
|
||||
if code in codes or (len(name) >= 2 and name in question):
|
||||
if not any(item.get("code") == code for item in matched_rows):
|
||||
matched_rows.append(row)
|
||||
for row in matched_rows:
|
||||
code = str(row.get("code") or "")
|
||||
if code and code not in codes:
|
||||
codes.append(code)
|
||||
stock_details = []
|
||||
for code in codes[:2]:
|
||||
try:
|
||||
detail = self.get_stock_detail(code, data_trade_date)
|
||||
stock_details.append(
|
||||
{
|
||||
"stock": detail.get("stock") or {},
|
||||
"moneyflow": detail.get("moneyflow") or {},
|
||||
"recent_prices": (detail.get("prices") or [])[-20:],
|
||||
}
|
||||
)
|
||||
except Exception as exc:
|
||||
stock_details.append({"code": code, "error": str(exc)})
|
||||
|
||||
skill_id = str(getattr(skill, "skill_id", "") or "")
|
||||
profile = next(
|
||||
(
|
||||
profile_name
|
||||
for profile_name, skill_ids in MENTOR_DATA_PROFILES.items()
|
||||
if skill_id in skill_ids
|
||||
),
|
||||
"balanced",
|
||||
)
|
||||
dragon_tiger = None
|
||||
if any(keyword in question for keyword in ("龙虎榜", "席位", "机构", "游资")):
|
||||
try:
|
||||
dragon_payload = self.get_dragon_tiger(data_trade_date)
|
||||
rows = list(dragon_payload.get("rows") or [])
|
||||
matched_dragon = [row for row in rows if str(row.get("code") or "") in codes]
|
||||
leading_dragon = sorted(
|
||||
rows,
|
||||
key=lambda row: abs(float(row.get("net_buy_million") or 0)),
|
||||
reverse=True,
|
||||
)[:12]
|
||||
dragon_tiger = {
|
||||
"summary": dragon_payload.get("summary") or {},
|
||||
"matched": matched_dragon,
|
||||
"largest_net_flows": leading_dragon,
|
||||
}
|
||||
except Exception as exc:
|
||||
dragon_tiger = {"error": str(exc)}
|
||||
|
||||
context: dict[str, Any] = {
|
||||
"data_trade_date": data_trade_date,
|
||||
"data_profile": profile,
|
||||
"overview": dashboard.get("overview") or {},
|
||||
"market_regime": regime,
|
||||
"recent_market_history": self.database.snapshot_summaries(data_trade_date, 10),
|
||||
"question_matched_stocks": matched_rows[:10],
|
||||
"stock_details": stock_details,
|
||||
}
|
||||
|
||||
ordered_limits = sorted(
|
||||
limits,
|
||||
key=lambda row: (
|
||||
float(row.get("streak") or 0),
|
||||
float(row.get("amount_billion") or 0),
|
||||
),
|
||||
reverse=True,
|
||||
)
|
||||
if profile in {"emotion", "balanced"}:
|
||||
context.update(
|
||||
{
|
||||
"limit_ladder": dashboard.get("ladders") or [],
|
||||
"limit_performance": dashboard.get("limit_performance") or [],
|
||||
"hot_sectors": (dashboard.get("sectors") or [])[:15],
|
||||
"sector_rotation": (dashboard.get("sector_rotation") or [])[:15],
|
||||
"limit_up_stocks": ordered_limits[:30],
|
||||
"broken_stocks": sorted(
|
||||
broken,
|
||||
key=lambda row: float(row.get("amount_billion") or 0),
|
||||
reverse=True,
|
||||
)[:20],
|
||||
"limit_down_stocks": down_limits[:20],
|
||||
"yesterday_limit_performance": sorted(
|
||||
yesterday_limits,
|
||||
key=lambda row: float(row.get("change") or 0),
|
||||
reverse=True,
|
||||
)[:20],
|
||||
}
|
||||
)
|
||||
elif profile == "first_board":
|
||||
context.update(
|
||||
{
|
||||
"first_board_environment": {
|
||||
"seal_rate": (dashboard.get("overview") or {}).get("seal_rate"),
|
||||
"broken_count": len(broken),
|
||||
"first_boards": [row for row in ordered_limits if int(row.get("streak") or 1) == 1][:35],
|
||||
"broken_stocks": sorted(
|
||||
broken,
|
||||
key=lambda row: float(row.get("amount_billion") or 0),
|
||||
reverse=True,
|
||||
)[:30],
|
||||
},
|
||||
"hot_sectors": (dashboard.get("sectors") or [])[:12],
|
||||
}
|
||||
)
|
||||
elif profile == "leader":
|
||||
context.update(
|
||||
{
|
||||
"limit_ladder": dashboard.get("ladders") or [],
|
||||
"multi_board_leaders": [
|
||||
row for row in ordered_limits if int(row.get("streak") or 0) >= 2
|
||||
][:25],
|
||||
"hot_sectors": (dashboard.get("sectors") or [])[:12],
|
||||
"sector_rotation": (dashboard.get("sector_rotation") or [])[:12],
|
||||
}
|
||||
)
|
||||
try:
|
||||
popularity = self.popularity(data_trade_date)
|
||||
context["popularity_core"] = {
|
||||
"consensus": [
|
||||
row for row in (popularity.get("combined") or [])
|
||||
if row.get("dual_source")
|
||||
][:10],
|
||||
"ths": (popularity.get("ths") or [])[:10],
|
||||
"eastmoney": (popularity.get("dc") or [])[:10],
|
||||
}
|
||||
except Exception:
|
||||
context["popularity_core"] = {"unavailable": True}
|
||||
elif profile == "trend":
|
||||
context.update(
|
||||
{
|
||||
"index_momentum": self._mentor_market_matrix(
|
||||
data_trade_date, MENTOR_INDEX_UNIVERSE
|
||||
),
|
||||
"sector_rotation": (dashboard.get("sector_rotation") or [])[:20],
|
||||
"hot_sectors": (dashboard.get("sectors") or [])[:20],
|
||||
"market_breadth": {
|
||||
key: (dashboard.get("overview") or {}).get(key)
|
||||
for key in ("up_count", "down_count", "flat_count", "amount_billion")
|
||||
},
|
||||
}
|
||||
)
|
||||
elif profile == "low_absorption":
|
||||
context.update(
|
||||
{
|
||||
"yesterday_limit_performance": sorted(
|
||||
yesterday_limits,
|
||||
key=lambda row: float(row.get("change") or 0),
|
||||
reverse=True,
|
||||
)[:35],
|
||||
"broken_stocks": broken[:20],
|
||||
"hot_sectors": (dashboard.get("sectors") or [])[:12],
|
||||
}
|
||||
)
|
||||
elif profile == "macro":
|
||||
context.update(
|
||||
{
|
||||
"broad_indexes": self._mentor_market_matrix(
|
||||
data_trade_date, MENTOR_INDEX_UNIVERSE
|
||||
),
|
||||
"core_etfs": self._mentor_market_matrix(
|
||||
data_trade_date, MENTOR_ETF_UNIVERSE
|
||||
),
|
||||
"market_style": {
|
||||
"amount_billion": (dashboard.get("overview") or {}).get("amount_billion"),
|
||||
"breadth": {
|
||||
"up": (dashboard.get("overview") or {}).get("up_count"),
|
||||
"down": (dashboard.get("overview") or {}).get("down_count"),
|
||||
},
|
||||
"top_sectors": (dashboard.get("sectors") or [])[:15],
|
||||
},
|
||||
"unavailable_data": [
|
||||
"政策原文与隔夜资讯尚未接入",
|
||||
"汇率、利率和商品宏观序列当前不可用",
|
||||
],
|
||||
}
|
||||
)
|
||||
if dragon_tiger is not None:
|
||||
context["dragon_tiger"] = dragon_tiger
|
||||
return context
|
||||
|
||||
def _mentor_market_matrix(
|
||||
self, trade_date: str, universe: tuple[tuple[str, str], ...]
|
||||
) -> list[dict[str, Any]]:
|
||||
ifind = getattr(self, "ifind", None)
|
||||
if not ifind or not ifind.configured:
|
||||
return []
|
||||
end = datetime.strptime(trade_date, "%Y%m%d")
|
||||
start = (end - timedelta(days=45)).strftime("%Y%m%d")
|
||||
names = {code: name for code, name in universe}
|
||||
try:
|
||||
rows = ifind.history(
|
||||
list(names), ["close", "volume", "amount"], start, trade_date, cache_ttl=600
|
||||
)
|
||||
except IfindError:
|
||||
return []
|
||||
grouped: dict[str, list[dict[str, Any]]] = {}
|
||||
for row in rows:
|
||||
code = str(row.get("thscode") or "").upper()
|
||||
if code in names:
|
||||
grouped.setdefault(code, []).append(row)
|
||||
result = []
|
||||
for code, name in universe:
|
||||
series = sorted(grouped.get(code, []), key=lambda row: str(row.get("time") or ""))
|
||||
closes = []
|
||||
for row in series:
|
||||
try:
|
||||
close = float(row.get("close") or 0)
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
if close > 0:
|
||||
closes.append(close)
|
||||
if not closes:
|
||||
continue
|
||||
def period_return(days: int) -> float | None:
|
||||
if len(closes) <= days or closes[-days - 1] <= 0:
|
||||
return None
|
||||
return round((closes[-1] / closes[-days - 1] - 1) * 100, 2)
|
||||
previous = closes[-2] if len(closes) > 1 else 0
|
||||
result.append(
|
||||
{
|
||||
"code": code,
|
||||
"name": name,
|
||||
"close": round(closes[-1], 3),
|
||||
"change": round((closes[-1] / previous - 1) * 100, 2) if previous else None,
|
||||
"return_5d": period_return(5),
|
||||
"return_10d": period_return(10),
|
||||
"return_20d": period_return(20),
|
||||
"latest_amount": series[-1].get("amount") if series else None,
|
||||
}
|
||||
)
|
||||
return result
|
||||
@@ -0,0 +1,6 @@
|
||||
"""Limit-up, broken-board, limit-down and prior-limit pool feature."""
|
||||
|
||||
from .repository import PoolRepositoryMixin
|
||||
from .service import PoolServiceMixin
|
||||
|
||||
__all__ = ["PoolRepositoryMixin", "PoolServiceMixin"]
|
||||
@@ -0,0 +1,27 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
class PoolRepositoryMixin:
|
||||
def save_reason_override(self, trade_date: str, code: str, reason: str) -> None:
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO reason_overrides (trade_date, code, reason, updated_at)
|
||||
VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(trade_date, code) DO UPDATE SET
|
||||
reason = excluded.reason,
|
||||
updated_at = excluded.updated_at
|
||||
""",
|
||||
(trade_date, code, reason, now),
|
||||
)
|
||||
|
||||
def reason_overrides(self, trade_date: str) -> dict[str, str]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"SELECT code, reason FROM reason_overrides WHERE trade_date = ?",
|
||||
(trade_date,),
|
||||
).fetchall()
|
||||
return {row["code"]: row["reason"] for row in rows}
|
||||
@@ -0,0 +1,174 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from datetime import datetime, time as dt_time
|
||||
from typing import Any
|
||||
|
||||
from backend.bootstrap.config import normalize_date, validate_stock_code
|
||||
from backend.data.providers.ifind_client import IfindError
|
||||
|
||||
|
||||
class PoolServiceMixin:
|
||||
def save_reason(self, trade_date: str, code: str, reason: str) -> None:
|
||||
normalized_date = normalize_date(trade_date)
|
||||
code = validate_stock_code(code)
|
||||
reason = reason.strip()
|
||||
if not reason or len(reason) > 200:
|
||||
raise ValueError("涨停原因应为 1 至 200 个字符。")
|
||||
self.database.save_reason_override(normalized_date, code, reason)
|
||||
|
||||
def _apply_reason_overrides(self, dashboard: dict[str, Any]) -> dict[str, Any]:
|
||||
trade_date = str(dashboard.get("meta", {}).get("trade_date", "")).replace("-", "")
|
||||
enrichment = self.database.get_data_snapshot("ifind_event_enrichment_v1", trade_date)
|
||||
if enrichment:
|
||||
self._merge_ifind_event_enrichment(dashboard, enrichment)
|
||||
else:
|
||||
self._schedule_ifind_event_enrichment(trade_date)
|
||||
overrides = self.database.reason_overrides(trade_date)
|
||||
if not overrides:
|
||||
return dashboard
|
||||
for key in ("limits", "broken", "down_limits"):
|
||||
for row in dashboard.get(key) or []:
|
||||
if row.get("code") in overrides:
|
||||
row["reason"] = overrides[row["code"]]
|
||||
row["reason_source"] = "manual"
|
||||
return dashboard
|
||||
|
||||
def _schedule_ifind_event_enrichment(self, trade_date: str) -> None:
|
||||
ifind = getattr(self, "ifind", None)
|
||||
if not ifind or not ifind.configured or not re.fullmatch(r"\d{8}", trade_date):
|
||||
return
|
||||
now = datetime.now().astimezone()
|
||||
if trade_date == now.strftime("%Y%m%d") and now.time().replace(tzinfo=None) < dt_time(15, 0):
|
||||
return
|
||||
self.jobs.submit(
|
||||
"market.ifind-event-enrichment",
|
||||
f"{trade_date}:v1",
|
||||
lambda: self._refresh_ifind_event_enrichment(trade_date),
|
||||
{"trade_date": trade_date, "trigger": "dashboard-enrichment"},
|
||||
)
|
||||
|
||||
def _refresh_ifind_event_enrichment(self, trade_date: str) -> None:
|
||||
if not self._ifind_event_lock.acquire(blocking=False):
|
||||
return
|
||||
try:
|
||||
if self.database.get_data_snapshot("ifind_event_enrichment_v1", trade_date):
|
||||
return
|
||||
ifind = getattr(self, "ifind", None)
|
||||
if not ifind or not ifind.configured:
|
||||
return
|
||||
current = datetime.strptime(trade_date, "%Y%m%d")
|
||||
display_date = f"{current.year}年{current.month}月{current.day}日"
|
||||
requests = {
|
||||
"limits": (
|
||||
f"{display_date}涨停股票,股票代码、股票简称、涨停原因、"
|
||||
"首次涨停时间、最终涨停时间、开板次数"
|
||||
),
|
||||
"broken": (
|
||||
f"{display_date}曾涨停但收盘未涨停的股票,股票代码、股票简称、"
|
||||
"涨停原因、首次涨停时间、开板次数"
|
||||
),
|
||||
"down_limits": (
|
||||
f"{display_date}跌停股票,股票代码、股票简称、跌停原因"
|
||||
),
|
||||
}
|
||||
result: dict[str, Any] = {
|
||||
"trade_date": trade_date,
|
||||
"generated_at": datetime.now().astimezone().isoformat(timespec="seconds"),
|
||||
"limits": {}, "broken": {}, "down_limits": {}, "partial": False,
|
||||
}
|
||||
for kind, query in requests.items():
|
||||
try:
|
||||
rows = ifind.wencai(query, "stock", cache_ttl=900)
|
||||
except IfindError:
|
||||
result["partial"] = True
|
||||
continue
|
||||
for raw in rows:
|
||||
code = self._ifind_row_code(raw)
|
||||
if not code:
|
||||
continue
|
||||
reason_tokens = (
|
||||
("跌停原因", "风险线索", "原因")
|
||||
if kind == "down_limits"
|
||||
else ("涨停原因类别", "涨停原因", "触板逻辑", "原因")
|
||||
)
|
||||
reason = str(self._ifind_field(raw, reason_tokens) or "").strip()
|
||||
first_time = self._normalize_ifind_event_time(
|
||||
self._ifind_field(raw, ("首次涨停时间", "首次触板时间", "首次封板时间"))
|
||||
)
|
||||
last_time = self._normalize_ifind_event_time(
|
||||
self._ifind_field(raw, ("最终涨停时间", "最后涨停时间", "最后封板时间"))
|
||||
)
|
||||
open_times = self._ifind_field(raw, ("开板次数", "打开涨停次数"))
|
||||
try:
|
||||
open_count = max(0, int(float(open_times))) if open_times not in (None, "") else None
|
||||
except (TypeError, ValueError):
|
||||
open_count = None
|
||||
result[kind][code] = {
|
||||
"reason": reason,
|
||||
"first_time": first_time,
|
||||
"last_time": last_time,
|
||||
"open_times": open_count,
|
||||
}
|
||||
if any(result[kind] for kind in ("limits", "broken", "down_limits")):
|
||||
self.database.save_data_snapshot(
|
||||
"ifind_event_enrichment_v1", trade_date, "ifind", result
|
||||
)
|
||||
finally:
|
||||
self._ifind_event_lock.release()
|
||||
|
||||
@staticmethod
|
||||
def _ifind_field(row: dict[str, Any], tokens: tuple[str, ...]) -> Any:
|
||||
for key, value in row.items():
|
||||
label = str(key or "")
|
||||
if any(token.casefold() == label.casefold() for token in tokens):
|
||||
return value
|
||||
for key, value in row.items():
|
||||
label = str(key or "")
|
||||
if any(token in label for token in tokens):
|
||||
return value
|
||||
return None
|
||||
|
||||
@classmethod
|
||||
def _ifind_row_code(cls, row: dict[str, Any]) -> str:
|
||||
value = cls._ifind_field(row, ("股票代码", "证券代码", "代码", "thscode"))
|
||||
match = re.search(r"(?<!\d)(\d{6})(?!\d)", str(value or ""))
|
||||
if match:
|
||||
return match.group(1)
|
||||
for value in row.values():
|
||||
match = re.search(r"(?<!\d)(\d{6})\.(?:SH|SZ|BJ)(?![A-Z])", str(value or ""), re.I)
|
||||
if match:
|
||||
return match.group(1)
|
||||
return ""
|
||||
|
||||
@staticmethod
|
||||
def _normalize_ifind_event_time(value: Any) -> str:
|
||||
text = str(value or "").strip()
|
||||
match = re.search(r"(?:^|\s)(\d{1,2}:\d{2}(?::\d{2})?)(?:$|\s)", text)
|
||||
if not match:
|
||||
match = re.search(r"(?<!\d)(\d{6})(?!\d)", text)
|
||||
if match:
|
||||
compact = match.group(1)
|
||||
return f"{compact[:2]}:{compact[2:4]}:{compact[4:]}"
|
||||
return ""
|
||||
parts = match.group(1).split(":")
|
||||
return ":".join(part.zfill(2) for part in parts)
|
||||
|
||||
@staticmethod
|
||||
def _merge_ifind_event_enrichment(
|
||||
dashboard: dict[str, Any], enrichment: dict[str, Any]
|
||||
) -> None:
|
||||
for kind in ("limits", "broken", "down_limits"):
|
||||
records = enrichment.get(kind) or {}
|
||||
for row in dashboard.get(kind) or []:
|
||||
event = records.get(str(row.get("code") or "")) or {}
|
||||
reason = str(event.get("reason") or "").strip()
|
||||
if reason:
|
||||
row["reason"] = reason
|
||||
row["reason_source"] = "market_event"
|
||||
if event.get("first_time"):
|
||||
row["first_time"] = event["first_time"]
|
||||
if event.get("last_time"):
|
||||
row["last_time"] = event["last_time"]
|
||||
if event.get("open_times") is not None:
|
||||
row["open_times"] = event["open_times"]
|
||||
@@ -0,0 +1,4 @@
|
||||
from .repository import PopularityRepositoryMixin
|
||||
from .service import PopularityServiceMixin
|
||||
|
||||
__all__ = ["PopularityRepositoryMixin", "PopularityServiceMixin"]
|
||||
@@ -0,0 +1,37 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
|
||||
class PopularityRepositoryMixin:
|
||||
def upsert_popularity_factors(self, rows: list[dict[str, Any]]) -> int:
|
||||
values = [
|
||||
(
|
||||
str(row.get("trade_date") or ""),
|
||||
str(row.get("ts_code") or ""),
|
||||
int(row["ths_rank"]) if row.get("ths_rank") not in (None, "") else None,
|
||||
int(row["dc_rank"]) if row.get("dc_rank") not in (None, "") else None,
|
||||
float(row.get("combined_score") or 0),
|
||||
int(row["rank_change"]) if row.get("rank_change") not in (None, "") else None,
|
||||
int(bool(row.get("dual_source"))),
|
||||
)
|
||||
for row in rows
|
||||
if row.get("trade_date") and row.get("ts_code")
|
||||
]
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO popularity_factors
|
||||
(trade_date, ts_code, ths_rank, dc_rank, combined_score,
|
||||
rank_change, dual_source)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(trade_date, ts_code) DO UPDATE SET
|
||||
ths_rank=excluded.ths_rank,
|
||||
dc_rank=excluded.dc_rank,
|
||||
combined_score=excluded.combined_score,
|
||||
rank_change=excluded.rank_change,
|
||||
dual_source=excluded.dual_source
|
||||
""",
|
||||
values,
|
||||
)
|
||||
return len(values)
|
||||
@@ -0,0 +1,11 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from backend.bootstrap.config import normalize_date
|
||||
from backend.features.market.insights import MarketInsightsService
|
||||
|
||||
|
||||
class PopularityServiceMixin:
|
||||
def popularity(self, trade_date: str, force: bool = False) -> dict[str, Any]:
|
||||
return self._market_insights().popularity(normalize_date(trade_date), force)
|
||||
@@ -0,0 +1,23 @@
|
||||
from .agent import ReviewAssistantError, stream_review_assistant
|
||||
from .http import ReviewHttpMixin
|
||||
from .repository import ReviewRepositoryMixin
|
||||
from .service import ReviewServiceMixin
|
||||
|
||||
__all__ = [
|
||||
"EMOTIONS",
|
||||
"ReviewAssistantError",
|
||||
"ReviewHttpMixin",
|
||||
"ReviewRepositoryMixin",
|
||||
"ReviewServiceMixin",
|
||||
"TRADE_ACTIONS",
|
||||
"TradeJournalService",
|
||||
"stream_review_assistant",
|
||||
]
|
||||
|
||||
|
||||
def __getattr__(name: str):
|
||||
if name in {"EMOTIONS", "TRADE_ACTIONS", "TradeJournalService"}:
|
||||
from . import trade_journal
|
||||
|
||||
return getattr(trade_journal, name)
|
||||
raise AttributeError(name)
|
||||
@@ -0,0 +1,61 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections.abc import Iterator
|
||||
from typing import Any
|
||||
|
||||
from backend.llm import transport as llm_transport
|
||||
|
||||
|
||||
class ReviewAssistantError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
def stream_review_assistant(
|
||||
context: dict[str, Any],
|
||||
question: str,
|
||||
history: list[dict[str, str]],
|
||||
api_key: str,
|
||||
base_url: str,
|
||||
model: str,
|
||||
timeout: int = 120,
|
||||
) -> Iterator[str]:
|
||||
if not api_key or not model:
|
||||
raise ReviewAssistantError("智能解读服务尚未配置。")
|
||||
messages = [{"role": "system", "content": _system_prompt(context)}]
|
||||
messages.extend(history[-12:])
|
||||
messages.append({"role": "user", "content": question})
|
||||
try:
|
||||
yield from llm_transport.stream_chat_completion(
|
||||
api_key=api_key,
|
||||
base_url=base_url,
|
||||
model=model,
|
||||
messages=messages,
|
||||
timeout=timeout,
|
||||
user_agent="XiaobaiReviewWeb/1.0",
|
||||
)
|
||||
except llm_transport.OpenAIEmptyResponseError as exc:
|
||||
raise ReviewAssistantError("智能解读未返回有效内容。") from exc
|
||||
except llm_transport.OpenAIHTTPError as exc:
|
||||
raise ReviewAssistantError(f"智能解读服务暂不可用({exc.code})。") from exc
|
||||
except llm_transport.OpenAITransportError as exc:
|
||||
raise ReviewAssistantError("智能解读连接中断,请稍后重试。") from exc
|
||||
|
||||
|
||||
def _system_prompt(context: dict[str, Any]) -> str:
|
||||
context_json = json.dumps(context, ensure_ascii=False, separators=(",", ":"))
|
||||
return f"""
|
||||
你是“小白复盘”的统一复盘助手。你负责把网页中已经存在的市场统计、策略跟踪、提醒、复盘笔记和手工交易日志连接起来,帮助用户复盘和形成下一步观察计划。
|
||||
|
||||
最高优先级规则:
|
||||
1. 只能使用下方“网页复盘数据”,数据缺失就明确说明,不得补造行情、交易或胜率。
|
||||
2. 不自动下单,不声称已执行任何操作,不修改策略、提醒、笔记或交易日志。
|
||||
3. 不承诺收益,不给无条件买卖指令。建议必须写成条件、失效条件和风险边界。
|
||||
4. 区分市场事实、用户记录和你的推断。引用数字时写明数据日期。
|
||||
5. 优先结合用户自己的策略跟踪与交易日志寻找可验证的重复模式;样本不足时明确标注。
|
||||
6. 使用中文,先直接回答,再给数据依据和下一步观察。避免空泛口号,不展示模型、接口或内部工程信息。
|
||||
7. 控制在 800 个中文字符以内,除非用户明确要求展开。
|
||||
|
||||
网页复盘数据:
|
||||
{context_json}
|
||||
""".strip()
|
||||
@@ -0,0 +1,83 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from datetime import date
|
||||
from http import HTTPStatus
|
||||
|
||||
from backend.bootstrap.config import normalize_date, validate_stock_code, validate_text
|
||||
from backend.features.review.agent import ReviewAssistantError
|
||||
|
||||
|
||||
class ReviewHttpMixin:
|
||||
def save_trade_entry(self) -> None:
|
||||
try:
|
||||
body = self.read_json_body()
|
||||
self.send_json(
|
||||
{"ok": True, **self.application_service.save_trade_entry(body)},
|
||||
HTTPStatus.CREATED,
|
||||
)
|
||||
except (ValueError, json.JSONDecodeError) as exc:
|
||||
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
|
||||
|
||||
def stream_assistant_chat(self) -> None:
|
||||
try:
|
||||
body = self.read_json_body()
|
||||
stream = self.application_service.assistant_stream(body)
|
||||
except (ValueError, json.JSONDecodeError) as exc:
|
||||
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
|
||||
return
|
||||
events = ({"type": "delta", "content": chunk} for chunk in stream)
|
||||
self.send_ndjson_stream(events, (ValueError, ReviewAssistantError))
|
||||
|
||||
def save_watchlist(self) -> None:
|
||||
try:
|
||||
body = self.read_json_body()
|
||||
code = validate_stock_code(str(body.get("code", "")))
|
||||
name = validate_text(body.get("name"), "股票名称", 30, required=True)
|
||||
sector = validate_text(body.get("sector"), "所属板块", 50)
|
||||
color = str(body.get("color") or "red")
|
||||
if color not in {"red", "blue", "green", "amber"}:
|
||||
raise ValueError("标记颜色不支持。")
|
||||
remark = validate_text(body.get("remark"), "跟踪备注", 240)
|
||||
service = self.application_service
|
||||
service.database.save_watchlist(
|
||||
service.current_user_id, code, name, sector, color, remark
|
||||
)
|
||||
self.send_json(
|
||||
{
|
||||
"ok": True,
|
||||
"items": service.database.list_watchlist(service.current_user_id),
|
||||
}
|
||||
)
|
||||
except (ValueError, json.JSONDecodeError) as exc:
|
||||
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
|
||||
|
||||
def save_note(self) -> None:
|
||||
try:
|
||||
body = self.read_json_body()
|
||||
code = str(body.get("code") or "").strip()
|
||||
if code:
|
||||
code = validate_stock_code(code)
|
||||
stock_name = validate_text(body.get("stock_name"), "股票名称", 30)
|
||||
trade_date = normalize_date(str(body.get("trade_date") or date.today().isoformat()))
|
||||
summary = validate_text(body.get("summary"), "盘面摘要", 500)
|
||||
content = validate_text(body.get("content"), "复盘内容", 5000)
|
||||
plan = validate_text(body.get("plan"), "明日计划", 2000)
|
||||
if not summary and not content and not plan:
|
||||
raise ValueError("每日复盘内容不能全部为空。")
|
||||
raw_id = body.get("id")
|
||||
note_id = int(raw_id) if raw_id else None
|
||||
service = self.application_service
|
||||
saved_id = service.database.save_note(
|
||||
service.current_user_id,
|
||||
code,
|
||||
stock_name,
|
||||
trade_date,
|
||||
content,
|
||||
plan,
|
||||
note_id,
|
||||
summary=summary,
|
||||
)
|
||||
self.send_json({"ok": True, "id": saved_id})
|
||||
except (ValueError, TypeError, json.JSONDecodeError) as exc:
|
||||
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
|
||||
@@ -0,0 +1,281 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
|
||||
class ReviewRepositoryMixin:
|
||||
def list_watchlist(self, user_id: int) -> list[dict[str, Any]]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"""
|
||||
SELECT code, name, sector, color, remark, created_at, updated_at
|
||||
FROM watchlist WHERE user_id = ? ORDER BY updated_at DESC, code
|
||||
""",
|
||||
(int(user_id),),
|
||||
).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
def save_watchlist(
|
||||
self, user_id: int, code: str, name: str, sector: str, color: str,
|
||||
remark: str | None = None,
|
||||
) -> None:
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
existing = connection.execute(
|
||||
"SELECT remark FROM watchlist WHERE user_id = ? AND code = ?",
|
||||
(int(user_id), code),
|
||||
).fetchone()
|
||||
saved_remark = (
|
||||
str(existing["remark"] or "") if remark is None and existing else str(remark or "")
|
||||
)
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO watchlist
|
||||
(user_id, code, name, sector, color, remark, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(user_id, code) DO UPDATE SET
|
||||
name = excluded.name,
|
||||
sector = excluded.sector,
|
||||
color = excluded.color,
|
||||
remark = excluded.remark,
|
||||
updated_at = excluded.updated_at
|
||||
""",
|
||||
(int(user_id), code, name, sector, color, saved_remark, now, now),
|
||||
)
|
||||
|
||||
def watchlist_price_history(
|
||||
self, codes: list[str], end_date: str, limit_per_code: int = 6
|
||||
) -> dict[str, list[dict[str, Any]]]:
|
||||
result: dict[str, list[dict[str, Any]]] = {}
|
||||
if not codes:
|
||||
return result
|
||||
with self.connect() as connection:
|
||||
for code in codes:
|
||||
rows = connection.execute(
|
||||
"""
|
||||
SELECT trade_date, ts_code, close, pct_chg
|
||||
FROM daily_bars
|
||||
WHERE substr(ts_code, 1, 6) = ? AND trade_date <= ?
|
||||
ORDER BY trade_date DESC LIMIT ?
|
||||
""",
|
||||
(str(code), end_date, int(limit_per_code)),
|
||||
).fetchall()
|
||||
result[str(code)] = [dict(row) for row in reversed(rows)]
|
||||
return result
|
||||
|
||||
def delete_watchlist(self, user_id: int, code: str) -> bool:
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"DELETE FROM watchlist WHERE user_id = ? AND code = ?",
|
||||
(int(user_id), code),
|
||||
)
|
||||
return cursor.rowcount > 0
|
||||
|
||||
def list_notes(
|
||||
self,
|
||||
user_id: int,
|
||||
code: str = "",
|
||||
trade_date: str = "",
|
||||
scope: str = "all",
|
||||
) -> list[dict[str, Any]]:
|
||||
clauses: list[str] = ["user_id = ?"]
|
||||
parameters: list[Any] = [int(user_id)]
|
||||
if scope == "daily":
|
||||
clauses.append("code = ''")
|
||||
elif scope == "stock":
|
||||
clauses.append("code <> ''")
|
||||
if code:
|
||||
clauses.append("code = ?")
|
||||
parameters.append(code)
|
||||
if trade_date:
|
||||
clauses.append("trade_date = ?")
|
||||
parameters.append(trade_date)
|
||||
where = f"WHERE {' AND '.join(clauses)}" if clauses else ""
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
f"""
|
||||
SELECT id, code, stock_name, trade_date, summary, content, plan, created_at, updated_at
|
||||
FROM review_notes {where}
|
||||
ORDER BY trade_date DESC, updated_at DESC, id DESC LIMIT 200
|
||||
""",
|
||||
parameters,
|
||||
).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
def save_note(
|
||||
self,
|
||||
user_id: int,
|
||||
code: str,
|
||||
stock_name: str,
|
||||
trade_date: str,
|
||||
content: str,
|
||||
plan: str,
|
||||
note_id: int | None = None,
|
||||
summary: str = "",
|
||||
) -> int:
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
if note_id:
|
||||
cursor = connection.execute(
|
||||
"""
|
||||
UPDATE review_notes
|
||||
SET code = ?, stock_name = ?, trade_date = ?, summary = ?, content = ?, plan = ?, updated_at = ?
|
||||
WHERE id = ? AND user_id = ?
|
||||
""",
|
||||
(code, stock_name, trade_date, summary, content, plan, now, note_id, int(user_id)),
|
||||
)
|
||||
if cursor.rowcount == 0:
|
||||
raise ValueError("复盘笔记不存在。")
|
||||
return note_id
|
||||
cursor = connection.execute(
|
||||
"""
|
||||
INSERT INTO review_notes
|
||||
(user_id, code, stock_name, trade_date, summary, content, plan, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(int(user_id), code, stock_name, trade_date, summary, content, plan, now, now),
|
||||
)
|
||||
return int(cursor.lastrowid)
|
||||
|
||||
def delete_note(self, user_id: int, note_id: int) -> bool:
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"DELETE FROM review_notes WHERE id = ? AND user_id = ?",
|
||||
(note_id, int(user_id)),
|
||||
)
|
||||
return cursor.rowcount > 0
|
||||
|
||||
def save_trade_entry(
|
||||
self,
|
||||
user_id: int,
|
||||
trade_date: str,
|
||||
code: str,
|
||||
name: str,
|
||||
action: str,
|
||||
price: float,
|
||||
quantity: int,
|
||||
position_pct: float,
|
||||
pnl_amount: float | None,
|
||||
pnl_pct: float | None,
|
||||
thesis: str,
|
||||
execution: str,
|
||||
emotion: str,
|
||||
tags: list[str],
|
||||
trade_id: int | None = None,
|
||||
) -> int:
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
tags_json = json.dumps(tags, ensure_ascii=False, separators=(",", ":"))
|
||||
with self.connect() as connection:
|
||||
if trade_id:
|
||||
cursor = connection.execute(
|
||||
"""
|
||||
UPDATE trade_entries SET
|
||||
trade_date=?, code=?, name=?, action=?, price=?, quantity=?,
|
||||
position_pct=?, pnl_amount=?, pnl_pct=?, thesis=?, execution=?,
|
||||
emotion=?, tags=?, updated_at=?
|
||||
WHERE id=? AND user_id=?
|
||||
""",
|
||||
(
|
||||
trade_date, code, name, action, price, quantity, position_pct,
|
||||
pnl_amount, pnl_pct, thesis, execution, emotion, tags_json, now,
|
||||
int(trade_id), int(user_id),
|
||||
),
|
||||
)
|
||||
if cursor.rowcount == 0:
|
||||
raise ValueError("交易记录不存在或无权修改。")
|
||||
return int(trade_id)
|
||||
cursor = connection.execute(
|
||||
"""
|
||||
INSERT INTO trade_entries
|
||||
(user_id, trade_date, code, name, action, price, quantity,
|
||||
position_pct, pnl_amount, pnl_pct, thesis, execution, emotion,
|
||||
tags, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
int(user_id), trade_date, code, name, action, price, quantity,
|
||||
position_pct, pnl_amount, pnl_pct, thesis, execution, emotion,
|
||||
tags_json, now, now,
|
||||
),
|
||||
)
|
||||
return int(cursor.lastrowid)
|
||||
|
||||
def list_trade_entries(
|
||||
self, user_id: int, start_date: str = "", end_date: str = "", code: str = "",
|
||||
limit: int = 300,
|
||||
) -> list[dict[str, Any]]:
|
||||
clauses = ["user_id = ?"]
|
||||
parameters: list[Any] = [int(user_id)]
|
||||
if start_date:
|
||||
clauses.append("trade_date >= ?")
|
||||
parameters.append(start_date)
|
||||
if end_date:
|
||||
clauses.append("trade_date <= ?")
|
||||
parameters.append(end_date)
|
||||
if code:
|
||||
clauses.append("code = ?")
|
||||
parameters.append(code)
|
||||
parameters.append(max(1, min(1000, int(limit))))
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
f"""
|
||||
SELECT * FROM trade_entries WHERE {' AND '.join(clauses)}
|
||||
ORDER BY trade_date DESC, id DESC LIMIT ?
|
||||
""",
|
||||
parameters,
|
||||
).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
def delete_trade_entry(self, user_id: int, trade_id: int) -> bool:
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"DELETE FROM trade_entries WHERE id = ? AND user_id = ?",
|
||||
(int(trade_id), int(user_id)),
|
||||
)
|
||||
return cursor.rowcount > 0
|
||||
|
||||
def save_assistant_exchange(
|
||||
self, user_id: int, question: str, answer: str, context_date: str
|
||||
) -> None:
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO assistant_messages
|
||||
(user_id, role, content, context_date, created_at)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
""",
|
||||
[
|
||||
(int(user_id), "user", question, context_date, now),
|
||||
(int(user_id), "assistant", answer, context_date, now),
|
||||
],
|
||||
)
|
||||
connection.execute(
|
||||
"""
|
||||
DELETE FROM assistant_messages WHERE user_id = ? AND id NOT IN (
|
||||
SELECT id FROM assistant_messages
|
||||
WHERE user_id = ? ORDER BY id DESC LIMIT 200
|
||||
)
|
||||
""",
|
||||
(int(user_id), int(user_id)),
|
||||
)
|
||||
|
||||
def list_assistant_messages(self, user_id: int, limit: int = 100) -> list[dict[str, Any]]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"""
|
||||
SELECT role, content, context_date, created_at FROM assistant_messages
|
||||
WHERE user_id = ? ORDER BY id DESC LIMIT ?
|
||||
""",
|
||||
(int(user_id), max(1, min(200, int(limit)))),
|
||||
).fetchall()
|
||||
return [dict(row) for row in reversed(rows)]
|
||||
|
||||
def delete_assistant_messages(self, user_id: int) -> int:
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"DELETE FROM assistant_messages WHERE user_id = ?", (int(user_id),)
|
||||
)
|
||||
return int(cursor.rowcount)
|
||||
@@ -0,0 +1,188 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, datetime, timedelta
|
||||
from typing import Any
|
||||
|
||||
from backend.bootstrap.config import normalize_date, tushare_code, validate_text
|
||||
from backend.data.providers.tushare_client import TushareError
|
||||
from backend.features.review.agent import ReviewAssistantError, stream_review_assistant
|
||||
|
||||
|
||||
class ReviewServiceMixin:
|
||||
def trade_entries(
|
||||
self, start_date: str = "", end_date: str = "", code: str = ""
|
||||
) -> dict[str, Any]:
|
||||
return self.trade_journal.list_entries(
|
||||
self.current_user_id, start_date, end_date, code
|
||||
)
|
||||
|
||||
def review_watchlist(self, trade_date: str) -> dict[str, Any]:
|
||||
normalized_date = normalize_date(trade_date)
|
||||
items = self.database.list_watchlist(self.current_user_id)
|
||||
if not items:
|
||||
return {"items": [], "trade_date": normalized_date}
|
||||
|
||||
resolved_date = normalized_date
|
||||
if self.configured:
|
||||
try:
|
||||
client = self._tushare_client()
|
||||
resolved_date, _ = client.resolve_trade_context(normalized_date)
|
||||
history = self.database.watchlist_price_history(
|
||||
[str(item["code"]) for item in items], resolved_date
|
||||
)
|
||||
missing_codes = [
|
||||
str(item["code"]) for item in items
|
||||
if len(history.get(str(item["code"])) or []) < 6
|
||||
]
|
||||
start_date = (
|
||||
datetime.strptime(resolved_date, "%Y%m%d") - timedelta(days=24)
|
||||
).strftime("%Y%m%d")
|
||||
for code in missing_codes:
|
||||
rows = client.query(
|
||||
"daily",
|
||||
{
|
||||
"ts_code": tushare_code(code),
|
||||
"start_date": start_date,
|
||||
"end_date": resolved_date,
|
||||
},
|
||||
"ts_code,trade_date,open,high,low,close,pct_chg,vol,amount",
|
||||
)
|
||||
if rows:
|
||||
self.database.upsert_daily_bars(rows)
|
||||
if missing_codes:
|
||||
history = self.database.watchlist_price_history(
|
||||
[str(item["code"]) for item in items], resolved_date
|
||||
)
|
||||
except (TushareError, ValueError):
|
||||
history = self.database.watchlist_price_history(
|
||||
[str(item["code"]) for item in items], resolved_date
|
||||
)
|
||||
else:
|
||||
history = self.database.watchlist_price_history(
|
||||
[str(item["code"]) for item in items], resolved_date
|
||||
)
|
||||
|
||||
auction_scores: dict[str, Any] = {}
|
||||
try:
|
||||
auction = self.auction_center(normalized_date, False)
|
||||
auction_scores = {
|
||||
str(row.get("code") or ""): row.get("attention_score")
|
||||
for row in (auction.get("watchlist_rows") or [])
|
||||
if row.get("available", True)
|
||||
}
|
||||
except (TushareError, ValueError):
|
||||
pass
|
||||
|
||||
enriched = []
|
||||
for item in items:
|
||||
code = str(item.get("code") or "")
|
||||
bars = history.get(code) or []
|
||||
latest = bars[-1] if bars else {}
|
||||
close = float(latest.get("close") or 0)
|
||||
base_close = float(bars[-6].get("close") or 0) if len(bars) >= 6 else 0
|
||||
enriched.append(
|
||||
{
|
||||
**item,
|
||||
"change": (
|
||||
round(float(latest.get("pct_chg") or 0), 2) if latest else None
|
||||
),
|
||||
"return_5d": (
|
||||
round((close / base_close - 1) * 100, 2)
|
||||
if close > 0 and base_close > 0 else None
|
||||
),
|
||||
"attention_score": auction_scores.get(code),
|
||||
"market_date": str(latest.get("trade_date") or ""),
|
||||
}
|
||||
)
|
||||
return {"items": enriched, "trade_date": resolved_date}
|
||||
|
||||
def save_trade_entry(self, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
trade_id = self.trade_journal.save(self.current_user_id, payload)
|
||||
return {"id": trade_id, **self.trade_entries()}
|
||||
|
||||
def delete_trade_entry(self, trade_id: int) -> dict[str, Any]:
|
||||
deleted = self.trade_journal.delete(self.current_user_id, trade_id)
|
||||
return {"deleted": deleted, **self.trade_entries()}
|
||||
|
||||
def assistant_messages(self) -> list[dict[str, Any]]:
|
||||
return self.database.list_assistant_messages(self.current_user_id)
|
||||
|
||||
def clear_assistant_messages(self) -> int:
|
||||
return self.database.delete_assistant_messages(self.current_user_id)
|
||||
|
||||
def assistant_stream(self, payload: dict[str, Any]):
|
||||
question = validate_text(payload.get("question"), "问题", 2000, required=True)
|
||||
trade_date = normalize_date(
|
||||
str(payload.get("trade_date") or date.today().isoformat())
|
||||
)
|
||||
context = self._assistant_context(trade_date)
|
||||
history = [
|
||||
{"role": item["role"], "content": str(item["content"])[:4000]}
|
||||
for item in self.assistant_messages()[-12:]
|
||||
if item.get("role") in {"user", "assistant"}
|
||||
]
|
||||
def generate():
|
||||
answer_parts: list[str] = []
|
||||
events = self.llm_gateway.stream(
|
||||
"assistant",
|
||||
"review-assistant-v1",
|
||||
lambda profile: stream_review_assistant(
|
||||
context,
|
||||
question,
|
||||
history,
|
||||
profile.api_key,
|
||||
profile.base_url,
|
||||
profile.model,
|
||||
),
|
||||
(ReviewAssistantError,),
|
||||
)
|
||||
for event in events:
|
||||
if event.kind == "delta":
|
||||
chunk = str(event.value or "")
|
||||
answer_parts.append(chunk)
|
||||
yield chunk
|
||||
elif event.kind == "complete":
|
||||
self.database.save_assistant_exchange(
|
||||
self.current_user_id,
|
||||
question,
|
||||
"".join(answer_parts).strip(),
|
||||
trade_date,
|
||||
)
|
||||
|
||||
return generate()
|
||||
|
||||
def _assistant_context(self, trade_date: str) -> dict[str, Any]:
|
||||
dashboard = self.get_dashboard(trade_date)
|
||||
actual_date = normalize_date(
|
||||
str((dashboard.get("meta") or {}).get("trade_date") or trade_date)
|
||||
)
|
||||
sentiment = self.sentiment_history(actual_date, 10)
|
||||
tracking = self.strategy_tracking.list_tracking(self.current_user_id, 5)
|
||||
alerts = self.alert_service.list_alerts(
|
||||
self.current_user_id, "all", date.today().isoformat()
|
||||
)
|
||||
trades = self.trade_journal.list_entries(
|
||||
self.current_user_id, end_date=actual_date
|
||||
)
|
||||
return {
|
||||
"data_date": actual_date,
|
||||
"market": {
|
||||
"overview": dashboard.get("overview") or {},
|
||||
"top_sectors": (dashboard.get("sectors") or [])[:8],
|
||||
"limit_performance": dashboard.get("limit_performance") or {},
|
||||
"sentiment_history": (sentiment.get("rows") or [])[-10:],
|
||||
},
|
||||
"personal": {
|
||||
"watchlist": self.database.list_watchlist(self.current_user_id)[:30],
|
||||
"review_notes": self.database.list_notes(
|
||||
self.current_user_id, scope="daily"
|
||||
)[:10],
|
||||
"strategy_tracking": {
|
||||
"summary": tracking.get("summary") or {},
|
||||
"batches": (tracking.get("batches") or [])[:5],
|
||||
},
|
||||
"alerts": (alerts.get("items") or [])[:20],
|
||||
"trade_summary": trades.get("summary") or {},
|
||||
"trade_entries": (trades.get("items") or [])[:30],
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from datetime import date
|
||||
from typing import Any
|
||||
|
||||
from backend.bootstrap.config import normalize_date, validate_stock_code, validate_text
|
||||
from backend.database.repositories import TradeJournalRepository
|
||||
|
||||
|
||||
TRADE_ACTIONS = {"buy": "买入", "sell": "卖出", "trim": "减仓", "add": "加仓", "watch": "观察"}
|
||||
EMOTIONS = {"calm": "平静", "confident": "笃定", "hesitant": "犹豫", "anxious": "焦虑", "impulsive": "冲动"}
|
||||
|
||||
|
||||
class TradeJournalService:
|
||||
def __init__(self, repository: TradeJournalRepository) -> None:
|
||||
self.repository = repository
|
||||
|
||||
def save(self, user_id: int, payload: dict[str, Any]) -> int:
|
||||
trade_id = int(payload.get("id") or 0)
|
||||
trade_date = normalize_date(str(payload.get("trade_date") or date.today().isoformat()))
|
||||
code = validate_stock_code(str(payload.get("code") or ""))
|
||||
name = validate_text(payload.get("name"), "股票名称", 40, required=True)
|
||||
action = str(payload.get("action") or "")
|
||||
if action not in TRADE_ACTIONS:
|
||||
raise ValueError("交易动作不支持。")
|
||||
emotion = str(payload.get("emotion") or "calm")
|
||||
if emotion not in EMOTIONS:
|
||||
raise ValueError("交易情绪不支持。")
|
||||
price = self._number(payload.get("price"), "成交价格", 0, 1000000, required=True)
|
||||
quantity = int(self._number(payload.get("quantity"), "成交数量", 0, 100000000))
|
||||
position_pct = self._number(payload.get("position_pct"), "仓位", 0, 100)
|
||||
pnl_amount = self._optional_number(payload.get("pnl_amount"), "盈亏金额", -1e12, 1e12)
|
||||
pnl_pct = self._optional_number(payload.get("pnl_pct"), "盈亏比例", -1000, 10000)
|
||||
thesis = validate_text(payload.get("thesis"), "交易逻辑", 2000)
|
||||
execution = validate_text(payload.get("execution"), "执行复核", 2000)
|
||||
raw_tags = payload.get("tags") or []
|
||||
if isinstance(raw_tags, str):
|
||||
raw_tags = [item.strip() for item in raw_tags.replace(",", ",").split(",")]
|
||||
if not isinstance(raw_tags, list):
|
||||
raise ValueError("交易标签格式不正确。")
|
||||
tags = [validate_text(item, "交易标签", 20) for item in raw_tags if str(item).strip()][:8]
|
||||
return self.repository.save_trade_entry(
|
||||
user_id, trade_date, code, name, action, price, quantity, position_pct,
|
||||
pnl_amount, pnl_pct, thesis, execution, emotion, tags, trade_id or None,
|
||||
)
|
||||
|
||||
def list_entries(
|
||||
self, user_id: int, start_date: str = "", end_date: str = "", code: str = ""
|
||||
) -> dict[str, Any]:
|
||||
start = normalize_date(start_date) if start_date else ""
|
||||
end = normalize_date(end_date) if end_date else date.today().strftime("%Y%m%d")
|
||||
if start and start > end:
|
||||
raise ValueError("开始日期不能晚于结束日期。")
|
||||
code = validate_stock_code(code) if code else ""
|
||||
items = self.repository.list_trade_entries(user_id, start, end, code)
|
||||
for item in items:
|
||||
item["tags"] = json.loads(item.get("tags") or "[]")
|
||||
item["action_label"] = TRADE_ACTIONS.get(item["action"], item["action"])
|
||||
item["emotion_label"] = EMOTIONS.get(item["emotion"], item["emotion"])
|
||||
realized = [item for item in items if item.get("pnl_pct") is not None]
|
||||
return {"items": items, "summary": self._summary(items, realized)}
|
||||
|
||||
def delete(self, user_id: int, trade_id: int) -> bool:
|
||||
return self.repository.delete_trade_entry(user_id, trade_id)
|
||||
|
||||
@staticmethod
|
||||
def _summary(items: list[dict[str, Any]], realized: list[dict[str, Any]]) -> dict[str, Any]:
|
||||
pnl_amounts = [float(item["pnl_amount"]) for item in realized if item.get("pnl_amount") is not None]
|
||||
positions = [float(item["position_pct"]) for item in items if float(item.get("position_pct") or 0) > 0]
|
||||
wins = sum(float(item.get("pnl_pct") or 0) > 0 for item in realized)
|
||||
return {
|
||||
"total": len(items),
|
||||
"realized": len(realized),
|
||||
"win_rate": round(wins / len(realized) * 100, 1) if realized else None,
|
||||
"pnl_amount": round(sum(pnl_amounts), 2) if pnl_amounts else None,
|
||||
"average_position": round(sum(positions) / len(positions), 1) if positions else None,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _number(value: Any, label: str, minimum: float, maximum: float, required: bool = False) -> float:
|
||||
if value in (None, ""):
|
||||
if required:
|
||||
raise ValueError(f"{label}不能为空。")
|
||||
return 0.0
|
||||
try:
|
||||
parsed = float(value)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError(f"{label}格式不正确。") from exc
|
||||
if parsed < minimum or parsed > maximum:
|
||||
raise ValueError(f"{label}超出允许范围。")
|
||||
return parsed
|
||||
|
||||
@classmethod
|
||||
def _optional_number(
|
||||
cls, value: Any, label: str, minimum: float, maximum: float
|
||||
) -> float | None:
|
||||
if value in (None, ""):
|
||||
return None
|
||||
return cls._number(value, label, minimum, maximum, required=True)
|
||||
@@ -0,0 +1,5 @@
|
||||
"""Sector rotation history and constituent detail feature."""
|
||||
|
||||
from .service import RotationServiceMixin
|
||||
|
||||
__all__ = ["RotationServiceMixin"]
|
||||
@@ -0,0 +1,165 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from backend.bootstrap.config import normalize_date, validate_text
|
||||
from backend.data.providers.tushare_client import TushareError
|
||||
from backend.features.sentiment.engine import (
|
||||
build_sentiment_history,
|
||||
latest_contiguous_history,
|
||||
)
|
||||
|
||||
|
||||
class RotationServiceMixin:
|
||||
def rotation_history(self, trade_date: str, limit: int = 9) -> dict[str, Any]:
|
||||
normalized_date = normalize_date(trade_date)
|
||||
# 板块轮动固定展示最近 9 个交易日,按由近到远排列。
|
||||
limit = 9
|
||||
snapshots = self.database.list_snapshot_payloads(normalized_date, 240)
|
||||
by_trade_date: dict[str, dict[str, Any]] = {}
|
||||
for snapshot in snapshots:
|
||||
meta = snapshot.get("meta") or {}
|
||||
actual_date = str(meta.get("trade_date") or snapshot.get("_snapshot_date") or "")
|
||||
compact_date = actual_date.replace("-", "")
|
||||
if len(compact_date) == 8:
|
||||
by_trade_date[compact_date] = snapshot
|
||||
|
||||
sentiment_dates = {
|
||||
str(row.get("trade_date") or "").replace("-", "")
|
||||
for row in latest_contiguous_history(build_sentiment_history(snapshots))
|
||||
}
|
||||
ordered_dates = sorted(
|
||||
date_key for date_key in by_trade_date
|
||||
if not sentiment_dates or date_key in sentiment_dates
|
||||
)[-limit:][::-1]
|
||||
rows = []
|
||||
for date_key in ordered_dates:
|
||||
snapshot = by_trade_date[date_key]
|
||||
sector_context = {
|
||||
str(item.get("name") or ""): item
|
||||
for item in snapshot.get("sectors") or []
|
||||
}
|
||||
sectors = []
|
||||
for item in (snapshot.get("sector_rotation") or [])[:12]:
|
||||
name = str(item.get("name") or "").strip()
|
||||
context = sector_context.get(name, {})
|
||||
sectors.append(
|
||||
{
|
||||
"name": name,
|
||||
"rank": int(item.get("rank") or len(sectors) + 1),
|
||||
"trend": item.get("trend") or "持平",
|
||||
"count": int(item.get("count") or 0),
|
||||
"strength": float(item.get("strength") or context.get("strength") or 0),
|
||||
"change": float(context.get("change") or 0),
|
||||
"leader": item.get("leader") or context.get("leader") or "--",
|
||||
}
|
||||
)
|
||||
rows.append(
|
||||
{
|
||||
"trade_date": f"{date_key[:4]}-{date_key[4:6]}-{date_key[6:]}",
|
||||
"sectors": sectors,
|
||||
}
|
||||
)
|
||||
return {
|
||||
"trade_date": rows[0]["trade_date"] if rows else normalized_date,
|
||||
"available_days": len(ordered_dates),
|
||||
"requested_days": limit,
|
||||
"rows": rows,
|
||||
}
|
||||
|
||||
def rotation_sector_members(self, trade_date: str, sector_name: str) -> dict[str, Any]:
|
||||
normalized_date = normalize_date(trade_date)
|
||||
sector_name = validate_text(sector_name, "板块名称", 60, required=True)
|
||||
dashboard = self.get_dashboard(normalized_date)
|
||||
actual_date = normalize_date(
|
||||
str((dashboard.get("meta") or {}).get("trade_date") or normalized_date)
|
||||
)
|
||||
cache_key = f"{actual_date}:{sector_name}"
|
||||
cached = self.database.get_data_snapshot("rotation_sector_members_v1", cache_key)
|
||||
if cached:
|
||||
cached["meta"] = {**(cached.get("meta") or {}), "cached": True}
|
||||
return cached
|
||||
if not self.configured:
|
||||
raise ValueError("板块成分数据暂不可用。")
|
||||
|
||||
representative = next(
|
||||
(
|
||||
item for item in dashboard.get("limits") or []
|
||||
if str(item.get("sector") or "").strip() == sector_name
|
||||
),
|
||||
None,
|
||||
)
|
||||
if not representative:
|
||||
raise ValueError("未找到该板块的代表股票,暂时无法核验成分股。")
|
||||
raw_code = str(representative.get("ts_code") or representative.get("code") or "")
|
||||
if "." in raw_code:
|
||||
ts_code = raw_code
|
||||
elif raw_code.startswith(("4", "8", "92")):
|
||||
ts_code = f"{raw_code}.BJ"
|
||||
elif raw_code.startswith(("6", "68", "90")):
|
||||
ts_code = f"{raw_code}.SH"
|
||||
else:
|
||||
ts_code = f"{raw_code}.SZ"
|
||||
client = self._tushare_client()
|
||||
try:
|
||||
industry = client.sw_stock_industry(ts_code, actual_date)
|
||||
sector_code = str(industry.get("l2_code") or "")
|
||||
members = client.sw_sector_members(sector_code, actual_date)
|
||||
except TushareError as exc:
|
||||
raise ValueError(f"该板块成分股暂不可用:{exc}") from exc
|
||||
|
||||
daily_rows = self.database.daily_bars_for_date(actual_date)
|
||||
if len(daily_rows) < 1000:
|
||||
try:
|
||||
daily_rows = client.query(
|
||||
"daily",
|
||||
{"trade_date": actual_date},
|
||||
"ts_code,trade_date,open,high,low,close,pct_chg,vol,amount",
|
||||
)
|
||||
if daily_rows:
|
||||
self.database.upsert_daily_bars(daily_rows)
|
||||
except TushareError:
|
||||
daily_rows = self.database.daily_bars_for_date(actual_date)
|
||||
daily_map = {str(item.get("ts_code") or ""): item for item in daily_rows}
|
||||
rows = []
|
||||
for member in members:
|
||||
member_code = str(member.get("ts_code") or "")
|
||||
quote = daily_map.get(member_code) or {}
|
||||
rows.append(
|
||||
{
|
||||
"code": member_code.split(".")[0],
|
||||
"ts_code": member_code,
|
||||
"name": str(member.get("name") or "--"),
|
||||
"change": quote.get("pct_chg"),
|
||||
"open": quote.get("open"),
|
||||
"close": quote.get("close"),
|
||||
"amount_billion": (
|
||||
round(float(quote.get("amount") or 0) / 100000, 2)
|
||||
if quote else None
|
||||
),
|
||||
"quoted": bool(quote),
|
||||
}
|
||||
)
|
||||
rows.sort(
|
||||
key=lambda item: (
|
||||
bool(item.get("quoted")),
|
||||
float(item.get("change") or -999),
|
||||
float(item.get("amount_billion") or 0),
|
||||
),
|
||||
reverse=True,
|
||||
)
|
||||
result = {
|
||||
"meta": {
|
||||
"trade_date": self._display_compact_date(actual_date),
|
||||
"sector_name": str(industry.get("l2_name") or sector_name),
|
||||
"sector_code": sector_code,
|
||||
"member_count": len(rows),
|
||||
"quoted_count": sum(bool(item.get("quoted")) for item in rows),
|
||||
"cached": False,
|
||||
},
|
||||
"rows": rows,
|
||||
}
|
||||
self.database.save_data_snapshot(
|
||||
"rotation_sector_members_v1", cache_key, "tushare", result
|
||||
)
|
||||
return result
|
||||
@@ -0,0 +1 @@
|
||||
"""Stock screening, custom selection, and strategy tracking feature."""
|
||||
@@ -0,0 +1,100 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
from backend.llm import transport as llm_transport
|
||||
from backend.features.screener.engine import FACTOR_FIELDS, REGIMES
|
||||
|
||||
|
||||
class LLMCompilerError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
def test_llm_connection(
|
||||
api_key: str,
|
||||
base_url: str,
|
||||
model: str,
|
||||
timeout: int = 30,
|
||||
) -> dict[str, Any]:
|
||||
if not api_key or not model:
|
||||
raise LLMCompilerError("API Key 或模型未配置。")
|
||||
try:
|
||||
result = llm_transport.chat_completion(
|
||||
api_key=api_key,
|
||||
base_url=base_url,
|
||||
model=model,
|
||||
messages=[{"role": "user", "content": "只回复 OK"}],
|
||||
timeout=timeout,
|
||||
user_agent="XiaobaiReviewWeb/0.5",
|
||||
)
|
||||
reply = str(result.content).strip()
|
||||
except llm_transport.OpenAIHTTPError as exc:
|
||||
raise LLMCompilerError(exc.describe("模型连接测试失败")) from exc
|
||||
except llm_transport.OpenAITransportError as exc:
|
||||
raise LLMCompilerError(f"模型连接测试失败:{exc}") from exc
|
||||
return {
|
||||
"ok": True,
|
||||
"model": model,
|
||||
"reply": reply[:100],
|
||||
"latency_ms": result.latency_ms,
|
||||
}
|
||||
|
||||
|
||||
def compile_strategy_with_llm(
|
||||
prompt: str,
|
||||
regime: str,
|
||||
api_key: str,
|
||||
base_url: str,
|
||||
model: str,
|
||||
timeout: int = 45,
|
||||
) -> dict[str, Any]:
|
||||
if not api_key or not model:
|
||||
raise LLMCompilerError("尚未配置 LLM API Key 或模型。")
|
||||
schema = {
|
||||
"name": "策略名称",
|
||||
"description": "策略说明",
|
||||
"regimes": [regime],
|
||||
"formula": {
|
||||
"universe": {"exclude_st": True, "listed_days_min": 120},
|
||||
"filters": [{"field": "return_5d", "op": ">=", "value": 0}],
|
||||
"score": [{"field": "sector_strength", "weight": 0.3, "direction": "desc"}],
|
||||
"limit": 15,
|
||||
"min_score": 0.55,
|
||||
},
|
||||
}
|
||||
system_prompt = (
|
||||
"你是A股量化策略编译器。只输出JSON对象,不输出Markdown。"
|
||||
"不得生成Python、SQL、网络请求或未提供的因子。"
|
||||
f"当前市场阶段为{REGIMES.get(regime, regime)}。"
|
||||
f"可用因子为:{json.dumps(FACTOR_FIELDS, ensure_ascii=False)}。"
|
||||
"运算符只能使用 >, >=, <, <=, ==, !=, between, in。"
|
||||
"score权重均大于0且不超过1,direction只能是asc或desc。"
|
||||
"退潮和冰点策略必须提高门槛并允许结果为空。"
|
||||
f"严格遵循以下结构:{json.dumps(schema, ensure_ascii=False)}"
|
||||
)
|
||||
try:
|
||||
result = llm_transport.chat_completion(
|
||||
api_key=api_key,
|
||||
base_url=base_url,
|
||||
model=model,
|
||||
messages=[
|
||||
{"role": "system", "content": system_prompt},
|
||||
{"role": "user", "content": prompt[:3000]},
|
||||
],
|
||||
timeout=timeout,
|
||||
user_agent="XiaobaiReviewWeb/0.4",
|
||||
)
|
||||
content = result.content.strip()
|
||||
if content.startswith("```"):
|
||||
content = content.strip("`")
|
||||
if content.startswith("json"):
|
||||
content = content[4:].strip()
|
||||
compiled = json.loads(content)
|
||||
except llm_transport.OpenAIHTTPError as exc:
|
||||
raise LLMCompilerError(exc.describe("LLM 策略编译失败")) from exc
|
||||
except (llm_transport.OpenAITransportError, json.JSONDecodeError) as exc:
|
||||
raise LLMCompilerError(f"LLM 策略编译失败:{exc}") from exc
|
||||
compiled["compiler"] = "llm"
|
||||
compiled["model"] = model
|
||||
return compiled
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,814 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from backend.features.sentiment.engine import build_sentiment_history
|
||||
|
||||
|
||||
def _optional_float(value: Any) -> float | None:
|
||||
if value in (None, ""):
|
||||
return None
|
||||
try:
|
||||
return float(value)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
class ScreenerRepositoryMixin:
|
||||
def upsert_benchmark_bars(self, rows: list[dict[str, Any]]) -> int:
|
||||
values = [
|
||||
(
|
||||
str(row.get("trade_date") or ""), str(row.get("ts_code") or ""),
|
||||
float(row.get("close") or 0), float(row.get("pct_chg") or 0),
|
||||
)
|
||||
for row in rows if row.get("trade_date") and row.get("ts_code")
|
||||
]
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO benchmark_bars (trade_date, ts_code, close, pct_chg)
|
||||
VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(trade_date, ts_code) DO UPDATE SET
|
||||
close=excluded.close, pct_chg=excluded.pct_chg
|
||||
""",
|
||||
values,
|
||||
)
|
||||
return len(values)
|
||||
|
||||
def upsert_daily_indicators(self, rows: list[dict[str, Any]]) -> int:
|
||||
values = [
|
||||
(
|
||||
str(row.get("trade_date") or ""), row.get("ts_code", ""),
|
||||
float(row.get("turnover_rate") or 0), float(row.get("volume_ratio") or 0),
|
||||
float(row.get("total_mv") or 0), float(row.get("circ_mv") or 0),
|
||||
_optional_float(row.get("pe_ttm")), _optional_float(row.get("pb")),
|
||||
_optional_float(row.get("ps_ttm")), _optional_float(row.get("dv_ttm")),
|
||||
)
|
||||
for row in rows if row.get("trade_date") and row.get("ts_code")
|
||||
]
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO daily_indicators
|
||||
(trade_date, ts_code, turnover_rate, volume_ratio, total_mv, circ_mv,
|
||||
pe_ttm, pb, ps_ttm, dv_ttm)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(trade_date, ts_code) DO UPDATE SET
|
||||
turnover_rate=excluded.turnover_rate, volume_ratio=excluded.volume_ratio,
|
||||
total_mv=excluded.total_mv, circ_mv=excluded.circ_mv,
|
||||
pe_ttm=excluded.pe_ttm, pb=excluded.pb,
|
||||
ps_ttm=excluded.ps_ttm, dv_ttm=excluded.dv_ttm
|
||||
""",
|
||||
values,
|
||||
)
|
||||
return len(values)
|
||||
|
||||
def upsert_fundamental_indicators(self, rows: list[dict[str, Any]]) -> int:
|
||||
values = [
|
||||
(
|
||||
str(row.get("end_date") or ""), str(row.get("ann_date") or ""),
|
||||
str(row.get("ts_code") or ""), _optional_float(row.get("roe")),
|
||||
_optional_float(row.get("roa")), _optional_float(row.get("roic")),
|
||||
_optional_float(row.get("grossprofit_margin")),
|
||||
_optional_float(row.get("netprofit_yoy")), _optional_float(row.get("or_yoy")),
|
||||
_optional_float(row.get("ocf_to_opincome")),
|
||||
)
|
||||
for row in rows
|
||||
if row.get("end_date") and row.get("ts_code")
|
||||
]
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO fundamental_indicators
|
||||
(end_date, ann_date, ts_code, roe, roa, roic, grossprofit_margin,
|
||||
netprofit_yoy, or_yoy, ocf_to_opincome)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(end_date, ts_code) DO UPDATE SET
|
||||
ann_date=excluded.ann_date, roe=excluded.roe, roa=excluded.roa,
|
||||
roic=excluded.roic, grossprofit_margin=excluded.grossprofit_margin,
|
||||
netprofit_yoy=excluded.netprofit_yoy, or_yoy=excluded.or_yoy,
|
||||
ocf_to_opincome=excluded.ocf_to_opincome
|
||||
""",
|
||||
values,
|
||||
)
|
||||
return len(values)
|
||||
|
||||
def upsert_moneyflow(self, rows: list[dict[str, Any]]) -> int:
|
||||
values = []
|
||||
for row in rows:
|
||||
if not row.get("trade_date") or not row.get("ts_code"):
|
||||
continue
|
||||
large_net = (
|
||||
float(row.get("buy_lg_amount") or 0) + float(row.get("buy_elg_amount") or 0)
|
||||
- float(row.get("sell_lg_amount") or 0) - float(row.get("sell_elg_amount") or 0)
|
||||
)
|
||||
medium_net = float(row.get("buy_md_amount") or 0) - float(row.get("sell_md_amount") or 0)
|
||||
small_net = float(row.get("buy_sm_amount") or 0) - float(row.get("sell_sm_amount") or 0)
|
||||
values.append((
|
||||
str(row["trade_date"]), row["ts_code"], float(row.get("net_mf_amount") or 0),
|
||||
large_net, medium_net, small_net,
|
||||
))
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO moneyflow_daily
|
||||
(trade_date, ts_code, net_mf_amount, large_net_amount, medium_net_amount, small_net_amount)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(trade_date, ts_code) DO UPDATE SET
|
||||
net_mf_amount=excluded.net_mf_amount, large_net_amount=excluded.large_net_amount,
|
||||
medium_net_amount=excluded.medium_net_amount, small_net_amount=excluded.small_net_amount
|
||||
""",
|
||||
values,
|
||||
)
|
||||
return len(values)
|
||||
|
||||
def upsert_earnings_events(self, rows: list[dict[str, Any]]) -> int:
|
||||
values = [
|
||||
(
|
||||
str(row.get("end_date") or ""),
|
||||
str(row.get("ann_date") or ""),
|
||||
str(row.get("ts_code") or ""),
|
||||
_optional_float(row.get("forecast_profit")),
|
||||
_optional_float(row.get("actual_profit")),
|
||||
_optional_float(row.get("surprise_pct")),
|
||||
_optional_float(row.get("revenue_yoy")),
|
||||
_optional_float(row.get("netprofit_yoy")),
|
||||
str(row.get("source") or ""),
|
||||
)
|
||||
for row in rows
|
||||
if row.get("end_date") and row.get("ann_date") and row.get("ts_code")
|
||||
]
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO earnings_events
|
||||
(end_date, ann_date, ts_code, forecast_profit, actual_profit,
|
||||
surprise_pct, revenue_yoy, netprofit_yoy, source)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(end_date, ann_date, ts_code) DO UPDATE SET
|
||||
forecast_profit=excluded.forecast_profit,
|
||||
actual_profit=excluded.actual_profit,
|
||||
surprise_pct=excluded.surprise_pct,
|
||||
revenue_yoy=excluded.revenue_yoy,
|
||||
netprofit_yoy=excluded.netprofit_yoy,
|
||||
source=excluded.source
|
||||
""",
|
||||
values,
|
||||
)
|
||||
return len(values)
|
||||
|
||||
def daily_indicator_dates(self, end_date: str = "", limit: int = 400) -> list[str]:
|
||||
where = "WHERE trade_date <= ?" if end_date else ""
|
||||
parameters: tuple[Any, ...] = (end_date, limit) if end_date else (limit,)
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
f"SELECT DISTINCT trade_date FROM daily_indicators {where} "
|
||||
"ORDER BY trade_date DESC LIMIT ?",
|
||||
parameters,
|
||||
).fetchall()
|
||||
return [row["trade_date"] for row in reversed(rows)]
|
||||
|
||||
def fundamental_periods(self) -> list[str]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"SELECT DISTINCT end_date FROM fundamental_indicators ORDER BY end_date"
|
||||
).fetchall()
|
||||
return [str(row["end_date"]) for row in rows]
|
||||
|
||||
def factor_dates(self, end_date: str = "", limit: int = 80) -> list[str]:
|
||||
where = "WHERE trade_date <= ?" if end_date else ""
|
||||
parameters: tuple[Any, ...] = (end_date, limit) if end_date else (limit,)
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
f"SELECT DISTINCT trade_date FROM daily_bars {where} ORDER BY trade_date DESC LIMIT ?",
|
||||
parameters,
|
||||
).fetchall()
|
||||
return [row["trade_date"] for row in reversed(rows)]
|
||||
|
||||
def factor_health_summary(self, end_date: str) -> dict[str, Any]:
|
||||
dividend_start = f"{max(0, int(end_date[:4] or 0) - 5)}0101"
|
||||
with self.connect() as connection:
|
||||
market = connection.execute(
|
||||
"SELECT EXISTS(SELECT 1 FROM daily_bars WHERE trade_date <= ? LIMIT 1)",
|
||||
(end_date,),
|
||||
).fetchone()[0]
|
||||
auction = connection.execute(
|
||||
"SELECT EXISTS(SELECT 1 FROM auction_factors WHERE trade_date <= ? LIMIT 1)",
|
||||
(end_date,),
|
||||
).fetchone()[0]
|
||||
benchmark_rows = connection.execute(
|
||||
"SELECT COUNT(*) FROM benchmark_bars WHERE ts_code = '000300.SH' AND trade_date <= ?",
|
||||
(end_date,),
|
||||
).fetchone()[0]
|
||||
indicator_date = connection.execute(
|
||||
"SELECT MAX(trade_date) FROM daily_indicators WHERE trade_date <= ?",
|
||||
(end_date,),
|
||||
).fetchone()[0]
|
||||
if indicator_date:
|
||||
valuation_rows, valuation_available = connection.execute(
|
||||
"""
|
||||
SELECT COUNT(*), COALESCE(MAX(pe_ttm IS NOT NULL), 0)
|
||||
FROM daily_indicators WHERE trade_date = ?
|
||||
""",
|
||||
(indicator_date,),
|
||||
).fetchone()
|
||||
else:
|
||||
valuation_rows, valuation_available = 0, 0
|
||||
dividend_years = connection.execute(
|
||||
"""
|
||||
SELECT COUNT(DISTINCT substr(trade_date, 1, 4))
|
||||
FROM daily_indicators
|
||||
WHERE trade_date <= ? AND trade_date >= ?
|
||||
""",
|
||||
(end_date, dividend_start),
|
||||
).fetchone()[0]
|
||||
fundamental_rows = connection.execute(
|
||||
"""
|
||||
SELECT COUNT(*) FROM fundamental_indicators fi
|
||||
INNER JOIN (
|
||||
SELECT ts_code, MAX(ann_date || ':' || end_date) AS latest_key
|
||||
FROM fundamental_indicators
|
||||
WHERE ann_date = '' OR ann_date <= ?
|
||||
GROUP BY ts_code
|
||||
) latest
|
||||
ON latest.ts_code = fi.ts_code
|
||||
AND latest.latest_key = (fi.ann_date || ':' || fi.end_date)
|
||||
""",
|
||||
(end_date,),
|
||||
).fetchone()[0]
|
||||
moneyflow_dates = connection.execute(
|
||||
"""
|
||||
SELECT COUNT(DISTINCT trade_date)
|
||||
FROM moneyflow_daily
|
||||
WHERE trade_date IN (
|
||||
SELECT DISTINCT trade_date
|
||||
FROM daily_bars
|
||||
WHERE trade_date <= ?
|
||||
ORDER BY trade_date DESC
|
||||
LIMIT 5
|
||||
)
|
||||
""",
|
||||
(end_date,),
|
||||
).fetchone()[0]
|
||||
earnings_rows = connection.execute(
|
||||
"""
|
||||
SELECT COUNT(*) FROM earnings_events
|
||||
WHERE ann_date <= ? AND ann_date >= replace(date(?, '-45 day'), '-', '')
|
||||
""",
|
||||
(end_date, f"{end_date[:4]}-{end_date[4:6]}-{end_date[6:8]}"),
|
||||
).fetchone()[0]
|
||||
popularity_rows = connection.execute(
|
||||
"SELECT COUNT(*) FROM popularity_factors WHERE trade_date = ?",
|
||||
(end_date,),
|
||||
).fetchone()[0]
|
||||
institution_rows = connection.execute(
|
||||
"SELECT COUNT(*) FROM lhb_institution_daily WHERE trade_date = ?",
|
||||
(end_date,),
|
||||
).fetchone()[0]
|
||||
return {
|
||||
"market": bool(market),
|
||||
"auction": bool(auction),
|
||||
"benchmark": int(benchmark_rows or 0) >= 60,
|
||||
"benchmark_rows": int(benchmark_rows or 0),
|
||||
"valuation": bool(valuation_available),
|
||||
"fundamental": int(fundamental_rows or 0) >= 100,
|
||||
"dividend_history": int(dividend_years or 0) >= 4,
|
||||
"valuation_rows": int(valuation_rows or 0),
|
||||
"fundamental_rows": int(fundamental_rows or 0),
|
||||
"dividend_years": int(dividend_years or 0),
|
||||
"moneyflow_history": int(moneyflow_dates or 0) >= 5,
|
||||
"moneyflow_dates": int(moneyflow_dates or 0),
|
||||
"earnings_events": int(earnings_rows or 0) > 0,
|
||||
"earnings_event_rows": int(earnings_rows or 0),
|
||||
"popularity": int(popularity_rows or 0) > 0,
|
||||
"popularity_rows": int(popularity_rows or 0),
|
||||
"institutions": int(institution_rows or 0) > 0,
|
||||
"institution_rows": int(institution_rows or 0),
|
||||
}
|
||||
|
||||
def load_factor_data(self, end_date: str, limit_dates: int = 80) -> dict[str, Any]:
|
||||
dates = self.factor_dates(end_date, limit_dates)
|
||||
if not dates:
|
||||
return {
|
||||
"dates": [], "bars": [], "master": [], "indicators": [],
|
||||
"indicator_history": [], "indicator_series": [], "fundamentals": [],
|
||||
"moneyflow": [], "moneyflow_history": [], "auction": [],
|
||||
"benchmarks": [], "fundamental_history": [],
|
||||
"earnings_events": [], "popularity": [], "institutions": [],
|
||||
}
|
||||
placeholders = ",".join("?" for _ in dates)
|
||||
with self.connect() as connection:
|
||||
bars = connection.execute(
|
||||
f"SELECT * FROM daily_bars WHERE trade_date IN ({placeholders}) ORDER BY trade_date, ts_code",
|
||||
dates,
|
||||
).fetchall()
|
||||
master = connection.execute("SELECT * FROM stock_master").fetchall()
|
||||
indicators = connection.execute(
|
||||
"""
|
||||
SELECT * FROM daily_indicators
|
||||
WHERE trade_date = (
|
||||
SELECT MAX(trade_date) FROM daily_indicators WHERE trade_date <= ?
|
||||
)
|
||||
""",
|
||||
(end_date,),
|
||||
).fetchall()
|
||||
indicator_history = connection.execute(
|
||||
"""
|
||||
SELECT di.* FROM daily_indicators di
|
||||
INNER JOIN (
|
||||
SELECT ts_code, substr(trade_date, 1, 4) AS year_key,
|
||||
MAX(trade_date) AS max_date
|
||||
FROM daily_indicators
|
||||
WHERE trade_date <= ? AND trade_date >= ?
|
||||
GROUP BY ts_code, substr(trade_date, 1, 4)
|
||||
) latest
|
||||
ON latest.ts_code = di.ts_code AND latest.max_date = di.trade_date
|
||||
ORDER BY di.trade_date, di.ts_code
|
||||
""",
|
||||
(end_date, str(max(0, int(end_date[:4] or 0) - 5)) + "0101"),
|
||||
).fetchall()
|
||||
indicator_series = connection.execute(
|
||||
f"""
|
||||
SELECT trade_date, ts_code, turnover_rate, volume_ratio,
|
||||
total_mv, circ_mv, pe_ttm, pb, ps_ttm, dv_ttm
|
||||
FROM daily_indicators
|
||||
WHERE trade_date IN ({placeholders})
|
||||
ORDER BY trade_date, ts_code
|
||||
""",
|
||||
dates,
|
||||
).fetchall()
|
||||
fundamentals = connection.execute(
|
||||
"""
|
||||
SELECT fi.* FROM fundamental_indicators fi
|
||||
INNER JOIN (
|
||||
SELECT ts_code, MAX(ann_date || ':' || end_date) AS latest_key
|
||||
FROM fundamental_indicators
|
||||
WHERE ann_date = '' OR ann_date <= ?
|
||||
GROUP BY ts_code
|
||||
) latest
|
||||
ON latest.ts_code = fi.ts_code
|
||||
AND latest.latest_key = (fi.ann_date || ':' || fi.end_date)
|
||||
""",
|
||||
(end_date,),
|
||||
).fetchall()
|
||||
fundamental_history = connection.execute(
|
||||
"""
|
||||
SELECT * FROM fundamental_indicators
|
||||
WHERE ann_date = '' OR ann_date <= ?
|
||||
ORDER BY ann_date, end_date, ts_code
|
||||
""",
|
||||
(end_date,),
|
||||
).fetchall()
|
||||
moneyflow = connection.execute(
|
||||
"""
|
||||
SELECT * FROM moneyflow_daily
|
||||
WHERE trade_date = (
|
||||
SELECT MAX(trade_date) FROM moneyflow_daily WHERE trade_date <= ?
|
||||
)
|
||||
""",
|
||||
(end_date,),
|
||||
).fetchall()
|
||||
flow_dates = dates[-min(5, len(dates)):]
|
||||
flow_placeholders = ",".join("?" for _ in flow_dates)
|
||||
moneyflow_history = connection.execute(
|
||||
f"""
|
||||
SELECT * FROM moneyflow_daily
|
||||
WHERE trade_date IN ({flow_placeholders})
|
||||
ORDER BY trade_date, ts_code
|
||||
""",
|
||||
flow_dates,
|
||||
).fetchall()
|
||||
auction = connection.execute(
|
||||
"""
|
||||
SELECT * FROM auction_factors
|
||||
WHERE trade_date = (
|
||||
SELECT MAX(trade_date) FROM auction_factors WHERE trade_date <= ?
|
||||
)
|
||||
""",
|
||||
(end_date,),
|
||||
).fetchall()
|
||||
benchmarks = connection.execute(
|
||||
f"""
|
||||
SELECT * FROM benchmark_bars
|
||||
WHERE ts_code = '000300.SH' AND trade_date IN ({placeholders})
|
||||
ORDER BY trade_date
|
||||
""",
|
||||
dates,
|
||||
).fetchall()
|
||||
earnings_events = connection.execute(
|
||||
"""
|
||||
SELECT * FROM earnings_events
|
||||
WHERE ann_date <= ?
|
||||
ORDER BY ann_date, end_date, ts_code
|
||||
""",
|
||||
(end_date,),
|
||||
).fetchall()
|
||||
popularity = connection.execute(
|
||||
"SELECT * FROM popularity_factors WHERE trade_date = ? ORDER BY ts_code",
|
||||
(end_date,),
|
||||
).fetchall()
|
||||
institutions = connection.execute(
|
||||
"SELECT * FROM lhb_institution_daily WHERE trade_date = ? ORDER BY ts_code",
|
||||
(end_date,),
|
||||
).fetchall()
|
||||
return {
|
||||
"dates": dates,
|
||||
"bars": [dict(row) for row in bars],
|
||||
"master": [dict(row) for row in master],
|
||||
"indicators": [dict(row) for row in indicators],
|
||||
"indicator_history": [dict(row) for row in indicator_history],
|
||||
"indicator_series": [dict(row) for row in indicator_series],
|
||||
"fundamentals": [dict(row) for row in fundamentals],
|
||||
"fundamental_history": [dict(row) for row in fundamental_history],
|
||||
"moneyflow": [dict(row) for row in moneyflow],
|
||||
"moneyflow_history": [dict(row) for row in moneyflow_history],
|
||||
"auction": [dict(row) for row in auction],
|
||||
"benchmarks": [dict(row) for row in benchmarks],
|
||||
"earnings_events": [dict(row) for row in earnings_events],
|
||||
"popularity": [dict(row) for row in popularity],
|
||||
"institutions": [dict(row) for row in institutions],
|
||||
}
|
||||
|
||||
def snapshot_summaries(self, end_date: str, limit: int = 10) -> list[dict[str, Any]]:
|
||||
try:
|
||||
from sentiment_engine import build_sentiment_history
|
||||
except ModuleNotFoundError:
|
||||
from .sentiment_engine import build_sentiment_history
|
||||
|
||||
series = build_sentiment_history(self.list_snapshot_payloads(end_date, 260))
|
||||
return [
|
||||
{
|
||||
"trade_date": row["trade_date"],
|
||||
"sentiment_score": row["score"],
|
||||
"seal_rate": row["seal_rate"],
|
||||
"limit_up_count": row["limit_up_count"],
|
||||
"limit_down_count": row["limit_down_count"],
|
||||
"broken_count": row["broken_count"],
|
||||
"up_count": row["up_count"],
|
||||
"down_count": row["down_count"],
|
||||
"amount_billion": row["amount_billion"],
|
||||
}
|
||||
for row in series[-limit:]
|
||||
]
|
||||
|
||||
def save_screener_strategy(
|
||||
self, user_id: int | None, name: str, description: str, regimes: list[str], formula: dict[str, Any],
|
||||
builtin: bool = False, strategy_id: int | None = None,
|
||||
) -> int:
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
regimes_json = json.dumps(regimes, ensure_ascii=False)
|
||||
formula_json = json.dumps(formula, ensure_ascii=False, separators=(",", ":"))
|
||||
with self.connect() as connection:
|
||||
if strategy_id:
|
||||
if builtin:
|
||||
cursor = connection.execute(
|
||||
"""
|
||||
UPDATE screener_strategies SET name=?, description=?, regimes=?, formula=?,
|
||||
builtin=1, user_id=NULL, updated_at=? WHERE id=? AND builtin=1
|
||||
""",
|
||||
(name, description, regimes_json, formula_json, now, strategy_id),
|
||||
)
|
||||
else:
|
||||
cursor = connection.execute(
|
||||
"""
|
||||
UPDATE screener_strategies SET name=?, description=?, regimes=?, formula=?,
|
||||
updated_at=? WHERE id=? AND builtin=0 AND user_id=?
|
||||
""",
|
||||
(name, description, regimes_json, formula_json, now, strategy_id, int(user_id or 0)),
|
||||
)
|
||||
if cursor.rowcount == 0:
|
||||
raise ValueError("选股策略不存在。")
|
||||
return strategy_id
|
||||
cursor = connection.execute(
|
||||
"""
|
||||
INSERT INTO screener_strategies
|
||||
(user_id, name, description, regimes, formula, builtin, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(None if builtin else int(user_id or 0), name, description, regimes_json, formula_json, int(builtin), now, now),
|
||||
)
|
||||
return int(cursor.lastrowid)
|
||||
|
||||
def list_screener_strategies(self, user_id: int | None = None) -> list[dict[str, Any]]:
|
||||
with self.connect() as connection:
|
||||
if user_id is None:
|
||||
rows = connection.execute(
|
||||
"SELECT * FROM screener_strategies WHERE builtin = 1 ORDER BY updated_at DESC, id"
|
||||
).fetchall()
|
||||
else:
|
||||
rows = connection.execute(
|
||||
"""
|
||||
SELECT * FROM screener_strategies
|
||||
WHERE builtin = 1 OR user_id = ?
|
||||
ORDER BY builtin DESC, updated_at DESC, id
|
||||
""",
|
||||
(int(user_id),),
|
||||
).fetchall()
|
||||
result = []
|
||||
for row in rows:
|
||||
item = dict(row)
|
||||
item["regimes"] = json.loads(item["regimes"])
|
||||
item["formula"] = json.loads(item["formula"])
|
||||
item["builtin"] = bool(item["builtin"])
|
||||
result.append(item)
|
||||
return result
|
||||
|
||||
def delete_screener_strategy(self, user_id: int, strategy_id: int) -> bool:
|
||||
with self.connect() as connection:
|
||||
row = connection.execute(
|
||||
"SELECT builtin, user_id FROM screener_strategies WHERE id = ?",
|
||||
(strategy_id,),
|
||||
).fetchone()
|
||||
if not row:
|
||||
raise ValueError("选股策略不存在。")
|
||||
if bool(row["builtin"]):
|
||||
raise ValueError("内置策略不能删除。")
|
||||
if int(row["user_id"] or 0) != int(user_id):
|
||||
raise ValueError("无权删除其他账号的策略。")
|
||||
cursor = connection.execute(
|
||||
"DELETE FROM screener_strategies WHERE id = ? AND builtin = 0 AND user_id = ?",
|
||||
(strategy_id, int(user_id)),
|
||||
)
|
||||
return cursor.rowcount > 0
|
||||
|
||||
def save_screener_run(
|
||||
self, user_id: int, trade_date: str, regime: str, strategy_name: str,
|
||||
formula: dict[str, Any], result: dict[str, Any], mode: str = "smart",
|
||||
) -> int:
|
||||
normalized_mode = mode if mode in {"smart", "curated", "quant"} else "smart"
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"""
|
||||
INSERT INTO screener_runs
|
||||
(user_id, trade_date, regime, mode, strategy_name, formula, result, created_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(None if int(user_id) == 0 else int(user_id), trade_date, regime,
|
||||
normalized_mode, strategy_name,
|
||||
json.dumps(formula, ensure_ascii=False, separators=(",", ":")),
|
||||
json.dumps(result, ensure_ascii=False, separators=(",", ":")), now),
|
||||
)
|
||||
return int(cursor.lastrowid)
|
||||
|
||||
@staticmethod
|
||||
def _screener_run_payload(row: sqlite3.Row) -> dict[str, Any] | None:
|
||||
try:
|
||||
result = json.loads(row["result"])
|
||||
except json.JSONDecodeError:
|
||||
return None
|
||||
result.setdefault("meta", {}).update(
|
||||
{
|
||||
"run_id": int(row["id"]),
|
||||
"trade_date": str(row["trade_date"] or ""),
|
||||
"regime": str(row["regime"] or ""),
|
||||
"mode": str(row["mode"] or "smart"),
|
||||
"strategy_name": str(row["strategy_name"] or ""),
|
||||
"created_at": row["created_at"],
|
||||
}
|
||||
)
|
||||
return result
|
||||
|
||||
def latest_screener_run(
|
||||
self, user_id: int, trade_date: str, mode: str = "",
|
||||
) -> dict[str, Any] | None:
|
||||
owner_clause = "user_id IS NULL" if int(user_id) == 0 else "user_id = ?"
|
||||
parameters: tuple[Any, ...] = () if int(user_id) == 0 else (int(user_id),)
|
||||
parameters += (trade_date,)
|
||||
mode_clause = ""
|
||||
if mode in {"smart", "curated", "quant"}:
|
||||
mode_clause = " AND mode = ?"
|
||||
parameters += (mode,)
|
||||
with self.connect() as connection:
|
||||
row = connection.execute(
|
||||
f"""
|
||||
SELECT id, trade_date, regime, mode, strategy_name, result, created_at
|
||||
FROM screener_runs
|
||||
WHERE {owner_clause} AND trade_date <= ?{mode_clause}
|
||||
ORDER BY id DESC LIMIT 1
|
||||
""",
|
||||
parameters,
|
||||
).fetchone()
|
||||
return self._screener_run_payload(row) if row else None
|
||||
|
||||
def latest_screener_runs(self, user_id: int, trade_date: str) -> dict[str, dict[str, Any]]:
|
||||
owner_clause = "user_id IS NULL" if int(user_id) == 0 else "user_id = ?"
|
||||
parameters: tuple[Any, ...] = () if int(user_id) == 0 else (int(user_id),)
|
||||
parameters += (trade_date,)
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
f"""
|
||||
SELECT runs.id, runs.trade_date, runs.regime, runs.mode,
|
||||
runs.strategy_name, runs.result, runs.created_at
|
||||
FROM screener_runs runs
|
||||
INNER JOIN (
|
||||
SELECT mode, MAX(id) AS id
|
||||
FROM screener_runs
|
||||
WHERE {owner_clause} AND trade_date <= ?
|
||||
GROUP BY mode
|
||||
) latest ON latest.id = runs.id
|
||||
""",
|
||||
parameters,
|
||||
).fetchall()
|
||||
results: dict[str, dict[str, Any]] = {}
|
||||
for row in rows:
|
||||
mode = str(row["mode"] or "smart")
|
||||
payload = self._screener_run_payload(row)
|
||||
if mode in {"smart", "curated", "quant"} and payload:
|
||||
results[mode] = payload
|
||||
return results
|
||||
|
||||
def latest_screener_context_runs(
|
||||
self, user_id: int, trade_date: str, limit: int = 60,
|
||||
) -> list[dict[str, Any]]:
|
||||
safe_limit = max(1, min(120, int(limit)))
|
||||
owner_clause = "user_id IS NULL" if int(user_id) == 0 else "user_id = ?"
|
||||
parameters: tuple[Any, ...] = () if int(user_id) == 0 else (int(user_id),)
|
||||
parameters += (trade_date, safe_limit)
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
f"""
|
||||
WITH ranked AS (
|
||||
SELECT id, trade_date, regime, mode, strategy_name, result, created_at,
|
||||
ROW_NUMBER() OVER (
|
||||
PARTITION BY
|
||||
mode,
|
||||
CASE WHEN mode = 'smart' THEN regime ELSE '' END,
|
||||
CASE WHEN mode IN ('smart', 'curated') THEN strategy_name ELSE '' END
|
||||
ORDER BY id DESC
|
||||
) AS context_rank
|
||||
FROM screener_runs
|
||||
WHERE {owner_clause} AND trade_date <= ?
|
||||
)
|
||||
SELECT id, trade_date, regime, mode, strategy_name, result, created_at
|
||||
FROM ranked
|
||||
WHERE context_rank = 1
|
||||
ORDER BY id DESC
|
||||
LIMIT ?
|
||||
""",
|
||||
parameters,
|
||||
).fetchall()
|
||||
return [
|
||||
payload
|
||||
for row in rows
|
||||
if (payload := self._screener_run_payload(row)) is not None
|
||||
]
|
||||
|
||||
def screener_runs_for_date(
|
||||
self, user_id: int, trade_date: str, limit: int = 80,
|
||||
) -> list[dict[str, Any]]:
|
||||
safe_limit = max(1, min(160, int(limit)))
|
||||
owner_clause = "user_id IS NULL" if int(user_id) == 0 else "user_id = ?"
|
||||
parameters: tuple[Any, ...] = () if int(user_id) == 0 else (int(user_id),)
|
||||
parameters += (trade_date, safe_limit)
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
f"""
|
||||
SELECT id, trade_date, regime, mode, strategy_name, result, created_at
|
||||
FROM screener_runs
|
||||
WHERE {owner_clause} AND trade_date = ?
|
||||
ORDER BY id DESC
|
||||
LIMIT ?
|
||||
""",
|
||||
parameters,
|
||||
).fetchall()
|
||||
result = []
|
||||
seen: set[tuple[str, str, str]] = set()
|
||||
for row in rows:
|
||||
key = (
|
||||
str(row["mode"] or "smart"),
|
||||
str(row["regime"] or ""),
|
||||
str(row["strategy_name"] or ""),
|
||||
)
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
payload = self._screener_run_payload(row)
|
||||
if payload is not None:
|
||||
result.append(payload)
|
||||
return result
|
||||
|
||||
def get_screener_run(self, user_id: int, run_id: int) -> dict[str, Any] | None:
|
||||
owner_clause = "user_id IS NULL" if int(user_id) == 0 else "user_id = ?"
|
||||
parameters: tuple[Any, ...] = (int(run_id),)
|
||||
if int(user_id) != 0:
|
||||
parameters += (int(user_id),)
|
||||
with self.connect() as connection:
|
||||
row = connection.execute(
|
||||
f"""
|
||||
SELECT id, trade_date, regime, mode, strategy_name, result, created_at
|
||||
FROM screener_runs WHERE id = ? AND {owner_clause}
|
||||
""",
|
||||
parameters,
|
||||
).fetchone()
|
||||
if not row:
|
||||
return None
|
||||
result = self._screener_run_payload(row)
|
||||
if result is None:
|
||||
return None
|
||||
result.setdefault("meta", {}).update(
|
||||
{
|
||||
"run_id": int(row["id"]),
|
||||
"trade_date": row["trade_date"],
|
||||
"mode": str(row["mode"] or "smart"),
|
||||
"created_at": row["created_at"],
|
||||
}
|
||||
)
|
||||
result["strategy_name"] = row["strategy_name"]
|
||||
result["regime"] = row["regime"]
|
||||
return result
|
||||
|
||||
def save_strategy_tracks(
|
||||
self,
|
||||
user_id: int,
|
||||
run_id: int,
|
||||
selection_date: str,
|
||||
strategy_name: str,
|
||||
candidates: list[dict[str, Any]],
|
||||
) -> int:
|
||||
now = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
values = []
|
||||
for item in candidates:
|
||||
ts_code = str(item.get("ts_code") or "").strip()
|
||||
code = str(item.get("code") or ts_code.split(".")[0]).strip()
|
||||
entry_price = float(item.get("price") or 0)
|
||||
if not ts_code or not code or entry_price <= 0:
|
||||
continue
|
||||
values.append(
|
||||
(
|
||||
int(user_id), int(run_id), selection_date, strategy_name, ts_code, code,
|
||||
str(item.get("name") or "--"), str(item.get("sector") or "其他"),
|
||||
entry_price, now, now,
|
||||
)
|
||||
)
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO strategy_tracks
|
||||
(user_id, run_id, selection_date, strategy_name, ts_code, code,
|
||||
name, sector, entry_price, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(user_id, run_id, ts_code) DO UPDATE SET
|
||||
name=excluded.name, sector=excluded.sector,
|
||||
entry_price=excluded.entry_price, updated_at=excluded.updated_at
|
||||
""",
|
||||
values,
|
||||
)
|
||||
return len(values)
|
||||
|
||||
def list_strategy_tracks(self, user_id: int, limit_batches: int = 12) -> list[dict[str, Any]]:
|
||||
limit_batches = max(1, min(50, int(limit_batches)))
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"""
|
||||
SELECT * FROM strategy_tracks
|
||||
WHERE user_id = ? AND run_id IN (
|
||||
SELECT run_id FROM strategy_tracks WHERE user_id = ?
|
||||
GROUP BY run_id ORDER BY run_id DESC LIMIT ?
|
||||
)
|
||||
ORDER BY run_id DESC, id
|
||||
""",
|
||||
(int(user_id), int(user_id), limit_batches),
|
||||
).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
def delete_strategy_track(self, user_id: int, track_id: int) -> bool:
|
||||
with self.connect() as connection:
|
||||
cursor = connection.execute(
|
||||
"DELETE FROM strategy_tracks WHERE id = ? AND user_id = ?",
|
||||
(int(track_id), int(user_id)),
|
||||
)
|
||||
return cursor.rowcount > 0
|
||||
|
||||
def load_tracking_bars(
|
||||
self, targets: list[tuple[str, str]], limit: int = 5
|
||||
) -> dict[tuple[str, str], list[dict[str, Any]]]:
|
||||
unique_targets = set(targets)
|
||||
if not unique_targets:
|
||||
return {}
|
||||
codes = sorted({ts_code for ts_code, _ in unique_targets})
|
||||
earliest_date = min(selection_date for _, selection_date in unique_targets)
|
||||
placeholders = ",".join("?" for _ in codes)
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
f"""
|
||||
SELECT ts_code, trade_date, open, high, low, close FROM daily_bars
|
||||
WHERE ts_code IN ({placeholders}) AND trade_date > ?
|
||||
ORDER BY ts_code, trade_date
|
||||
""",
|
||||
[*codes, earliest_date],
|
||||
).fetchall()
|
||||
by_code: dict[str, list[dict[str, Any]]] = {}
|
||||
for row in rows:
|
||||
item = dict(row)
|
||||
by_code.setdefault(str(item["ts_code"]), []).append(item)
|
||||
row_limit = max(1, min(20, int(limit)))
|
||||
return {
|
||||
(ts_code, selection_date): [
|
||||
row for row in by_code.get(ts_code, []) if row["trade_date"] > selection_date
|
||||
][:row_limit]
|
||||
for ts_code, selection_date in unique_targets
|
||||
}
|
||||
@@ -0,0 +1,435 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import re
|
||||
from datetime import date, datetime
|
||||
from typing import Any
|
||||
|
||||
from backend.bootstrap.config import normalize_date, validate_text
|
||||
from backend.data.providers.tushare_client import TushareError
|
||||
from backend.llm import LLMGatewayError
|
||||
from backend.features.screener.compiler import (
|
||||
LLMCompilerError,
|
||||
compile_strategy_with_llm,
|
||||
)
|
||||
from backend.features.screener.engine import (
|
||||
FACTOR_FIELDS,
|
||||
FACTOR_GROUPS,
|
||||
REGIMES,
|
||||
FactorDataService,
|
||||
compile_local_strategy,
|
||||
)
|
||||
|
||||
|
||||
SCREENER_LIBRARY_VERSION = 8
|
||||
|
||||
|
||||
def automatic_screener_jobs(
|
||||
strategies: list[dict[str, Any]], regime_id: str
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Build the close-of-day jobs; only stage screening is regime-gated."""
|
||||
smart_strategy = next(
|
||||
(
|
||||
item for item in strategies
|
||||
if item.get("formula", {}).get("meta", {}).get("library") != "curated"
|
||||
and regime_id in (item.get("regimes") or [])
|
||||
),
|
||||
None,
|
||||
)
|
||||
curated = [
|
||||
item for item in strategies
|
||||
if item.get("formula", {}).get("meta", {}).get("library") == "curated"
|
||||
]
|
||||
jobs = ([{"mode": "smart", "strategy": smart_strategy}] if smart_strategy else [])
|
||||
jobs.extend({"mode": "curated", "strategy": item} for item in curated)
|
||||
return jobs
|
||||
|
||||
|
||||
class ScreenerServiceMixin:
|
||||
@staticmethod
|
||||
def _strategy_missing_data(
|
||||
strategy: dict[str, Any], factor_dates: list[str], factor_health: dict[str, Any]
|
||||
) -> list[str]:
|
||||
formula = strategy.get("formula") or {}
|
||||
meta = formula.get("meta") or {}
|
||||
used_fields = {
|
||||
str(item.get("field") or "")
|
||||
for item in list(formula.get("filters") or []) + list(formula.get("score") or [])
|
||||
}
|
||||
valuation_fields = {"pe_ttm", "pb", "ps_ttm", "dividend_yield_ttm", "total_mv_billion"}
|
||||
fundamental_fields = {"roe", "roa", "roic", "gross_margin", "netprofit_yoy", "revenue_yoy", "ocf_to_opincome"}
|
||||
auction_fields = {"auction_change", "auction_amount_million", "auction_turnover_rate", "auction_volume_ratio"}
|
||||
missing = []
|
||||
required_history = max(21, min(260, int(meta.get("history_days") or 21)))
|
||||
if len(factor_dates) < required_history:
|
||||
missing.append(f"历史行情(需{required_history}日)")
|
||||
if used_fields & valuation_fields and not factor_health["valuation"]:
|
||||
missing.append("估值数据")
|
||||
if used_fields & fundamental_fields and not factor_health["fundamental"]:
|
||||
missing.append("财务质量")
|
||||
if meta.get("requires_valuation") and not factor_health["valuation"]:
|
||||
missing.append("估值数据")
|
||||
if meta.get("requires_fundamental") and not factor_health["fundamental"]:
|
||||
missing.append("财务质量")
|
||||
if "dividend_years" in used_fields and not factor_health["dividend_history"]:
|
||||
missing.append("历年分红")
|
||||
if used_fields & auction_fields and not factor_health["auction"]:
|
||||
missing.append("竞价数据")
|
||||
if meta.get("requires_benchmark") and not factor_health.get("benchmark"):
|
||||
missing.append("沪深300基准")
|
||||
if meta.get("requires_moneyflow_history") and not factor_health.get("moneyflow_history"):
|
||||
missing.append("近5日资金流")
|
||||
if meta.get("requires_earnings_events") and not factor_health.get("earnings_events"):
|
||||
missing.append("业绩预告与快报")
|
||||
if meta.get("requires_popularity") and not factor_health.get("popularity"):
|
||||
missing.append("当日人气榜")
|
||||
if meta.get("requires_institutions") and not factor_health.get("institutions"):
|
||||
missing.append("龙虎榜机构席位")
|
||||
return list(dict.fromkeys(missing))
|
||||
|
||||
def screener_setup(self, trade_date: str) -> dict[str, Any]:
|
||||
normalized_date = normalize_date(trade_date)
|
||||
regime = self.screener.detect_regime(normalized_date)
|
||||
factor_dates = self.database.factor_dates(normalized_date, 300)
|
||||
auction_dates = self.database.auction_factor_dates(normalized_date, 100)
|
||||
factor_health = self.screener.factor_health(normalized_date)
|
||||
strategies = self.database.list_screener_strategies(self.current_user_id)
|
||||
for strategy in strategies:
|
||||
missing = self._strategy_missing_data(strategy, factor_dates, factor_health)
|
||||
strategy["data_ready"] = not missing
|
||||
strategy["missing_data"] = missing
|
||||
automatic_results = self.database.screener_runs_for_date(0, normalized_date)
|
||||
personal_results = self.database.screener_runs_for_date(
|
||||
self.current_user_id, normalized_date
|
||||
)
|
||||
recent_results = [
|
||||
*[item for item in automatic_results if item.get("meta", {}).get("mode") in {"smart", "curated"}],
|
||||
*[item for item in personal_results if item.get("meta", {}).get("mode") == "quant"],
|
||||
]
|
||||
latest_results: dict[str, dict[str, Any]] = {}
|
||||
for result in reversed(recent_results):
|
||||
mode = str(result.get("meta", {}).get("mode") or "smart")
|
||||
latest_results[mode] = result
|
||||
automatic_status = self.database.get_data_snapshot(
|
||||
"screener_auto_v1", normalized_date
|
||||
) or {}
|
||||
return {
|
||||
"trade_date": normalized_date,
|
||||
"regime": regime,
|
||||
"regimes": [{"id": key, "label": value} for key, value in REGIMES.items()],
|
||||
"strategies": strategies,
|
||||
"factor_fields": [{"id": key, "label": value} for key, value in FACTOR_FIELDS.items()],
|
||||
"factor_groups": [
|
||||
{
|
||||
"name": name,
|
||||
"fields": [{"id": field, "label": FACTOR_FIELDS[field]} for field in fields],
|
||||
}
|
||||
for name, fields in FACTOR_GROUPS.items()
|
||||
],
|
||||
"operators": [">", ">=", "<", "<=", "==", "between"],
|
||||
"factor_data": {
|
||||
"date_count": len(factor_dates),
|
||||
"start_date": factor_dates[0] if factor_dates else "",
|
||||
"end_date": factor_dates[-1] if factor_dates else "",
|
||||
"ready": len(factor_dates) >= 21,
|
||||
"auction_date_count": len(auction_dates),
|
||||
"auction_ready": bool(auction_dates and auction_dates[-1] == factor_dates[-1]) if factor_dates else False,
|
||||
"health": factor_health,
|
||||
},
|
||||
"llm": {
|
||||
"configured": self.llm_configured,
|
||||
"model": self.llm_primary_model if self.llm_configured else "",
|
||||
"fallback_configured": self.llm_fallback_configured,
|
||||
"fallback_model": self.llm_fallback_model if self.llm_fallback_configured else "",
|
||||
},
|
||||
"latest_results": latest_results,
|
||||
"recent_results": recent_results,
|
||||
"automatic_status": automatic_status,
|
||||
# Kept during the client transition for compatibility with older frontends.
|
||||
"latest_result": latest_results.get("smart"),
|
||||
}
|
||||
|
||||
def screener_tracking(self, limit: int = 12) -> dict[str, Any]:
|
||||
return self.strategy_tracking.list_tracking(self.current_user_id, limit)
|
||||
|
||||
def add_screener_tracking(self, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
try:
|
||||
run_id = int(payload.get("run_id") or 0)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError("选股批次无效。") from exc
|
||||
code = str(payload.get("code") or "").strip()
|
||||
if run_id <= 0 or not re.fullmatch(r"\d{6}", code):
|
||||
raise ValueError("选股批次或股票代码无效。")
|
||||
return self.strategy_tracking.add_candidate(self.current_user_id, run_id, code)
|
||||
|
||||
def remove_screener_tracking(self, track_id: int) -> dict[str, Any]:
|
||||
return self.strategy_tracking.remove_candidate(self.current_user_id, track_id)
|
||||
|
||||
def refresh_screener_tracking(self, trade_date: str) -> dict[str, Any]:
|
||||
normalized_date = normalize_date(trade_date)
|
||||
notice = ""
|
||||
if self.configured:
|
||||
try:
|
||||
FactorDataService(self.database, self._tushare_client()).sync(
|
||||
normalized_date, 15
|
||||
)
|
||||
except TushareError:
|
||||
notice = "最新日线暂未补齐,已按现有数据更新跟踪。"
|
||||
else:
|
||||
notice = "公共行情尚未配置,已按现有数据更新跟踪。"
|
||||
return {
|
||||
"tracking": self.screener_tracking(),
|
||||
"notice": notice,
|
||||
}
|
||||
|
||||
def sync_screener_data(self, trade_date: str, lookback: int = 45) -> dict[str, Any]:
|
||||
if not self.configured:
|
||||
raise ValueError("请先配置 Tushare Token。")
|
||||
normalized_date = normalize_date(trade_date)
|
||||
lookback = max(25, min(260, int(lookback)))
|
||||
with self.sync_lock:
|
||||
return FactorDataService(self.database, self._tushare_client()).sync(
|
||||
normalized_date, lookback
|
||||
)
|
||||
|
||||
def _schedule_automatic_screeners(
|
||||
self, trade_date: str, snapshot: dict[str, Any] | None = None
|
||||
) -> bool:
|
||||
normalized_date = normalize_date(trade_date)
|
||||
now = datetime.now().astimezone()
|
||||
if (
|
||||
normalized_date != now.strftime("%Y%m%d")
|
||||
or now.weekday() >= 5
|
||||
or now.time().replace(tzinfo=None) < datetime.strptime("15:10", "%H:%M").time()
|
||||
or self.auto_screener_lock.locked()
|
||||
):
|
||||
return False
|
||||
snapshot = snapshot or self.database.get_snapshot(normalized_date) or {}
|
||||
actual_date = str((snapshot.get("meta") or {}).get("trade_date") or "").replace("-", "")
|
||||
if actual_date != normalized_date:
|
||||
return False
|
||||
marker = self.database.get_data_snapshot("screener_auto_v1", normalized_date) or {}
|
||||
if (
|
||||
marker.get("status") == "complete"
|
||||
and int(marker.get("library_version") or 0) == SCREENER_LIBRARY_VERSION
|
||||
):
|
||||
return False
|
||||
last_attempt = self._auto_screener_last_attempt.get(normalized_date)
|
||||
if last_attempt and (now - last_attempt).total_seconds() < 600:
|
||||
return False
|
||||
self._auto_screener_last_attempt[normalized_date] = now
|
||||
return self.jobs.submit(
|
||||
"screener.automatic",
|
||||
f"{normalized_date}:v{SCREENER_LIBRARY_VERSION}",
|
||||
lambda: self.run_automatic_screeners(normalized_date),
|
||||
{"trade_date": normalized_date, "trigger": "post-close"},
|
||||
)
|
||||
|
||||
def run_automatic_screeners(self, trade_date: str) -> dict[str, Any]:
|
||||
normalized_date = normalize_date(trade_date)
|
||||
with self.auto_screener_lock:
|
||||
started_at = datetime.now().astimezone().isoformat(timespec="seconds")
|
||||
status: dict[str, Any] = {
|
||||
"trade_date": normalized_date,
|
||||
"library_version": SCREENER_LIBRARY_VERSION,
|
||||
"status": "running",
|
||||
"started_at": started_at,
|
||||
"completed": [],
|
||||
"skipped": [],
|
||||
"failed": [],
|
||||
}
|
||||
self.database.save_data_snapshot(
|
||||
"screener_auto_v1", normalized_date, "system", status
|
||||
)
|
||||
try:
|
||||
factor_sync = FactorDataService(
|
||||
self.database, self._tushare_client()
|
||||
).sync(normalized_date, 260)
|
||||
factor_dates = self.database.factor_dates(normalized_date, 300)
|
||||
if not factor_dates or factor_dates[-1] != normalized_date:
|
||||
raise ValueError("当日收盘行情尚未入库")
|
||||
factor_health = self.screener.factor_health(normalized_date)
|
||||
regime = self.screener.detect_regime(normalized_date)
|
||||
regime_id = str(regime.get("id") or "repair")
|
||||
strategies = self.database.list_screener_strategies(None)
|
||||
jobs = automatic_screener_jobs(strategies, regime_id)
|
||||
existing = {
|
||||
(
|
||||
str(item.get("meta", {}).get("mode") or "smart"),
|
||||
str(item.get("meta", {}).get("strategy_name") or ""),
|
||||
)
|
||||
for item in self.database.screener_runs_for_date(0, normalized_date)
|
||||
if int(item.get("meta", {}).get("library_version") or 0)
|
||||
== SCREENER_LIBRARY_VERSION
|
||||
}
|
||||
required_history = max(
|
||||
[
|
||||
int((job["strategy"].get("formula", {}).get("meta", {}) or {}).get("history_days") or 80)
|
||||
for job in jobs if job.get("strategy")
|
||||
] or [80]
|
||||
)
|
||||
factors, actual_date = self.screener.build_factors(
|
||||
normalized_date, history_days=required_history
|
||||
)
|
||||
if actual_date != normalized_date:
|
||||
raise ValueError("当日因子尚未完成收盘定格")
|
||||
for job in jobs:
|
||||
strategy = job["strategy"]
|
||||
mode = str(job["mode"])
|
||||
name = str(strategy.get("name") or "未命名策略")
|
||||
if (mode, name) in existing:
|
||||
status["completed"].append({"mode": mode, "name": name, "cached": True})
|
||||
continue
|
||||
missing = self._strategy_missing_data(
|
||||
strategy, factor_dates, factor_health
|
||||
)
|
||||
if missing:
|
||||
status["skipped"].append(
|
||||
{"mode": mode, "name": name, "reason": "、".join(missing)}
|
||||
)
|
||||
continue
|
||||
try:
|
||||
formula = copy.deepcopy(strategy.get("formula") or {})
|
||||
formula.setdefault("meta", {})["library_version"] = (
|
||||
SCREENER_LIBRARY_VERSION
|
||||
)
|
||||
result = self.screener.screen(
|
||||
0,
|
||||
normalized_date,
|
||||
formula,
|
||||
regime_id,
|
||||
name,
|
||||
False,
|
||||
None,
|
||||
mode,
|
||||
factors,
|
||||
actual_date,
|
||||
)
|
||||
status["completed"].append(
|
||||
{
|
||||
"mode": mode,
|
||||
"name": name,
|
||||
"candidate_count": len(result.get("candidates") or []),
|
||||
}
|
||||
)
|
||||
except Exception as exc:
|
||||
status["failed"].append(
|
||||
{"mode": mode, "name": name, "reason": str(exc)}
|
||||
)
|
||||
status.update(
|
||||
{
|
||||
"status": "complete" if not status["failed"] else "partial",
|
||||
"finished_at": datetime.now().astimezone().isoformat(timespec="seconds"),
|
||||
"factor_sync": factor_sync,
|
||||
"regime": regime,
|
||||
}
|
||||
)
|
||||
except Exception as exc:
|
||||
status.update(
|
||||
{
|
||||
"status": "failed",
|
||||
"finished_at": datetime.now().astimezone().isoformat(timespec="seconds"),
|
||||
"error": str(exc),
|
||||
}
|
||||
)
|
||||
self.database.save_data_snapshot(
|
||||
"screener_auto_v1", normalized_date, "system", status
|
||||
)
|
||||
return status
|
||||
|
||||
def compile_screener_strategy(self, prompt: str, regime: str) -> dict[str, Any]:
|
||||
prompt = prompt.strip()
|
||||
if not prompt or len(prompt) > 3000:
|
||||
raise ValueError("策略描述应为 1 至 3000 个字符。")
|
||||
if regime not in REGIMES:
|
||||
raise ValueError("市场阶段不支持。")
|
||||
notice = ""
|
||||
source = self.llm_source
|
||||
if source == "platform":
|
||||
try:
|
||||
gateway_result = self.llm_gateway.call(
|
||||
"screener",
|
||||
"strategy-compiler-v1",
|
||||
lambda profile: compile_strategy_with_llm(
|
||||
prompt,
|
||||
regime,
|
||||
profile.api_key,
|
||||
profile.base_url,
|
||||
profile.model,
|
||||
),
|
||||
(LLMCompilerError,),
|
||||
)
|
||||
compiled = gateway_result.value
|
||||
if gateway_result.role == "fallback":
|
||||
compiled["compiler"] = "llm_fallback"
|
||||
notice = "智能策略生成服务已自动切换。"
|
||||
except LLMGatewayError as exc:
|
||||
if exc.code != "unavailable":
|
||||
raise
|
||||
compiled = compile_local_strategy(prompt, regime)
|
||||
notice = "智能策略生成暂不可用,已使用本地模板。"
|
||||
else:
|
||||
compiled = compile_local_strategy(prompt, regime)
|
||||
notice = "智能策略生成暂不可用,已使用本地模板。"
|
||||
compiled["formula"] = self.screener.validate_formula(compiled["formula"])
|
||||
compiled["notice"] = notice
|
||||
return compiled
|
||||
|
||||
def save_screener_strategy(self, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
name = validate_text(payload.get("name"), "策略名称", 60, required=True)
|
||||
description = validate_text(payload.get("description"), "策略说明", 1000)
|
||||
regimes = payload.get("regimes") or []
|
||||
if not isinstance(regimes, list) or not regimes or any(item not in REGIMES for item in regimes):
|
||||
raise ValueError("策略适用阶段不正确。")
|
||||
formula = self.screener.validate_formula(payload.get("formula") or {})
|
||||
strategy_id = self.database.save_screener_strategy(
|
||||
self.current_user_id, name, description, regimes, formula
|
||||
)
|
||||
return {
|
||||
"id": strategy_id,
|
||||
"strategies": self.database.list_screener_strategies(self.current_user_id),
|
||||
}
|
||||
|
||||
def delete_screener_strategy(self, strategy_id: int) -> dict[str, Any]:
|
||||
deleted = self.database.delete_screener_strategy(self.current_user_id, strategy_id)
|
||||
return {
|
||||
"deleted": deleted,
|
||||
"strategies": self.database.list_screener_strategies(self.current_user_id),
|
||||
}
|
||||
|
||||
def run_screener(self, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
trade_date = normalize_date(str(payload.get("trade_date") or date.today().isoformat()))
|
||||
regime = str(payload.get("regime") or "")
|
||||
if regime not in REGIMES:
|
||||
raise ValueError("市场阶段不支持。")
|
||||
strategy_name = validate_text(payload.get("strategy_name"), "策略名称", 60, required=True)
|
||||
formula = payload.get("formula") or {}
|
||||
requested_mode = str(payload.get("mode") or "").strip()
|
||||
if requested_mode and requested_mode not in {"smart", "curated", "quant"}:
|
||||
raise ValueError("选股模式不受支持。")
|
||||
if requested_mode:
|
||||
mode = requested_mode
|
||||
else:
|
||||
meta = formula.get("meta") if isinstance(formula, dict) else {}
|
||||
library = str((meta or {}).get("library") or "")
|
||||
category = str((meta or {}).get("category") or "")
|
||||
if library == "curated":
|
||||
mode = "curated"
|
||||
elif library == "quant" or (library == "custom" and category == "量化公式"):
|
||||
mode = "quant"
|
||||
else:
|
||||
mode = "smart"
|
||||
realtime_snapshot = None
|
||||
dashboard = self.get_dashboard(trade_date)
|
||||
if self.configured and dashboard.get("meta", {}).get("realtime"):
|
||||
try:
|
||||
realtime_snapshot = self._tushare_client().realtime_factor_snapshot(trade_date)
|
||||
except TushareError as exc:
|
||||
raise ValueError(f"实时选股行情不可用,已停止筛选:{exc}") from exc
|
||||
result = self.screener.screen(
|
||||
self.current_user_id, trade_date, formula, regime, strategy_name,
|
||||
bool(payload.get("run_backtest", True)),
|
||||
realtime_snapshot,
|
||||
mode,
|
||||
)
|
||||
return result
|
||||
@@ -0,0 +1,486 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
|
||||
def _meta(
|
||||
category: str,
|
||||
quality: str,
|
||||
frequency: str,
|
||||
risk: str,
|
||||
data_group: str,
|
||||
history_days: int,
|
||||
backtest_days: int,
|
||||
take_profit: float,
|
||||
stop_loss: float,
|
||||
**extra: Any,
|
||||
) -> dict[str, Any]:
|
||||
return {
|
||||
"library": "curated",
|
||||
"category": category,
|
||||
"quality": quality,
|
||||
"frequency": frequency,
|
||||
"risk": risk,
|
||||
"data_group": data_group,
|
||||
"history_days": history_days,
|
||||
"backtest_days": backtest_days,
|
||||
"take_profit": take_profit,
|
||||
"stop_loss": stop_loss,
|
||||
**extra,
|
||||
}
|
||||
|
||||
|
||||
ADVANCED_CURATED_STRATEGIES = [
|
||||
{
|
||||
"name": "中期动量·强者恒强",
|
||||
"description": "用60日至5日前的中期动量识别持续强势,同时剔除当日无法正常成交的涨停标的。",
|
||||
"regimes": ["repair", "fermentation", "climax", "divergence"],
|
||||
"formula": {
|
||||
"meta": _meta("动量反转", "A-", "每周", "中", "历史行情", 80, 10, 8, -5),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 180},
|
||||
"filters": [
|
||||
{"field": "close", "op": "between", "value": [3, 100]},
|
||||
{"field": "momentum_60_5_rank", "op": ">=", "value": 0.90},
|
||||
{"field": "is_limit_up_today", "op": "==", "value": 0},
|
||||
],
|
||||
"score": [
|
||||
{"field": "momentum_60_5", "weight": 0.55, "direction": "desc"},
|
||||
{"field": "relative_strength", "weight": 0.25, "direction": "desc"},
|
||||
{"field": "amount_billion", "weight": 0.20, "direction": "desc"},
|
||||
],
|
||||
"limit": 25,
|
||||
"min_score": 0.50,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "强者回调",
|
||||
"description": "在中期强势股池中寻找回踩20日线、短期超卖且近20日无跌停的牛回头候选。",
|
||||
"regimes": ["repair", "fermentation", "divergence"],
|
||||
"formula": {
|
||||
"meta": _meta("动量反转", "A-", "每日", "中", "历史行情", 80, 10, 8, -5),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 180},
|
||||
"filters": [
|
||||
{"field": "momentum_60_5_rank", "op": ">=", "value": 0.70},
|
||||
{"field": "return_5d_rank", "op": "<=", "value": 0.20},
|
||||
{"field": "above_ma20", "op": "==", "value": 1},
|
||||
{"field": "rsi_6", "op": "<=", "value": 30},
|
||||
{"field": "no_limit_down_20d", "op": "==", "value": 1},
|
||||
],
|
||||
"score": [
|
||||
{"field": "momentum_60_5", "weight": 0.42, "direction": "desc"},
|
||||
{"field": "return_5d", "weight": 0.33, "direction": "asc"},
|
||||
{"field": "amount_billion", "weight": 0.25, "direction": "desc"},
|
||||
],
|
||||
"limit": 20,
|
||||
"min_score": 0.48,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "超跌反转",
|
||||
"description": "筛选短期极端回撤、充分换手但尚未形成长期单边下跌的修复候选。",
|
||||
"regimes": ["ice", "repair"],
|
||||
"formula": {
|
||||
"meta": _meta("动量反转", "B+", "每日", "高", "行情与财务", 80, 5, 8, -5),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 180},
|
||||
"filters": [
|
||||
{"field": "return_5d_rank", "op": "<=", "value": 0.05},
|
||||
{"field": "turnover_5d", "op": ">=", "value": 30},
|
||||
{"field": "return_60d", "op": ">=", "value": -40},
|
||||
{"field": "financial_risk", "op": "==", "value": 0},
|
||||
{"field": "is_limit_down_today", "op": "==", "value": 0},
|
||||
],
|
||||
"score": [
|
||||
{"field": "return_5d", "weight": 0.45, "direction": "asc"},
|
||||
{"field": "turnover_5d", "weight": 0.30, "direction": "desc"},
|
||||
{"field": "amount_billion", "weight": 0.25, "direction": "desc"},
|
||||
],
|
||||
"limit": 10,
|
||||
"min_score": 0.50,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "相对强度新高",
|
||||
"description": "以个股相对沪深300的强度线识别弱市领涨和结构性抱团标的。",
|
||||
"regimes": ["ice", "repair", "fermentation", "divergence"],
|
||||
"formula": {
|
||||
"meta": _meta("动量反转", "A", "每周", "中", "行情与指数", 130, 20, 12, -7, requires_benchmark=True),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 250},
|
||||
"filters": [
|
||||
{"field": "amount_billion", "op": ">=", "value": 1},
|
||||
{"field": "rs_high_120", "op": "==", "value": 1},
|
||||
{"field": "excess_return_60d", "op": ">=", "value": 10},
|
||||
{"field": "ma60_slope", "op": ">", "value": 0},
|
||||
],
|
||||
"score": [
|
||||
{"field": "excess_return_60d", "weight": 0.50, "direction": "desc"},
|
||||
{"field": "ma60_slope", "weight": 0.25, "direction": "desc"},
|
||||
{"field": "amount_billion", "weight": 0.25, "direction": "desc"},
|
||||
],
|
||||
"limit": 20,
|
||||
"min_score": 0.52,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "均线多头排列",
|
||||
"description": "使用5、10、20、60日均线多头结构、20日线斜率和250日位置确认趋势。",
|
||||
"regimes": ["repair", "fermentation", "climax", "divergence"],
|
||||
"formula": {
|
||||
"meta": _meta("趋势追踪", "A-", "每周", "中低", "历史行情", 260, 20, 12, -7),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 365},
|
||||
"filters": [
|
||||
{"field": "ma_bull_alignment", "op": "==", "value": 1},
|
||||
{"field": "ma20_slope_5d", "op": ">", "value": 0},
|
||||
{"field": "drawdown_from_high_250", "op": "<=", "value": 20},
|
||||
],
|
||||
"score": [
|
||||
{"field": "ma20_slope_5d", "weight": 0.38, "direction": "desc"},
|
||||
{"field": "drawdown_from_high_250", "weight": 0.32, "direction": "asc"},
|
||||
{"field": "relative_strength", "weight": 0.30, "direction": "desc"},
|
||||
],
|
||||
"limit": 30,
|
||||
"min_score": 0.50,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "唐奇安通道突破",
|
||||
"description": "收盘突破前20日高点,并以突破幅度、量能和突破前振幅过滤假突破。",
|
||||
"regimes": ["repair", "fermentation", "divergence"],
|
||||
"formula": {
|
||||
"meta": _meta("趋势追踪", "A-", "每日", "中", "历史行情", 80, 20, 12, -7),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 180},
|
||||
"filters": [
|
||||
{"field": "donchian_breakout_pct", "op": ">=", "value": 2},
|
||||
{"field": "volume_ratio_5d", "op": ">=", "value": 1.8},
|
||||
{"field": "range_20d", "op": "<=", "value": 35},
|
||||
],
|
||||
"score": [
|
||||
{"field": "volume_ratio_5d", "weight": 0.40, "direction": "desc"},
|
||||
{"field": "donchian_breakout_pct", "weight": 0.35, "direction": "desc"},
|
||||
{"field": "range_20d", "weight": 0.25, "direction": "asc"},
|
||||
],
|
||||
"limit": 15,
|
||||
"min_score": 0.52,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "周线趋势·日线买点",
|
||||
"description": "周线MACD位于多头区间,日线金叉或回踩20日线收阳时确认多周期共振。",
|
||||
"regimes": ["repair", "fermentation", "divergence"],
|
||||
"formula": {
|
||||
"meta": _meta("趋势追踪", "A", "每周", "中低", "多周期行情", 180, 20, 12, -7),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 365},
|
||||
"filters": [
|
||||
{"field": "weekly_trend_signal", "op": "==", "value": 1},
|
||||
{"field": "daily_buy_trigger", "op": "==", "value": 1},
|
||||
{"field": "weekly_amount_trend", "op": "==", "value": 1},
|
||||
],
|
||||
"score": [
|
||||
{"field": "ma20_slope_5d", "weight": 0.35, "direction": "desc"},
|
||||
{"field": "relative_strength", "weight": 0.35, "direction": "desc"},
|
||||
{"field": "amount_billion", "weight": 0.30, "direction": "desc"},
|
||||
],
|
||||
"limit": 20,
|
||||
"min_score": 0.52,
|
||||
},
|
||||
},
|
||||
]
|
||||
|
||||
ADVANCED_CURATED_STRATEGIES.extend(
|
||||
[
|
||||
{
|
||||
"name": "空间板",
|
||||
"description": "识别当日新晋市场最高板,并要求所属方向具备足够的涨停支撑。",
|
||||
"regimes": ["repair", "fermentation"],
|
||||
"formula": {
|
||||
"meta": _meta("连板接力", "B+", "每日", "很高", "涨停结构", 80, 3, 8, -6),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 120},
|
||||
"filters": [
|
||||
{"field": "is_market_height", "op": "==", "value": 1},
|
||||
{"field": "new_space_board", "op": "==", "value": 1},
|
||||
{"field": "sector_limit_count", "op": ">=", "value": 3},
|
||||
],
|
||||
"score": [
|
||||
{"field": "limit_streak", "weight": 0.50, "direction": "desc"},
|
||||
{"field": "sector_limit_count", "weight": 0.30, "direction": "desc"},
|
||||
{"field": "amount_billion", "weight": 0.20, "direction": "desc"},
|
||||
],
|
||||
"limit": 5,
|
||||
"min_score": 0.45,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "龙头首阴",
|
||||
"description": "筛选三板以上强势股断板后的首次缩量阴线,并结合板块强度观察承接质量。",
|
||||
"regimes": ["fermentation", "climax"],
|
||||
"formula": {
|
||||
"meta": _meta("低吸反核", "B", "每日", "很高", "涨停结构", 80, 5, 8, -6),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 120},
|
||||
"filters": [
|
||||
{"field": "max_continuous_board_10d", "op": ">=", "value": 3},
|
||||
{"field": "dragon_first_yin", "op": "==", "value": 1},
|
||||
{"field": "yin_day_pct", "op": ">=", "value": -7},
|
||||
{"field": "vol_vs_previous", "op": "<=", "value": 0.8},
|
||||
],
|
||||
"score": [
|
||||
{"field": "max_continuous_board_10d", "weight": 0.45, "direction": "desc"},
|
||||
{"field": "vol_vs_previous", "weight": 0.30, "direction": "asc"},
|
||||
{"field": "sector_strength", "weight": 0.25, "direction": "desc"},
|
||||
],
|
||||
"limit": 5,
|
||||
"min_score": 0.48,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "断板反包",
|
||||
"description": "连板断板后1至3日内,以涨停收复断板高点和量能确认N字反包。",
|
||||
"regimes": ["repair", "fermentation"],
|
||||
"formula": {
|
||||
"meta": _meta("低吸反核", "B+", "每日", "高", "涨停结构", 80, 3, 8, -6),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 120},
|
||||
"filters": [
|
||||
{"field": "broken_reversal", "op": "==", "value": 1},
|
||||
{"field": "days_since_broken", "op": "between", "value": [1, 3]},
|
||||
{"field": "close_above_broken_high", "op": "==", "value": 1},
|
||||
{"field": "vol_vs_broken_day", "op": ">=", "value": 1},
|
||||
],
|
||||
"score": [
|
||||
{"field": "days_since_broken", "weight": 0.35, "direction": "asc"},
|
||||
{"field": "vol_vs_broken_day", "weight": 0.35, "direction": "desc"},
|
||||
{"field": "sector_strength", "weight": 0.30, "direction": "desc"},
|
||||
],
|
||||
"limit": 5,
|
||||
"min_score": 0.46,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "核按钮反核",
|
||||
"description": "近5日强势股盘中深水急杀后收回,并以长下影和非放量结构确认承接。",
|
||||
"regimes": ["repair", "fermentation"],
|
||||
"formula": {
|
||||
"meta": _meta("低吸反核", "B+", "每日", "很高", "历史行情", 80, 5, 8, -6),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 120},
|
||||
"filters": [
|
||||
{"field": "recent_limit_up_5d", "op": ">=", "value": 1},
|
||||
{"field": "intraday_min_pct", "op": "<=", "value": -7},
|
||||
{"field": "pct_chg", "op": ">=", "value": -3},
|
||||
{"field": "lower_shadow_ratio", "op": ">=", "value": 2},
|
||||
{"field": "vol_vs_previous", "op": "<=", "value": 1.1},
|
||||
],
|
||||
"score": [
|
||||
{"field": "lower_shadow_ratio", "weight": 0.42, "direction": "desc"},
|
||||
{"field": "intraday_min_pct", "weight": 0.30, "direction": "asc"},
|
||||
{"field": "sector_strength", "weight": 0.28, "direction": "desc"},
|
||||
],
|
||||
"limit": 5,
|
||||
"min_score": 0.48,
|
||||
},
|
||||
},
|
||||
]
|
||||
)
|
||||
|
||||
ADVANCED_CURATED_STRATEGIES.extend(
|
||||
[
|
||||
{
|
||||
"name": "景气-趋势-拥挤三维行业打分",
|
||||
"description": "以行业财务景气、价格趋势和交易拥挤度合成行业得分,再选取行业内动量与成交承载靠前的公司。",
|
||||
"regimes": ["ice", "repair", "fermentation", "climax", "divergence", "retreat"],
|
||||
"formula": {
|
||||
"meta": _meta(
|
||||
"行业轮动", "A-", "双周", "中", "行业、财务与交易拥挤", 80, 20, 12, -7,
|
||||
requires_fundamental=True,
|
||||
),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 180},
|
||||
"filters": [
|
||||
{"field": "sector_composite_score", "op": ">=", "value": 0.58},
|
||||
{"field": "sector_crowding_rank", "op": "<=", "value": 0.90},
|
||||
{"field": "sector_stock_momentum_rank", "op": ">=", "value": 0.50},
|
||||
{"field": "amount_billion", "op": ">=", "value": 1},
|
||||
],
|
||||
"score": [
|
||||
{"field": "sector_composite_score", "weight": 0.55, "direction": "desc"},
|
||||
{"field": "sector_stock_momentum_rank", "weight": 0.25, "direction": "desc"},
|
||||
{"field": "sector_crowding_rank", "weight": 0.20, "direction": "asc"},
|
||||
],
|
||||
"limit": 12,
|
||||
"min_score": 0.50,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "大小盘/成长价值风格切换(元策略)",
|
||||
"description": "比较大小盘与成长价值组合近20日相对表现,动态选择当前占优风格中的匹配标的。",
|
||||
"regimes": ["ice", "repair", "fermentation", "climax", "divergence", "retreat"],
|
||||
"formula": {
|
||||
"meta": _meta(
|
||||
"元策略", "A-", "每周", "中低", "行情、估值与财务", 80, 20, 12, -7,
|
||||
requires_fundamental=True, requires_valuation=True,
|
||||
),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 250},
|
||||
"filters": [
|
||||
{"field": "style_fit_score", "op": ">=", "value": 0.65},
|
||||
{"field": "amount_billion", "op": ">=", "value": 1},
|
||||
],
|
||||
"score": [
|
||||
{"field": "style_fit_score", "weight": 0.70, "direction": "desc"},
|
||||
{"field": "relative_strength", "weight": 0.30, "direction": "desc"},
|
||||
],
|
||||
"limit": 20,
|
||||
"min_score": 0.52,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "业绩超预期漂移(SUE/PEAD)",
|
||||
"description": "以业绩预告和业绩快报的同报告期差异识别超预期事件,并限定在公告后的首个交易窗口。",
|
||||
"regimes": ["ice", "repair", "fermentation", "climax", "divergence", "retreat"],
|
||||
"formula": {
|
||||
"meta": _meta(
|
||||
"业绩事件", "A-", "事件驱动", "中", "业绩预告与快报", 80, 20, 12, -7,
|
||||
requires_earnings_events=True,
|
||||
),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 180},
|
||||
"filters": [
|
||||
{"field": "earnings_surprise_pct", "op": ">=", "value": 10},
|
||||
{"field": "revenue_yoy", "op": ">", "value": 0},
|
||||
{"field": "earnings_event_quality", "op": "==", "value": 1},
|
||||
{"field": "earnings_days_since_announce", "op": "between", "value": [1, 5]},
|
||||
],
|
||||
"score": [
|
||||
{"field": "earnings_surprise_pct", "weight": 0.60, "direction": "desc"},
|
||||
{"field": "relative_strength", "weight": 0.25, "direction": "desc"},
|
||||
{"field": "amount_billion", "weight": 0.15, "direction": "desc"},
|
||||
],
|
||||
"limit": 15,
|
||||
"min_score": 0.50,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "多因子综合打分(IC动态加权)",
|
||||
"description": "将价值、成长、质量、动量和交易情绪标准化,并按近期横截面有效性动态合成综合分。",
|
||||
"regimes": ["ice", "repair", "fermentation", "climax", "divergence", "retreat"],
|
||||
"formula": {
|
||||
"meta": _meta(
|
||||
"多因子", "A-", "每周", "中", "行情、估值与财务", 260, 20, 12, -7,
|
||||
requires_fundamental=True, requires_valuation=True,
|
||||
),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 250},
|
||||
"filters": [
|
||||
{"field": "multi_factor_composite", "op": ">=", "value": 0.65},
|
||||
{"field": "financial_risk", "op": "==", "value": 0},
|
||||
{"field": "amount_billion", "op": ">=", "value": 1},
|
||||
],
|
||||
"score": [
|
||||
{"field": "multi_factor_composite", "weight": 0.75, "direction": "desc"},
|
||||
{"field": "relative_strength", "weight": 0.15, "direction": "desc"},
|
||||
{"field": "amount_billion", "weight": 0.10, "direction": "desc"},
|
||||
],
|
||||
"limit": 30,
|
||||
"min_score": 0.55,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "热度突增潜伏(另类数据)",
|
||||
"description": "从同花顺和东方财富人气榜中寻找排名快速跃升、但价格尚未明显兑现的观察候选。",
|
||||
"regimes": ["ice", "repair", "fermentation", "climax", "divergence", "retreat"],
|
||||
"formula": {
|
||||
"meta": _meta(
|
||||
"热度观察", "B+", "每日", "高", "人气榜与行情", 80, 10, 10, -7,
|
||||
requires_popularity=True, backtestable=False,
|
||||
),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 120},
|
||||
"filters": [
|
||||
{"field": "popularity_score", "op": ">=", "value": 15},
|
||||
{"field": "return_10d", "op": "<=", "value": 5},
|
||||
{"field": "recent_limit_up_5d", "op": "==", "value": 0},
|
||||
{"field": "amount_billion", "op": ">=", "value": 0.5},
|
||||
],
|
||||
"score": [
|
||||
{"field": "popularity_score", "weight": 0.50, "direction": "desc"},
|
||||
{"field": "popularity_rank_change", "weight": 0.25, "direction": "desc"},
|
||||
{"field": "popularity_dual_source", "weight": 0.10, "direction": "desc"},
|
||||
{"field": "amount_billion", "weight": 0.15, "direction": "desc"},
|
||||
],
|
||||
"limit": 10,
|
||||
"min_score": 0.48,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "机构榜溢价",
|
||||
"description": "筛选龙虎榜机构专用席位低位净买入的公司,并以席位数量和成交承载确认信号。",
|
||||
"regimes": ["ice", "repair", "fermentation", "climax", "divergence", "retreat"],
|
||||
"formula": {
|
||||
"meta": _meta(
|
||||
"资金席位", "B+", "每日", "中高", "龙虎榜机构席位", 80, 10, 10, -7,
|
||||
requires_institutions=True,
|
||||
),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 180},
|
||||
"filters": [
|
||||
{"field": "institution_net_buy_million", "op": ">=", "value": 30},
|
||||
{"field": "institution_seat_count", "op": ">=", "value": 1},
|
||||
{"field": "return_60d", "op": "<=", "value": 30},
|
||||
{"field": "previous_limit_streak", "op": "<=", "value": 2},
|
||||
],
|
||||
"score": [
|
||||
{"field": "institution_net_buy_million", "weight": 0.55, "direction": "desc"},
|
||||
{"field": "institution_seat_count", "weight": 0.15, "direction": "desc"},
|
||||
{"field": "relative_position_60", "weight": 0.20, "direction": "asc"},
|
||||
{"field": "amount_billion", "weight": 0.10, "direction": "desc"},
|
||||
],
|
||||
"limit": 10,
|
||||
"min_score": 0.48,
|
||||
},
|
||||
},
|
||||
]
|
||||
)
|
||||
|
||||
ADVANCED_CURATED_STRATEGIES.extend(
|
||||
[
|
||||
{
|
||||
"name": "行业动量轮动",
|
||||
"description": "选择20日涨幅居前的行业,并在行业内部保留趋势与成交承载更强的前排公司。",
|
||||
"regimes": ["ice", "repair", "fermentation", "climax", "divergence", "retreat"],
|
||||
"formula": {
|
||||
"meta": _meta("行业轮动", "A-", "双周", "中", "行业与历史行情", 80, 20, 12, -7),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 180},
|
||||
"filters": [
|
||||
{"field": "sector_momentum_rank", "op": ">=", "value": 0.90},
|
||||
{"field": "sector_stock_momentum_rank", "op": ">=", "value": 0.80},
|
||||
{"field": "amount_billion", "op": ">=", "value": 1},
|
||||
],
|
||||
"score": [
|
||||
{"field": "sector_return_20d", "weight": 0.38, "direction": "desc"},
|
||||
{"field": "return_20d", "weight": 0.32, "direction": "desc"},
|
||||
{"field": "total_mv_billion", "weight": 0.18, "direction": "desc"},
|
||||
{"field": "amount_billion", "weight": 0.12, "direction": "desc"},
|
||||
],
|
||||
"limit": 12,
|
||||
"min_score": 0.48,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": "主力资金行业流入",
|
||||
"description": "寻找近5日主力资金持续净流入、行业涨幅尚未充分兑现的板块前排。",
|
||||
"regimes": ["ice", "repair", "fermentation", "climax", "divergence", "retreat"],
|
||||
"formula": {
|
||||
"meta": _meta(
|
||||
"行业轮动", "B+", "每周", "中高", "行业与资金流", 80, 10, 10, -7,
|
||||
requires_moneyflow_history=True,
|
||||
),
|
||||
"universe": {"exclude_st": True, "listed_days_min": 180},
|
||||
"filters": [
|
||||
{"field": "sector_flow_rank", "op": ">=", "value": 0.85},
|
||||
{"field": "sector_net_flow_5d_million", "op": ">", "value": 0},
|
||||
{"field": "sector_return_5d", "op": "<=", "value": 8},
|
||||
{"field": "flow_to_circ_mv_5d", "op": ">", "value": 0},
|
||||
{"field": "amount_billion", "op": ">=", "value": 1},
|
||||
],
|
||||
"score": [
|
||||
{"field": "flow_to_circ_mv_5d", "weight": 0.42, "direction": "desc"},
|
||||
{"field": "sector_net_flow_5d_million", "weight": 0.30, "direction": "desc"},
|
||||
{"field": "sector_return_5d", "weight": 0.16, "direction": "asc"},
|
||||
{"field": "amount_billion", "weight": 0.12, "direction": "desc"},
|
||||
],
|
||||
"limit": 15,
|
||||
"min_score": 0.48,
|
||||
},
|
||||
},
|
||||
]
|
||||
)
|
||||
@@ -0,0 +1,134 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from backend.database.repositories import StrategyTrackingRepository
|
||||
|
||||
|
||||
class StrategyTrackingService:
|
||||
def __init__(self, repository: StrategyTrackingRepository) -> None:
|
||||
self.repository = repository
|
||||
|
||||
def record_run(
|
||||
self,
|
||||
user_id: int,
|
||||
run_id: int,
|
||||
selection_date: str,
|
||||
strategy_name: str,
|
||||
candidates: list[dict[str, Any]],
|
||||
) -> int:
|
||||
return self.repository.save_strategy_tracks(
|
||||
user_id, run_id, selection_date, strategy_name, candidates
|
||||
)
|
||||
|
||||
def add_candidate(self, user_id: int, run_id: int, code: str) -> dict[str, Any]:
|
||||
run = self.repository.get_screener_run(user_id, run_id)
|
||||
if not run:
|
||||
run = self.repository.get_screener_run(0, run_id)
|
||||
if not run:
|
||||
raise ValueError("选股结果不存在或不属于当前账号。")
|
||||
normalized_code = str(code or "").strip().split(".")[0]
|
||||
candidate = next(
|
||||
(
|
||||
item for item in run.get("candidates", [])
|
||||
if str(item.get("code") or item.get("ts_code") or "").split(".")[0]
|
||||
== normalized_code
|
||||
),
|
||||
None,
|
||||
)
|
||||
if not candidate:
|
||||
raise ValueError("该股票不在本次选股结果中。")
|
||||
added = self.record_run(
|
||||
user_id,
|
||||
run_id,
|
||||
str(run.get("meta", {}).get("trade_date") or ""),
|
||||
str(run.get("strategy_name") or "未命名策略"),
|
||||
[candidate],
|
||||
)
|
||||
return {"added": added, "tracking": self.list_tracking(user_id)}
|
||||
|
||||
def remove_candidate(self, user_id: int, track_id: int) -> dict[str, Any]:
|
||||
deleted = self.repository.delete_strategy_track(user_id, track_id)
|
||||
return {"deleted": deleted, "tracking": self.list_tracking(user_id)}
|
||||
|
||||
def list_tracking(self, user_id: int, limit_batches: int = 12) -> dict[str, Any]:
|
||||
tracks = self.repository.list_strategy_tracks(user_id, limit_batches)
|
||||
if not tracks:
|
||||
return {"batches": [], "summary": self._summary([])}
|
||||
|
||||
bars = self.repository.load_tracking_bars(
|
||||
[(item["ts_code"], item["selection_date"]) for item in tracks], 5
|
||||
)
|
||||
batches: dict[int, dict[str, Any]] = {}
|
||||
all_items: list[dict[str, Any]] = []
|
||||
for track in tracks:
|
||||
key = (track["ts_code"], track["selection_date"])
|
||||
metrics = self.calculate_metrics(float(track["entry_price"]), bars.get(key, []))
|
||||
item = {
|
||||
"id": track["id"],
|
||||
"code": track["code"],
|
||||
"name": track["name"],
|
||||
"sector": track["sector"],
|
||||
"entry_price": round(float(track["entry_price"]), 2),
|
||||
**metrics,
|
||||
}
|
||||
all_items.append(item)
|
||||
batch = batches.setdefault(
|
||||
int(track["run_id"]),
|
||||
{
|
||||
"run_id": int(track["run_id"]),
|
||||
"selection_date": track["selection_date"],
|
||||
"strategy_name": track["strategy_name"],
|
||||
"items": [],
|
||||
},
|
||||
)
|
||||
batch["items"].append(item)
|
||||
|
||||
ordered = list(batches.values())
|
||||
for batch in ordered:
|
||||
batch["summary"] = self._summary(batch["items"])
|
||||
return {"batches": ordered, "summary": self._summary(all_items)}
|
||||
|
||||
@staticmethod
|
||||
def calculate_metrics(entry_price: float, bars: list[dict[str, Any]]) -> dict[str, Any]:
|
||||
valid = [row for row in bars[:5] if float(row.get("close") or 0) > 0]
|
||||
if entry_price <= 0 or not valid:
|
||||
return {
|
||||
"observed_days": 0,
|
||||
"status": "等待 T+1",
|
||||
"t1_open": None,
|
||||
"t1_close": None,
|
||||
"t3_close": None,
|
||||
"t5_close": None,
|
||||
"max_gain": None,
|
||||
"max_drawdown": None,
|
||||
}
|
||||
|
||||
def change(price: Any) -> float:
|
||||
return round((float(price or 0) / entry_price - 1) * 100, 2)
|
||||
|
||||
observed = len(valid)
|
||||
return {
|
||||
"observed_days": observed,
|
||||
"status": "已完成" if observed >= 5 else f"跟踪中 {observed}/5",
|
||||
"t1_open": change(valid[0]["open"]),
|
||||
"t1_close": change(valid[0]["close"]),
|
||||
"t3_close": change(valid[2]["close"]) if observed >= 3 else None,
|
||||
"t5_close": change(valid[4]["close"]) if observed >= 5 else None,
|
||||
"max_gain": max(change(row["high"]) for row in valid),
|
||||
"max_drawdown": min(change(row["low"]) for row in valid),
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _summary(items: list[dict[str, Any]]) -> dict[str, Any]:
|
||||
completed = [item for item in items if item.get("t5_close") is not None]
|
||||
t1 = [float(item["t1_close"]) for item in items if item.get("t1_close") is not None]
|
||||
t5 = [float(item["t5_close"]) for item in completed]
|
||||
return {
|
||||
"total": len(items),
|
||||
"observed": len(t1),
|
||||
"completed": len(completed),
|
||||
"t1_win_rate": round(sum(value > 0 for value in t1) / len(t1) * 100, 1) if t1 else None,
|
||||
"t5_win_rate": round(sum(value > 0 for value in t5) / len(t5) * 100, 1) if t5 else None,
|
||||
"average_t5": round(sum(t5) / len(t5), 2) if t5 else None,
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
"""Market sentiment cycle and history feature."""
|
||||
|
||||
from .engine import (
|
||||
COMPONENT_WEIGHTS,
|
||||
SENTIMENT_ENGINE_VERSION,
|
||||
apply_sentiment_to_dashboard,
|
||||
build_sentiment_history,
|
||||
latest_contiguous_history,
|
||||
)
|
||||
from .service import SentimentServiceMixin
|
||||
|
||||
__all__ = [
|
||||
"COMPONENT_WEIGHTS",
|
||||
"SENTIMENT_ENGINE_VERSION",
|
||||
"SentimentServiceMixin",
|
||||
"apply_sentiment_to_dashboard",
|
||||
"build_sentiment_history",
|
||||
"latest_contiguous_history",
|
||||
]
|
||||
@@ -0,0 +1,490 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from copy import deepcopy
|
||||
from statistics import mean, median
|
||||
from typing import Any
|
||||
|
||||
from backend.data.numbers import non_nan_number as _number
|
||||
|
||||
|
||||
COMPONENT_WEIGHTS = {
|
||||
"breadth": 20,
|
||||
"limit_ecology": 25,
|
||||
"profit_effect": 30,
|
||||
"ladder_structure": 15,
|
||||
"liquidity": 10,
|
||||
}
|
||||
|
||||
SENTIMENT_ENGINE_VERSION = 2
|
||||
|
||||
|
||||
def _clamp(value: float, lower: float = 0.0, upper: float = 100.0) -> float:
|
||||
return min(upper, max(lower, value))
|
||||
|
||||
|
||||
def _linear(value: float, low: float, high: float) -> float:
|
||||
if high <= low:
|
||||
return 50.0
|
||||
return _clamp((value - low) / (high - low) * 100)
|
||||
|
||||
|
||||
def _percentile(value: float, history: list[float]) -> float:
|
||||
if not history:
|
||||
return 50.0
|
||||
below = sum(item < value for item in history)
|
||||
equal = sum(item == value for item in history)
|
||||
return _clamp((below + equal * 0.5) / len(history) * 100)
|
||||
|
||||
|
||||
def _adaptive_score(value: float, fixed: float, history: list[float]) -> float:
|
||||
if len(history) < 20:
|
||||
return fixed
|
||||
return fixed * 0.25 + _percentile(value, history[-250:]) * 0.75
|
||||
|
||||
|
||||
def _trade_date(payload: dict[str, Any]) -> str:
|
||||
meta = payload.get("meta") or {}
|
||||
return str(meta.get("trade_date") or payload.get("_snapshot_date") or "").replace("-", "")
|
||||
|
||||
|
||||
def _deduplicate_snapshots(snapshots: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||
by_trade_date: dict[str, dict[str, Any]] = {}
|
||||
for payload in snapshots:
|
||||
trade_date = _trade_date(payload)
|
||||
if trade_date:
|
||||
by_trade_date[trade_date] = payload
|
||||
return [by_trade_date[key] for key in sorted(by_trade_date)]
|
||||
|
||||
|
||||
def _snapshot_stats(payload: dict[str, Any]) -> dict[str, Any]:
|
||||
overview = payload.get("overview") or {}
|
||||
meta = payload.get("meta") or {}
|
||||
limits = list(payload.get("limits") or [])
|
||||
broken = list(payload.get("broken") or [])
|
||||
down_limits = list(payload.get("down_limits") or [])
|
||||
yesterday = list(payload.get("yesterday_limits") or [])
|
||||
|
||||
limit_up = len(limits) if limits else int(_number(overview.get("limit_up_count")))
|
||||
broken_count = len(broken) if broken else int(_number(overview.get("broken_count")))
|
||||
limit_down = len(down_limits) if down_limits else int(_number(overview.get("limit_down_count")))
|
||||
streaks = [max(1, int(_number(row.get("streak"), 1))) for row in limits]
|
||||
first_board = sum(streak == 1 for streak in streaks)
|
||||
second_board = sum(streak == 2 for streak in streaks)
|
||||
three_plus = sum(streak >= 3 for streak in streaks)
|
||||
max_height = max(streaks, default=0)
|
||||
present_levels = set(streaks)
|
||||
ladder_completeness = (
|
||||
sum(level in present_levels for level in range(1, max_height + 1)) / max_height * 100
|
||||
if max_height else 0.0
|
||||
)
|
||||
|
||||
up_count = int(_number(overview.get("up_count")))
|
||||
down_count = int(_number(overview.get("down_count")))
|
||||
flat_count = int(_number(overview.get("flat_count")))
|
||||
active_count = up_count + down_count
|
||||
breadth_ratio = up_count / max(active_count, 1) * 100
|
||||
seal_rate = _number(overview.get("seal_rate"))
|
||||
if not seal_rate and limit_up + broken_count:
|
||||
seal_rate = limit_up / (limit_up + broken_count) * 100
|
||||
|
||||
previous_limit_count = len(yesterday)
|
||||
previous_positive_count = sum(_number(row.get("current_change")) > 0 for row in yesterday)
|
||||
previous_positive_rate = previous_positive_count / max(previous_limit_count, 1) * 100
|
||||
advanced_count = sum(row.get("outcome") == "晋级" for row in yesterday)
|
||||
advance_rate = advanced_count / max(previous_limit_count, 1) * 100
|
||||
average_previous_change = (
|
||||
mean(_number(row.get("current_change")) for row in yesterday) if yesterday else 0.0
|
||||
)
|
||||
median_previous_change = (
|
||||
median(_number(row.get("current_change")) for row in yesterday) if yesterday else 0.0
|
||||
)
|
||||
severe_loss_count = sum(_number(row.get("current_change")) <= -5 for row in yesterday)
|
||||
severe_loss_rate = severe_loss_count / max(previous_limit_count, 1) * 100
|
||||
previous_down_count = sum(row.get("outcome") == "跌停" for row in yesterday)
|
||||
high_previous = [row for row in yesterday if int(_number(row.get("prior_streak"), 1)) >= 2]
|
||||
high_positive_rate = (
|
||||
sum(_number(row.get("current_change")) > 0 for row in high_previous)
|
||||
/ max(len(high_previous), 1)
|
||||
* 100
|
||||
)
|
||||
|
||||
amount_billion = _number(overview.get("amount_billion"))
|
||||
limit_amount_billion = sum(_number(row.get("amount_billion")) for row in limits)
|
||||
return {
|
||||
"trade_date": _trade_date(payload),
|
||||
"previous_trade_date": str(meta.get("previous_trade_date") or "").replace("-", ""),
|
||||
"up_count": up_count,
|
||||
"down_count": down_count,
|
||||
"flat_count": flat_count,
|
||||
"breadth_ratio": round(breadth_ratio, 1),
|
||||
"limit_up_count": limit_up,
|
||||
"first_board_count": first_board,
|
||||
"second_board_count": second_board,
|
||||
"three_plus_count": three_plus,
|
||||
"max_height": max_height,
|
||||
"ladder_completeness": round(ladder_completeness, 1),
|
||||
"broken_count": broken_count,
|
||||
"limit_down_count": limit_down,
|
||||
"seal_rate": round(seal_rate, 1),
|
||||
"previous_limit_count": previous_limit_count,
|
||||
"previous_positive_count": previous_positive_count,
|
||||
"previous_positive_rate": round(previous_positive_rate, 1),
|
||||
"advance_rate": round(advance_rate, 1),
|
||||
"average_previous_change": round(average_previous_change, 2),
|
||||
"median_previous_change": round(median_previous_change, 2),
|
||||
"severe_loss_count": severe_loss_count,
|
||||
"severe_loss_rate": round(severe_loss_rate, 1),
|
||||
"previous_down_count": previous_down_count,
|
||||
"high_positive_rate": round(high_positive_rate, 1),
|
||||
"amount_billion": round(amount_billion, 1),
|
||||
"limit_amount_billion": round(limit_amount_billion, 2),
|
||||
}
|
||||
|
||||
|
||||
def _sentiment_label(score: float) -> str:
|
||||
if score >= 80:
|
||||
return "情绪高涨"
|
||||
if score >= 60:
|
||||
return "情绪偏强"
|
||||
if score >= 40:
|
||||
return "情绪中性"
|
||||
if score >= 20:
|
||||
return "情绪偏弱"
|
||||
return "情绪冰点"
|
||||
|
||||
|
||||
def _phase_signal(score: float, momentum: float, profit_score: float) -> str:
|
||||
if score < 25:
|
||||
return "修复" if momentum > 3 else "冰点"
|
||||
if score < 45:
|
||||
return "修复" if momentum > 3 else "退潮"
|
||||
if score >= 80:
|
||||
return "高潮" if momentum >= -2 and profit_score >= 60 else "分化"
|
||||
if score >= 65:
|
||||
return "分化" if momentum < -3 or profit_score < 50 else "发酵"
|
||||
if momentum < -5:
|
||||
return "退潮"
|
||||
return "发酵" if momentum >= 0 and profit_score >= 45 else "分化"
|
||||
|
||||
|
||||
def _confirmed_phase(
|
||||
previous: dict[str, Any] | None,
|
||||
score: float,
|
||||
day_change: float,
|
||||
systemic_health: float,
|
||||
profit_score: float,
|
||||
ecology_score: float,
|
||||
phase_signal: str,
|
||||
extreme_ice: bool,
|
||||
fermentation_signal_count: int,
|
||||
) -> tuple[str, str]:
|
||||
if previous is None:
|
||||
return phase_signal, "首个连续交易日,采用原始阶段信号"
|
||||
previous_phase = str(previous.get("phase") or phase_signal)
|
||||
if extreme_ice:
|
||||
return "冰点", "市场宽度与跌停数量触发极端冰点"
|
||||
|
||||
recovery = day_change >= 6 and score >= 25 and systemic_health >= 24
|
||||
fermentation_confirmed = fermentation_signal_count >= 2
|
||||
climax_ready = (
|
||||
score >= 80
|
||||
and profit_score >= 60
|
||||
and systemic_health >= 60
|
||||
and ecology_score >= 70
|
||||
)
|
||||
|
||||
if previous_phase == "冰点":
|
||||
return ("修复", "冰点后首次有效回升") if recovery else ("冰点", "冰点尚未形成有效修复")
|
||||
|
||||
if previous_phase == "退潮":
|
||||
if score < 25:
|
||||
return "冰点", "退潮继续下探至冰点区间"
|
||||
return ("修复", "退潮后出现有效回升") if recovery else ("退潮", "退潮尚未形成有效修复")
|
||||
|
||||
if previous_phase == "修复":
|
||||
if score < 25:
|
||||
return "冰点", "修复失败并重新跌入冰点区间"
|
||||
if day_change <= -6 and score < 45:
|
||||
return "退潮", "修复失败且温度显著回落"
|
||||
if fermentation_confirmed:
|
||||
return "发酵", "发酵条件连续两个交易日成立"
|
||||
return "修复", "修复延续,等待发酵确认"
|
||||
|
||||
if previous_phase == "发酵":
|
||||
if score < 25:
|
||||
return "冰点", "发酵阶段出现极端情绪坍塌"
|
||||
if score < 45 and (day_change < 0 or systemic_health < 35):
|
||||
return "退潮", "发酵阶段温度与系统健康度同步转弱"
|
||||
if climax_ready:
|
||||
return "高潮", "温度、赚钱效应与涨停生态共同达到高潮条件"
|
||||
if phase_signal in {"分化", "退潮"} or day_change <= -6:
|
||||
return "分化", "发酵阶段出现降温或赚钱效应弱化"
|
||||
return "发酵", "发酵状态延续"
|
||||
|
||||
if previous_phase == "高潮":
|
||||
if score < 25:
|
||||
return "冰点", "高潮后出现极端情绪坍塌"
|
||||
if climax_ready:
|
||||
return "高潮", "高潮条件继续成立"
|
||||
if score < 45 or systemic_health < 30:
|
||||
return "退潮", "高潮后风险快速释放"
|
||||
return "分化", "高潮条件消退,进入分化"
|
||||
|
||||
if previous_phase == "分化":
|
||||
if score < 25:
|
||||
return "冰点", "分化继续恶化至冰点区间"
|
||||
if score < 45 or systemic_health < 30:
|
||||
return "退潮", "分化后温度或系统健康度继续下降"
|
||||
if fermentation_confirmed:
|
||||
return "发酵", "分化转强条件连续两个交易日成立"
|
||||
return "分化", "分化延续,等待方向确认"
|
||||
|
||||
return phase_signal, "采用原始阶段信号"
|
||||
|
||||
|
||||
def build_sentiment_history(snapshots: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||
payloads = _deduplicate_snapshots(snapshots)
|
||||
raw_rows = [_snapshot_stats(payload) for payload in payloads]
|
||||
results: list[dict[str, Any]] = []
|
||||
|
||||
for index, stats in enumerate(raw_rows):
|
||||
previous = raw_rows[:index]
|
||||
limit_history = [float(row["limit_up_count"]) for row in previous]
|
||||
down_limit_history = [float(row["limit_down_count"]) for row in previous]
|
||||
height_history = [float(row["max_height"]) for row in previous]
|
||||
three_plus_history = [float(row["three_plus_count"]) for row in previous]
|
||||
amount_history = [float(row["amount_billion"]) for row in previous[-20:] if row["amount_billion"]]
|
||||
|
||||
breadth_score = _clamp(float(stats["breadth_ratio"]))
|
||||
limit_strength = _adaptive_score(
|
||||
float(stats["limit_up_count"]),
|
||||
_linear(float(stats["limit_up_count"]), 10, 100),
|
||||
limit_history,
|
||||
)
|
||||
down_relief = 100 - _adaptive_score(
|
||||
float(stats["limit_down_count"]),
|
||||
_linear(float(stats["limit_down_count"]), 0, 50),
|
||||
down_limit_history,
|
||||
)
|
||||
seal_quality = _linear(float(stats["seal_rate"]), 35, 90)
|
||||
systemic_health = breadth_score * 0.60 + down_relief * 0.40
|
||||
systemic_gate = 1.0 if systemic_health >= 35 else 0.35 + systemic_health / 35 * 0.65
|
||||
ecology_base_score = limit_strength * 0.35 + seal_quality * 0.35 + down_relief * 0.30
|
||||
# Systemic risk is applied once to the final temperature. Reapplying it here
|
||||
# would count market breadth and limit-down pressure twice.
|
||||
limit_ecology_score = ecology_base_score
|
||||
|
||||
if stats["previous_limit_count"]:
|
||||
positive_score = float(stats["previous_positive_rate"])
|
||||
average_change_score = _clamp(50 + float(stats["average_previous_change"]) * 6)
|
||||
median_change_score = _clamp(50 + float(stats["median_previous_change"]) * 7)
|
||||
advance_score = _clamp(float(stats["advance_rate"]) * 2.5)
|
||||
severe_loss_safety = _clamp(100 - float(stats["severe_loss_rate"]) * 3)
|
||||
down_safety = _clamp(100 - float(stats["previous_down_count"]) / stats["previous_limit_count"] * 700)
|
||||
tail_safety_score = severe_loss_safety * 0.70 + down_safety * 0.30
|
||||
profit_effect_score = (
|
||||
positive_score * 0.30
|
||||
+ median_change_score * 0.25
|
||||
+ average_change_score * 0.10
|
||||
+ advance_score * 0.20
|
||||
+ tail_safety_score * 0.15
|
||||
)
|
||||
else:
|
||||
profit_effect_score = 50.0
|
||||
|
||||
max_height_score = _adaptive_score(
|
||||
float(stats["max_height"]),
|
||||
_linear(float(stats["max_height"]), 1, 7),
|
||||
height_history,
|
||||
)
|
||||
continuation_rate = (
|
||||
(float(stats["second_board_count"]) + float(stats["three_plus_count"]))
|
||||
/ max(float(stats["limit_up_count"]), 1)
|
||||
* 100
|
||||
)
|
||||
three_plus_density = float(stats["three_plus_count"]) / max(float(stats["limit_up_count"]), 1) * 100
|
||||
three_plus_score = _adaptive_score(
|
||||
float(stats["three_plus_count"]),
|
||||
_clamp(three_plus_density * 5),
|
||||
three_plus_history,
|
||||
)
|
||||
ladder_structure_score = (
|
||||
max_height_score * 0.30
|
||||
+ _clamp(continuation_rate * 3) * 0.25
|
||||
+ three_plus_score * 0.25
|
||||
+ float(stats["ladder_completeness"]) * 0.20
|
||||
)
|
||||
|
||||
amount_baseline = mean(amount_history) if amount_history else float(stats["amount_billion"] or 1)
|
||||
amount_ratio = float(stats["amount_billion"]) / max(amount_baseline, 1)
|
||||
amount_score = _clamp(50 + (amount_ratio - 1) * 100)
|
||||
limit_amount_share = float(stats["limit_amount_billion"]) / max(float(stats["amount_billion"]), 1) * 100
|
||||
liquidity_score = amount_score * 0.70 + _clamp(limit_amount_share * 20) * 0.30
|
||||
|
||||
component_scores = {
|
||||
"breadth": breadth_score,
|
||||
"limit_ecology": limit_ecology_score,
|
||||
"profit_effect": profit_effect_score,
|
||||
"ladder_structure": ladder_structure_score,
|
||||
"liquidity": liquidity_score,
|
||||
}
|
||||
raw_score = sum(component_scores[key] * weight / 100 for key, weight in COMPONENT_WEIGHTS.items())
|
||||
score = round(
|
||||
raw_score * systemic_gate
|
||||
)
|
||||
extreme_ice = float(stats["breadth_ratio"]) <= 15 and float(stats["limit_down_count"]) >= 100
|
||||
if extreme_ice:
|
||||
score = min(score, 15)
|
||||
elif float(stats["breadth_ratio"]) <= 25 and float(stats["limit_down_count"]) >= 50:
|
||||
score = min(score, 24)
|
||||
previous_scores: list[float] = []
|
||||
expected_date = str(stats.get("previous_trade_date") or "")
|
||||
for prior_result in reversed(results):
|
||||
if not expected_date or str(prior_result.get("trade_date") or "") != expected_date:
|
||||
break
|
||||
previous_scores.append(float(prior_result["score"]))
|
||||
expected_date = str(prior_result.get("previous_trade_date") or "")
|
||||
if len(previous_scores) == 3:
|
||||
break
|
||||
momentum = score - mean(previous_scores) if previous_scores else 0.0
|
||||
direction = "升温" if momentum > 3 else "降温" if momentum < -3 else "持平"
|
||||
normalization = "历史百分位" if len(previous) >= 20 else "固定锚点"
|
||||
previous_result = (
|
||||
results[-1]
|
||||
if results and str(stats.get("previous_trade_date") or "") == str(results[-1].get("trade_date") or "")
|
||||
else None
|
||||
)
|
||||
day_change = score - float(previous_result["score"]) if previous_result else 0.0
|
||||
ema_score = round(
|
||||
score if not previous_result
|
||||
else score * 0.5 + float(previous_result.get("ema_score", previous_result["score"])) * 0.5,
|
||||
1,
|
||||
)
|
||||
phase_signal = _phase_signal(score, momentum, profit_effect_score)
|
||||
fermentation_ready = (
|
||||
phase_signal == "发酵"
|
||||
and score >= 45
|
||||
and profit_effect_score >= 45
|
||||
and systemic_health >= 35
|
||||
and not extreme_ice
|
||||
)
|
||||
previous_fermentation_count = int(previous_result.get("fermentation_signal_count") or 0) if previous_result else 0
|
||||
fermentation_signal_count = previous_fermentation_count + 1 if fermentation_ready else 0
|
||||
phase, transition_reason = _confirmed_phase(
|
||||
previous_result,
|
||||
score,
|
||||
day_change,
|
||||
systemic_health,
|
||||
profit_effect_score,
|
||||
limit_ecology_score,
|
||||
phase_signal,
|
||||
extreme_ice,
|
||||
fermentation_signal_count,
|
||||
)
|
||||
previous_phase = str(previous_result.get("phase") or "") if previous_result else ""
|
||||
if phase not in {"修复", "分化"}:
|
||||
fermentation_signal_count = 0
|
||||
elif phase == "分化" and previous_phase != "分化":
|
||||
fermentation_signal_count = 0
|
||||
|
||||
components = {
|
||||
"breadth": {
|
||||
"label": "市场宽度",
|
||||
"score": round(breadth_score, 1),
|
||||
"weight": COMPONENT_WEIGHTS["breadth"],
|
||||
"summary": f"上涨占比 {stats['breadth_ratio']:.1f}%",
|
||||
},
|
||||
"limit_ecology": {
|
||||
"label": "涨停生态",
|
||||
"score": round(limit_ecology_score, 1),
|
||||
"weight": COMPONENT_WEIGHTS["limit_ecology"],
|
||||
"summary": (
|
||||
f"涨停 {stats['limit_up_count']} · 跌停 {stats['limit_down_count']} · "
|
||||
f"封板 {stats['seal_rate']:.1f}%"
|
||||
),
|
||||
},
|
||||
"profit_effect": {
|
||||
"label": "赚钱效应",
|
||||
"score": round(profit_effect_score, 1),
|
||||
"weight": COMPONENT_WEIGHTS["profit_effect"],
|
||||
"summary": (
|
||||
f"昨涨停红盘 {stats['previous_positive_rate']:.1f}% · "
|
||||
f"中位 {stats['median_previous_change']:+.2f}% · "
|
||||
f"重亏 {stats['severe_loss_rate']:.1f}%"
|
||||
if stats["previous_limit_count"] else "缺少前一交易日样本"
|
||||
),
|
||||
},
|
||||
"ladder_structure": {
|
||||
"label": "连板结构",
|
||||
"score": round(ladder_structure_score, 1),
|
||||
"weight": COMPONENT_WEIGHTS["ladder_structure"],
|
||||
"summary": f"最高 {stats['max_height']} 板 · 三板以上 {stats['three_plus_count']} 家",
|
||||
},
|
||||
"liquidity": {
|
||||
"label": "成交活跃度",
|
||||
"score": round(liquidity_score, 1),
|
||||
"weight": COMPONENT_WEIGHTS["liquidity"],
|
||||
"summary": f"成交 {stats['amount_billion']:.1f} 亿 · 均值比 {amount_ratio:.2f}",
|
||||
},
|
||||
}
|
||||
results.append(
|
||||
{
|
||||
**stats,
|
||||
"score": score,
|
||||
"ema_score": ema_score,
|
||||
"label": _sentiment_label(score),
|
||||
"phase": phase,
|
||||
"phase_signal": phase_signal,
|
||||
"transition_reason": transition_reason,
|
||||
"fermentation_signal_count": fermentation_signal_count,
|
||||
"day_change": round(day_change, 1),
|
||||
"direction": direction,
|
||||
"momentum": round(momentum, 1),
|
||||
"normalization": "250日历史百分位" if len(previous) >= 20 else normalization,
|
||||
"history_days": len(previous) + 1,
|
||||
"systemic_health": round(systemic_health, 1),
|
||||
"risk_multiplier": round(systemic_gate, 3),
|
||||
"components": components,
|
||||
}
|
||||
)
|
||||
return results
|
||||
|
||||
|
||||
def latest_contiguous_history(series: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||
if not series:
|
||||
return []
|
||||
contiguous = [series[-1]]
|
||||
for row in reversed(series[:-1]):
|
||||
expected_previous = str(contiguous[0].get("previous_trade_date") or "")
|
||||
if not expected_previous or expected_previous != str(row.get("trade_date") or ""):
|
||||
break
|
||||
contiguous.insert(0, row)
|
||||
return contiguous
|
||||
|
||||
|
||||
def apply_sentiment_to_dashboard(
|
||||
dashboard: dict[str, Any],
|
||||
historical_snapshots: list[dict[str, Any]] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
result = deepcopy(dashboard)
|
||||
history = list(historical_snapshots or [])
|
||||
history.append(result)
|
||||
series = build_sentiment_history(history)
|
||||
target_date = _trade_date(result)
|
||||
sentiment = next((row for row in reversed(series) if row["trade_date"] == target_date), None)
|
||||
if not sentiment:
|
||||
return result
|
||||
overview = dict(result.get("overview") or {})
|
||||
overview.update(
|
||||
{
|
||||
"sentiment_score": sentiment["score"],
|
||||
"sentiment_trend_score": sentiment["ema_score"],
|
||||
"sentiment_label": sentiment["label"],
|
||||
"sentiment_phase": sentiment["phase"],
|
||||
"sentiment_direction": sentiment["direction"],
|
||||
"sentiment_components": sentiment["components"],
|
||||
"sentiment_engine_version": SENTIMENT_ENGINE_VERSION,
|
||||
}
|
||||
)
|
||||
result["overview"] = overview
|
||||
return result
|
||||
@@ -0,0 +1,39 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from backend.bootstrap.config import normalize_date
|
||||
from backend.features.sentiment.engine import (
|
||||
COMPONENT_WEIGHTS,
|
||||
apply_sentiment_to_dashboard,
|
||||
build_sentiment_history,
|
||||
latest_contiguous_history,
|
||||
)
|
||||
|
||||
|
||||
class SentimentServiceMixin:
|
||||
def _enrich_dashboard_sentiment(
|
||||
self,
|
||||
dashboard: dict[str, Any],
|
||||
end_date: str,
|
||||
) -> dict[str, Any]:
|
||||
history = self.database.list_snapshot_payloads(end_date, 260)
|
||||
return apply_sentiment_to_dashboard(dashboard, history)
|
||||
|
||||
def sentiment_history(self, trade_date: str, limit: int = 20) -> dict[str, Any]:
|
||||
normalized_date = normalize_date(trade_date)
|
||||
limit = max(10, min(120, int(limit)))
|
||||
full_series = build_sentiment_history(
|
||||
self.database.list_snapshot_payloads(normalized_date, 240)
|
||||
)
|
||||
series = latest_contiguous_history(full_series)
|
||||
rows = series[-limit:]
|
||||
return {
|
||||
"trade_date": rows[-1]["trade_date"] if rows else normalized_date,
|
||||
"available_days": len(series),
|
||||
"stored_days": len(full_series),
|
||||
"requested_days": limit,
|
||||
"rows": rows,
|
||||
"weights": COMPONENT_WEIGHTS,
|
||||
"normalization": rows[-1]["normalization"] if rows else "固定锚点",
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user