Compare commits

..
Author SHA1 Message Date
031eefab4d fix(HEL-386): 清理任务按 ISO 截止时间删除 job_runs/src_calls
YYYYMMDD 与 ISO 字符串比较会把同年保留期内记录全部误删。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-02 12:26:09 +08:00
3498dd7a4b feat(HEL-382): 搭建 datahub 底座和盘后正式数据链路
新增独立 xiaobai-datahub 服务(SQLite WAL、Tushare 盘后发布、/v1 契约和管理后台),不改现站页面与数据链路。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-02 12:05:26 +08:00
c2ebc0ab91 docs(HEL-353): 恢复普通 README 并移除多媒体展示
放弃宣传稿与 docs/images 素材,按当前仓库真实结构重写常规说明。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-01 21:58:35 +08:00
71a6d68ed7 docs(HEL-277): 用 25 组真实素材替换 README 占位
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-01 21:39:08 +08:00
cda13a787f docs(HEL-272): 用终版项目介绍更新仓库 README
将总工审核通过的项目介绍写入根目录 README,保留本机启动、Docker 与系统配置等必要说明,技术栈保持公开。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-30 20:58:16 +08:00
总管andmultica-agent 6b688fd473 修复(HEL-235B): 可靠同步受限权限工作目录
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 23:29:11 +08:00
总管andmultica-agent 1e8da5fee2 部署(HEL-235B): 补严分叉与镜像标签校验
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 23:27:18 +08:00
总工andmultica-agent 51f410d942 部署(HEL-235B): 服务器本地目录纳入 Git 管理,固化 main 校验构建流程
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 23:22:59 +08:00
总工 a8732f51be 部署集成(HEL-252): 合入登录页动态小人 4a63ccd 2026-08-29 21:59:09 +08:00
4a63ccd10f feat(HEL-251): 按确认稿实现登录页动态小K线人物
在桌面登录门户品牌空白带加入红绿小K线角色,支持焦点、密码遮挡、登录反馈和减少动态效果,且不移动原有文案与行情图。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 21:46:05 +08:00
总工 865d8b0516 部署集成(HEL-244): 合入切换账号返回修复 d2a165a 2026-08-29 21:01:12 +08:00
d2a165ada8 fix(HEL-243): 当前账号可点击返回,避免切换死路
当前账号行原先只显示对号、不可点击,单账号时只能关网页。现改为整行继续使用且不重新签发会话,并带回切换前的业务页。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 20:44:18 +08:00
总工 e29d5115fa 部署集成(HEL-241): 合入桌面登录门户返工 aef8a05(含 28px 底部缺口修复) 2026-08-29 20:09:41 +08:00
aef8a059f2 fix(HEL-240): 去掉登录页底部状态栏缺口
登录页覆盖全局 body 底边距,让左栏品牌面板铺满视口高度。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 20:04:22 +08:00
f905b44675 feat(HEL-240): 按确认样图还原桌面登录页左右骨架
把登录门户改回确认稿的品牌行、中下部主标题、底部行情标签和 K 线背景,并让超宽屏按比例展开左栏。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 19:21:13 +08:00
541fb48c1c feat(HEL-238): 按确认样图重做手机端系统管理,并加入部署基线门禁
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 17:34:50 +08:00
总工andmultica-agent 2a2d205a38 恢复集成(HEL-237): 并入 HEL-235 安全构建入口工具链(仅构建工具与文档,无运行时影响)
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 16:01:54 +08:00
总工andmultica-agent 34cb32d78f 恢复集成(HEL-237): 以完整基线 89b8d33 为底合入登录线 cefc869
事故根因:HEL-214/221/226/233 均直接基于 main(8a5e78f) 构建部署,
绕过了 .11 正式线(HEL-183/188/190/191/192/193/199/207/208 + HEL-164
共 14 个已验收提交),导致问天工具行右对齐、侧栏等高、管理员刷新
结果、快照补档、收盘日线修复等整体丢失。

本合并以 deploy 前最后完整基线 89b8d33(镜像 official-limit-guard-d9ee725)
为底,合入 cefc869(HEL-221 情绪周期等高、HEL-226 登录门户与免密切换、
HEL-233 移动端系统管理五页),三处 CSS 缓存版本统一刷新为 20260829-hel237,
architecture-inventory 按合并后源码重新生成。

Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 15:51:48 +08:00
总管andmultica-agent 89b8d33de7 fix(HEL-164): 排除构建中的数据库备份
Co-authored-by: multica-agent <github@multica.ai>
2026-08-28 11:32:21 +00:00
总管andmultica-agent d9ee725744 fix(HEL-164): 拒绝缓存不完整涨停快照
Co-authored-by: multica-agent <github@multica.ai>
2026-08-28 09:44:51 +00:00
总管andmultica-agent 1cb2745867 feat(HEL-164): 显示管理员刷新实际结果
Co-authored-by: multica-agent <github@multica.ai>
2026-08-28 09:29:06 +00:00
总管andmultica-agent f27471238a fix(HEL-164): 撤回盘后刷新改动并恢复原逻辑
Co-authored-by: multica-agent <github@multica.ai>
2026-08-28 09:05:42 +00:00
总工andmultica-agent 6d7a839202 chore(HEL-208): 刷新 architecture-inventory 以对齐 HEL-207 行情改动
Co-authored-by: multica-agent <github@multica.ai>
2026-08-28 08:09:49 +00:00
总工 fc1e5b89e4 merge(HEL-208): 集成收盘行情修复 cf206c7 到 .11 正式线(基于 09a935a) 2026-08-28 08:09:28 +00:00
cf206c7de9 fix(HEL-207): 收盘后改走日线,禁止误调 rt_k,回退旧快照记失败
将实时窗口与调度窗口统一到 15:05;盘后优先日线并补关闭盘后同步;沿用旧快照时 sync/job 记 failed。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-28 08:00:08 +00:00
总工andmultica-agent 09a935aac4 merge(HEL-193): 以 .11 线上基线 013ed29 整合行情历史补档 8e94c7b(HEL-190/HEL-199)
- 合入 agent/agent/ebb4e3e0638a:真实交易日历补最近60日快照 + sys.path 引导返工
- architecture-inventory 于合并后重新生成

Co-authored-by: multica-agent <github@multica.ai>
2026-08-27 16:13:34 +00:00
8e94c7b429 fix(HEL-199): 为补档工具补上仓库根 sys.path 引导
使 python3 tools/backfill_recent_snapshots.py --help 在干净环境下可直接运行,并同步文档运行示例为容器内执行。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-27 15:12:42 +00:00
总工 013ed29ffb merge(HEL-192): 集成问天解势修复到 .11 正式线(含 HEL-183 顶栏修复) 2026-08-27 14:58:11 +00:00
7ad445bc9f fix(HEL-190): 按真实交易日历补齐最近60日快照,修复断档后只显示当天
保留连续性过滤,新增可审计补档工具与备份步骤;周末/节假日与真缺档分开处理,支持重复执行与部分失败续跑。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-27 14:50:32 +00:00
ef86b31f6b fix(HEL-191): 补回问天解势知识文件兜底并消除 Failed to fetch
宿主机 data 挂载会遮盖镜像内 heaven_knowledge.json;增加不受挂载影响的 seed,
并将缺文件/坏 JSON 转为结构化中文错误,前端展示可读提示而非 Failed to fetch。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-27 14:47:27 +00:00
94e6f618c8 施工(HEL-188): 刷新 architecture-inventory 以对齐 HEL-183 运行时资源指纹
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-27 14:05:18 +00:00
f68f950106 施工(HEL-183): 问天工具行右对齐并统一 logo/收起/问师顶距与页头签
恢复顶栏工具行 margin-left:auto,问天隐藏行情条后仍靠右;logo 区与收起按钮对齐顶栏/状态条高度;问师恢复页顶 14px;问天三签改用全站 segmented 页头。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-27 13:41:23 +00:00
104 changed files with 9326 additions and 752 deletions
+1
View File
@@ -9,6 +9,7 @@ __pycache__/
*.log
runtime/
data/cache/
data/backups/
data/private-mentor-skills/
data/*.db
data/*.db-shm
+3
View File
@@ -8,6 +8,9 @@ data/*.db
data/*.db-shm
data/*.db-wal
data/backups/
datahub-data/
xiaobai-datahub/data/
xiaobai-datahub/.venv/
data/*.bak
data/*.backup
*.log
+52 -65
View File
@@ -14,13 +14,29 @@
v
xiaobai-review 容器 :8765
|-- /app 只读应用代码
| `-- backend/features/heaven/assets/heaven_knowledge.json
| 镜像内 seed(不受 data 挂载遮盖)
`-- /app/data 宿主机 ./data 持久化挂载
|-- review.db
|-- iching_zh.json
`-- heaven_knowledge.json 优先读取;缺失时回退到上方 seed
```
账号、加密后的公共数据 Token、平台模型 API Key、生辰资料、行情快照和复盘数据均在
`data/review.db`。解密密钥来自 `.env` 中的 `APP_ENCRYPTION_KEY`。数据库与
密钥必须成对备份,任意一个丢失都无法恢复账号内的加密资料。
问天静态知识文件:
- `data/iching_zh.json``data/heaven_knowledge.json` 纳入 Git 与镜像构建;
`.dockerignore` 不排除这两个文件(只排除 `data/*.db``data/cache/` 等运行时产物)。
- Compose 把宿主机 `./data` 整目录挂到 `/app/data`,会遮盖镜像里同路径文件。
因此宿主机 `data/` 应保留上述两个 JSON;若只缺 `heaven_knowledge.json`
服务会回退读取镜像内
`backend/features/heaven/assets/heaven_knowledge.json`,解势仍可用。
- 持久化位置:正式环境以宿主机项目目录下的 `./data/heaven_knowledge.json` 为准;
补文件后无需改代码,重启容器即可加载。
管理员私有的问师 Skill 保存在宿主机 `data/private-mentor-skills/`。该目录随 `data`
挂载进入容器,但被 Git 与 Docker 构建上下文排除,不会进入 Gitea 或镜像。私有 Skill
只对管理员账号返回和开放调用,也会随本指南的 `data` 备份一起保存。
@@ -149,87 +165,58 @@ docker compose restart xiaobai-review
docker compose down
```
### 镜像构建的唯一安全入口(2026-08 HEL-235 起
### 服务器本地目录更新与构建(日常推荐
生产机 `192.168.200.11` `/opt/1panel/docker/compose/xiaobaifupan` 只是历史文件树:
不是 Git 仓库、内容停在旧提交、与线上镜像不一致,且其 `compose.yaml` 会把构建结果打进
`xiaobai-review:latest`。**禁止在该目录(或任何服务器工作树)里 `docker build` /
`docker compose build`**,否则会把已上线功能悄悄打回旧版。
唯一安全构建方式是在有仓库检出、能免密 SSH 到部署机的机器上运行:
```bash
tools/build_image.sh <提交号> <镜像tag>
# 示例:tools/build_image.sh cefc86917d89 verify-hel235-cefc869
```
该脚本的行为约束:
-`git fetch`,再把提交号解析为完整 SHA,解析失败立即中止,绝不使用本地脏状态或服务器旧目录;
- 镜像 tag 必须以 `-<提交短号7位>` 结尾(如 `hel234-cefc869`),禁止 `latest``rollback-*`
- 通过 `git archive <提交> | ssh 部署机 docker build -` 流式构建,服务器上不存在构建用工作树;
- 构建后回读镜像 label 里的 `org.opencontainers.image.revision`,与预期提交不一致则删除镜像并中止;
- 每次构建在部署机 `~/xiaobai-build/BUILD_LOG.tsv` 留痕,可追溯每个镜像的来源提交。
构建只产出镜像,不启动、不替换任何容器;换版用新 tag 起新容器,回滚用既有镜像 tag 重跑。
### 使用 Gitea 更新程序(旧方式,生产机禁用)
代码仓库为:
生产机 `192.168.200.11``/opt/1panel/docker/compose/xiaobaifupan` 自 2026-08-29HEL-235B
起已是受 Git 管理的工作目录,只跟踪 Gitea `main`(仓库
`http://192.168.200.36:3200/leefer/xiaobai-review.git`)。由于目录顶层归 root
`.git` 存放在部署账号家目录(外部 Git 目录方案):
```text
http://192.168.200.36:3200/leefer/xiaobaifupan.git
~/xiaobai-build/repos/xiaobai-review.git Git 元数据(分支/历史/索引)
/opt/1panel/docker/compose/xiaobaifupan 工作目录(程序文件本体)
~/xiaobai-build/update-from-main.sh 一键更新+构建入口
~/xiaobai-git 便捷查看(status/log/diff
```
首次在服务器部署代码时,可以直接克隆到目标目录
日常更新只需要在服务器上执行一条命令
```bash
sudo mkdir -p /opt/xiaobai-review
sudo chown "$USER":"$USER" /opt/xiaobai-review
git clone http://192.168.200.36:3200/leefer/xiaobaifupan.git /opt/xiaobai-review
cd /opt/xiaobai-review
~/xiaobai-build/update-from-main.sh # 更新到 main 并构建 main-<短号> 镜像
~/xiaobai-build/update-from-main.sh verify-tag main-a8732f5 # 部署前复核镜像与 main 一致
```
私有仓库会提示输入 Gitea 用户名和密码或访问令牌。不要把密码写入仓库 URL、
`compose.yaml` 或脚本。然后把原 `.env``data/` 放回该目录;这两项已被 Git
忽略,后续拉取代码不会覆盖数据库与密钥。
脚本在构建前强制完成五道校验,任一不符立即停止、不产出镜像:
如需部署管理员私有问师,通过 NAS 文件管理器将本地
`data/private-mentor-skills/` 复制到服务器项目的同名 `data` 目录,并保持目录仅由
部署账号和容器运行用户读取。该内容不会通过 Gitea 同步。
1. `git fetch` 成功(连不上 Gitea 即停);
2. 必须在 `main` 分支(智能体不得用功能分支直接当正式线);
3. 工作区无未提交改动、无多余文件;
4. 只允许快进合并到 `origin/main`(分叉即停);main 新增/删除顶层文件时会给出
需管理员执行的精确清单(目录顶层归 root);
5. 构建后回读镜像 `org.opencontainers.image.revision`,与 `main` 提交不一致则删除镜像。
每次更新前先创建 SQLite 一致性备份,再拉取并重建容器(注意:`docker compose up -d --build`
从服务器本地工作树构建,仅适用于来源可信的全新环境;生产机 `192.168.200.11` 禁用,
请用 `tools/build_image.sh` 构建后换容器):
镜像 tag 固定为 `main-<提交短号7位>`(不带提交号的模糊 tag 一律禁止);每次构建在
`~/xiaobai-build/BUILD_LOG.tsv` 留痕。构建只产出镜像,不启动、不替换容器;换版与
回滚步骤见 `~/xiaobai-build/README.md`
```bash
cd /opt/xiaobai-review
docker compose exec -T xiaobai-review python -c "import sqlite3; s=sqlite3.connect('/app/data/review.db'); d=sqlite3.connect('/app/data/review-before-update.db'); s.backup(d); d.close(); s.close()"
git pull --ff-only origin main
docker compose up -d --build
docker compose ps
curl --fail http://127.0.0.1:8765/api/health
```
`compose.yaml` 的镜像名与 revision 标签同样做了强校验:直接 `docker compose up -d --build`
会因缺少 `XIAOBAI_GIT_REV` / `XIAOBAI_GIT_SHORT` 变量而拒绝执行,避免再出现构建进
`latest` 的模糊版本。需要用 compose 时先 `export` 这两个变量(值以
`~/xiaobai-build/xiaobai-git rev-parse HEAD` 为准),或直接用上面的脚本。
`docker compose up -d --build` 会原地替换应用容器,不删除宿主机的 `data` 目录。
数据库迁移会在新容器启动时自动执行。若 `git pull --ff-only` 提示本地代码有修改,
先用 `git status` 查明原因,不要用强制重置覆盖 `.env``data`
### 智能体高级入口:Git 归档流式构建
### 不使用 Git 时更新(生产机禁用)
有仓库检出、能免密 SSH 到部署机的智能体可以用 `tools/build_image.sh <提交号> <镜像tag>`
从任意明确提交流式构建(`git archive | ssh docker build`),tag 同样必须以
`-<提交短号7位>` 结尾,构建后回读 revision 校验并留痕。用于在服务器不便拉取时的
应急构建;日常正式线仍应走 `main`
`docker compose build` 会从服务器本地目录构建,来源提交不可追溯。生产机
`192.168.200.11` 上禁止使用本节方式,一律改用上一节的 `tools/build_image.sh`
### 历史方式(已废弃)
重新上传代码后执行:
```bash
docker compose down
docker compose build --pull
docker compose up -d
```
`docker compose down` 不会删除宿主机的 `data` 目录。不要使用带有手工删除
`data` 目录的清理命令。
早期文档建议在服务器重新 `git clone` 一份或手工上传代码后 `docker compose up --build`
这两条路径已废弃:服务器上**只允许存在一个受管工作目录**(上述
`/opt/1panel/docker/compose/xiaobaifupan`),任何脱离 Git 校验的本地构建都会把
来源提交变成不可追溯状态,禁止使用。
## 7. 备份与恢复
+4 -1
View File
@@ -23,7 +23,10 @@ COPY requirements.txt ./
RUN python -m pip install --no-cache-dir -r requirements.txt
COPY --chown=xiaobai:xiaobai . .
RUN mkdir -p /app/data && chown -R xiaobai:xiaobai /app/data
RUN mkdir -p /app/data && chown -R xiaobai:xiaobai /app/data \
&& test -f /app/data/heaven_knowledge.json \
&& test -f /app/data/iching_zh.json \
&& test -f /app/backend/features/heaven/assets/heaven_knowledge.json
USER xiaobai
+104 -42
View File
@@ -1,76 +1,138 @@
# 小白复盘 Web
# 小白复盘
一个面向 A 股盘后复盘的本地 Web 工作台。后端使用 Python 访问 Tushare Pro,前端不依赖构建工具
面向 A 股盘后复盘的本地 Web 工作台。收盘后把涨停、炸板、连板梯队、板块轮动、集合竞价、龙虎榜等数据整理成可浏览的复盘界面;不接券商、不代为下单,也不提供个股推荐
本目录是经过保真迁移、结构治理和用户人工验收的唯一正式源码,不依赖父目录旧程序或失败版本
目录职责见[ARCHITECTURE.md](ARCHITECTURE.md),产品与维护文档见[docs/README.md](docs/README.md)。
本目录是唯一正式源码。模块边界见 [ARCHITECTURE.md](ARCHITECTURE.md),产品与维护文档见 [docs/README.md](docs/README.md)
当前包含集合竞价、涨停池、炸板池、跌停板、昨日涨停、涨停表现、市场天梯、板块轮动、题材库、人气热榜、龙虎榜和个人复盘工作区。交易日快照与同步记录保存在本地 SQLite 数据库 `data/review.db`
## 主要功能
集合竞价中心采用盘前生命周期:9:15 前显示预告,9:15–9:25 明确等待最终竞价,9:25–9:30 自动读取并重试最终竞价筛选,9:30 后停止更新并冻结为复盘归档。当前 Tushare 只提供 9:25 最终竞价快照,不将其表述为动态虚拟撮合行情
登录后左侧共 16 个页面,另有一个内嵌页「策略持续跟踪」。交易日快照保存在本地 SQLite 数据库 `data/review.db`
第三阶段加入了机构席位、席位别名、个股复权日 K、资金流、自选股、涨停原因修订、个股笔记、每日复盘和历史数据回补。
- **情绪周期**:0–100 情绪温度与阶段判定(默认首页)
- **涨停池 / 炸板池 / 跌停板 / 昨日涨停 / 涨停表现**:封板结构、炸板、跌停与昨日反馈
- **市场天梯**:按连板高度排列的市场梯队
- **板块轮动**:近若干交易日板块热力与成分下钻
- **集合竞价**:盘前生命周期;9:30 后停止更新并冻结为复盘归档。当前数据源提供 9:25 最终竞价快照,不是动态虚拟撮合行情
- **题材库 / 人气热榜 / 龙虎榜**:题材成分、双榜人气、席位与游资档案
- **智能选股**(会员):六阶段策略、精选策略库、自然语言编译为受控公式后的确定性筛选与滚动回测;候选需手动加入后才进入五交易日跟踪
- **问师**(会员):按选定的游资思维 Skill 单师对话;新增公开角色时在 `游资skills` 下增加含 `SKILL.md` 的目录,并在 `游资skills/mentor_catalog.json` 登记。管理员私有角色放在 `data/private-mentor-skills`(不进 Git / 镜像)
- **问天**(会员,冻结区,勿改代码):观势 / 观气 / 观心。卦象、干支、节气与气机由本地程序确定性计算,大模型只负责文字解释
- **我的复盘**:手工交易日志、每日复盘、提醒中心与复盘助手;不接券商、不自动下单
股票代码在桌面端悬停后会显示分时与日 K 快速预览,默认优先展示日 K;移动端点击代码后从底部打开预览面板。股票详情以及板块、题材、指数详情均可在日 K 与最新分时之间切换。日 K 复用个股详情缓存;分时优先使用 iFinD,东方财富仅作隔离的展示兜底,并使用短时内存缓存。图表数据不写入主行情不参与情绪、选股或问天计算;不可用时明确显示“分时不可用”,不会用日 K 模拟分时走势
全局能力:日间 / 夜间主题、股票代码悬停预览日 K 与分时、`Ctrl + K` 全局搜索。图表数据不写入主行情,也不参与情绪、选股或问天计算。
智能选股包含六阶段盘后候选、29 套精选策略、自定义公式 DSL、自然语言公式编译、候选排名和滚动回测。阶段与精选策略在当日行情更新后由后台确定性计算;自定义选股由用户手动执行,LLM 只负责编译自然语言条件,不参与候选筛选。竞价、估值、财务、资金、人气和席位等字段按已登记的数据可用性进入因子库,缺失时明确显示覆盖问题。
## 技术栈
候选只有经用户手动加入后才进入五交易日持续跟踪,展示 T+1 开盘/收盘、T+3、T+5、最大涨幅与最大回撤。提醒中心支持手工日期提醒,并在策略首日反馈和五日跟踪完成时生成账号私有的站内提醒。
| 层面 | 说明 |
| --- | --- |
| 运行时 | Python 3.12;标准库 `ThreadingHTTPServer`,无独立 Web 框架 |
| 依赖 | `requirements.txt` 仅含 `cryptography`;问天历法使用仓库内 `vendor/lunar_python` |
| 数据库 | SQLiteWAL),默认文件 `data/review.db` |
| 前端 | 原生 HTML / CSS / JavaScript,无打包、无构建步骤 |
| 部署 | Docker / Docker Compose,或本机直接运行 `server.py` |
| 安全 | 账号密码 scrypt 哈希;行情 Token 与模型密钥用 `APP_ENCRYPTION_KEY` 加密后存库 |
问师模块会读取当前复盘、近十日市场情绪、涨跌停、昨日反馈、板块轮动、市场阶段、龙虎榜和指定个股数据,再按选中的游资思维 Skill 进行单师对话。对话记录按账号、老师和交易日期保存在服务端;主模型不可用时自动切换辅助模型。
## 环境要求
新增公开问师角色时,在 `游资skills` 下增加一个包含 `SKILL.md` 的独立目录,并在 `游资skills/mentor_catalog.json` 中登记素材等级与结构质检。管理员私有角色放在 `data/private-mentor-skills`,该目录不进入 Git 或 Docker 镜像,且只会出现在管理员的问师列表中。系统会从 Skill 的 frontmatter、一级标题、核心模型和引用语中自动生成角色信息,无需修改注册代码。
- Python 3.12(与 `Dockerfile` 一致)
- 本机启动:能执行 `python` / `pip`
- Docker 部署:Docker Engine 24+Compose v2`docker compose`
- 行情:部署者自行申请并配置 Tushare Pro Token;部分分时优先使用同花顺 iFinD(可选)
- 问师、问天解释、复盘助手、自然语言编译公式:需配置 OpenAI 兼容接口;未配置时市场数据页仍可用
问天模块包含三个相互独立的部分:观势以市场数据生成三才六爻,用于观察“势”,行情缺失或自动取象明显偏差时可显式手动校准六爻,人工结果与自动来源严格区分;观气依据干支、精确节气、五运六气及客主加临关系观察“运”,行业五行仅作传统取象归类;观心先准备1秒,再完成5轮“吸3秒、顿2秒、呼4秒”,随后以六次三枚铜钱起卦、察念和解卦完成一次不输入问题的问心仪式。卦象、干支、节气与气机关系均由本地确定性程序计算,LLM只负责解释,不参与起卦或改动结果。
## 安装与启动
问天模块使用项目本地的 `lunar-python` 计算历法,并使用 `data/iching_zh.json` 中的固定六十四卦、卦辞和爻辞。第三方授权见 `THIRD_PARTY_NOTICES.md`
仓库根目录即为运行目录(`server.py``requirements.txt` 都在根目录)
“我的复盘”包含结构化手工交易日志,可记录方向、价格、数量、仓位、盈亏、逻辑、执行、情绪和标签,不接券商也不自动下单。顶部“复盘助手”以流式方式读取市场统计、策略跟踪、提醒、个人复盘和交易日志;对话按账号保存,只提供分析和条件化计划。
## 启动
```powershell
cd app
```bash
python -m pip install -r requirements.txt
python server.py
```
浏览器打开 `http://127.0.0.1:8765`,首次使用先注册账号。首个账号自动成为管理员,后续账号默认为普通用户。主行情不再回退演示数据:盘前、非交易日或临时取数失败时沿用最近真实收盘快照;没有任何真实快照时提示等待管理员完成首次同步。
默认监听 `127.0.0.1:8765`(仅本机可访问)。浏览器打开该地址,首次使用先注册账号;第一个账号自动成为管理员,之后注册的默认为普通用户。
需要后台启动本地验收端口时,使用`tools/start_local.ps1`。该工具把日志、进程号和Python缓存
统一写入`runtime/`,不在源码根目录产生运行文件:
主行情不再回退演示数据:盘前、非交易日或临时取数失败时沿用最近真实收盘快照;没有任何真实快照时,页面会提示等待管理员完成首次同步。
可选参数:
```bash
python server.py --host 127.0.0.1 --port 8765
```
Windows 下若需要后台启动,并把日志、进程号和 Python 缓存写入 `runtime/`(不在源码根目录产生运行文件):
```powershell
powershell -ExecutionPolicy Bypass -File tools/start_local.ps1 -Port 8797
powershell -ExecutionPolicy Bypass -File tools/start_local.ps1
```
局域网 Docker 部署使用 `Dockerfile``compose.yaml`,完整的迁移、持久化、
防火墙、备份和恢复步骤见 [DOCKER_DEPLOY.md](DOCKER_DEPLOY.md)。
该脚本默认端口为 `8797`。统一验收:
账号密码使用 scrypt 哈希;公共 Tushare Token、平台模型密钥以及原始生辰资料均使用 `APP_ENCRYPTION_KEY` 加密后保存在 SQLite。公共数据和平台模型归系统所有,生辰资料仍按账号隔离。普通用户不配置 LLM,只有管理员授权的有效会员可以使用平台模型。请将 `.env` 与数据库一起备份,丢失加密密钥后无法恢复这些资料。
```bash
python tools/verify_baseline.py
```
## 系统与账号配置
涉及运行时或前端时再加 `--e2e`Playwright)。
管理员通过页面右上角“系统管理”保存公共 Tushare Token、平台主/辅助模型、会员每日额度和后台刷新开关。所有用户读取同一份 SQLite 行情快照,不再分别配置行情 Token。已有个人凭据中的 Tushare Token 会在升级时迁移到系统配置并从个人凭据移除。
## Docker 使用
局域网或服务器部署使用仓库根目录的 `Dockerfile``compose.yaml`。容器监听 `8765`,默认以非 root 用户运行,并把宿主机 `./data` 挂到容器内 `/app/data`
1. 复制 `.env.example``.env`,填入 `APP_ENCRYPTION_KEY` 以及行情 / 模型等初始化配置。密钥不会返回到浏览器。
2. `compose.yaml` 构建时要求带上当前 Git 提交号,避免打出无版本标签的镜像:
```bash
export XIAOBAI_GIT_REV="$(git rev-parse HEAD)"
export XIAOBAI_GIT_SHORT="$(git rev-parse --short=7 HEAD)"
docker compose build
docker compose up -d
```
3. 检查健康接口:
```bash
docker compose ps
curl http://127.0.0.1:8765/api/health
```
健康响应类似 `{"ok": true, "storage": "sqlite", "account_required": true}`
完整的迁移、持久化、防火墙、备份、恢复与正式线构建入口见 [DOCKER_DEPLOY.md](DOCKER_DEPLOY.md)。`.env` 必须与数据库成对备份;丢失 `APP_ENCRYPTION_KEY` 后无法恢复加密资料。
## 目录与配置
```text
TUSHARE_TOKEN=你的Token
server.py 进程入口
backend/ 服务、路由、数据、任务、LLM
frontend/ 无构建前端(shared + pages
config/ 页面 / API / 任务等注册表
data/ SQLite 与私有数据(数据库文件不进 Git)
runtime/ 本地日志、PID、缓存(不进 Git)
tools/ 启动、验收与构建辅助脚本
游资skills/ 公开问师角色
vendor/ 本地第三方库(含 lunar-python
Dockerfile
compose.yaml
.env.example 环境变量模板(复制为 .env 后填写)
```
`.env` 中的 Tushare 和平台 LLM 配置只用于初始化系统配置,密钥不会返回到浏览器。后台刷新只在交易时段更新 SQLite 快照,不会主动刷新或重绘用户页面;用户点击页面“刷新”时读取最新快照。管理员也可点“后台刷新”立即启动一次后台同步,当前页面仍保持不变
管理员通过页面右上角「系统管理」保存公共 Tushare Token、平台主/辅助模型、会员每日额度和后台刷新开关。所有用户读取同一份 SQLite 行情快照。`.env` 中的 Tushare 和平台 LLM 配置只用于初始化系统配置。
普通用户在账号设置中维护个人资料、查看会员状态和修改密码,不配置个人 LLM。有效会员自动使用平台模型;管理员可在“系统管理”中手动开通、续期、停用会员。平台模型受管理员设置的每日调用次数限制,管理员账号始终可用。
普通用户在账号设置中维护个人资料、查看会员状态和修改密码,不配置个人 LLM。有效会员使用平台模型;管理员可开通、续期、停用会员。平台模型受每日调用次数限制,管理员账号始终可用。
Tushare 各接口有独立积分权限。程序优先使用 `limit_list_d` 获取涨跌停明细;该接口不可用时,会尝试通过日线和每日涨跌停价格推算。
相关文档:
## 隔离实时聚合验证
- [ARCHITECTURE.md](ARCHITECTURE.md) — 模块边界
- [docs/README.md](docs/README.md) — 交接手册入口
- [DOCKER_DEPLOY.md](DOCKER_DEPLOY.md) — Docker 部署、备份与恢复
- [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) — 第三方授权(含问天历法库)
- [AGENTS.md](AGENTS.md) — 维护约束
`backend/data/realtime.py`用于验证东方财富、同花顺和选股宝网页数据源。它不写入 SQLite 主行情快照,也不参与情绪评分或智能选股;当 Tushare 实时指数权限不可用时,观势会使用东方财富三大指数和板块外显,并继续使用 Tushare 的板块成分内核与个股数据。
## 注意事项与免责声明
登录后可调用:
```text
GET /api/realtime-aggregate/health?sector=元器件
```
返回内容包括东方财富三大指数及板块快照、指数时间差、同花顺和选股宝可用性、每个来源的耗时与错误。盘中指数时间差不超过15秒,收盘后不超过120秒。`ready=true` 仅表示本次验证满足聚合层约束,不代表这些网页内部接口具有长期稳定性或商业使用授权
- 本项目是个人研究与复盘工具,全部数据、指标、候选与文字分析均不构成投资建议、证券推荐或买卖要约。
- 不接券商、不代为下单。交易日志只做手工记录与统计,不代表实际成交。
- 情绪温度、阶段判定、连板梯队、策略筛选等均为基于公开数据的统计与规则计算,不预测走势,不保证收益。
- 「问天」属于传统文化视角的观察工具,不具备预测功能,不得作为投资依据;该模块为冻结区,不要改其代码。
- 行情来自第三方接口,可能延迟、缺失或口径调整;不可用时页面会明确提示,请以交易所与券商正式披露为准。
- 不要把服务端口直接暴露到公网。不要把 Token、密码、密钥、数据库或 `.env` 提交进 Git。
- 股市有风险,入市需谨慎。投资决策及其后果由使用者本人承担
@@ -41,13 +41,16 @@ class DashboardMixin:
raise TushareError(f"No daily data returned for {trade_date}")
notices: list[str] = []
limit_data_source = "official"
try:
limit_rows = self._load_limit_lists(trade_date)
previous_limit_rows = self._load_limit_type(previous_trade_date, "U")
if not limit_rows:
limit_data_source = "derived"
notices.append("涨跌停高级接口当日数据尚未更新,已使用日线数据推算。")
limit_rows = self._derive_limits(trade_date, daily)
except TushareError as exc:
limit_data_source = "derived"
notices.append(f"涨跌停高级接口不可用,已使用日线数据推算:{exc}")
limit_rows = self._derive_limits(trade_date, daily)
previous_daily = self._load_daily(previous_trade_date)
@@ -79,6 +82,7 @@ class DashboardMixin:
"trade_date": _display_date(trade_date),
"previous_trade_date": _display_date(previous_trade_date),
"source": "tushare",
"limit_data_source": limit_data_source,
"updated_at": datetime.now().astimezone().isoformat(timespec="seconds"),
"notice": "".join(notices),
},
@@ -0,0 +1,117 @@
{
"version": "2026.08.05-5",
"sources": {
"zhouyi": {
"title": "周易经文与十翼",
"scope": "卦辞、爻辞、彖传、象传",
"kind": "public_domain_primary",
"note": "观势与观心只引用本项目已校录的卦爻原文,不把现代网络释文当作原典。"
},
"jingfang": {
"title": "京氏易传",
"scope": "八宫与纳甲体系来源",
"kind": "public_domain_traditional",
"note": "确定性程序采用京房纳甲、八宫世应的通行排法。"
},
"huozhulin": {
"title": "火珠林",
"scope": "纳甲筮法、六亲与日月关系",
"kind": "public_domain_traditional",
"note": "用于观心规则脉络,不直接复制后世简化断语。"
},
"zengshan": {
"title": "增删卜易",
"scope": "用神、世应、动变、日月旺衰",
"kind": "public_domain_traditional",
"note": "只采用可明确编码且有一致输入条件的规则;争议规则单独标记。"
},
"neijing": {
"title": "黄帝内经·素问运气七篇",
"scope": "五运、司天在泉、主客气与运气关系",
"kind": "public_domain_primary",
"note": "观气将原典关系转成当日自我观察语言,不宣称对股价存在因果作用。"
}
},
"trend": {
"method": "本卦说明当下结构,实际动爻说明变化关节,之卦说明所趋结构;多动爻全部保留,不以固定口诀删去用户实际得到的爻。",
"rules": {
"stable": "无动爻时以本卦整体、上下卦关系和大象为主,说明结构的延续条件,不把静止等同于永远不变。",
"single": "一爻动时以该爻的时位、爻辞和象辞为变化核心,并用之卦检查变化后的结构。",
"multiple": "多爻动时逐一保留相关爻义,先找共同方向与冲突,再结合之卦给出有条件的倾向;不得用固定套话把不同动爻压成同一结论。"
}
},
"fortune": {
"principle": "先立中运与司天在泉的年纲,再察当前客气加临主气,最后以日辰说明当日触发;不使用产品权重推导传统结论。",
"movement": {
"太过": "太过表示该运之气偏于有余,解释时同时观察其本气表现与对所胜、所生关系的牵动,不直接等同于吉或凶。",
"不及": "不及表示该运之气偏于不足,解释时同时观察其所不胜来乘与所生受累的可能,不直接等同于弱势结论。"
},
"qi": {
"厥阴风木": "厥阴取风木之动,侧重疏泄、升发、变化与不定;偏盛时可表现为动摇、急变或升散不收。",
"少阴君火": "少阴取君火之明与热,侧重显化、温煦和内在驱动;偏盛时容易躁热,受制时则显而不畅。",
"太阴湿土": "太阴取湿土之濡与承载,侧重黏滞、蓄积和转化;偏盛时容易困重迟缓,得化时则能承接。",
"少阳相火": "少阳取相火之行与枢转,侧重外达、加速和往来;偏盛时容易浮越躁动,受阻时表现为枢机不利。",
"阳明燥金": "阳明取燥金之收与清肃,侧重收敛、裁决和边界;偏盛时容易干急严峻,得润时则清明有序。",
"太阳寒水": "太阳取寒水之藏与凝,侧重潜藏、收引和下行;偏盛时容易凝滞退缩,得温时则蓄势有根。"
},
"relations": {
"same": "客主同气表示同类气相并,重点看是否相得而彰,还是同气偏盛而亢;不能机械判为有利。",
"guest_generates_host": "客生主表示来气生助时令本气,气机较易衔接;仍需观察生助是否过度及年纲是否承接。",
"host_generates_guest": "主生客表示时令本气向来气流转,有相生也有外泄;不能只取相生而忽略主气受耗。",
"guest_controls_host": "客克主表示来气制约主气,传统称客胜为从;重点解释外来变化居上及原有节律受制。",
"host_controls_guest": "主克客表示主气制约来气,传统称主胜为逆;重点解释时令与来气相持而不把相克直接断凶。"
},
"day_trigger": "日辰只说明当日关系如何被触发,不与中运、司天在泉或主客气并列重复计权。",
"industry_boundary": "五行对应行业只作传统取象:可以说明本次已经出现的五行之气对相应行业形成的象征性关注、节奏或约束,但不得读取或猜测行业实时行情,不得预测涨跌,也不得把取象写成投资推荐。",
"personal_boundary": "personal.natal_day_master才是用户本命日主;today_relative_to_natal_day_master中的pillars是当日历法,stem_relations只是当日年、月、日三柱天干相对本命日主的确定性关系标签。只能使用本次检索到的关系释义,不得自行重算十神、扩展五行生克、使用藏干、库气或支的燥湿属性,也不得把当日日柱称为用户命局,或由这些字段推断命局中某一十神偏重、身强身弱或喜用神。",
"personal_relations": {
"比肩": "比肩作为当日天干关系标签,只提示用户可能更在意自主判断、同类比较或坚持原有立场;不能据此判断命局强弱或现实事件。",
"劫财": "劫财作为当日天干关系标签,只提示用户留意精力、注意力或可支配资源在同类事项间的分流与竞争感;不等同于破财或他人争夺。",
"食神": "食神作为当日天干关系标签,只提示用户留意表达、输出、舒缓与完成感;不等同于收益或确定的轻松结果。",
"伤官": "伤官作为当日天干关系标签,只提示用户留意质疑规则、急于表达或追求自主空间的倾向;不等同于冲突或违规。",
"偏财": "偏财作为当日天干关系标签,只提示用户留意机会分配、灵活取舍与非固定资源的吸引力;不等同于意外获利。",
"正财": "正财作为当日天干关系标签,只提示用户更关注可核对的结果、资源边界和务实落地;不等同于必得收益或现金变化。",
"七杀": "七杀作为当日天干关系标签,只提示用户留意紧迫感、外部压力和快速决断冲动;不等同于危险必然发生。",
"正官": "正官作为当日天干关系标签,只提示用户更在意规则、责任、秩序和可交付标准;不等同于结果必然受控。",
"偏印": "偏印作为当日天干关系标签,只提示用户留意内省、非惯常信息和反复推敲的倾向;不等同于退缩、失眠或方向错误。",
"正印": "正印作为当日天干关系标签,只提示用户更在意依据、支持、学习和安全边界;不等同于必然获得帮助。"
}
},
"heart": {
"presets": {
"trade": "关于我心中的这笔交易,此刻最需要看清的机会、阻碍与风险是什么?",
"mind": "此刻影响我交易判断的情绪、执念或盲点是什么?",
"unthemed": "不设具体问题,只观此刻一念。"
},
"focus": {
"trade": "以世爻、应爻、妻财爻及实际动变为主要检索对象,同时检查兄弟、官鬼和子孙的生克,不把任何单一六亲固定判吉凶。",
"mind": "以世爻和实际动爻为主,观察官鬼所示压力、子孙所示舒解及内外生克;不把心境问题强行翻译成价格方向。",
"unthemed": "不强选事项用神,以本卦、世爻、实际动爻和之卦作一般观照,不猜测用户没有提出的问题。",
"custom": "先依据用户明确写出的股票交易问题选择相关六亲;无法明确归类时退回世爻、动爻和卦变的一般解释,不擅自补全问题。"
},
"evidence_order": [
"用户问题与预设来源",
"本卦及卦宫",
"世应与所问相关六亲",
"月建日辰、旬空及冲合生克",
"实际动爻与变爻",
"之卦与整体卦义",
"六神辅助象义"
],
"limits": "六神只作辅助象义;空亡、月破、日冲、合冲刑害均需结合用神、世应和动变,不得单项宣布结果。",
"semantics": {
"self_response": "世爻表示求测者当前立场与承受状态,应爻表示所问事项的外部一端或对照面。应爻不是固定的合作方、庄家或资金方;只有用户问题明确给出该角色时,才可作对应解释。",
"calendar": "月建与日辰用于判断爻在起卦时刻的承受、生扶和制约。旬空表示该爻所象征的条件当下可能未落实、难发挥或有名无实,但不能单凭旬空判失败,也不能用填实日期预测何时涨跌或行动。月破、日冲、六合、六冲、六害和相刑同样必须与世应、相关六亲及动变合看。",
"movement": "动爻说明关系正在变化;变爻说明变化后的承接方向。回头生、回头克和原变爻生克只描述力量关系,不自动对应现实中的借贷、融资、合作或某个具体人物。进神退神只说明同类地支变化的进退趋势,不直接宣布价格方向。",
"six_spirits": "六神只补充表达色彩,不单独定成败。青龙不必然有利,白虎不必然紧急或凶险,朱雀不必然等同口舌,玄武不必然等同欺骗,勾陈与螣蛇也不得脱离爻位、六亲和动变独断。",
"timing_boundary": "观心不作应期预测。可以说明某项条件在起卦时刻尚未落实或受制,但不得给出未来若干日、某干支日、出空或填实后必然发生什么。",
"relatives": {
"兄弟": "兄弟是与卦宫五行同类的关系。在股票交易问题中可作为竞争、同类力量或资源分流的候选象义,但不直接等同合作方、亏损或他人拿走资金。",
"子孙": "子孙是卦宫所生的关系,可作为舒缓、产出、执行后的释放或对压力的制衡候选象义,但不直接等同收益、资金提供方或确定的利好。",
"妻财": "妻财是卦宫所克的关系,在股票交易问题中可作为价值、收益预期、持仓利益或可支配资源的候选象义,但不直接等同现金、融资、自有资金或必得之财。",
"官鬼": "官鬼是克制卦宫的关系,可作为压力、风险、规则约束或担忧的候选象义,但不直接等同借贷、坏消息、疾病或必然损失。",
"父母": "父母是生助卦宫的关系,可作为信息、依据、计划、规则、凭据或保护条件的候选象义,但不直接等同政策、合同或某一条消息。"
}
}
}
}
+13 -4
View File
@@ -3,15 +3,24 @@ from __future__ import annotations
import json
from http import HTTPStatus
from backend.features.heaven.knowledge import HeavenKnowledgeError
class HeavenHttpMixin:
def _send_heaven_client_error(self, exc: Exception) -> None:
payload: dict = {"error": str(exc)}
code = getattr(exc, "error_code", None)
if code:
payload["code"] = str(code)
self.send_json(payload, HTTPStatus.BAD_REQUEST)
def heaven_hexagram(self) -> None:
try:
body = self.read_json_body()
result = self.application_service.heaven_hexagram(body.get("lines"))
self.send_json({"ok": True, "hexagram": result})
except (ValueError, json.JSONDecodeError) as exc:
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
self._send_heaven_client_error(exc)
def heaven_personal(self) -> None:
try:
@@ -19,12 +28,12 @@ class HeavenHttpMixin:
result = self.application_service.heaven_personal(body)
self.send_json({"ok": True, "personal": result})
except (ValueError, json.JSONDecodeError) as exc:
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
self._send_heaven_client_error(exc)
def heaven_interpret(self) -> None:
try:
body = self.read_json_body()
result = self.application_service.heaven_interpret(body)
self.send_json({"ok": True, **result})
except (ValueError, json.JSONDecodeError) as exc:
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
except (HeavenKnowledgeError, ValueError, json.JSONDecodeError) as exc:
self._send_heaven_client_error(exc)
+53 -2
View File
@@ -2,12 +2,23 @@ from __future__ import annotations
import json
from functools import lru_cache
from pathlib import Path
from typing import Any
from backend.bootstrap.config import APP_DIR
KNOWLEDGE_FILE = APP_DIR / "data" / "heaven_knowledge.json"
# Baked into the image outside the ./data bind mount so volume overlay cannot hide it.
KNOWLEDGE_SEED_FILE = Path(__file__).resolve().parent / "assets" / "heaven_knowledge.json"
class HeavenKnowledgeError(ValueError):
"""Structured knowledge-file failure surfaced to HTTP as Chinese API errors."""
def __init__(self, message: str, *, code: str) -> None:
super().__init__(message)
self.error_code = code
def prepare_heaven_context(mode: str, calculation: dict[str, Any]) -> dict[str, Any]:
@@ -379,9 +390,49 @@ def _line_record(line: dict[str, Any]) -> dict[str, Any]:
}
def resolve_heaven_knowledge_path() -> Path:
"""Prefer the persisted data-dir file; fall back to the image-baked seed."""
if KNOWLEDGE_FILE.is_file():
return KNOWLEDGE_FILE
if KNOWLEDGE_SEED_FILE.is_file():
return KNOWLEDGE_SEED_FILE
raise HeavenKnowledgeError(
"问天知识文件缺失:未找到 heaven_knowledge.json。"
"请确认宿主机 data 目录或镜像内 seed 文件完整。",
code="heaven_knowledge_missing",
)
@lru_cache(maxsize=1)
def _knowledge_catalog() -> dict[str, Any]:
payload = json.loads(KNOWLEDGE_FILE.read_text(encoding="utf-8"))
path = resolve_heaven_knowledge_path()
try:
raw = path.read_text(encoding="utf-8")
except OSError as exc:
raise HeavenKnowledgeError(
f"问天知识文件无法读取({path.name}):{exc.strerror or exc}",
code="heaven_knowledge_missing",
) from exc
try:
payload = json.loads(raw)
except json.JSONDecodeError as exc:
raise HeavenKnowledgeError(
f"问天知识文件 JSON 损坏({path.name}),无法解析:"
f"{exc.lineno} 行附近。",
code="heaven_knowledge_invalid",
) from exc
if not isinstance(payload, dict):
raise HeavenKnowledgeError(
f"问天知识文件格式不正确({path.name}):根节点必须是对象。",
code="heaven_knowledge_invalid",
)
if not payload.get("version") or not isinstance(payload.get("sources"), dict):
raise ValueError("问天知识库格式不完整。")
raise HeavenKnowledgeError(
f"问天知识库格式不完整({path.name}):缺少 version 或 sources。",
code="heaven_knowledge_invalid",
)
return payload
def clear_heaven_knowledge_cache() -> None:
_knowledge_catalog.cache_clear()
+202
View File
@@ -0,0 +1,202 @@
"""Auditable recent-trading-day snapshot backfill helpers.
Planning and backup stay free of provider imports so feature boundary tests remain green.
The service layer supplies open trading dates from the live calendar and executes sync.
"""
from __future__ import annotations
import sqlite3
from datetime import date, datetime, timedelta
from pathlib import Path
from typing import Any, Iterable
MAX_RANGE_TRADING_DAYS = 15
MAX_RECENT_TRADING_DAYS = 60
DEFAULT_RECENT_TRADING_DAYS = 60
# Tables touched by a successful historical dashboard sync. User / token / model
# tables must never appear here.
SNAPSHOT_BACKFILL_WRITE_TABLES = frozenset(
{
"dashboard_snapshots",
"data_snapshots",
"sync_runs",
}
)
def clamp_recent_lookback(lookback: int) -> int:
value = int(lookback)
if value < 1:
raise ValueError("回补交易日数量至少为 1。")
if value > MAX_RECENT_TRADING_DAYS:
raise ValueError(f"单次最多回补最近 {MAX_RECENT_TRADING_DAYS} 个交易日。")
return value
def calendar_window_start(end_date: str, lookback: int) -> str:
"""Natural-day lower bound large enough to cover lookback open sessions."""
end = datetime.strptime(end_date, "%Y%m%d").date()
span = max(40, int(lookback * 2) + 20)
return (end - timedelta(days=span)).strftime("%Y%m%d")
def select_open_trade_dates(
calendar_rows: Iterable[dict[str, Any]],
end_date: str,
lookback: int,
) -> list[str]:
"""Pick the last ``lookback`` open SSE sessions on or before ``end_date``."""
lookback = clamp_recent_lookback(lookback)
end = normalize_compact_date(end_date)
open_dates = sorted(
{
normalize_compact_date(str(row.get("cal_date") or ""))
for row in calendar_rows
if int(row.get("is_open") or 0) == 1 and row.get("cal_date")
}
)
open_dates = [item for item in open_dates if item <= end]
if not open_dates:
raise ValueError("交易日历未返回可用交易日,请检查行情 Token。")
return open_dates[-lookback:]
def select_open_trade_dates_in_range(
calendar_rows: Iterable[dict[str, Any]],
start_date: str,
end_date: str,
*,
maximum: int = MAX_RANGE_TRADING_DAYS,
) -> tuple[list[str], list[str]]:
"""Return (open_dates, skipped_non_trading_days) inside an inclusive range."""
start = normalize_compact_date(start_date)
end = normalize_compact_date(end_date)
if start > end:
raise ValueError("开始日期不能晚于结束日期。")
open_set = {
normalize_compact_date(str(row.get("cal_date") or ""))
for row in calendar_rows
if int(row.get("is_open") or 0) == 1 and row.get("cal_date")
}
open_dates: list[str] = []
skipped: list[str] = []
cursor = datetime.strptime(start, "%Y%m%d").date()
last = datetime.strptime(end, "%Y%m%d").date()
while cursor <= last:
compact = cursor.strftime("%Y%m%d")
if compact in open_set:
open_dates.append(compact)
else:
skipped.append(compact)
cursor += timedelta(days=1)
if len(open_dates) > maximum:
raise ValueError(f"单次最多回补 {maximum} 个交易日。")
return open_dates, skipped
def classify_snapshot_coverage(
trade_dates: list[str],
existing_dates: Iterable[str],
) -> dict[str, Any]:
present_set = {
normalize_compact_date(item)
for item in existing_dates
if item
}
present = [item for item in trade_dates if item in present_set]
missing = [item for item in trade_dates if item not in present_set]
return {
"trade_dates": list(trade_dates),
"present": present,
"missing": missing,
"present_count": len(present),
"missing_count": len(missing),
}
def create_sqlite_backup(
source_path: Path,
backup_dir: Path,
*,
label: str = "pre-backfill",
stamped_at: datetime | None = None,
) -> Path:
"""Create a timestamped SQLite backup via the native backup API."""
source = Path(source_path)
if not source.exists():
raise FileNotFoundError(f"数据库不存在:{source}")
stamp = (stamped_at or datetime.now().astimezone()).strftime("%Y%m%d-%H%M%S")
safe_label = "".join(ch if ch.isalnum() or ch in "-_" else "-" for ch in label).strip("-") or "backup"
backup_dir = Path(backup_dir)
backup_dir.mkdir(parents=True, exist_ok=True)
target = backup_dir / f"review-{safe_label}-{stamp}.db"
source_conn = sqlite3.connect(f"file:{source}?mode=ro", uri=True)
try:
target_conn = sqlite3.connect(target)
try:
source_conn.backup(target_conn)
target_conn.commit()
finally:
target_conn.close()
finally:
source_conn.close()
return target
def display_date(compact: str) -> str:
value = normalize_compact_date(compact)
return f"{value[:4]}-{value[4:6]}-{value[6:8]}"
def normalize_compact_date(value: str) -> str:
compact = str(value or "").replace("-", "").strip()
if len(compact) != 8 or not compact.isdigit():
raise ValueError("日期格式应为 YYYY-MM-DD。")
datetime.strptime(compact, "%Y%m%d")
return compact
def build_backfill_audit(
*,
mode: str,
end_date: str,
lookback: int | None,
coverage: dict[str, Any],
skipped_non_trading_days: list[str] | None = None,
backup_path: str | None = None,
dry_run: bool = False,
results: list[dict[str, Any]] | None = None,
) -> dict[str, Any]:
results = list(results or [])
succeeded = [row for row in results if row.get("status") == "success"]
skipped = [row for row in results if row.get("status") == "skipped"]
failed = [row for row in results if row.get("status") == "failed"]
return {
"ok": not failed,
"mode": mode,
"dry_run": dry_run,
"end_date": display_date(end_date),
"lookback": lookback,
"backup_path": backup_path,
"write_tables": sorted(SNAPSHOT_BACKFILL_WRITE_TABLES),
"trade_dates": [display_date(item) for item in coverage.get("trade_dates") or []],
"present": [display_date(item) for item in coverage.get("present") or []],
"missing": [display_date(item) for item in coverage.get("missing") or []],
"skipped_non_trading_days": [
display_date(item) for item in (skipped_non_trading_days or [])
],
"present_count": int(coverage.get("present_count") or 0),
"missing_count": int(coverage.get("missing_count") or 0),
"results": results,
"succeeded_count": len(succeeded),
"skipped_count": len(skipped),
"failed_count": len(failed),
"created_dates": [
str(row.get("trade_date") or "")
for row in succeeded
if row.get("action") == "created"
],
}
+25
View File
@@ -227,6 +227,31 @@ class MarketRepositoryMixin:
result.append(payload)
return result
def list_snapshot_trade_dates(
self,
start_date: str = "",
end_date: str = "",
) -> list[str]:
clauses: list[str] = []
parameters: list[Any] = []
if start_date:
clauses.append("trade_date >= ?")
parameters.append(start_date)
if end_date:
clauses.append("trade_date <= ?")
parameters.append(end_date)
where = f"WHERE {' AND '.join(clauses)}" if clauses else ""
with self.connect() as connection:
rows = connection.execute(
f"""
SELECT trade_date FROM dashboard_snapshots
{where}
ORDER BY trade_date
""",
parameters,
).fetchall()
return [str(row["trade_date"]) for row in rows]
def start_sync(self, trade_date: str, source: str) -> int:
started_at = datetime.now().astimezone().isoformat(timespec="seconds")
with self.connect() as connection:
+234 -22
View File
@@ -3,9 +3,11 @@ from __future__ import annotations
import copy
import re
from datetime import date, datetime, time as dt_time, timedelta
from pathlib import Path
from typing import Any
from backend.bootstrap.config import (
DATA_DIR,
normalize_date,
tushare_code,
validate_stock_code,
@@ -13,6 +15,17 @@ from backend.bootstrap.config import (
)
from backend.data.providers.ifind_client import IfindError
from backend.data.providers.tushare_client import TushareClient, TushareError
from backend.features.market.backfill_history import (
DEFAULT_RECENT_TRADING_DAYS,
MAX_RANGE_TRADING_DAYS,
build_backfill_audit,
calendar_window_start,
classify_snapshot_coverage,
create_sqlite_backup,
display_date,
select_open_trade_dates,
select_open_trade_dates_in_range,
)
from backend.features.market.charts import ChartDataError
from backend.features.market.insights import MarketInsightsService
from backend.features.sentiment.engine import SENTIMENT_ENGINE_VERSION
@@ -185,6 +198,11 @@ class MarketServiceMixin:
raise TushareError("公共行情尚未配置")
dashboard = self._tushare_client().dashboard(normalized_date)
if (dashboard.get("meta") or {}).get("limit_data_source") == "derived":
raise TushareError(
str((dashboard.get("meta") or {}).get("notice") or "官方涨跌停数据尚未返回")
)
dashboard["meta"]["source"] = source
dashboard["meta"]["requested_date"] = self._display_compact_date(normalized_date)
dashboard = self._enrich_dashboard_sentiment(dashboard, normalized_date)
@@ -890,31 +908,226 @@ class MarketServiceMixin:
"intraday": intraday_points,
}
def backfill(self, start_date: str, end_date: str) -> list[dict[str, Any]]:
start = datetime.strptime(normalize_date(start_date), "%Y%m%d").date()
end = datetime.strptime(normalize_date(end_date), "%Y%m%d").date()
if start > end:
raise ValueError("开始日期不能晚于结束日期。")
weekdays = []
current = start
while current <= end:
if current.weekday() < 5:
weekdays.append(current)
current += timedelta(days=1)
if len(weekdays) > 15:
raise ValueError("单次最多回补 15 个工作日。")
results = []
for day in weekdays:
dashboard = self.sync_dashboard(day.strftime("%Y%m%d"))
def backfill(
self,
start_date: str = "",
end_date: str = "",
*,
lookback: int | None = None,
dry_run: bool = False,
force: bool = False,
create_backup: bool = True,
) -> dict[str, Any]:
"""Backfill dashboard snapshots for real trading days only.
- Date-range mode keeps the admin UI contract (max 15 open sessions).
- Recent mode fills the last N open sessions (default/max 60).
Weekends and holidays are reported as skipped non-trading days, not errors.
"""
if not self.configured:
raise ValueError("公共行情尚未配置,无法回补历史快照。")
normalized_end = normalize_date(end_date or date.today().isoformat())
if lookback is not None or not (start_date and end_date):
target_lookback = (
DEFAULT_RECENT_TRADING_DAYS if lookback is None else int(lookback)
)
return self.backfill_recent_trading_days(
end_date=normalized_end,
lookback=target_lookback,
dry_run=dry_run,
force=force,
create_backup=create_backup,
)
return self._backfill_date_range(
start_date=normalize_date(start_date),
end_date=normalized_end,
dry_run=dry_run,
force=force,
create_backup=create_backup,
)
def backfill_recent_trading_days(
self,
end_date: str = "",
lookback: int = DEFAULT_RECENT_TRADING_DAYS,
*,
dry_run: bool = False,
force: bool = False,
create_backup: bool = True,
) -> dict[str, Any]:
normalized_end = normalize_date(end_date or date.today().isoformat())
trade_dates = self._load_recent_open_trade_dates(normalized_end, lookback)
existing = self.database.list_snapshot_trade_dates(
trade_dates[0], trade_dates[-1]
)
coverage = classify_snapshot_coverage(trade_dates, existing)
return self._execute_snapshot_backfill(
mode="recent",
end_date=normalized_end,
lookback=lookback,
coverage=coverage,
skipped_non_trading_days=[],
dry_run=dry_run,
force=force,
create_backup=create_backup,
)
def _backfill_date_range(
self,
start_date: str,
end_date: str,
*,
dry_run: bool = False,
force: bool = False,
create_backup: bool = True,
) -> dict[str, Any]:
window_start = calendar_window_start(end_date, MAX_RANGE_TRADING_DAYS)
calendar_rows = self._tushare_client().query(
"trade_cal",
{
"exchange": "SSE",
"start_date": min(window_start, start_date),
"end_date": end_date,
},
"cal_date,is_open,pretrade_date",
)
trade_dates, skipped = select_open_trade_dates_in_range(
calendar_rows,
start_date,
end_date,
maximum=MAX_RANGE_TRADING_DAYS,
)
if not trade_dates:
raise ValueError("选定区间内没有交易日,周末或节假日无需回补。")
existing = self.database.list_snapshot_trade_dates(trade_dates[0], trade_dates[-1])
coverage = classify_snapshot_coverage(trade_dates, existing)
return self._execute_snapshot_backfill(
mode="range",
end_date=end_date,
lookback=None,
coverage=coverage,
skipped_non_trading_days=skipped,
dry_run=dry_run,
force=force,
create_backup=create_backup,
)
def _load_recent_open_trade_dates(self, end_date: str, lookback: int) -> list[str]:
start_date = calendar_window_start(end_date, lookback)
calendar_rows = self._tushare_client().query(
"trade_cal",
{
"exchange": "SSE",
"start_date": start_date,
"end_date": end_date,
},
"cal_date,is_open,pretrade_date",
)
return select_open_trade_dates(calendar_rows, end_date, lookback)
def _execute_snapshot_backfill(
self,
*,
mode: str,
end_date: str,
lookback: int | None,
coverage: dict[str, Any],
skipped_non_trading_days: list[str],
dry_run: bool,
force: bool,
create_backup: bool,
) -> dict[str, Any]:
targets = list(coverage["trade_dates"] if force else coverage["missing"])
backup_path: str | None = None
if create_backup and not dry_run and targets:
backup = create_sqlite_backup(
Path(self.database.path),
DATA_DIR / "backups",
label=f"pre-{mode}-backfill",
)
backup_path = str(backup)
results: list[dict[str, Any]] = []
if dry_run:
for trade_date in coverage["trade_dates"]:
exists = trade_date in coverage["present"]
if exists and not force:
status = "skipped"
action = "exists"
else:
status = "planned"
action = "refresh" if exists else "create"
results.append(
{
"requested_date": display_date(trade_date),
"trade_date": display_date(trade_date),
"status": status,
"action": action,
}
)
return build_backfill_audit(
mode=mode,
end_date=end_date,
lookback=lookback,
coverage=coverage,
skipped_non_trading_days=skipped_non_trading_days,
backup_path=backup_path,
dry_run=True,
results=results,
)
present_before = set(coverage["present"])
for trade_date in targets:
existed = trade_date in present_before
try:
dashboard = self.sync_dashboard(trade_date)
actual = normalize_date(
str(dashboard.get("meta", {}).get("trade_date") or trade_date)
)
results.append(
{
"requested_date": display_date(trade_date),
"trade_date": display_date(actual),
"status": "success",
"action": "refreshed" if existed else "created",
"source": dashboard.get("meta", {}).get("source"),
"records": self._record_count(dashboard),
}
)
except Exception as exc:
results.append(
{
"requested_date": display_date(trade_date),
"trade_date": display_date(trade_date),
"status": "failed",
"action": "refresh" if existed else "create",
"error": str(exc),
}
)
for trade_date in coverage["present"]:
if force:
continue
results.append(
{
"requested_date": day.isoformat(),
"trade_date": dashboard["meta"]["trade_date"],
"source": dashboard["meta"]["source"],
"records": self._record_count(dashboard),
"requested_date": display_date(trade_date),
"trade_date": display_date(trade_date),
"status": "skipped",
"action": "exists",
}
)
return results
results.sort(key=lambda row: str(row.get("requested_date") or ""))
return build_backfill_audit(
mode=mode,
end_date=end_date,
lookback=lookback,
coverage=coverage,
skipped_non_trading_days=skipped_non_trading_days,
backup_path=backup_path,
dry_run=False,
results=results,
)
def _stock_identity(self, code: str, trade_date: str) -> tuple[str, str]:
snapshot = self.database.get_snapshot(trade_date) or {}
@@ -955,4 +1168,3 @@ class MarketServiceMixin:
len(dashboard.get(key) or [])
for key in ("limits", "broken", "down_limits", "yesterday_limits")
)
+3 -1
View File
@@ -26,13 +26,15 @@ class SystemHttpMixin:
def start_background_refresh(self) -> None:
try:
body = self.read_json_body(allow_empty=True)
started = self.application_service.request_background_sync(
refresh = self.application_service.request_background_sync(
str(body.get("trade_date") or date.today().isoformat())
)
started = bool(refresh.get("started"))
self.send_json(
{
"ok": True,
"started": started,
"job_key": str(refresh.get("job_key") or ""),
"message": "后台刷新已开始" if started else "已有后台刷新任务正在运行",
},
HTTPStatus.ACCEPTED,
+8 -2
View File
@@ -29,11 +29,17 @@ class SystemRoutesMixin:
def backfill_data(self) -> None:
try:
body = self.read_json_body()
results = self.application_service.backfill(
lookback_raw = body.get("lookback")
lookback = int(lookback_raw) if lookback_raw not in (None, "") else None
audit = self.application_service.backfill(
str(body.get("start_date") or ""),
str(body.get("end_date") or ""),
lookback=lookback,
dry_run=bool(body.get("dry_run")),
force=bool(body.get("force")),
create_backup=body.get("create_backup", True) is not False,
)
self.send_json({"ok": True, "results": results})
self.send_json({"ok": True, **audit, "results": audit.get("results") or []})
except ValueError as exc:
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
except Exception as exc:
+14 -3
View File
@@ -7,6 +7,16 @@ from datetime import date
from backend.bootstrap.config import normalize_date
def _verified_dashboard_result(dashboard: dict[str, object]) -> dict[str, object]:
meta = dashboard.get("meta") or {}
if isinstance(meta, dict) and meta.get("carried_forward"):
return {
"status": "failed",
"error": str(meta.get("notice") or "未获取到所选日期的最新行情"),
}
return dashboard
class JobServiceMixin:
def start_background_jobs(self) -> threading.Thread:
return self.jobs.start_scheduler(
@@ -20,15 +30,16 @@ class JobServiceMixin:
workers_stopped = self.jobs.wait_for_idle(timeout_seconds)
return scheduler_stopped and workers_stopped
def request_background_sync(self, trade_date: str) -> bool:
def request_background_sync(self, trade_date: str) -> dict[str, object]:
normalized = normalize_date(trade_date)
key = f"manual:{normalized}:{time.time_ns()}"
return self.jobs.submit(
started = self.jobs.submit(
"market.refresh",
key,
lambda: self.sync_dashboard(normalized),
lambda: _verified_dashboard_result(self.sync_dashboard(normalized)),
{"trade_date": normalized, "trigger": "administrator"},
)
return {"started": started, "job_key": key if started else ""}
def _background_refresh_tick(self) -> None:
if not (
+45
View File
@@ -0,0 +1,45 @@
# Optional overlay. Does not replace the existing xiaobai-review service.
# Start later (总工部署时) with:
# docker compose -f compose.yaml -f compose.datahub.yaml up -d
#
# Required .env keys: DATAHUB_ENCRYPTION_KEY, DATAHUB_TOKEN, DATAHUB_ADMIN_PASSWORD, TUSHARE_TOKEN
services:
xiaobai-datahub:
build:
context: ./xiaobai-datahub
dockerfile: Dockerfile
image: xiaobai-datahub:local
container_name: xiaobai-datahub
ports:
- "0.0.0.0:8766:8766/tcp"
env_file:
- ./xiaobai-datahub/.env
environment:
DATAHUB_ENCRYPTION_KEY: "${DATAHUB_ENCRYPTION_KEY:?DATAHUB_ENCRYPTION_KEY must be set}"
DATAHUB_TOKEN: "${DATAHUB_TOKEN:?DATAHUB_TOKEN must be set}"
DATAHUB_ADMIN_PASSWORD: "${DATAHUB_ADMIN_PASSWORD:?DATAHUB_ADMIN_PASSWORD must be set}"
TUSHARE_TOKEN: "${TUSHARE_TOKEN:-}"
DATAHUB_DB_PATH: /app/data/datahub.db
DATAHUB_BACKUP_DIR: /app/data/backups
TZ: Asia/Shanghai
PYTHONUTF8: "1"
volumes:
- type: bind
source: ./datahub-data
target: /app/data
restart: unless-stopped
init: true
read_only: true
tmpfs:
- /tmp:size=64m,mode=1777
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
stop_grace_period: 30s
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
+3 -1
View File
@@ -3,7 +3,9 @@ services:
build:
context: .
dockerfile: Dockerfile
image: xiaobai-review:latest
labels:
org.opencontainers.image.revision: "${XIAOBAI_GIT_REV:?必须先设置 XIAOBAI_GIT_REV=当前 main 完整提交号(或改用 tools/update_from_main.sh}"
image: xiaobai-review:main-${XIAOBAI_GIT_SHORT:?必须先设置 XIAOBAI_GIT_SHORT=7位提交短号(或改用 tools/update_from_main.sh}
container_name: xiaobai-review
ports:
- "0.0.0.0:8765:8765/tcp"
+40 -40
View File
@@ -374,11 +374,11 @@
}
],
"css_layers": [
"/shared/tokens.css?v=20260829-1",
"/shared/tokens.css?v=20260829-hel240",
"/shared/base.css?v=20260806-1",
"/shared/shell.css?v=20260820-8",
"/shared/auth.css?v=20260829-1",
"/shared/components/controls.css?v=20260820-2",
"/shared/shell.css?v=20260829-hel237",
"/shared/auth.css?v=20260829-hel240b",
"/shared/components/controls.css?v=20260829-hel237",
"/shared/components/navigation.css?v=20260820-1",
"/shared/components/cards.css?v=20260820-1",
"/shared/components/tables.css?v=20260820-1",
@@ -394,8 +394,8 @@
"/pages/popularity/foundation.css?v=20260820-1",
"/pages/dragon-tiger/foundation.css?v=20260820-1",
"/pages/screener/foundation.css?v=20260820-4",
"/pages/mentor/foundation.css?v=20260820-2",
"/pages/heaven/foundation.css?v=20260806-2",
"/pages/mentor/foundation.css?v=20260827-hel183",
"/pages/heaven/foundation.css?v=20260827-hel183",
"/pages/review/foundation.css?v=20260820-4"
],
"frontend_composition": {
@@ -440,8 +440,8 @@
"code_hotspots": [
{
"path": "frontend/pages/heaven/foundation.css",
"bytes": 185936,
"lines": 11734
"bytes": 182616,
"lines": 11494
},
{
"path": "frontend/pages/screener/foundation.css",
@@ -450,13 +450,13 @@
},
{
"path": "frontend/pages/heaven/page.js",
"bytes": 97189,
"lines": 2069
"bytes": 97268,
"lines": 2070
},
{
"path": "frontend/shared/shell.css",
"bytes": 63550,
"lines": 3757
"bytes": 63659,
"lines": 3763
},
{
"path": "backend/features/heaven/engine.py",
@@ -465,8 +465,8 @@
},
{
"path": "frontend/index.html",
"bytes": 48037,
"lines": 663
"bytes": 48254,
"lines": 664
},
{
"path": "backend/features/screener/catalog.py",
@@ -490,8 +490,8 @@
},
{
"path": "backend/data/providers/tushare_dashboard.py",
"bytes": 28051,
"lines": 644
"bytes": 28234,
"lines": 648
},
{
"path": "backend/data/providers/tushare_industries.py",
@@ -505,8 +505,8 @@
},
{
"path": "frontend/pages/heaven/page.html",
"bytes": 19747,
"lines": 262
"bytes": 19885,
"lines": 269
},
{
"path": "frontend/pages/screener/page.html",
@@ -545,8 +545,8 @@
},
{
"path": "frontend/shared/admin.js",
"bytes": 14145,
"lines": 261
"bytes": 14410,
"lines": 268
},
{
"path": "backend/features/heaven/market_context.py",
@@ -555,8 +555,13 @@
},
{
"path": "frontend/shared/session.js",
"bytes": 12848,
"lines": 283
"bytes": 13219,
"lines": 289
},
{
"path": "frontend/shared/dashboard.js",
"bytes": 12894,
"lines": 274
},
{
"path": "backend/features/market/insights_auction_data.py",
@@ -578,11 +583,6 @@
"bytes": 10539,
"lines": 244
},
{
"path": "frontend/shared/dashboard.js",
"bytes": 9993,
"lines": 220
},
{
"path": "backend/data/providers/tushare_sectors.py",
"bytes": 9876,
@@ -673,16 +673,16 @@
"bytes": 5451,
"lines": 118
},
{
"path": "frontend/pages.config.js",
"bytes": 5385,
"lines": 130
},
{
"path": "frontend/pages/market/search.js",
"bytes": 5384,
"lines": 131
},
{
"path": "frontend/pages.config.js",
"bytes": 5380,
"lines": 130
},
{
"path": "frontend/pages/auction/page.html",
"bytes": 5350,
@@ -773,6 +773,11 @@
"bytes": 2299,
"lines": 57
},
{
"path": "backend/jobs/service.py",
"bytes": 2219,
"lines": 60
},
{
"path": "backend/features/screener/regime.py",
"bytes": 2202,
@@ -804,9 +809,9 @@
"lines": 45
},
{
"path": "backend/jobs/service.py",
"bytes": 1746,
"lines": 49
"path": "backend/features/system/routes.py",
"bytes": 1791,
"lines": 46
},
{
"path": "backend/features/alerts/routes.py",
@@ -833,11 +838,6 @@
"bytes": 1455,
"lines": 48
},
{
"path": "backend/features/system/routes.py",
"bytes": 1423,
"lines": 40
},
{
"path": "backend/features/themes/routes.py",
"bytes": 1337,
+69
View File
@@ -0,0 +1,69 @@
# 行情历史补档(最近 60 个交易日)
用于修复 `dashboard_snapshots` 断档导致情绪周期 / 主题轮动 / 智能选股只剩当天的问题。
保留 `latest_contiguous_history` 连续性规则;通过真实交易日历回补缺失交易日快照。
## 适用场景
- 库中已有稀疏历史快照,但最近一个真实交易日缺失,接口 `available_days=1`
- 需要可重复执行、可审计、可回退的补档,而不是迁库或放宽算法。
## 前置
1. 使用与线上一致的代码分支。
2. 管理员账号已配置可用的公共 Tushare Token。
3. 只操作目标环境自己的 `data/review.db`;禁止 `.36``.11` 互拷。
## 上线步骤(总工执行)
在目标环境容器内执行(应用根目录;宿主机也可直接跑,脚本已自带仓库根 `sys.path` 引导):
```bash
# 1) 只读规划:区分已有、真正缺档;不会写入
docker compose exec xiaobai-review python tools/backfill_recent_snapshots.py --account <管理员账号> --lookback 60 --dry-run --json
# 2) 正式补档:先走 SQLite backup API 写 data/backups/review-pre-recent-backfill-*.db
# 再对缺失交易日调用现有 sync_dashboard
docker compose exec xiaobai-review python tools/backfill_recent_snapshots.py --account <管理员账号> --lookback 60 --json
# 3) 验证
# GET /api/sentiment/history?trade_date=YYYY-MM-DD&limit=60
# 期望 available_days >= 20,且不再只有 1 天
```
管理端日期区间回补(`/api/backfill`)已改为只处理交易日历中的开市日,周末/节假日会进入
`skipped_non_trading_days`,不再当成错误;单次仍限制 15 个交易日。最近 60 日请用本工具。
## 写入边界
只会通过现有同步路径写入:
- `dashboard_snapshots`
- 同步审计表 `sync_runs`
- 必要时的 `data_snapshots`(仅当请求日被解析到其他交易日)
不得改动用户、Token、模型绑定或系统配置表。
## 回滚
1. 优先按审计结果的 `created_dates` 精确删除新增行:
```sql
DELETE FROM dashboard_snapshots WHERE trade_date IN ('YYYYMMDD', ...);
```
2. 若需整库回退,停止写入后用补档前备份覆盖:
```bash
# 示例:把 data/backups/review-pre-recent-backfill-YYYYMMDD-HHMMSS.db
# 复制回 data/review.db 后重启容器
```
3. 代码回退:对该提交执行 Git revert 后重新部署镜像。
## 验收要点
- dry-run 与正式执行可重复跑;已有交易日默认跳过。
- 周末、节假日出现在 `skipped_non_trading_days`,不计入失败。
- 部分交易日同步失败时,其他日期仍会继续,并在审计结果中标 `failed`
- 情绪周期、主题轮动 9 列、智能选股置信度随连续交易日恢复。
+7 -6
View File
@@ -34,11 +34,11 @@
document.documentElement.style.colorScheme = theme;
})();
</script>
<link rel="stylesheet" href="/shared/tokens.css?v=20260829-1">
<link rel="stylesheet" href="/shared/tokens.css?v=20260829-hel240">
<link rel="stylesheet" href="/shared/base.css?v=20260806-1">
<link rel="stylesheet" href="/shared/shell.css?v=20260820-8">
<link rel="stylesheet" href="/shared/auth.css?v=20260829-1">
<link rel="stylesheet" href="/shared/components/controls.css?v=20260820-2">
<link rel="stylesheet" href="/shared/shell.css?v=20260829-hel237">
<link rel="stylesheet" href="/shared/auth.css?v=20260829-hel240b">
<link rel="stylesheet" href="/shared/components/controls.css?v=20260829-hel237">
<link rel="stylesheet" href="/shared/components/navigation.css?v=20260820-1">
<link rel="stylesheet" href="/shared/components/cards.css?v=20260820-1">
<link rel="stylesheet" href="/shared/components/tables.css?v=20260820-1">
@@ -54,8 +54,8 @@
<link rel="stylesheet" href="/pages/popularity/foundation.css?v=20260820-1">
<link rel="stylesheet" href="/pages/dragon-tiger/foundation.css?v=20260820-1">
<link rel="stylesheet" href="/pages/screener/foundation.css?v=20260820-4">
<link rel="stylesheet" href="/pages/mentor/foundation.css?v=20260820-2">
<link rel="stylesheet" href="/pages/heaven/foundation.css?v=20260806-2">
<link rel="stylesheet" href="/pages/mentor/foundation.css?v=20260827-hel183">
<link rel="stylesheet" href="/pages/heaven/foundation.css?v=20260827-hel183">
<link rel="stylesheet" href="/pages/review/foundation.css?v=20260820-4">
</head>
<body>
@@ -611,6 +611,7 @@
<label class="form-field"><span>iFinD Refresh Token</span><input id="systemIfindTokenInput" type="password" autocomplete="off" maxlength="2048" placeholder="留空保留现有 Token"></label>
<label class="switch-control"><input id="systemBackgroundRefresh" type="checkbox"><span>启用交易时段后台刷新</span></label>
<p class="form-hint">所有用户读取同一份后台快照,页面不会随后台任务自动重绘。</p>
<div id="adminRefreshStatus" class="admin-refresh-status" data-tone="idle" role="status" aria-live="polite"><i data-lucide="circle-dot"></i><span>尚未手动刷新</span></div>
<div class="dialog-actions admin-inline-actions"><button id="adminRefreshButton" class="button" type="button"><i data-lucide="refresh-cw"></i>立即后台刷新</button><button class="button primary" type="submit">保存行情配置</button></div>
</form>
<section class="settings-section">
+127 -24
View File
@@ -19,41 +19,144 @@
document.documentElement.style.colorScheme = theme;
})();
</script>
<link rel="stylesheet" href="/shared/tokens.css?v=20260829-1">
<link rel="stylesheet" href="/shared/tokens.css?v=20260829-hel251">
<link rel="stylesheet" href="/shared/base.css?v=20260806-1">
<link rel="stylesheet" href="/shared/auth.css?v=20260829-1">
<link rel="stylesheet" href="/shared/auth.css?v=20260829-hel251">
<link rel="stylesheet" href="/shared/components/controls.css?v=20260820-2">
</head>
<body class="login-portal">
<button id="loginThemeToggle" class="login-theme-toggle" type="button">🌙 夜间</button>
<aside class="login-brand" aria-hidden="true">
<div class="login-brand-mark"><span class="login-brand-glyph"></span></div>
<p class="login-brand-kicker">收盘之后 · 复盘开始</p>
<h1 class="login-brand-title">小白复盘</h1>
<p class="login-brand-lead">看懂情绪周期,把复盘变成下一次的先手。</p>
<dl class="login-brand-stats">
<div class="login-stat">
<dt>市场情绪</dt>
<dd>72 <span class="login-stat-tag">高热</span></dd>
<div class="login-brand-header">
<div class="login-brand-mark"><span class="login-brand-glyph"></span></div>
<div class="login-brand-identity">
<p class="login-brand-name">小白复盘</p>
<p class="login-brand-subtitle">A股个人复盘工作台</p>
</div>
<div class="login-stat">
<dt>涨停</dt>
<dd>63</dd>
</div>
<div class="login-mascots" id="loginMascots" aria-hidden="true" data-mood="idle">
<div class="login-mascots-stage">
<div class="login-mascot is-red">
<div class="login-mascot-breathe">
<div class="login-mascot-lean">
<svg class="login-mascot-figure" viewBox="0 0 96 210" focusable="false">
<ellipse class="login-mascot-shadow" cx="48" cy="200" rx="26" ry="5.5"></ellipse>
<line class="login-mascot-wick" x1="48" y1="10" x2="48" y2="50"></line>
<rect class="login-mascot-body" x="16" y="50" width="64" height="126" rx="14"></rect>
<line class="login-mascot-wick" x1="48" y1="176" x2="48" y2="190"></line>
<g class="login-mascot-face">
<circle class="login-mascot-eye" cx="36" cy="94" r="10"></circle>
<circle class="login-mascot-eye" cx="60" cy="94" r="10"></circle>
<circle class="login-mascot-pupil" cx="36" cy="94" r="4.2"></circle>
<circle class="login-mascot-pupil" cx="60" cy="94" r="4.2"></circle>
<rect class="login-mascot-lid" x="24" y="82" width="48" height="24" rx="10"></rect>
<path class="login-mascot-happy" d="M28 96 Q36 88 44 96"></path>
<path class="login-mascot-happy" d="M52 96 Q60 88 68 96"></path>
<circle class="login-mascot-sad" cx="36" cy="96" r="2.2"></circle>
<circle class="login-mascot-sad" cx="60" cy="96" r="2.2"></circle>
</g>
<ellipse class="login-mascot-arm is-left" cx="16" cy="128" rx="8" ry="5.5"></ellipse>
<ellipse class="login-mascot-arm is-right" cx="80" cy="128" rx="8" ry="5.5"></ellipse>
<ellipse class="login-mascot-hand is-left" cx="16" cy="128" rx="9" ry="7"></ellipse>
<ellipse class="login-mascot-hand is-right" cx="80" cy="128" rx="9" ry="7"></ellipse>
</svg>
</div>
</div>
</div>
<div class="login-mascot is-green">
<div class="login-mascot-breathe">
<div class="login-mascot-lean">
<div class="login-mascot-figure">
<svg class="login-mascot-front" viewBox="0 0 84 175" focusable="false">
<ellipse class="login-mascot-shadow" cx="42" cy="166" rx="22" ry="4.5"></ellipse>
<line class="login-mascot-wick" x1="42" y1="8" x2="42" y2="42"></line>
<rect class="login-mascot-body" x="15" y="42" width="54" height="104" rx="12"></rect>
<line class="login-mascot-wick" x1="42" y1="146" x2="42" y2="158"></line>
<g class="login-mascot-face">
<circle class="login-mascot-eye" cx="32" cy="76" r="8.5"></circle>
<circle class="login-mascot-eye" cx="52" cy="76" r="8.5"></circle>
<circle class="login-mascot-pupil" cx="32" cy="76" r="3.6"></circle>
<circle class="login-mascot-pupil" cx="52" cy="76" r="3.6"></circle>
<rect class="login-mascot-lid" x="22" y="66" width="40" height="20" rx="8"></rect>
<path class="login-mascot-happy" d="M25 78 Q32 71 39 78"></path>
<path class="login-mascot-happy" d="M45 78 Q52 71 59 78"></path>
<circle class="login-mascot-sad" cx="32" cy="78" r="1.9"></circle>
<circle class="login-mascot-sad" cx="52" cy="78" r="1.9"></circle>
</g>
<ellipse class="login-mascot-arm is-left" cx="15" cy="108" rx="7" ry="4.5"></ellipse>
<ellipse class="login-mascot-arm is-right" cx="69" cy="108" rx="7" ry="4.5"></ellipse>
</svg>
<svg class="login-mascot-back" viewBox="0 0 84 175" focusable="false">
<ellipse class="login-mascot-shadow" cx="42" cy="166" rx="22" ry="4.5"></ellipse>
<line class="login-mascot-wick" x1="42" y1="8" x2="42" y2="42"></line>
<rect class="login-mascot-body" x="15" y="42" width="54" height="104" rx="12"></rect>
<line class="login-mascot-wick" x1="42" y1="146" x2="42" y2="158"></line>
<rect class="login-mascot-slit" x="40.5" y="58" width="3" height="72" rx="1.5"></rect>
<ellipse class="login-mascot-arm is-left" cx="15" cy="108" rx="7" ry="4.5"></ellipse>
<ellipse class="login-mascot-arm is-right" cx="69" cy="108" rx="7" ry="4.5"></ellipse>
</svg>
</div>
</div>
</div>
</div>
</div>
<div class="login-stat">
<dt>跌停</dt>
<dd>4</dd>
</div>
<div class="login-stat">
<dt>两市成交</dt>
<dd>1.02万亿</dd>
</div>
</dl>
</div>
<div class="login-brand-copy">
<p class="login-brand-kicker">收盘之后 · 复盘开始</p>
<h1 class="login-brand-title">看懂情绪周期,把复盘变成下一次的先手。</h1>
<p class="login-brand-lead">情绪周期、涨停梯队、主题轮动、竞价、龙虎榜、人气榜、交易复盘,集中在一个安静的复盘空间。</p>
</div>
<div class="login-brand-market">
<svg class="login-brand-chart" viewBox="0 0 480 168" focusable="false">
<defs>
<linearGradient id="loginChartFade" x1="0" x2="0" y1="0" y2="1">
<stop offset="0%" stop-color="#d7e4ff" stop-opacity="0.18"></stop>
<stop offset="100%" stop-color="#d7e4ff" stop-opacity="0"></stop>
</linearGradient>
</defs>
<path class="login-chart-area" d="M8 118 C 52 108, 78 96, 112 102 S 168 128, 204 112 S 268 78, 312 86 S 372 118, 428 92 L 472 84 L 472 168 L 8 168 Z"></path>
<g class="login-candles">
<g class="is-up" transform="translate(36 0)"><line x1="8" y1="58" x2="8" y2="128"></line><rect x="2" y="72" width="12" height="40"></rect></g>
<g class="is-down" transform="translate(68 0)"><line x1="8" y1="64" x2="8" y2="132"></line><rect x="2" y="86" width="12" height="28"></rect></g>
<g class="is-up" transform="translate(100 0)"><line x1="8" y1="48" x2="8" y2="118"></line><rect x="2" y="60" width="12" height="44"></rect></g>
<g class="is-down" transform="translate(132 0)"><line x1="8" y1="70" x2="8" y2="136"></line><rect x="2" y="92" width="12" height="26"></rect></g>
<g class="is-up" transform="translate(164 0)"><line x1="8" y1="42" x2="8" y2="110"></line><rect x="2" y="54" width="12" height="38"></rect></g>
<g class="is-up" transform="translate(196 0)"><line x1="8" y1="36" x2="8" y2="98"></line><rect x="2" y="48" width="12" height="32"></rect></g>
<g class="is-down" transform="translate(228 0)"><line x1="8" y1="58" x2="8" y2="128"></line><rect x="2" y="78" width="12" height="36"></rect></g>
<g class="is-up" transform="translate(260 0)"><line x1="8" y1="40" x2="8" y2="104"></line><rect x="2" y="52" width="12" height="36"></rect></g>
<g class="is-down" transform="translate(292 0)"><line x1="8" y1="66" x2="8" y2="134"></line><rect x="2" y="88" width="12" height="30"></rect></g>
<g class="is-up" transform="translate(324 0)"><line x1="8" y1="44" x2="8" y2="112"></line><rect x="2" y="58" width="12" height="40"></rect></g>
<g class="is-down" transform="translate(356 0)"><line x1="8" y1="72" x2="8" y2="138"></line><rect x="2" y="96" width="12" height="24"></rect></g>
<g class="is-up" transform="translate(388 0)"><line x1="8" y1="38" x2="8" y2="108"></line><rect x="2" y="50" width="12" height="42"></rect></g>
<g class="is-up" transform="translate(420 0)"><line x1="8" y1="32" x2="8" y2="96"></line><rect x="2" y="44" width="12" height="34"></rect></g>
</g>
<path class="login-chart-line" d="M8 118 C 52 108, 78 96, 112 102 S 168 128, 204 112 S 268 78, 312 86 S 372 118, 428 92 L 472 84"></path>
</svg>
<dl class="login-brand-stats">
<div class="login-stat">
<dt>市场情绪</dt>
<dd>72 <span class="login-stat-tag">高热</span></dd>
</div>
<div class="login-stat">
<dt>涨停</dt>
<dd>63</dd>
</div>
<div class="login-stat">
<dt>跌停</dt>
<dd>4</dd>
</div>
<div class="login-stat login-stat-wide">
<dt>两市成交</dt>
<dd>1.02万亿</dd>
</div>
</dl>
</div>
<p class="login-brand-disclaimer">股市有风险,投资需谨慎 · 本工具仅供个人复盘学习使用</p>
</aside>
<main class="login-stage">
<button id="loginThemeToggle" class="login-theme-toggle" type="button">🌙 夜间</button>
<section class="login-card" id="loginCard" aria-live="polite"></section>
</main>
<script src="/shared/api.js?v=20260803-2"></script>
<script src="/login/page.js?v=20260829-1"></script>
<script src="/login/page.js?v=20260829-hel251"></script>
</body>
</html>
+360 -23
View File
@@ -13,6 +13,9 @@
loading: false,
confirmingId: null,
error: "",
username: "",
password: "",
passwordVisible: false,
};
function escapeHtml(value) {
@@ -52,48 +55,90 @@
state.error = message || "";
}
function membershipLabel(account) {
if (account.role === "admin") return account.membership?.subscribed ? "管理员 · 会员" : "管理员";
return account.membership?.subscribed ? "会员" : "普通用户";
function formatLastUsed(value) {
if (!value) return "";
const parsed = new Date(value);
if (Number.isNaN(parsed.getTime())) return "";
const now = new Date();
const hh = String(parsed.getHours()).padStart(2, "0");
const mm = String(parsed.getMinutes()).padStart(2, "0");
if (parsed.toDateString() === now.toDateString()) return `今天 ${hh}:${mm}`;
const yesterday = new Date(now);
yesterday.setDate(now.getDate() - 1);
if (parsed.toDateString() === yesterday.toDateString()) return `昨天 ${hh}:${mm}`;
return `${parsed.getMonth() + 1}${parsed.getDate()}`;
}
function chipsFor(account, current) {
const chips = [];
if (current) chips.push('<span class="login-chip login-chip-current">当前</span>');
if (account.role === "admin") chips.push('<span class="login-chip">管理员</span>');
if (account.membership?.subscribed) chips.push('<span class="login-chip login-chip-member">会员</span>');
else if (account.role !== "admin") chips.push('<span class="login-chip">普通用户</span>');
return chips.join("");
}
function returnPath() {
const raw = new URLSearchParams(global.location.search).get("next") || "";
if (!raw) return "/";
try {
const url = new URL(raw, global.location.origin);
if (url.origin !== global.location.origin) return "/";
const path = url.pathname || "/";
if (path === "/login" || path.startsWith("/login/")) return "/";
return `${path}${url.search}${url.hash}` || "/";
} catch (_error) {
return "/";
}
}
function enterApp() {
const next = new URLSearchParams(global.location.search).get("next");
global.location.replace(next && next.startsWith("/") ? next : "/");
global.location.replace(returnPath());
}
function formMarkup(options) {
const registering = state.mode === "register";
const submitLabel = options.submitLabel
|| (state.loading ? "正在登录..." : registering ? "注册并进入" : options.add ? "添加并进入" : "登录");
const lead = options.lead;
const hint = options.hint;
const invalid = state.error ? " is-invalid" : "";
const passwordType = state.passwordVisible ? "text" : "password";
const passwordToggle = state.passwordVisible ? "隐藏" : "显示";
return [
options.back
? '<button class="login-back" type="button" data-login-action="picker">返回账号列表</button>'
: "",
`<h2 class="login-card-title">${escapeHtml(options.title)}</h2>`,
`<p class="login-card-lead">${escapeHtml(options.lead)}</p>`,
`<p class="login-card-lead">${escapeHtml(lead)}</p>`,
'<div class="login-tabs" role="tablist">',
`<button class="login-tab${state.mode === "login" ? " is-active" : ""}" type="button" data-auth-mode="login">登录</button>`,
`<button class="login-tab${state.mode === "register" ? " is-active" : ""}" type="button" data-auth-mode="register">注册</button>`,
"</div>",
'<form class="login-form" id="loginForm">',
'<label class="form-field"><span>账号名</span><input id="loginUsername" type="text" minlength="3" maxlength="30" autocomplete="username" required></label>',
`<label class="form-field"><span>密码</span><input id="loginPassword" type="password" minlength="8" maxlength="128" autocomplete="${registering ? "new-password" : "current-password"}" required></label>`,
`<label class="form-field"><span>账号名</span><input id="loginUsername" type="text" minlength="3" maxlength="30" autocomplete="username" placeholder="请输入账号名" value="${escapeHtml(state.username)}" required></label>`,
`<label class="form-field"><span>密码</span><span class="login-password-wrap"><input id="loginPassword" class="${invalid.trim()}" type="${passwordType}" minlength="8" maxlength="128" autocomplete="${registering ? "new-password" : "current-password"}" placeholder="请输入密码" value="${escapeHtml(state.password)}" required><button class="login-password-toggle" type="button" data-login-action="toggle-password" aria-pressed="${state.passwordVisible ? "true" : "false"}" aria-label="${state.passwordVisible ? "隐藏密码" : "显示密码"}">${passwordToggle}</button></span></label>`,
`<label class="form-field" id="loginConfirmField"${registering ? "" : " hidden"}><span>确认密码</span><input id="loginPasswordConfirm" type="password" minlength="8" maxlength="128" autocomplete="new-password"${registering ? " required" : ""}></label>`,
state.error ? `<p class="login-error">${escapeHtml(state.error)}</p>` : '<p class="login-error" hidden></p>',
`<button class="button primary login-submit" type="submit"${state.loading ? " disabled" : ""}>`,
state.loading ? '<span class="login-spinner" aria-hidden="true"></span>' : "",
`<span>${escapeHtml(submitLabel)}</span></button>`,
"</form>",
'<p class="login-hint">密码连续输错 5 次将锁定 10 分钟。还没有账号?切换到「注册」创建。</p>',
`<p class="login-hint">${escapeHtml(hint)}</p>`,
].join("");
}
function accountRow(account) {
const current = Number(account.user_id) === Number(state.currentUserId);
const confirming = Number(state.confirmingId) === Number(account.user_id);
const classes = `login-account-row${current ? " is-current" : ""}${confirming ? " is-confirming" : ""}`;
if (state.view === "manage" && confirming) {
const managing = state.view === "manage";
const classes = [
"login-account-row",
current ? "is-current" : "",
confirming ? "is-confirming" : "",
!managing ? "is-switchable" : "",
].filter(Boolean).join(" ");
if (managing && confirming) {
return [
`<div class="${classes}" data-user-id="${account.user_id}">`,
`<p class="login-confirm-copy">移除「${escapeHtml(account.username)}」的本机记录?</p>`,
@@ -103,16 +148,25 @@
"</div></div>",
].join("");
}
const action = state.view === "manage"
? `<button class="login-account-remove" type="button" data-confirm-id="${account.user_id}">移除</button>`
const glyph = escapeHtml(String(account.username || "账").slice(0, 1));
const tone = Number(account.user_id || 0) % 4;
const used = formatLastUsed(account.last_used_at);
const action = managing
? `<button class="login-account-remove" type="button" data-confirm-id="${account.user_id}" aria-label="移除 ${escapeHtml(account.username)}"><svg width="16" height="16" viewBox="0 0 16 16" aria-hidden="true"><path fill="currentColor" d="M6 2h4l.5 1H14v1H2V3h3.5L6 2zm1 4v6H6V6h1zm3 0v6H9V6h1zM3.5 5H13l-.7 8.2A1.5 1.5 0 0 1 10.81 14H5.19a1.5 1.5 0 0 1-1.49-1.8L3.5 5z"></path></svg></button>`
: current
? '<span class="login-account-check" aria-hidden="true">✓</span>'
: `<button class="login-account-enter" type="button" data-switch-id="${account.user_id}">进入</button>`;
? '<span class="login-account-action"><span class="login-account-check" aria-hidden="true">✓</span>继续使用</span>'
: "";
const switchAttr = !managing && !current ? ` data-switch-id="${account.user_id}"` : "";
const resumeAttr = !managing && current ? ` data-resume-id="${account.user_id}"` : "";
return [
`<div class="${classes}" data-user-id="${account.user_id}">`,
`<div class="${classes}" data-user-id="${account.user_id}"${switchAttr}${resumeAttr}>`,
`<span class="login-avatar tone-${tone}" aria-hidden="true">${glyph}</span>`,
'<div class="login-account-meta">',
'<div class="login-account-name">',
`<strong>${escapeHtml(account.username)}</strong>`,
`<span>${escapeHtml(membershipLabel(account))}${current ? " · 当前" : ""}</span>`,
chipsFor(account, current),
"</div>",
used ? `<span class="login-account-used">上次登录 ${escapeHtml(used)}</span>` : "",
"</div>",
action,
"</div>",
@@ -123,10 +177,15 @@
const count = state.accounts.length;
const managing = state.view === "manage";
return [
`<h2 class="login-card-title">${managing ? "管理账号记录" : "选择账号"}</h2>`,
`<p class="login-card-lead">这台电脑已记录 ${count} 个账号,可直接进入,无需再次输入密码。</p>`,
managing
? '<button class="login-manage" type="button" data-login-action="picker">完成</button>'
? ""
: '<button class="login-back" type="button" data-login-action="resume">返回复盘</button>',
`<h2 class="login-card-title">${managing ? "管理账号记录" : "选择账号"}</h2>`,
`<p class="login-card-lead">${managing
? "移除只删除这台电脑上的登录记录,不会注销账号"
: `这台电脑已记录 ${count} 个账号,点选即可进入,无需再次输入密码。`}</p>`,
managing
? '<div class="login-manage-toolbar"><p class="login-manage-hint">点击右侧图标移除对应记录</p><button class="login-manage-done" type="button" data-login-action="picker">完成</button></div>'
: "",
`<div class="login-account-list">${state.accounts.map(accountRow).join("")}</div>`,
managing
@@ -136,7 +195,9 @@
? ""
: '<button class="login-manage" type="button" data-login-action="manage">管理已记录的账号</button>',
state.error ? `<p class="login-error">${escapeHtml(state.error)}</p>` : "",
'<p class="login-privacy"><span class="login-lock" aria-hidden="true">🔒</span>账号记录仅保存在这台电脑的浏览器中</p>',
managing
? '<p class="login-privacy"><span class="login-lock" aria-hidden="true">🔒</span>移除后再次登录该账号需重新输入密码</p>'
: '<p class="login-privacy"><span class="login-lock" aria-hidden="true">🔒</span>账号记录仅保存在这台电脑的浏览器中</p>',
].join("");
}
@@ -145,7 +206,10 @@
if (state.view === "first" || state.view === "add") {
card.innerHTML = formMarkup({
title: state.view === "add" ? "添加账号" : "欢迎回来",
lead: "登录后进入你的复盘空间",
lead: state.view === "add" ? "登录另一个账号,添加后可随时一键切换" : "登录后进入你的复盘空间",
hint: state.view === "add"
? "添加后账号会保存在这台电脑,方便随时切换。"
: "密码连续输错 5 次将锁定 10 分钟。还没有账号?切换到「注册」创建。",
add: state.view === "add",
back: state.view === "add",
});
@@ -153,6 +217,7 @@
card.innerHTML = pickerMarkup();
}
bindCard();
if (mascots) mascots.sync();
}
function bindCard() {
@@ -166,6 +231,14 @@
card.querySelectorAll("[data-login-action]").forEach((button) => {
button.addEventListener("click", () => {
const action = button.dataset.loginAction;
if (action === "toggle-password") {
togglePasswordVisible();
return;
}
if (action === "resume") {
resumeCurrentAccount();
return;
}
if (action === "picker") {
state.view = state.accounts.length ? "picker" : "first";
state.confirmingId = null;
@@ -184,8 +257,12 @@
card.querySelectorAll("[data-switch-id]").forEach((button) => {
button.addEventListener("click", () => switchAccount(Number(button.dataset.switchId)));
});
card.querySelectorAll("[data-resume-id]").forEach((button) => {
button.addEventListener("click", () => resumeCurrentAccount());
});
card.querySelectorAll("[data-confirm-id]").forEach((button) => {
button.addEventListener("click", () => {
button.addEventListener("click", (event) => {
event.stopPropagation();
state.confirmingId = Number(button.dataset.confirmId);
render();
});
@@ -212,6 +289,8 @@
event.preventDefault();
const username = document.querySelector("#loginUsername").value.trim();
const password = document.querySelector("#loginPassword").value;
state.username = username;
state.password = password;
if (state.mode === "register" && password !== document.querySelector("#loginPasswordConfirm").value) {
setError("两次输入的密码不一致。");
render();
@@ -222,11 +301,13 @@
render();
try {
await api.request(`/api/auth/${state.mode}`, "POST", { username, password });
await celebrateLogin();
enterApp();
} catch (error) {
state.loading = false;
setError(error.message || "账号操作失败");
render();
mascots.fail();
}
}
@@ -244,6 +325,27 @@
}
}
async function resumeCurrentAccount() {
state.loading = true;
setError("");
render();
try {
const session = await api.request("/api/auth/me");
const sessionUserId = session.user?.id;
const matches = Boolean(session.authenticated) && (
!state.currentUserId || Number(sessionUserId) === Number(state.currentUserId)
);
if (!matches) {
throw new Error("当前会话已失效,请重新登录");
}
enterApp();
} catch (error) {
state.loading = false;
setError(error.message || "当前会话已失效,请重新登录");
render();
}
}
async function forgetAccount(userId) {
try {
await api.request("/api/auth/forget", "POST", { user_id: userId });
@@ -258,6 +360,241 @@
}
}
function togglePasswordVisible() {
state.passwordVisible = !state.passwordVisible;
const input = document.querySelector("#loginPassword");
const button = document.querySelector(".login-password-toggle");
if (input) {
input.type = state.passwordVisible ? "text" : "password";
input.focus();
}
if (button) {
button.textContent = state.passwordVisible ? "隐藏" : "显示";
button.setAttribute("aria-pressed", state.passwordVisible ? "true" : "false");
button.setAttribute("aria-label", state.passwordVisible ? "隐藏密码" : "显示密码");
}
mascots.sync();
}
function reducedMotion() {
return Boolean(global.matchMedia && global.matchMedia("(prefers-reduced-motion: reduce)").matches);
}
function finePointer() {
return Boolean(global.matchMedia && global.matchMedia("(pointer: fine)").matches);
}
function wait(ms) {
return new Promise((resolve) => global.setTimeout(resolve, ms));
}
async function celebrateLogin() {
mascots.succeed();
if (!reducedMotion()) await wait(720);
}
const mascots = (() => {
const root = document.querySelector("#loginMascots");
const red = root && root.querySelector(".login-mascot.is-red");
const green = root && root.querySelector(".login-mascot.is-green");
const motion = {
pupilX: 0,
pupilY: 0,
targetX: 0,
targetY: 0,
leanRed: 0,
leanGreen: 0,
targetLeanRed: 0,
targetLeanGreen: 0,
};
let mood = "idle";
let locked = "";
let blinkTimer = 0;
let failTimer = 0;
let raf = 0;
function setVars() {
if (!red || !green) return;
const pupilX = `${motion.pupilX.toFixed(2)}px`;
const pupilY = `${motion.pupilY.toFixed(2)}px`;
red.style.setProperty("--pupil-x", pupilX);
red.style.setProperty("--pupil-y", pupilY);
green.style.setProperty("--pupil-x", pupilX);
green.style.setProperty("--pupil-y", pupilY);
red.style.setProperty("--lean", `${motion.leanRed.toFixed(2)}deg`);
green.style.setProperty("--lean", `${motion.leanGreen.toFixed(2)}deg`);
}
function poseFor(next) {
if (next === "account" || next === "busy") {
motion.targetX = 3.2;
motion.targetY = 0.8;
motion.targetLeanRed = 7;
motion.targetLeanGreen = 5.6;
return;
}
if (next === "password") {
motion.targetX = 0;
motion.targetY = 0;
motion.targetLeanRed = 0;
motion.targetLeanGreen = 0;
return;
}
if (next === "fail") {
motion.targetX = 0;
motion.targetY = 2.8;
motion.targetLeanRed = 8;
motion.targetLeanGreen = 8;
return;
}
if (next === "success") {
motion.targetX = 0;
motion.targetY = 0;
motion.targetLeanRed = 0;
motion.targetLeanGreen = 0;
return;
}
if (!finePointer()) {
motion.targetX = 2.4;
motion.targetY = 0.4;
motion.targetLeanRed = 4;
motion.targetLeanGreen = 3.2;
return;
}
motion.targetLeanRed = 0;
motion.targetLeanGreen = 0;
}
function applyMood(next) {
if (!root) return;
const changed = next !== mood;
if (changed) {
mood = next;
root.dataset.mood = next;
poseFor(next);
} else if (next !== "idle") {
poseFor(next);
}
if (reducedMotion()) {
motion.pupilX = motion.targetX;
motion.pupilY = motion.targetY;
motion.leanRed = motion.targetLeanRed;
motion.leanGreen = motion.targetLeanGreen;
setVars();
}
}
function focusedControl() {
const active = document.activeElement;
if (!active || !card.contains(active)) return "";
if (active.classList.contains("login-password-toggle")) return "loginPassword";
return active.id || "";
}
function desiredMood() {
if (locked === "success") return "success";
if (state.loading) return "busy";
const focused = focusedControl();
if (focused === "loginPassword" || focused === "loginPasswordConfirm") return "password";
if (focused === "loginUsername") return "account";
if (locked === "fail") return "fail";
return "idle";
}
function sync() {
applyMood(desiredMood());
}
function succeed() {
locked = "success";
applyMood("success");
}
function fail() {
locked = "fail";
applyMood("fail");
global.clearTimeout(failTimer);
failTimer = global.setTimeout(() => {
if (locked === "fail") locked = "";
sync();
}, 900);
}
function blink() {
if (!root || reducedMotion()) return;
if (mood !== "idle" && mood !== "account" && mood !== "busy") return;
root.classList.remove("is-blinking");
void root.offsetWidth;
root.classList.add("is-blinking");
global.setTimeout(() => root.classList.remove("is-blinking"), 160);
}
function scheduleBlink() {
global.clearTimeout(blinkTimer);
if (reducedMotion()) return;
const waitMs = 4000 + Math.random() * 2000;
blinkTimer = global.setTimeout(() => {
blink();
scheduleBlink();
}, waitMs);
}
function onMouseMove(event) {
if (reducedMotion() || !finePointer()) return;
if (desiredMood() !== "idle") return;
const rect = root.getBoundingClientRect();
const cx = rect.left + rect.width * 0.42;
const cy = rect.top + rect.height * 0.42;
const dx = event.clientX - cx;
const dy = event.clientY - cy;
const dist = Math.hypot(dx, dy) || 1;
const cap = 3.5;
motion.targetX = (dx / dist) * Math.min(cap, Math.abs(dx) / 90);
motion.targetY = (dy / dist) * Math.min(cap, Math.abs(dy) / 90);
const tilt = Math.max(-5, Math.min(5, (dx / Math.max(global.innerWidth, 1)) * 10));
motion.targetLeanRed = tilt;
motion.targetLeanGreen = tilt * 0.8;
}
function tick() {
if (!root) return;
if (!reducedMotion()) {
motion.pupilX += (motion.targetX - motion.pupilX) * 0.18;
motion.pupilY += (motion.targetY - motion.pupilY) * 0.18;
motion.leanRed += (motion.targetLeanRed - motion.leanRed) * 0.18;
motion.leanGreen += (motion.targetLeanGreen - motion.leanGreen) * 0.18;
setVars();
} else {
motion.pupilX = motion.targetX;
motion.pupilY = motion.targetY;
motion.leanRed = motion.targetLeanRed;
motion.leanGreen = motion.targetLeanGreen;
setVars();
}
raf = global.requestAnimationFrame(tick);
}
function onFocusChange() {
global.requestAnimationFrame(sync);
}
if (root) {
document.addEventListener("focusin", onFocusChange);
document.addEventListener("focusout", onFocusChange);
global.addEventListener("mousemove", onMouseMove, { passive: true });
if (!reducedMotion()) {
raf = global.requestAnimationFrame(tick);
scheduleBlink();
} else {
poseFor("idle");
setVars();
}
sync();
}
return { sync, succeed, fail };
})();
themeButton.addEventListener("click", () => {
applyTheme(document.documentElement.dataset.theme === "dark" ? "light" : "dark", true);
});
+415 -8
View File
@@ -3192,8 +3192,7 @@
.m-sys-grid div {
padding: 12px;
border-radius: 8px;
background: var(--surface);
box-shadow: var(--elevation-card);
background: var(--surface-muted);
}
.m-sys-grid span {
@@ -3209,16 +3208,424 @@
color: var(--text-primary);
}
.m-sys-account-actions {
display: grid;
.m-sys-home {
padding: 0 0 12px;
display: flex;
flex-direction: column;
gap: 8px;
margin-top: 16px;
}
.m-page[data-page^="system/"] .m-btn-primary {
margin-bottom: 8px;
.m-sys-body {
padding: 12px 12px 20px;
display: flex;
flex-direction: column;
gap: 12px;
}
.m-sys-home .m-card,
.m-sys-body .m-card {
margin-left: 0;
margin-right: 0;
}
.m-sys-profile-card {
display: flex;
gap: 12px;
align-items: flex-start;
padding: 14px;
}
.m-sys-avatar {
flex: 0 0 auto;
width: 48px;
height: 48px;
border-radius: 50%;
display: flex;
align-items: center;
justify-content: center;
background: var(--action-soft);
color: var(--action);
font-size: var(--font-size-page-title);
font-weight: 600;
}
.m-sys-profile-card strong {
display: block;
font-size: var(--font-size-card-title);
color: var(--text-primary);
}
.m-sys-profile-meta {
margin: 4px 0 8px;
font-size: var(--font-size-caption);
color: var(--text-secondary);
}
.m-sys-badges {
display: flex;
flex-wrap: wrap;
gap: 6px;
}
.m-sys-badge {
display: inline-flex;
align-items: center;
height: 22px;
padding: 0 8px;
border-radius: 999px;
font-size: var(--font-size-aux);
font-weight: 600;
background: var(--surface-muted);
color: var(--text-secondary);
}
.m-sys-badge--admin {
background: var(--action-soft);
color: var(--action);
}
.m-sys-badge--ok {
background: var(--market-down-soft);
color: var(--market-down);
}
.m-sys-group-title {
margin: 4px 0 0;
font-size: var(--font-size-caption);
font-weight: 600;
color: var(--text-tertiary);
}
.m-sys-list {
padding: 0;
overflow: hidden;
}
.m-sys-row {
width: 100%;
display: flex;
align-items: center;
gap: 12px;
padding: 10px 12px;
border: 0;
background: transparent;
color: inherit;
text-align: left;
cursor: pointer;
-webkit-tap-highlight-color: transparent;
}
.m-sys-row + .m-sys-row {
border-top: 1px solid var(--border);
}
.m-sys-row:active {
background: var(--surface-hover);
}
.m-sys-row-icon {
flex: 0 0 auto;
width: 36px;
height: 36px;
border-radius: 10px;
display: inline-flex;
align-items: center;
justify-content: center;
background: var(--action-soft);
color: var(--action);
}
.m-sys-row--danger .m-sys-row-icon {
background: var(--market-up-soft);
color: var(--market-up);
}
.m-sys-row-body {
flex: 1;
min-width: 0;
}
.m-sys-row-body strong {
display: block;
font-size: var(--font-size-body);
font-weight: 600;
color: var(--text-primary);
}
.m-sys-row-body small {
display: block;
margin-top: 2px;
font-size: var(--font-size-caption);
color: var(--text-secondary);
}
.m-sys-row-chevron {
flex: 0 0 auto;
color: var(--text-tertiary);
}
.m-sys-foot {
margin: 8px 0 0;
text-align: center;
font-size: var(--font-size-caption);
color: var(--text-tertiary);
}
.m-sys-notice {
padding: 12px;
border-radius: 12px;
background: var(--surface-muted);
}
.m-sys-notice p {
margin: 0;
font-size: var(--font-size-label);
color: var(--text-secondary);
line-height: 1.55;
}
.m-sys-notice .m-sys-badges {
margin-top: 8px;
}
.m-sys-section {
padding: 12px;
}
.m-sys-section > strong,
.m-sys-section-title {
display: block;
margin-bottom: 10px;
font-size: var(--font-size-card-title);
color: var(--text-primary);
}
.m-sys-hint {
margin: 8px 0 0;
font-size: var(--font-size-caption);
color: var(--text-tertiary);
line-height: 1.45;
}
.m-sys-status-list {
display: grid;
gap: 8px;
}
.m-sys-status-item {
display: flex;
align-items: center;
justify-content: space-between;
gap: 8px;
font-size: var(--font-size-body);
color: var(--text-primary);
}
.m-sys-dot {
width: 8px;
height: 8px;
border-radius: 50%;
background: var(--text-tertiary);
}
.m-sys-dot--ok {
background: var(--market-down);
}
.m-sys-switch-row {
display: flex;
align-items: center;
justify-content: space-between;
gap: 12px;
}
.m-sys-switch-row strong {
display: block;
font-size: var(--font-size-body);
margin-bottom: 0;
}
.m-btn-outline,
.m-btn-outline-danger {
width: 100%;
height: 44px;
display: flex;
align-items: center;
justify-content: center;
gap: 8px;
border-radius: 8px;
background: var(--surface);
font-size: var(--font-size-body);
font-weight: 600;
cursor: pointer;
-webkit-tap-highlight-color: transparent;
}
.m-btn-outline {
border: 1px solid var(--action);
color: var(--action);
}
.m-btn-outline:active {
background: var(--action-soft);
}
.m-btn-outline-danger {
border: 1px solid var(--market-up);
color: var(--market-up);
}
.m-btn-outline-danger:active {
background: var(--market-up-soft);
}
.m-btn-outline:disabled,
.m-btn-outline-danger:disabled {
opacity: 0.45;
cursor: default;
pointer-events: none;
}
.m-sys-model-card .m-sys-badges,
.m-sys-user-row .m-sys-badges {
margin-top: 6px;
}
.m-sys-model-card {
display: flex;
align-items: flex-start;
justify-content: space-between;
gap: 8px;
width: 100%;
padding: 12px;
border: 0;
background: transparent;
text-align: left;
cursor: pointer;
}
.m-sys-model-card + .m-sys-model-card {
border-top: 1px solid var(--border);
}
.m-sys-model-card:active {
background: var(--surface-hover);
}
.m-sys-user-row {
display: flex;
align-items: center;
gap: 10px;
padding: 12px;
}
.m-sys-user-row + .m-sys-user-row {
border-top: 1px solid var(--border);
}
.m-sys-user-row .m-btn-outline {
width: auto;
height: 32px;
padding: 0 12px;
flex: 0 0 auto;
}
.m-sys-pair {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 8px;
}
.m-sheet-root.is-dialog .m-sheet {
display: none;
}
.m-dialog {
position: absolute;
left: 50%;
top: 42%;
width: calc(100% - 48px);
max-width: 320px;
transform: translate(-50%, -46%) scale(0.96);
border-radius: 12px;
background: var(--surface);
box-shadow: var(--elevation-float);
padding: 18px 16px 14px;
z-index: 3;
opacity: 0;
transition: opacity var(--motion-enter) var(--ease-enter),
transform var(--motion-enter) var(--ease-enter);
}
.m-sheet-root.is-open .m-dialog {
opacity: 1;
transform: translate(-50%, -50%) scale(1);
}
.m-dialog h2 {
margin: 0 0 8px;
font-size: var(--font-size-card-title);
color: var(--text-primary);
text-align: center;
}
.m-dialog p {
margin: 0 0 16px;
font-size: var(--font-size-label);
color: var(--text-secondary);
line-height: 1.5;
text-align: center;
}
.m-dialog-actions {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 8px;
}
.m-dialog-actions .m-btn-outline,
.m-dialog-actions .m-btn-outline-danger,
.m-dialog-actions .m-btn-primary {
height: 40px;
}
.m-sys-sheet-actions {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 8px;
margin-top: 12px;
}
.m-sys-test-row {
display: flex;
align-items: center;
gap: 10px;
margin: 8px 0 12px;
}
.m-sys-test-row .m-btn-outline {
width: auto;
flex: 0 0 auto;
height: 36px;
padding: 0 12px;
}
.m-sys-test-row [data-model-test-status] {
flex: 1;
font-size: var(--font-size-caption);
color: var(--text-secondary);
}
.m-sys-hero {
padding: 14px;
}
.m-sys-hero strong {
display: block;
font-size: var(--font-size-page-title);
margin-bottom: 6px;
}
.m-page[data-page^="system/"] .m-card {
margin-bottom: 12px;
margin-bottom: 0;
}
+504 -159
View File
@@ -91,6 +91,14 @@
"sticky-note": '<path d="M16 3H5a2 2 0 0 0-2 2v14a2 2 0 0 0 2 2h14a2 2 0 0 0 2-2V8Z"/><path d="M15 3v4a2 2 0 0 0 2 2h4"/>',
bell: '<path d="M6 8a6 6 0 0 1 12 0c0 7 3 9 3 9H3s3-2 3-9"/><path d="M10.3 21a1.94 1.94 0 0 0 3.4 0"/>',
lock: '<rect width="18" height="11" x="3" y="11" rx="2" ry="2"/><path d="M7 11V7a5 5 0 0 1 10 0v4"/>',
user: '<path d="M19 21v-2a4 4 0 0 0-4-4H9a4 4 0 0 0-4 4v2"/><circle cx="12" cy="7" r="4"/>',
gem: '<path d="M6 3h12l4 6-10 13L2 9Z"/><path d="M11 3 8 9l4 13 4-13-3-6"/><path d="M2 9h20"/>',
users: '<path d="M16 21v-2a4 4 0 0 0-4-4H6a4 4 0 0 0-4 4v2"/><circle cx="9" cy="7" r="4"/><path d="M22 21v-2a4 4 0 0 0-3-3.87"/><path d="M16 3.13a4 4 0 0 1 0 7.75"/>',
"sliders-horizontal": '<line x1="21" x2="14" y1="4" y2="4"/><line x1="10" x2="3" y1="4" y2="4"/><line x1="21" x2="12" y1="12" y2="12"/><line x1="8" x2="3" y1="12" y2="12"/><line x1="21" x2="16" y1="20" y2="20"/><line x1="12" x2="3" y1="20" y2="20"/><line x1="14" x2="14" y1="2" y2="6"/><line x1="8" x2="8" y1="10" y2="14"/><line x1="16" x2="16" y1="18" y2="22"/>',
sun: '<circle cx="12" cy="12" r="4"/><path d="M12 2v2"/><path d="M12 20v2"/><path d="m4.93 4.93 1.41 1.41"/><path d="m17.66 17.66 1.41 1.41"/><path d="M2 12h2"/><path d="M20 12h2"/><path d="m6.34 17.66-1.41 1.41"/><path d="m19.07 4.93-1.41 1.41"/>',
moon: '<path d="M12 3a6 6 0 0 0 9 9 9 9 0 1 1-9-9Z"/>',
"arrow-left-right": '<path d="M8 3 4 7l4 4"/><path d="M4 7h16"/><path d="m16 21 4-4-4-4"/><path d="M20 17H4"/>',
"log-out": '<path d="M9 21H5a2 2 0 0 1-2-2V5a2 2 0 0 1 2-2h4"/><polyline points="16 17 21 12 16 7"/><line x1="21" x2="9" y1="12" y2="12"/>',
};
function icon(name, size) {
@@ -191,6 +199,10 @@
account: null,
admin: null,
adminTab: "market",
models: [],
accounts: [],
editingModelId: "",
editingUserId: "",
},
};
@@ -1594,11 +1606,43 @@
/* ---------------------------------------------------------------- helpers shared by new pages */
function bindConfirmAction(onConfirm) {
const ok = document.querySelector("[data-confirm-ok]");
if (ok && typeof onConfirm === "function") {
ok.addEventListener("click", function () {
closeSheet();
onConfirm();
});
}
}
function openCenteredDialog(content) {
const root = ensureSheetRoot();
sheetToken += 1;
root.classList.add("is-dialog");
root.innerHTML =
'<div class="m-sheet-backdrop" data-sheet-backdrop></div>' +
'<div class="m-dialog" role="dialog" aria-modal="true">' + content + "</div>";
global.requestAnimationFrame(function () { root.classList.add("is-open"); });
}
function openConfirmSheet(title, body, options) {
const opts = options || {};
const confirmLabel = opts.confirmLabel || "确定";
const cancelLabel = opts.cancelLabel || "取消";
const danger = Boolean(opts.danger);
if (opts.centered) {
openCenteredDialog(
"<h2>" + escapeHtml(title) + "</h2>" +
(body ? "<p>" + escapeHtml(body) + "</p>" : "") +
'<div class="m-dialog-actions">' +
'<button class="m-btn-outline" type="button" data-sheet-close>' + escapeHtml(cancelLabel) + "</button>" +
'<button class="' + (danger ? "m-btn-outline-danger" : "m-btn-primary") + '" type="button" data-confirm-ok>' + escapeHtml(confirmLabel) + "</button>" +
"</div>"
);
bindConfirmAction(opts.onConfirm);
return;
}
openSheet(
'<div class="m-sheet-head"><h2>' + escapeHtml(title) + '</h2>' +
'<button class="m-sheet-close" type="button" data-sheet-close aria-label="关闭">' + icon("close", 20) + '</button></div>' +
@@ -1610,13 +1654,7 @@
'</div></div>',
{ detail: false }
);
const ok = document.querySelector('[data-confirm-ok]');
if (ok && typeof opts.onConfirm === 'function') {
ok.addEventListener('click', function () {
closeSheet();
opts.onConfirm();
});
}
bindConfirmAction(opts.onConfirm);
}
function nextSeq() {
@@ -3123,6 +3161,7 @@
function openSheet(content, opts) {
const root = ensureSheetRoot();
sheetToken += 1;
root.classList.remove("is-dialog");
root.innerHTML =
'<div class="m-sheet-backdrop" data-sheet-backdrop></div>' +
'<div class="m-sheet' + (opts && opts.detail ? " m-sheet--detail" : "") + '" role="dialog" aria-modal="true">' +
@@ -3138,12 +3177,16 @@
if (!root) return;
const token = sheetToken;
root.classList.remove("is-open");
root.classList.remove("is-dialog");
global.setTimeout(function () {
if (sheetToken === token && !root.classList.contains("is-open")) root.innerHTML = "";
if (sheetToken === token && !root.classList.contains("is-open")) {
root.innerHTML = "";
}
}, 340);
}
function bindSheetDrag(root) {
if (root.classList.contains("is-dialog")) return;
const sheet = root.querySelector(".m-sheet");
const backdrop = root.querySelector(".m-sheet-backdrop");
const handle = root.querySelector(".m-sheet-handle");
@@ -4863,7 +4906,7 @@
return '<table class="m-table"><thead><tr>' + head + "</tr></thead><tbody>" + body + "</tbody></table>";
}
/* ---------------------------------------------------------------- 系统管理(恢复桌面端已有能力,禁止再走占位页 */
/* ---------------------------------------------------------------- 系统管理(HEL-238 按确认样图重排 */
function isSystemPage(key) {
return String(key || state.key || "").indexOf("system/") === 0;
@@ -4889,13 +4932,27 @@
return new Intl.DateTimeFormat("zh-CN", { year: "numeric", month: "2-digit", day: "2-digit" }).format(parsed);
}
function currentUser() {
return global.MobileSession && global.MobileSession.state ? global.MobileSession.state.user : null;
}
function currentUsername() {
const user = currentUser();
return user && user.username ? user.username : "当前账号";
}
function currentMembership() {
const user = currentUser();
return (user && user.membership) || {};
}
function setupSystemPage(key) {
state.key = key;
state.requestedDate = todayString();
state.sort = { key: "", dir: null };
state.sortTable = { cols: null, reapply: null };
state.detail = null;
if (key === "system/admin") state.system.adminTab = "market";
if (key === "system/admin") state.system.adminTab = state.system.adminTab || "market";
document.getElementById("m-view").classList.add("m-view-feature");
global.MobileRouter.updateHeader({ title: findLabel(key) || key, back: true, actions: "" });
document.getElementById("m-view").innerHTML = complexFrame(key, complexScroll(skeletonHtml(6)));
@@ -4917,6 +4974,9 @@
if (seq !== state.seq || state.key !== key) return;
if (key === "system/admin" || key === "system/members") {
state.system.admin = payload || {};
state.system.models = ((payload.llm && payload.llm.models) || []).map(function (item) {
return Object.assign({}, item);
});
renderSystemAdmin(key);
} else {
state.system.account = payload || {};
@@ -4929,9 +4989,115 @@
});
}
function currentUsername() {
const user = global.MobileSession && global.MobileSession.state ? global.MobileSession.state.user : null;
return user && user.username ? user.username : "当前账号";
function formatLastUsed(value) {
if (!value) return "";
const parsed = new Date(value);
if (Number.isNaN(parsed.getTime())) return "";
const now = new Date();
const hh = String(parsed.getHours()).padStart(2, "0");
const mm = String(parsed.getMinutes()).padStart(2, "0");
const sameDay = parsed.toDateString() === now.toDateString();
const yesterday = new Date(now);
yesterday.setDate(now.getDate() - 1);
if (sameDay) return "今天 " + hh + ":" + mm;
if (parsed.toDateString() === yesterday.toDateString()) return "昨天 " + hh + ":" + mm;
return membershipDateLabel(value) + " " + hh + ":" + mm;
}
function systemRowHtml(item) {
return '<button class="m-sys-row" type="button" data-route="#/feature/' + item.key + '">' +
'<span class="m-sys-row-icon">' + icon(item.icon, 18) + "</span>" +
'<span class="m-sys-row-body"><strong>' + escapeHtml(item.label) + "</strong><small>" + escapeHtml(item.hint) + "</small></span>" +
'<span class="m-sys-row-chevron">' + icon("chevron-right", 16) + "</span>" +
"</button>";
}
function systemThemeRowHtml() {
const dark = document.getElementById("m-app") && document.getElementById("m-app").dataset.theme === "dark";
return '<div class="m-sys-row" data-theme-row>' +
'<span class="m-sys-row-icon" data-theme-row-icon>' + icon(dark ? "moon" : "sun", 18) + "</span>" +
'<span class="m-sys-row-body"><strong>外观主题</strong><small data-theme-row-label>' + (dark ? "当前:夜间模式" : "当前:日间模式") + "</small></span>" +
'<button class="m-theme-switch" type="button" data-theme-toggle role="switch" aria-checked="' + (dark ? "true" : "false") + '" aria-label="切换日间/夜间模式"><span class="m-theme-switch-thumb"></span></button>' +
"</div>";
}
function renderSystemHome() {
state.key = "system";
document.getElementById("m-view").classList.remove("m-view-feature");
global.MobileRouter.updateHeader({ title: "系统管理", back: false, actions: "" });
const view = document.getElementById("m-view");
view.innerHTML = '<div class="m-sys-home" data-system-page="home">' + skeletonHtml(4) + "</div>";
const seq = nextSeq();
Promise.all([
global.MobileAPI.request("/api/account/status").catch(function () { return {}; }),
global.MobileSession.listAccounts().catch(function () { return { accounts: [] }; })
]).then(function (results) {
if (seq !== state.seq || state.key !== "system") return;
state.system.account = results[0] || {};
state.system.accounts = (results[1] && results[1].accounts) || [];
paintSystemHome();
});
}
function paintSystemHome() {
const user = currentUser() || {};
const membership = currentMembership();
const account = state.system.account || {};
const access = account.llm_access || {};
const status = access.membership || membership;
const username = user.username || currentUsername();
const avatar = String(username).slice(0, 1);
const remembered = (state.system.accounts || []).some(function (item) {
return String(item.user_id) === String(user.id);
});
const currentGrant = (state.system.accounts || []).find(function (item) {
return String(item.user_id) === String(user.id);
});
const lastUsed = formatLastUsed(currentGrant && currentGrant.last_used_at);
const metaParts = [];
if (lastUsed) metaParts.push("上次登录:" + lastUsed);
if (remembered) metaParts.push("本机已记住");
const badges = [];
if (global.MobileSession.isAdmin()) badges.push('<span class="m-sys-badge m-sys-badge--admin">管理员</span>');
if (status.subscribed) badges.push('<span class="m-sys-badge m-sys-badge--ok">会员有效</span>');
else if (!global.MobileSession.isAdmin()) badges.push('<span class="m-sys-badge">普通用户</span>');
const accountRows = [
{ key: "system/profile", icon: "user", label: "账号资料", hint: "出生信息 · 加密保存" },
{ key: "system/password", icon: "lock", label: "修改密码", hint: "建议定期更换" },
{ key: "system/membership", icon: "gem", label: "会员状态", hint: "有效期与智能分析额度" }
];
const adminRows = [
{ key: "system/admin", icon: "sliders-horizontal", label: "系统设置", hint: "行情数据 · 模型池" },
{ key: "system/members", icon: "users", label: "会员管理", hint: "开通 · 续期 · 额度" }
];
const html =
'<div class="m-sys-home" data-system-page="home">' +
'<div class="m-card m-sys-profile-card"><span class="m-sys-avatar">' + escapeHtml(avatar) + "</span><div>" +
"<strong>" + escapeHtml(username) + "</strong>" +
(metaParts.length ? '<p class="m-sys-profile-meta">' + escapeHtml(metaParts.join(" · ")) + "</p>" : "") +
(badges.length ? '<div class="m-sys-badges">' + badges.join("") + "</div>" : "") +
"</div></div>" +
'<h3 class="m-sys-group-title">账号</h3>' +
'<div class="m-card m-sys-list">' + accountRows.map(systemRowHtml).join("") + "</div>" +
'<h3 class="m-sys-group-title">偏好</h3>' +
'<div class="m-card m-sys-list">' + systemThemeRowHtml() + "</div>" +
(global.MobileSession.isAdmin()
? '<h3 class="m-sys-group-title">管理员专区</h3><div class="m-card m-sys-list">' + adminRows.map(systemRowHtml).join("") + "</div>"
: "") +
'<h3 class="m-sys-group-title">其他</h3>' +
'<div class="m-card m-sys-list">' +
'<button class="m-sys-row" type="button" data-system-switch>' +
'<span class="m-sys-row-icon">' + icon("arrow-left-right", 18) + "</span>" +
'<span class="m-sys-row-body"><strong>切换账号</strong><small>本机免密进入其他账号</small></span>' +
'<span class="m-sys-row-chevron">' + icon("chevron-right", 16) + "</span></button>" +
'<button class="m-sys-row m-sys-row--danger" type="button" data-system-logout>' +
'<span class="m-sys-row-icon">' + icon("log-out", 18) + "</span>" +
'<span class="m-sys-row-body"><strong>退出登录</strong><small>退出后需要重新登录</small></span>' +
'<span class="m-sys-row-chevron">' + icon("chevron-right", 16) + "</span></button>" +
"</div>" +
'<p class="m-sys-foot">' + (global.MobileSession.isAdmin() ? "小白复盘 · 内网个人版" : "系统设置与会员管理仅管理员可见") + "</p>" +
"</div>";
document.getElementById("m-view").innerHTML = html;
}
function renderSystemProfile() {
@@ -4947,9 +5113,10 @@
const gender = birth.gender || "unspecified";
const configured = Boolean(payload.birth_profile_configured);
const html =
'<div class="m-form-body" data-system-page="profile">' +
'<div class="m-card"><strong>' + escapeHtml(currentUsername()) + "</strong><p class=\"m-sys-lead\">出生信息仅对当前账号可见并加密保存。</p></div>" +
'<p class="m-sys-lead">原始信息加密保存且不在观气页回显;智能解读只使用排盘后的派生结果。</p>' +
'<div class="m-sys-body" data-system-page="profile">' +
'<div class="m-sys-notice"><p>出生信息仅对当前账号可见并加密保存。原始信息不会在观气页回显,智能解读只使用排盘后的派生结果。</p>' +
'<div class="m-sys-badges"><span class="m-sys-badge ' + (configured ? "m-sys-badge--ok" : "") + '">' + (configured ? "已加密保存" : "尚未设置") + "</span></div></div>" +
'<div class="m-card m-sys-section"><strong>命理资料</strong>' +
formFieldHtml("出生日期", dateInputHtml("m-sys-birth-date", birthDate), true) +
formFieldHtml("出生时间", '<input id="m-sys-birth-time" type="time" value="' + escapeHtml(birthTime) + '">', true) +
formFieldHtml("性别", '<select id="m-sys-birth-gender">' +
@@ -4957,12 +5124,10 @@
'<option value="male"' + (gender === "male" ? " selected" : "") + ">男</option>" +
'<option value="female"' + (gender === "female" ? " selected" : "") + ">女</option>" +
"</select>", false) +
'<p class="m-sys-lead">资料状态:' + (configured ? "已加密保存" : "尚未设置") + "</p>" +
'<button class="m-btn-primary m-btn-danger" type="button" data-system-delete-birth' + (configured ? "" : " disabled") + ">删除资料</button>" +
'<div class="m-card m-sys-account-actions">' +
'<button class="m-btn-primary" type="button" data-system-switch>切换账号</button>' +
'<button class="m-btn-primary m-btn-danger" type="button" data-system-logout>退出当前账号</button>' +
"</div></div>";
"</div>" +
'<button class="m-btn-outline-danger" type="button" data-system-delete-birth' + (configured ? "" : " disabled") + ">删除命理资料</button>" +
'<p class="m-sys-hint">删除后智能解读将无法使用出生信息,执行前会再次确认。</p>' +
"</div>";
const page = document.querySelector(".m-page");
if (page) {
page.innerHTML = '<div class="m-scroll" id="m-scroll">' + html + "</div>" +
@@ -4972,17 +5137,16 @@
function renderSystemPassword() {
const html =
'<div class="m-form-body" data-system-page="password">' +
'<p class="m-sys-lead">仅修改当前账号密码,不会保存在这台设备上。</p>' +
formFieldHtml("当前密码", '<input id="m-sys-password-current" type="password" autocomplete="current-password">', true) +
formFieldHtml("密码", '<input id="m-sys-password-new" type="password" minlength="8" maxlength="128" autocomplete="new-password">', true) +
formFieldHtml("确认新密码", '<input id="m-sys-password-confirm" type="password" minlength="8" maxlength="128" autocomplete="new-password">', true) +
"</div>";
'<div class="m-sys-body" data-system-page="password">' +
'<div class="m-sys-notice"><p>仅修改当前账号的登录密码,密码不会保存在这台设备上。修改成功后下次登录需使用新密码。</p></div>' +
'<div class="m-card m-sys-section"><strong>设置新密码</strong>' +
formFieldHtml("当前密码", '<input id="m-sys-password-current" type="password" autocomplete="current-password" placeholder="输入现在的密码">', true, '<p class="m-field-error" hidden data-field-error="current"></p>') +
formFieldHtml("新密码", '<input id="m-sys-password-new" type="password" minlength="8" maxlength="128" autocomplete="new-password" placeholder="8-128 位">', true, '<p class="m-sys-hint">建议字母与数字混合,不要与其他网站重复。</p><p class="m-field-error" hidden data-field-error="new"></p>') +
formFieldHtml("确认新密码", '<input id="m-sys-password-confirm" type="password" minlength="8" maxlength="128" autocomplete="new-password" placeholder="再输入一次新密码">', true, '<p class="m-field-error" hidden data-field-error="confirm"></p>') +
'<button class="m-btn-primary" type="button" data-system-save-password>更新密码</button>' +
"</div></div>";
const page = document.querySelector(".m-page");
if (page) {
page.innerHTML = '<div class="m-scroll" id="m-scroll">' + html + "</div>" +
'<div class="m-form-bar"><button class="m-btn-primary" type="button" data-system-save-password>更新密码</button></div>';
}
if (page) page.innerHTML = '<div class="m-scroll" id="m-scroll">' + html + "</div>";
}
function renderSystemMembership() {
@@ -4995,25 +5159,28 @@
? number(membership.remaining_days) + " 天"
: (membership.is_admin || membership.subscribed ? "长期有效" : "--");
const detail = membership.subscribed
? ((membership.plan || "会员") + (membership.expires_at ? " · 有效至 " + membershipDateLabel(membership.expires_at) : " · 长期有效"))
? ((membership.plan || "会员") + (membership.expires_at ? " · 有效至 " + membershipDateLabel(membership.expires_at) : " · 长期有效") + (membership.remaining_days != null ? ",剩余 " + number(membership.remaining_days) + " 天。" : "。"))
: membership.is_admin
? "管理员拥有智能功能管理权限,但不会因此显示为已开通会员。"
: "开通会员后可使用智能选股、问师、问天、复盘助手等智能功能。";
const quota = "会员默认每日智能分析额度 " + number(access.daily_limit) + " 次,由管理员统一设置。";
const usage = membership.active ? ("今日已用 " + number(access.used_today) + " 次") : "今日智能分析:--";
const usageSummary = membership.active ? (number(access.used_today) + " / " + number(access.daily_limit)) : "--";
const usedToday = membership.active ? (number(access.used_today) + " 次") : "--";
const remainingCalls = membership.is_admin ? "不限" : membership.active ? (number(access.remaining_calls) + " 次") : "--";
const html =
'<div class="m-form-body" data-system-page="membership">' +
'<div class="m-card"><strong>' + escapeHtml(badge) + "</strong><p class=\"m-sys-lead\">" + escapeHtml(detail) + "</p><p class=\"m-sys-lead\">" + escapeHtml(quota) + "</p></div>" +
'<div class="m-sys-body" data-system-page="membership">' +
'<div class="m-card m-sys-hero"><strong>' + escapeHtml(badge) + '</strong>' +
(membership.subscribed ? '<div class="m-sys-badges"><span class="m-sys-badge m-sys-badge--ok">已开通</span></div>' : "") +
'<p class="m-sys-lead">' + escapeHtml(detail) + "</p></div>" +
'<div class="m-card m-sys-section"><strong>今日智能分析</strong>' +
'<div class="m-sys-grid">' +
"<div><span>开通状态</span><strong>" + escapeHtml(stateLabel) + "</strong></div>" +
"<div><span>剩余时长</span><strong>" + escapeHtml(remaining) + "</strong></div>" +
"<div><span>今日智能分析</span><strong>" + escapeHtml(usageSummary) + "</strong></div>" +
"<div><span>剩余智能分析</span><strong>" + escapeHtml(remainingCalls) + "</strong></div>" +
"<div><span>今日已用</span><strong>" + escapeHtml(usedToday) + "</strong></div>" +
"<div><span>今日剩余</span><strong>" + escapeHtml(remainingCalls) + "</strong></div>" +
"</div>" +
'<div class="m-card"><p class="m-sys-lead">' + escapeHtml(usage) + "</p>" +
'<p class="m-sys-lead">行情、搜索、自选与复盘:普通用户可用。智能选股、问师、问天、复盘助手:仅会员可用。</p></div>' +
'<p class="m-sys-hint">会员每日智能分析额度 ' + number(access.daily_limit) + " 次,每日 0 点自动重置,由管理员统一设置。</p></div>" +
'<div class="m-card m-sys-section"><strong>权益说明</strong>' +
'<p class="m-sys-lead">全部用户可用:行情、搜索、自选股、交易日志与复盘。</p>' +
'<p class="m-sys-lead">会员专属:智能选股、问师、问天、复盘助手等智能功能。</p></div>' +
"</div>";
systemFill(html);
}
@@ -5026,6 +5193,10 @@
"</div>";
}
function statusDot(ok) {
return '<span class="m-sys-dot' + (ok ? " m-sys-dot--ok" : "") + '"></span>';
}
function renderSystemAdmin(key) {
if (key === "system/members") {
renderSystemMembers();
@@ -5035,74 +5206,86 @@
const data = payload.data || {};
const ifind = data.ifind || {};
const llm = payload.llm || {};
const status = "Tushare " + (data.configured ? "已配置" : "未配置") +
" · iFinD " + (ifind.configured ? "已配置" : "未配置") +
" · " + number(data.snapshot_dates) + " 个交易日";
const refreshLabel = data.background_refresh_enabled ? "后台刷新已启用" : "后台刷新已暂停";
const tab = state.system.adminTab === "models" ? "models" : "market";
const marketHtml =
'<div class="m-form-body" data-system-admin-panel="market">' +
'<div class="m-card"><strong>公共行情</strong><p class="m-sys-lead">' + escapeHtml(status) + "</p><p class=\"m-sys-lead\">" + escapeHtml(refreshLabel) + "</p></div>" +
formFieldHtml("Tushare Token", '<input id="m-sys-token" type="password" autocomplete="off" minlength="20" placeholder="留空保留现有 Token">', false) +
formFieldHtml("iFinD Refresh Token", '<input id="m-sys-ifind" type="password" autocomplete="off" maxlength="2048" placeholder="留空保留现有 Token">', false) +
formFieldHtml("交易时段后台刷新", '<input id="m-sys-bg-refresh" type="checkbox"' + (data.background_refresh_enabled ? " checked" : "") + ">", false) +
'<p class="m-sys-lead">所有用户读取同一份后台快照,页面不会随后台任务自动重绘。</p>' +
'<button class="m-btn-primary" type="button" data-system-refresh>立即后台刷新</button>' +
'<div class="m-card"><strong>历史数据回补</strong></div>' +
'<div class="m-sys-body" data-system-admin-panel="market">' +
'<div class="m-card m-sys-section"><strong>数据源状态</strong>' +
'<div class="m-sys-status-list">' +
'<div class="m-sys-status-item"><span>Tushare</span><span>' + statusDot(data.configured) + (data.configured ? " 已配置" : " 未配置") + "</span></div>" +
'<div class="m-sys-status-item"><span>iFinD</span><span>' + statusDot(ifind.configured) + (ifind.configured ? " 已配置" : " 未配置") + "</span></div>" +
'<div class="m-sys-status-item"><span>行情快照</span><strong>' + number(data.snapshot_dates) + " 个交易日</strong></div>" +
'<div class="m-sys-status-item"><span>后台刷新</span><span>' + statusDot(data.background_refresh_enabled) + (data.background_refresh_enabled ? " 已启用" : " 已暂停") + "</span></div>" +
"</div></div>" +
'<div class="m-card m-sys-section"><strong>数据源密钥</strong>' +
formFieldHtml("Tushare Token", '<input id="m-sys-token" type="password" autocomplete="off" minlength="20" placeholder="留空则保留现有 Token">', false) +
formFieldHtml("iFinD Refresh Token", '<input id="m-sys-ifind" type="password" autocomplete="off" maxlength="2048" placeholder="留空则保留现有 Token">', false) +
'<button class="m-btn-primary" type="button" data-system-save-market>保存密钥</button></div>' +
'<div class="m-card m-sys-section"><strong>后台刷新</strong>' +
'<div class="m-sys-switch-row"><div><strong>交易时段自动刷新</strong><p class="m-sys-hint">开启后后台定时更新快照</p></div>' +
'<button class="m-theme-switch" type="button" data-system-toggle-refresh role="switch" aria-checked="' + (data.background_refresh_enabled ? "true" : "false") + '" aria-label="交易时段自动刷新"><span class="m-theme-switch-thumb"></span></button></div>' +
'<input id="m-sys-bg-refresh" type="checkbox"' + (data.background_refresh_enabled ? " checked" : "") + ' hidden>' +
'<button class="m-btn-outline" type="button" data-system-refresh>立即刷新一次</button>' +
'<p class="m-sys-hint">所有用户读取同一份快照,刷新不影响当前页面内容。</p></div>' +
'<div class="m-card m-sys-section"><strong>历史数据回补</strong>' +
formFieldHtml("开始日期", dateInputHtml("m-sys-backfill-start", ""), false) +
formFieldHtml("结束日期", dateInputHtml("m-sys-backfill-end", ""), false) +
'<button class="m-btn-primary" type="button" data-system-backfill>开始回补</button>' +
'<button class="m-btn-outline" type="button" data-system-backfill>开始回补</button>' +
'<p class="m-sys-hint">回补用于补齐缺失的历史行情,开始前会再次确认;回补期间页面可正常使用。</p></div>' +
"</div>";
const models = state.system.models || [];
const modelsHtml =
'<div class="m-form-body" data-system-admin-panel="models">' +
'<div class="m-sys-body" data-system-admin-panel="models">' +
'<div class="m-card m-sys-section"><strong>模型分工</strong>' +
formFieldHtml("主模型", '<select id="m-sys-primary-model"></select>', false) +
formFieldHtml("辅助模型", '<select id="m-sys-fallback-model"></select>', false) +
'<div id="m-sys-model-list">' + renderModelPoolHtml(llm.models || []) + "</div>" +
'<button class="m-btn-primary" type="button" data-system-add-model>添加模型</button>' +
'<p class="m-sys-hint">主模型不可用时自动改用辅助模型</p>' +
'<button class="m-btn-primary" type="button" data-system-save-models>保存分工</button></div>' +
'<div class="m-card m-sys-section"><strong>模型池 · ' + models.length + " 个</strong>" +
'<div id="m-sys-model-list">' + renderModelPoolHtml(models) + "</div>" +
'<p class="m-sys-hint">点任意模型卡片进入编辑:改名称、地址、密钥、测试连接或删除</p>' +
'<button class="m-btn-primary" type="button" data-system-add-model>+ 添加模型</button></div>' +
"</div>";
const page = document.querySelector(".m-page");
if (!page) return;
const bar = tab === "models"
? '<div class="m-form-bar"><button class="m-btn-primary" type="button" data-system-save-models>保存模型池</button></div>'
: '<div class="m-form-bar"><button class="m-btn-primary" type="button" data-system-save-market>保存行情配置</button></div>';
page.innerHTML = adminTabHtml() + '<div class="m-scroll" id="m-scroll">' + (tab === "models" ? modelsHtml : marketHtml) + "</div>" + bar;
page.innerHTML = adminTabHtml() + '<div class="m-scroll" id="m-scroll">' + (tab === "models" ? modelsHtml : marketHtml) + "</div>";
if (tab === "models") updateSystemModelRoleOptions(llm.primary_model_id || "", llm.fallback_model_id || "");
}
function hostOfUrl(url) {
try {
return new URL(url).host;
} catch (error) {
return String(url || "").replace(/^https?:\/\//, "").split("/")[0] || "--";
}
}
function renderModelPoolHtml(models) {
if (!models.length) {
return '<div class="m-state"><p>模型池为空,请先添加模型</p></div>';
}
return models.map(function (item, index) {
return '<article class="m-card" data-model-id="' + escapeHtml(item.id) + '">' +
"<strong>" + escapeHtml(item.name || ("模型 " + (index + 1))) + "</strong>" +
'<p class="m-sys-lead">' + (item.configured ? "已保存密钥" : "待配置") + "</p>" +
formFieldHtml("显示名称", '<input data-model-field="name" maxlength="50" value="' + escapeHtml(item.name || "") + '">', true) +
formFieldHtml("API Base URL", '<input data-model-field="base_url" type="url" value="' + escapeHtml(item.base_url || "https://api.openai.com/v1") + '">', true) +
formFieldHtml("模型标识", '<input data-model-field="model" maxlength="100" value="' + escapeHtml(item.model || "") + '">', true) +
formFieldHtml("API Key", '<input data-model-field="api_key" type="password" autocomplete="off" maxlength="300" placeholder="' + (item.configured ? "留空保留已保存的 Key" : "输入 API Key") + '">', !item.configured) +
'<button class="m-btn-primary" type="button" data-system-test-model>测试连接</button>' +
'<p class="m-sys-lead" data-model-test-status>未测试</p>' +
'<button class="m-btn-primary m-btn-danger" type="button" data-system-delete-model>删除模型</button>' +
"</article>";
const llm = (state.system.admin && state.system.admin.llm) || {};
return models.map(function (item) {
const badges = [];
if (item.id === llm.primary_model_id) badges.push('<span class="m-sys-badge m-sys-badge--admin">主模型</span>');
if (item.id === llm.fallback_model_id) badges.push('<span class="m-sys-badge">辅助</span>');
badges.push('<span class="m-sys-badge' + (item.configured ? " m-sys-badge--ok" : "") + '">' + (item.configured ? "已配置" : "待配置") + "</span>");
return '<button class="m-sys-model-card" type="button" data-model-id="' + escapeHtml(item.id) + '" data-system-edit-model>' +
"<div><strong>" + escapeHtml(item.name || "未命名模型") + "</strong>" +
'<div class="m-sys-badges">' + badges.join("") + "</div>" +
'<p class="m-sys-hint">' + escapeHtml(hostOfUrl(item.base_url) + " · " + (item.model || "未填写标识")) + "</p></div>" +
'<span class="m-sys-row-chevron">' + icon("chevron-right", 16) + "</span></button>";
}).join("");
}
function collectSystemModelPool() {
const models = (state.system.admin && state.system.admin.llm && state.system.admin.llm.models) || [];
const saved = new Map(models.map(function (item) { return [item.id, item]; }));
return Array.prototype.map.call(document.querySelectorAll("#m-sys-model-list [data-model-id]"), function (row) {
function fieldValue(name) {
const input = row.querySelector("[data-model-field='" + name + "']");
return String(input && input.value != null ? input.value : "").trim();
}
return (state.system.models || []).map(function (item) {
return {
id: row.dataset.modelId,
name: fieldValue("name"),
base_url: fieldValue("base_url"),
model: fieldValue("model"),
api_key: fieldValue("api_key"),
configured: Boolean(saved.get(row.dataset.modelId) && saved.get(row.dataset.modelId).configured),
id: item.id,
name: item.name || "",
base_url: item.base_url || "",
model: item.model || "",
api_key: item.api_key || "",
configured: Boolean(item.configured)
};
});
}
@@ -5122,6 +5305,40 @@
fallback.value = models.some(function (item) { return item.id === fallbackId; }) && fallbackId !== keepPrimary ? fallbackId : "";
}
function openModelEditSheet(modelId) {
const models = state.system.models || [];
const item = models.find(function (row) { return row.id === modelId; }) || {};
state.system.editingModelId = modelId;
openSheet(
'<div class="m-sheet-head"><h2>编辑模型</h2>' +
'<button class="m-sheet-close" type="button" data-sheet-close aria-label="关闭">' + icon("close", 20) + "</button></div>" +
'<div class="m-sheet-body" data-model-id="' + escapeHtml(modelId) + '">' +
formFieldHtml("显示名称", '<input data-model-field="name" maxlength="50" value="' + escapeHtml(item.name || "") + '">', true) +
formFieldHtml("API Base URL", '<input data-model-field="base_url" type="url" value="' + escapeHtml(item.base_url || "https://api.openai.com/v1") + '">', true) +
formFieldHtml("模型标识", '<input data-model-field="model" maxlength="100" value="' + escapeHtml(item.model || "") + '">', true) +
formFieldHtml("API Key", '<input data-model-field="api_key" type="password" autocomplete="off" maxlength="300" placeholder="' + (item.configured ? "留空则保留已保存的 Key" : "输入 API Key") + '">', !item.configured) +
'<div class="m-sys-test-row"><button class="m-btn-outline" type="button" data-system-test-model>测试连接</button>' +
'<span data-model-test-status>未测试</span></div>' +
'<div class="m-sys-sheet-actions">' +
'<button class="m-btn-outline-danger" type="button" data-system-delete-model>删除模型</button>' +
'<button class="m-btn-primary" type="button" data-system-save-model>保存</button>' +
"</div></div>"
);
}
function readModelSheetFields(root) {
function fieldValue(name) {
const input = root.querySelector("[data-model-field='" + name + "']");
return String(input && input.value != null ? input.value : "").trim();
}
return {
name: fieldValue("name"),
base_url: fieldValue("base_url"),
model: fieldValue("model"),
api_key: fieldValue("api_key")
};
}
function renderSystemMembers() {
const payload = state.system.admin || {};
const membership = payload.membership || {};
@@ -5129,39 +5346,77 @@
const userHtml = users.map(function (user) {
const admin = user.role === "admin";
const member = Boolean(user.membership_subscribed);
const identity = [admin ? "管理员" : "", member ? "会员有效" : "普通用户"].filter(Boolean).join(" · ");
const badges = [];
if (admin) badges.push('<span class="m-sys-badge m-sys-badge--admin">管理员</span>');
if (member) badges.push('<span class="m-sys-badge m-sys-badge--ok">会员</span>');
else badges.push('<span class="m-sys-badge">普通用户</span>');
const expiry = member
? (user.membership_expires_at ? "有效至 " + membershipDateLabel(user.membership_expires_at) : "永久有效")
: user.membership_status === "suspended"
? "会员已停用"
: "尚未开通";
return '<article class="m-card" data-admin-user="' + number(user.id) + '">' +
"<strong>" + escapeHtml(user.username) + "</strong>" +
'<p class="m-sys-lead">' + escapeHtml(identity) + " · 今日调用 " + number(user.used_today) + "</p>" +
'<p class="m-sys-lead">' + escapeHtml(expiry) + "</p>" +
formFieldHtml("状态", '<select data-member-status>' +
'<option value="inactive"' + (user.membership_status === "inactive" ? " selected" : "") + ">未开通</option>" +
'<option value="active"' + (user.membership_status === "active" ? " selected" : "") + ">有效</option>" +
'<option value="suspended"' + (user.membership_status === "suspended" ? " selected" : "") + ">停用</option>" +
"</select>", false) +
formFieldHtml("开通 / 续期时长", '<select data-member-duration><option value="">选择时长</option>' +
'<option value="1_month">1个月</option><option value="3_months">3个月</option>' +
'<option value="12_months">12个月</option><option value="3_years">3年</option>' +
'<option value="permanent">永久</option></select>', false) +
'<button class="m-btn-primary" type="button" data-system-save-member>应用</button>' +
"</article>";
return '<div class="m-sys-user-row" data-admin-user="' + number(user.id) + '">' +
'<div class="m-sys-row-body"><strong>' + escapeHtml(user.username) + "</strong>" +
'<div class="m-sys-badges">' + badges.join("") + "</div>" +
'<p class="m-sys-hint">' + escapeHtml(expiry) + " · 今日已用 " + number(user.used_today) + " 次</p></div>" +
'<button class="m-btn-outline" type="button" data-system-open-member>管理</button></div>';
}).join("") || '<div class="m-state"><p>暂无注册用户</p></div>';
const html =
'<div class="m-form-body" data-system-page="members">' +
'<div class="m-card"><strong>会员调用额度</strong><p class="m-sys-lead">每日自动重置</p></div>' +
'<div class="m-sys-body" data-system-page="members">' +
'<div class="m-card m-sys-section"><strong>全局额度</strong>' +
formFieldHtml("会员每日智能分析上限", '<input id="m-sys-member-limit" type="number" min="1" max="1000" value="' + (number(membership.member_daily_limit) || 50) + '">', false) +
'<button class="m-btn-primary" type="button" data-system-save-limit>保存调用额度</button>' +
'<div class="m-card"><strong>会员账号</strong><p class="m-sys-lead">手动开通与续期</p></div>' +
'<p class="m-sys-hint">对所有会员生效,每日 0 点自动重置。</p>' +
'<button class="m-btn-primary" type="button" data-system-save-limit>保存额度</button></div>' +
'<div class="m-card m-sys-section"><strong>会员账号 · ' + users.length + " 个</strong>" +
userHtml +
'<p class="m-sys-hint">点「管理」为对应账号开通、续期或停用会员。</p></div>' +
"</div>";
systemFill(html);
}
function openMemberManageSheet(userId) {
const users = (state.system.admin && state.system.admin.users) || [];
const user = users.find(function (item) { return String(item.id) === String(userId); });
if (!user) return;
state.system.editingUserId = String(userId);
const member = Boolean(user.membership_subscribed);
const expiry = member
? (user.membership_expires_at ? "有效至 " + membershipDateLabel(user.membership_expires_at) : "永久有效")
: user.membership_status === "suspended" ? "会员已停用" : "尚未开通";
openSheet(
'<div class="m-sheet-head"><h2>管理会员 · ' + escapeHtml(user.username) + "</h2>" +
'<button class="m-sheet-close" type="button" data-sheet-close aria-label="关闭">' + icon("close", 20) + "</button></div>" +
'<div class="m-sheet-body" data-admin-user="' + number(user.id) + '">' +
'<p class="m-sys-lead">当前状态:' + escapeHtml(expiry) + " · 今日已用 " + number(user.used_today) + " 次</p>" +
formFieldHtml("会员状态", '<select data-member-status>' +
'<option value="inactive"' + (user.membership_status === "inactive" ? " selected" : "") + ">未开通</option>" +
'<option value="active"' + (user.membership_status === "active" ? " selected" : "") + ">有效</option>" +
'<option value="suspended"' + (user.membership_status === "suspended" ? " selected" : "") + ">停用</option>" +
"</select>", false) +
formFieldHtml("开通 / 续期时长", '<select data-member-duration><option value="">选择时长</option>' +
'<option value="1_month">1个月</option><option value="3_months">3个月</option>' +
'<option value="12_months">12个月</option><option value="3_years">3年</option>' +
'<option value="permanent">永久</option></select>', false) +
'<p class="m-sys-hint">从当前时间开始顺延;已有会员则叠加续期。</p>' +
'<div class="m-sys-sheet-actions">' +
'<button class="m-btn-outline" type="button" data-sheet-close>取消</button>' +
'<button class="m-btn-primary" type="button" data-system-save-member>应用</button>' +
"</div></div>"
);
}
function setFieldError(inputId, message) {
const input = document.getElementById(inputId);
const field = input && input.closest(".m-form-field");
if (!field) return;
field.classList.toggle("is-invalid", Boolean(message));
const error = field.querySelector("[data-field-error]");
if (error) {
error.hidden = !message;
error.textContent = message || "";
}
}
function saveSystemBirth() {
const birthDate = (document.getElementById("m-sys-birth-date") || {}).value;
const birthTime = (document.getElementById("m-sys-birth-time") || {}).value;
@@ -5176,7 +5431,7 @@
gender: (document.getElementById("m-sys-birth-gender") || {}).value || "unspecified",
trade_date: todayString(),
}).then(function () {
showToast("个人命理资料已保存到当前账号");
showToast("个人命理资料已保存");
loadSystem();
}).catch(function (error) {
showToast(error && error.message ? error.message : "个人命理资料保存失败");
@@ -5186,12 +5441,12 @@
}
function deleteSystemBirth() {
openConfirmSheet("删除资料", "确定删除当前账号保存的个人命理资料吗?", {
openConfirmSheet("删除命理资料", "删除后智能解读将无法使用你的出生信息,此操作不可恢复。", {
danger: true,
centered: true,
confirmLabel: "删除",
onConfirm: function () {
global.MobileAPI.request("/api/account/birth-profile", "DELETE").then(function () {
closeSheet();
showToast("个人命理资料已删除");
loadSystem();
}).catch(function (error) {
@@ -5205,6 +5460,23 @@
const current = (document.getElementById("m-sys-password-current") || {}).value || "";
const next = (document.getElementById("m-sys-password-new") || {}).value || "";
const confirm = (document.getElementById("m-sys-password-confirm") || {}).value || "";
setFieldError("m-sys-password-current", "");
setFieldError("m-sys-password-new", "");
setFieldError("m-sys-password-confirm", "");
let invalid = false;
if (!current) {
setFieldError("m-sys-password-current", "请输入当前密码");
invalid = true;
}
if (next.length < 8 || next.length > 128) {
setFieldError("m-sys-password-new", "新密码长度应为 8 至 128 位");
invalid = true;
}
if (next !== confirm) {
setFieldError("m-sys-password-confirm", "两次输入的密码不一致,请重新输入。");
invalid = true;
}
if (invalid) return;
const button = document.querySelector("[data-system-save-password]");
if (button) button.disabled = true;
global.MobileAPI.request("/api/account/password", "POST", {
@@ -5212,14 +5484,13 @@
new_password: next,
confirm_password: confirm,
}).then(function () {
const formIds = ["m-sys-password-current", "m-sys-password-new", "m-sys-password-confirm"];
formIds.forEach(function (id) {
["m-sys-password-current", "m-sys-password-new", "m-sys-password-confirm"].forEach(function (id) {
const input = document.getElementById(id);
if (input) input.value = "";
});
showToast("密码已更新");
}).catch(function (error) {
showToast(error && error.message ? error.message : "密码更新失败");
showToast("更新失败:" + (error && error.message ? error.message : "密码更新失败"));
}).then(function () {
if (button) button.disabled = false;
});
@@ -5232,10 +5503,10 @@
function logoutSystemAccount() {
openConfirmSheet("退出当前账号", "退出后需要重新登录。本机已记录的其他账号仍可直接切换。", {
danger: true,
centered: true,
confirmLabel: "退出",
onConfirm: function () {
global.MobileSession.logout().then(function () {
closeSheet();
global.MobileRouter.replace("#/auth");
}).catch(function (error) {
showToast(error && error.message ? error.message : "退出失败");
@@ -5250,9 +5521,8 @@
global.MobileAPI.request("/api/admin/settings", "POST", {
tushare_token: ((document.getElementById("m-sys-token") || {}).value || "").trim(),
ifind_refresh_token: ((document.getElementById("m-sys-ifind") || {}).value || "").trim(),
background_refresh_enabled: Boolean((document.getElementById("m-sys-bg-refresh") || {}).checked),
}).then(function () {
showToast("行情配置已保存");
showToast("行情密钥已保存");
loadSystem();
}).catch(function (error) {
showToast(error && error.message ? error.message : "系统配置保存失败");
@@ -5261,30 +5531,65 @@
});
}
function toggleSystemRefresh() {
const enabled = !Boolean((state.system.admin && state.system.admin.data && state.system.admin.data.background_refresh_enabled));
global.MobileAPI.request("/api/admin/settings", "POST", {
background_refresh_enabled: enabled,
}).then(function () {
showToast(enabled ? "后台刷新已启用" : "后台刷新已暂停");
loadSystem();
}).catch(function (error) {
showToast(error && error.message ? error.message : "后台刷新设置失败");
});
}
function saveSystemModels() {
const button = document.querySelector("[data-system-save-models]");
if (button) button.disabled = true;
global.MobileAPI.request("/api/admin/settings", "POST", {
models: collectSystemModelPool(),
primary_model_id: (document.getElementById("m-sys-primary-model") || {}).value || "",
fallback_model_id: (document.getElementById("m-sys-fallback-model") || {}).value || "",
}).then(function () {
showToast("模型已保存");
showToast("模型分工已保存");
loadSystem();
}).catch(function (error) {
showToast(error && error.message ? error.message : "模型保存失败");
showToast(error && error.message ? error.message : "模型分工保存失败");
}).then(function () {
if (button) button.disabled = false;
});
}
function persistSystemModels(models, message) {
return global.MobileAPI.request("/api/admin/settings", "POST", { models: models }).then(function () {
showToast(message || "模型池已保存");
closeSheet();
loadSystem();
});
}
function saveEditedSystemModel() {
const sheet = document.querySelector(".m-sheet-body[data-model-id]");
if (!sheet) return;
const id = sheet.dataset.modelId;
const fields = readModelSheetFields(sheet);
const models = collectSystemModelPool().map(function (item) {
if (item.id !== id) return item;
return Object.assign({}, item, fields);
});
persistSystemModels(models, "模型已保存").catch(function (error) {
showToast(error && error.message ? error.message : "模型保存失败");
});
}
function addSystemModel() {
const models = collectSystemModelPool();
const id = "model-" + Date.now() + "-" + Math.floor(Math.random() * 10000);
models.push({ id: id, name: "模型 " + (models.length + 1), base_url: "https://api.openai.com/v1", model: "", api_key: "", configured: false });
const created = { id: id, name: "模型 " + (models.length + 1), base_url: "https://api.openai.com/v1", model: "", api_key: "", configured: false };
state.system.models = models.concat([created]);
const list = document.getElementById("m-sys-model-list");
if (list) list.innerHTML = renderModelPoolHtml(models);
if (list) list.innerHTML = renderModelPoolHtml(state.system.models);
updateSystemModelRoleOptions(id, (document.getElementById("m-sys-fallback-model") || {}).value || "");
openModelEditSheet(id);
}
function deleteSystemModel(row) {
@@ -5292,25 +5597,36 @@
const id = row.dataset.modelId;
const primary = (document.getElementById("m-sys-primary-model") || {}).value;
const fallback = (document.getElementById("m-sys-fallback-model") || {}).value;
if (id === primary || id === fallback) {
const llm = (state.system.admin && state.system.admin.llm) || {};
if (id === primary || id === fallback || id === llm.primary_model_id || id === llm.fallback_model_id) {
showToast("请先为主模型或辅助模型选择其他模型,再删除当前模型");
return;
}
const models = collectSystemModelPool().filter(function (item) { return item.id !== id; });
const list = document.getElementById("m-sys-model-list");
if (list) list.innerHTML = renderModelPoolHtml(models);
updateSystemModelRoleOptions(primary, fallback);
openConfirmSheet("删除模型?", "删除后该模型将从模型池移除,此操作不可恢复。", {
danger: true,
centered: true,
confirmLabel: "删除",
onConfirm: function () {
const models = collectSystemModelPool().filter(function (item) { return item.id !== id; });
persistSystemModels(models, "模型已删除").catch(function (error) {
showToast(error && error.message ? error.message : "删除失败");
});
}
});
}
function testSystemModel(row) {
if (!row) return;
const status = row.querySelector("[data-model-test-status]");
const button = row.querySelector("[data-system-test-model]");
const profile = collectSystemModelPool().find(function (item) { return item.id === row.dataset.modelId; }) || {};
const status = document.querySelector("[data-model-test-status]");
const button = document.querySelector("[data-system-test-model]");
const sheet = document.querySelector(".m-sheet-body[data-model-id]");
const id = (sheet && sheet.dataset.modelId) || row.dataset.modelId;
const fields = sheet ? readModelSheetFields(sheet) : {};
const profile = Object.assign({}, collectSystemModelPool().find(function (item) { return item.id === id; }) || {}, fields, { id: id });
if (button) button.disabled = true;
if (status) status.textContent = "连接中";
global.MobileAPI.request("/api/admin/settings/test", "POST", { model_id: row.dataset.modelId, profile: profile }).then(function (payload) {
if (status) status.textContent = "已连通 · " + number(payload.result && payload.result.latency_ms) + " ms";
global.MobileAPI.request("/api/admin/settings/test", "POST", { model_id: id, profile: profile }).then(function (payload) {
if (status) status.textContent = "上次测试:成功 · " + number(payload.result && payload.result.latency_ms) + "ms";
}).catch(function (error) {
if (status) status.textContent = error && error.message ? error.message : "测试失败";
}).then(function () {
@@ -5331,19 +5647,27 @@
}
function startSystemBackfill() {
const button = document.querySelector("[data-system-backfill]");
if (button) button.disabled = true;
global.MobileAPI.request("/api/backfill", "POST", {
start_date: (document.getElementById("m-sys-backfill-start") || {}).value,
end_date: (document.getElementById("m-sys-backfill-end") || {}).value,
}).then(function (payload) {
const count = payload && payload.results ? payload.results.length : 0;
showToast("历史回补完成,共处理 " + count + " 个工作日");
loadSystem();
}).catch(function (error) {
showToast(error && error.message ? error.message : "回补失败");
}).then(function () {
if (button) button.disabled = false;
const startDate = (document.getElementById("m-sys-backfill-start") || {}).value;
const endDate = (document.getElementById("m-sys-backfill-end") || {}).value;
openConfirmSheet("开始历史回补?", "将按选定日期补齐缺失行情,回补期间页面仍可使用。", {
confirmLabel: "开始回补",
centered: true,
onConfirm: function () {
const button = document.querySelector("[data-system-backfill]");
if (button) button.disabled = true;
global.MobileAPI.request("/api/backfill", "POST", {
start_date: startDate,
end_date: endDate,
}).then(function (payload) {
const count = payload && payload.results ? payload.results.length : 0;
showToast("历史回补完成,共处理 " + count + " 个工作日");
loadSystem();
}).catch(function (error) {
showToast(error && error.message ? error.message : "回补失败");
}).then(function () {
if (button) button.disabled = false;
});
}
});
}
@@ -5364,21 +5688,35 @@
function saveSystemMember(card) {
if (!card) return;
const button = card.querySelector("[data-system-save-member]");
if (button) button.disabled = true;
global.MobileAPI.request("/api/admin/membership", "POST", {
user_id: card.dataset.adminUser,
status: (card.querySelector("[data-member-status]") || {}).value,
duration: (card.querySelector("[data-member-duration]") || {}).value,
}).then(function (payload) {
if (state.system.admin) state.system.admin.users = payload.users || [];
showToast("会员状态已更新");
renderSystemMembers();
}).catch(function (error) {
showToast(error && error.message ? error.message : "会员状态保存失败");
}).then(function () {
if (button) button.disabled = false;
});
const status = (card.querySelector("[data-member-status]") || {}).value;
const apply = function () {
const button = card.querySelector("[data-system-save-member]");
if (button) button.disabled = true;
global.MobileAPI.request("/api/admin/membership", "POST", {
user_id: card.dataset.adminUser,
status: status,
duration: (card.querySelector("[data-member-duration]") || {}).value,
}).then(function (payload) {
if (state.system.admin) state.system.admin.users = payload.users || [];
closeSheet();
showToast("会员状态已更新");
renderSystemMembers();
}).catch(function (error) {
showToast(error && error.message ? error.message : "会员状态保存失败");
}).then(function () {
if (button) button.disabled = false;
});
};
if (status === "suspended") {
openConfirmSheet("停用该会员?", "停用后该账号将无法使用会员智能功能,可稍后重新开通。", {
danger: true,
centered: true,
confirmLabel: "停用",
onConfirm: apply
});
return;
}
apply();
}
/* ---------------------------------------------------------------- events */
@@ -5522,8 +5860,12 @@
if (event.target.closest("[data-system-switch]")) { switchSystemAccount(); return; }
if (event.target.closest("[data-system-logout]")) { logoutSystemAccount(); return; }
if (event.target.closest("[data-system-save-market]")) { saveSystemMarket(); return; }
if (event.target.closest("[data-system-toggle-refresh]")) { toggleSystemRefresh(); return; }
if (event.target.closest("[data-system-save-models]")) { saveSystemModels(); return; }
if (event.target.closest("[data-system-save-model]")) { saveEditedSystemModel(); return; }
if (event.target.closest("[data-system-add-model]")) { addSystemModel(); return; }
const editModel = event.target.closest("[data-system-edit-model]");
if (editModel) { openModelEditSheet(editModel.dataset.modelId); return; }
const deleteModel = event.target.closest("[data-system-delete-model]");
if (deleteModel) { deleteSystemModel(deleteModel.closest("[data-model-id]")); return; }
const testModel = event.target.closest("[data-system-test-model]");
@@ -5531,6 +5873,8 @@
if (event.target.closest("[data-system-refresh]")) { startSystemRefresh(); return; }
if (event.target.closest("[data-system-backfill]")) { startSystemBackfill(); return; }
if (event.target.closest("[data-system-save-limit]")) { saveSystemMemberLimit(); return; }
const openMember = event.target.closest("[data-system-open-member]");
if (openMember) { openMemberManageSheet(openMember.closest("[data-admin-user]").dataset.adminUser); return; }
const saveMember = event.target.closest("[data-system-save-member]");
if (saveMember) { saveSystemMember(saveMember.closest("[data-admin-user]")); return; }
@@ -5691,5 +6035,6 @@
global.MobilePages = {
render: renderPage,
has: function (key) { return Boolean(pageConfig(key) || isComplexPage(key)); },
renderSystemHome: renderSystemHome,
};
})(window);
+6 -2
View File
@@ -187,9 +187,9 @@
const dark = document.getElementById("m-app").dataset.theme === "dark";
const toggle = document.querySelector("[data-theme-toggle]");
if (toggle) toggle.setAttribute("aria-checked", dark ? "true" : "false");
const rowIcon = document.querySelector(".m-theme-row-icon");
const rowIcon = document.querySelector(".m-theme-row-icon, [data-theme-row-icon]");
if (rowIcon) rowIcon.innerHTML = icon(dark ? "moon" : "sun");
const rowBodySmall = document.querySelector(".m-theme-row-body small");
const rowBodySmall = document.querySelector(".m-theme-row-body small, [data-theme-row-label]");
if (rowBodySmall) rowBodySmall.textContent = dark ? "当前:夜间模式" : "当前:日间模式";
}
@@ -206,6 +206,10 @@
replace(DEFAULT_HASH);
return;
}
if (key === "system" && global.MobilePages && typeof global.MobilePages.renderSystemHome === "function") {
global.MobilePages.renderSystemHome();
return;
}
const items = visibleHubItems(hub);
updateHeader({ title: hub.title, back: false });
const section = key === "system" ? themeToggleSection() : "";
+1 -1
View File
@@ -95,7 +95,7 @@
"/shared/table.js?v=20260803-1",
"/shared/theme.js?v=20260803-1",
"/shared/dashboard.js?v=20260820-1",
"/shared/session.js?v=20260803-1",
"/shared/session.js?v=20260829-hel243",
"/shared/admin.js?v=20260803-1",
"/app.js?v=20260803-2",
];
+3 -243
View File
@@ -93,28 +93,6 @@ body[data-active-view="heavenView"] .workspace-view {
display: none;
}
:where(#heavenView) .heaven-tabs {
display: flex;
border-bottom: 1px solid var(--line);
}
:where(#heavenView) .heaven-tab {
border-bottom: 3px solid transparent;
background: transparent;
cursor: pointer;
}
:where(#heavenView) .heaven-tab.active {
border-bottom-color: var(--coral);
}
:where(#heavenView) .heaven-tab:hover {
border-bottom-color: var(--coral);
}
:where(#heavenView) .heaven-panel {
display: none;
}
@@ -1875,60 +1853,6 @@ body[data-active-view="heavenView"] .workspace-view {
color: var(--heaven-ink-faint);
}
#heavenView .heaven-tab {
font-family: var(--heaven-serif);
height: 52px;
position: relative;
padding: 0px 2px;
border: 0px;
color: var(--heaven-ink-soft);
font-size: 14px;
font-weight: 600;
}
#heavenView .heaven-tab::after {
content: "";
position: absolute;
right: 0px;
bottom: 0px;
left: 0px;
height: 2px;
background: var(--heaven-cinnabar);
opacity: 0;
transform: scaleX(0.3);
transition: opacity 220ms ease, transform 260ms var(--ease-out);
}
#heavenView .heaven-tab.active {
color: var(--heaven-ink);
}
#heavenView .heaven-tab:hover {
color: var(--heaven-ink);
}
#heavenView .heaven-tab.active::after {
opacity: 1;
transform: scaleX(1);
}
:where(#heavenView) .heaven-proverb {
margin: 0px;
@@ -1960,7 +1884,7 @@ body[data-active-view="heavenView"] .workspace-view {
}
#heavenView .button:focus-visible,
#heavenView .heaven-tab:focus-visible,
#heavenView .segment:focus-visible,
#heavenView summary:focus-visible {
outline: 2px solid var(--heaven-cinnabar);
@@ -2623,12 +2547,6 @@ body[data-active-view="heavenView"] .workspace-view {
padding: 0px 14px;
}
#heavenView .heaven-tabs {
gap: 22px;
padding: 0px 14px;
}
.heaven-proverb {
padding: 8px 14px;
@@ -3093,7 +3011,7 @@ body[data-active-view="heavenView"] .workspace-view {
#heavenView.heaven-data-loading .heaven-panel,
#heavenView.heaven-data-loading .heaven-proverb,
#heavenView.heaven-data-loading .heaven-tabs {
#heavenView.heaven-data-loading .heaven-page-head {
opacity: 0.42;
pointer-events: none;
@@ -3695,7 +3613,7 @@ body[data-active-view="heavenView"] .workspace-view {
@media (max-width: 900px) {
#heavenView .heaven-panel > ,
#heavenView .heaven-proverb,
#heavenView .heaven-tabs,
#heavenView .heaven-page-head,
#heavenView .heaven-toolbar {
width: min(100% - 28px, 1280px);
}
@@ -3746,22 +3664,6 @@ body[data-active-view="heavenView"] .workspace-view {
display: none;
}
#heavenView .heaven-tabs {
display: grid;
grid-template-columns: repeat(3, minmax(0px, 1fr));
gap: 0px;
padding: 0px;
}
#heavenView .heaven-tab {
width: 100%;
min-width: 0px;
}
#heavenFortunePanel .fortune-heading {
display: flex;
@@ -3867,18 +3769,6 @@ body[data-active-view="heavenView"] .workspace-view {
}
@media (max-width: 520px) {
.heaven-tabs {
gap: 0px;
padding: 0px 8px;
}
.heaven-tab {
min-width: 0px;
flex: 1 1 0%;
}
.heaven-controls {
grid-template-columns: 1fr;
@@ -6232,26 +6122,6 @@ body[data-active-view="heavenView"] .workspace-view {
padding: 12px 20px;
}
#heavenView .heaven-tabs {
align-items: stretch;
gap: 30px;
border-color: var(--heaven-rule);
background: rgba(253, 252, 248, 0.96);
width: min(100% - 40px, 1280px);
margin-right: auto;
margin-left: auto;
min-height: 54px;
padding: 0px 20px;
}
#heavenView .heaven-proverb {
margin-right: auto;
@@ -6297,12 +6167,6 @@ body[data-active-view="heavenView"] .workspace-view {
padding: 10px 14px;
}
#heavenView .heaven-tabs {
min-height: 52px;
padding: 0px 14px;
}
#heavenView .heaven-proverb {
padding: 9px 14px;
}
@@ -6668,18 +6532,6 @@ body[data-active-view="heavenView"] .workspace-view {
color: var(--wt-faint);
}
:root[data-theme="light"] #heavenView .wt-tabs .wt-tab {
color: var(--wt-muted);
}
:root[data-theme="light"] #heavenView .wt-tabs .wt-tab small {
color: var(--wt-faint);
}
:root[data-theme="light"] #heavenView .wt-tabs .wt-tab.on {
color: var(--wt-gold-bright);
}
:root[data-theme="light"] #heavenView .wt-empty {
color: var(--wt-muted);
}
@@ -7128,88 +6980,6 @@ body[data-active-view="heavenView"] .workspace-view {
letter-spacing: 4px;
}
.wt-tabs {
display: flex;
justify-content: center;
gap: 34px;
margin-top: 20px;
}
.wt-tabs .wt-tab {
position: relative;
padding: 8px 4px;
border: 0px;
background: transparent;
color: rgba(216, 210, 189, 0.5);
font-size: 15px;
letter-spacing: 3px;
transition: color 0.2s;
}
.wt-tabs .wt-tab small {
display: block;
margin-top: 3px;
color: rgba(216, 210, 189, 0.3);
font-family: inherit;
font-size: 10px;
letter-spacing: 1px;
}
.wt-tabs .wt-tab::after {
content: "";
position: absolute;
bottom: -2px;
left: 50%;
width: 0px;
height: 1.5px;
background: var(--wt-gold);
transform: translateX(-50%);
transition: 0.25s;
}
.wt-tabs .wt-tab.on {
color: var(--wt-gold-bright);
}
.wt-tabs .wt-tab.on::after {
width: 100%;
}
.wt-tabs .wt-tab:disabled {
cursor: default;
}
.wt-tabs .wt-tab:focus {
outline: none;
}
.wt-tabs .wt-tab:focus-visible {
box-shadow: rgba(201, 165, 92, 0.45) 0px 2px 0px;
}
.heaven-proverb {
margin: 8px 0px 0px;
@@ -9072,16 +8842,6 @@ body[data-active-view="heavenView"] .workspace-view {
gap: 5px;
}
.wt-tabs {
gap: 16px;
}
.wt-tabs .wt-tab {
font-size: 13px;
letter-spacing: 2px;
}
.heaven-proverb {
text-align: center;
}
+13 -6
View File
@@ -1,16 +1,23 @@
<section id="heavenView" class="workspace-view member-feature-view heaven-shell wt">
<div class="member-gate" hidden><div class="member-gate-icon"><i data-lucide="lock-keyhole"></i></div><div><strong>问天仅对会员开放</strong><span>开通会员后可使用观势、观气、观心及平台解读。会员状态可从顶部账号标识进入。</span></div></div>
<div class="section-toolbar redesigned-page-head heaven-page-head">
<div class="section-title-group">
<h2>问天</h2>
<span class="section-subtitle"><b id="heavenDataDate">--</b></span>
</div>
<div class="toolbar-controls">
<div class="segmented" role="group" aria-label="问天模块">
<button class="segment active on" type="button" data-heaven-panel="trend" aria-current="page">观势</button>
<button class="segment" type="button" data-heaven-panel="fortune">观气</button>
<button class="segment" type="button" data-heaven-panel="heart">观心</button>
</div>
</div>
</div>
<header class="wt-head">
<div class="wt-title-line">
<h1 class="wt-serif">问 天</h1>
<span id="heavenDataDate">--</span>
</div>
<div class="verse wt-serif">观天之道 · 执天之行</div>
<nav class="wt-tabs" aria-label="问天模块">
<button class="wt-tab wt-serif on" type="button" data-heaven-panel="trend" aria-current="page">观势<small>三才六爻 · 量化成卦</small></button>
<button class="wt-tab wt-serif" type="button" data-heaven-panel="fortune">观气<small>五运六气 · 日辰生克</small></button>
<button class="wt-tab wt-serif" type="button" data-heaven-panel="heart">观心<small>静心占卜 · 第一念</small></button>
</nav>
</header>
<p class="heaven-proverb wt-serif">遇事不决可问春风,春风不语即随本心</p>
<div id="heavenNotice" class="inline-notice" role="status" hidden></div>
+2 -1
View File
@@ -1363,7 +1363,8 @@ async function interpretHeaven(mode) {
} catch (error) {
stopHeavenReadingAnimation();
state.heavenReadingLoading = false;
state.heavenReadingError = error.message || "问天解读失败";
const detail = error?.payload?.message || error?.payload?.error || error.message;
state.heavenReadingError = detail || "问天解读失败";
renderHeavenReadingDialog();
showHeavenNotice(state.heavenReadingError);
showToast(state.heavenReadingError);
+1 -1
View File
@@ -49,7 +49,7 @@ body[data-active-view="mentorView"] .app-page-context span {
height: 100%;
min-height: 0;
flex-direction: column;
padding: 0;
padding: var(--page-pad-y) 0 0;
color: var(--qp-text-1);
font-family: "PingFang SC", "Microsoft YaHei", system-ui, sans-serif;
}
+8 -1
View File
@@ -7,7 +7,14 @@ async function backfillData() {
start_date: document.querySelector("#backfillStart").value,
end_date: document.querySelector("#backfillEnd").value,
});
showToast(`历史回补完成,共处理 ${payload.results.length} 个工作日`);
const failed = (payload.failed_count || 0);
const skipped = (payload.skipped_non_trading_days || []).length;
const suffix = failed
? `,失败 ${failed}`
: skipped
? `,跳过 ${skipped} 个非交易日`
: "";
showToast(`历史回补完成,共处理 ${payload.results.length} 个交易日${suffix}`);
state.sentimentHistory = null;
state.sentimentHistoryKey = "";
if (state.activeView === "sentimentCycleView") {
+19 -2
View File
@@ -46,12 +46,29 @@
}
}
function readableRequestError(error) {
const message = String(error?.message || "");
if (
error instanceof TypeError
|| /failed to fetch|networkerror|load failed|network request failed/i.test(message)
) {
return "网络请求失败,服务暂时不可用,请稍后重试。";
}
return message || "请求失败";
}
async function request(url, method = "GET", body = null, options = {}) {
const response = await fetch(url, requestOptions(method, body, options.signal));
let response;
try {
response = await fetch(url, requestOptions(method, body, options.signal));
} catch (error) {
throw new ApiError(readableRequestError(error), 0, null);
}
const payload = await parseJson(response);
handleUnauthorized(response, url);
if (!response.ok || payload.error) {
throw new ApiError(payload.error || "请求失败", response.status, payload);
const message = payload.message || payload.error || "请求失败";
throw new ApiError(message, response.status, payload);
}
return payload;
}
+812 -67
View File
File diff suppressed because it is too large Load Diff
+10 -2
View File
@@ -660,7 +660,9 @@ body.sidebar-collapsed .sidebar-collapse-button .lucide {
text-align: left;
min-height: 38px;
height: var(--size-statusbar);
min-height: var(--size-statusbar);
display: flex;
@@ -670,7 +672,7 @@ body.sidebar-collapsed .sidebar-collapse-button .lucide {
margin: auto -8px -8px;
padding: 10px 16px;
padding: 0 16px;
border-right: 0px;
@@ -691,6 +693,12 @@ body.sidebar-collapsed .sidebar-collapse-button .lucide {
font-size: 12px;
}
.sidebar-collapse-button .lucide {
width: 14px;
height: 14px;
}
body.sidebar-collapsed .sidebar-collapse-button span {
display: none;
}
+58 -4
View File
@@ -40,17 +40,71 @@ async function loadDashboard(force = false, background = false, showOverlay = tr
async function startAdminRefresh() {
const buttons = [document.querySelector("#syncButton"), document.querySelector("#adminRefreshButton")].filter(Boolean);
buttons.forEach((button) => { button.disabled = true; });
const requestedDate = elements.tradeDate.value;
setAdminRefreshStatus("running", `正在刷新 ${requestedDate} 的行情,请稍候…`, "loader-circle");
try {
const payload = await apiRequest("/api/admin/refresh", "POST", { trade_date: elements.tradeDate.value });
showToast(payload.message || "后台刷新已提交");
setStatus("后台刷新运行中,当前页面保持不变");
const payload = await apiRequest("/api/admin/refresh", "POST", { trade_date: requestedDate });
if (!payload.started || !payload.job_key) {
setAdminRefreshStatus("warning", "已有刷新任务正在运行,请稍后再试。", "clock-3");
showToast(payload.message || "已有后台刷新任务正在运行");
return;
}
setStatus(`正在刷新 ${requestedDate} 的行情`);
const job = await waitForAdminRefresh(payload.job_key);
if (job.status === "failed") {
const reason = job.message || job.error_code || "数据源未返回结果";
setAdminRefreshStatus("failure", `刷新失败:${reason}`, "circle-x");
setStatus("后台刷新失败");
showToast("后台刷新失败");
return;
}
const query = new URLSearchParams({ trade_date: requestedDate });
const dashboard = await apiRequest(`/api/dashboard?${query}`);
applyDashboard(dashboard);
const meta = dashboard.meta || {};
const actualDate = String(meta.trade_date || "").slice(0, 10);
const requestedCompact = requestedDate.replaceAll("-", "");
const actualCompact = actualDate.replaceAll("-", "");
const updated = formatTimestamp(meta.updated_at);
if (actualCompact !== requestedCompact || meta.carried_forward) {
const reason = meta.notice ? `${meta.notice}` : "";
setAdminRefreshStatus("warning", `刷新已完成,但没有获取到 ${requestedDate} 的最新行情;当前仍是 ${actualDate || "未知日期"}${reason}`, "triangle-alert");
showToast("刷新完成,但未获取到所选日期的最新行情");
} else if (meta.notice) {
setAdminRefreshStatus("warning", `已刷新到 ${actualDate}${updated}),但数据源提示:${meta.notice}`, "triangle-alert");
showToast(`已刷新到 ${actualDate},请留意数据源提示`);
} else {
setAdminRefreshStatus("success", `刷新成功:已获取 ${actualDate} 的最新行情,更新时间 ${updated}`, "circle-check");
showToast(`刷新成功:已获取 ${actualDate} 的最新行情`);
}
} catch (error) {
showToast(error.message || "后台刷新启动失败");
const message = error.message || "后台刷新失败";
setAdminRefreshStatus("failure", `刷新失败:${message}`, "circle-x");
setStatus("后台刷新失败");
showToast(message);
} finally {
buttons.forEach((button) => { button.disabled = false; });
}
}
function setAdminRefreshStatus(tone, message, icon = "circle-dot") {
const status = document.querySelector("#adminRefreshStatus");
if (!status) return;
status.dataset.tone = tone;
status.innerHTML = `<i data-lucide="${icon}"></i><span>${escapeHtml(message)}</span>`;
refreshIcons();
}
async function waitForAdminRefresh(jobKey) {
for (let attempt = 0; attempt < 120; attempt += 1) {
const payload = await apiRequest("/api/admin/settings");
const job = (payload.data?.jobs || []).find((item) => item.idempotency_key === jobKey);
if (job && ["success", "failed"].includes(job.status)) return job;
await new Promise((resolve) => setTimeout(resolve, 1000));
}
throw new Error("刷新等待超时,请稍后重试");
}
function applyDashboard(payload, background = false) {
state.dashboard = payload;
const selectedDate = payload.meta.requested_date || payload.meta.trade_date;
+7 -1
View File
@@ -196,7 +196,13 @@ async function changeAccountPassword(event) {
async function switchAccount() {
toggleAccountDropdown(false);
window.location.assign("/login/");
const params = new URLSearchParams();
const next = `${window.location.pathname}${window.location.search}${window.location.hash}`;
if (next.startsWith("/") && !next.startsWith("//") && next !== "/login" && !next.startsWith("/login/") && !next.startsWith("/login?")) {
params.set("next", next);
}
const query = params.toString();
window.location.assign("/login/" + (query ? `?${query}` : ""));
}
+10 -4
View File
@@ -1294,7 +1294,9 @@ body.sidebar-collapsed {
}
.sidebar-brand {
min-height: 55px;
height: var(--size-topbar);
min-height: var(--size-topbar);
display: flex;
@@ -1304,7 +1306,7 @@ body.sidebar-collapsed {
margin: 0px -8px 7px;
padding: 0px 16px;
padding: 0 16px;
border-bottom: 1px solid var(--r2-line-soft);
@@ -2146,13 +2148,17 @@ body.sidebar-collapsed .status-bar {
}
.module-nav .sidebar-brand {
height: var(--size-topbar);
min-height: var(--size-topbar);
display: flex;
align-items: center;
gap: 8px;
padding: 14px 16px;
padding: 0 16px;
border-bottom: 1px solid var(--line-soft);
}
@@ -3494,7 +3500,7 @@ body.sidebar-collapsed .status-bar {
flex: 0 0 auto;
align-items: center;
gap: var(--header-action-gap);
margin-left: 0;
margin-left: auto;
overflow: visible;
}
+29
View File
@@ -147,6 +147,31 @@
--duration-fast: 150ms;
--duration-normal: 220ms;
--login-brand-gradient: linear-gradient(165deg, #0c1e4a, #16307c, #2153cc);
--login-brand-share: 34%;
--login-brand-min: 420px;
--login-brand-cap: 560px;
--login-brand-wide-share: 29.2%;
--login-brand-pad: 36px 40px 24px;
--login-brand-mark-size: 44px;
--login-brand-name-size: 16px;
--login-hero-size: 28px;
--login-card-width: 408px;
--login-card-pad: 32px;
--login-card-title-size: 22px;
--login-account-row-min: 72px;
--login-account-avatar: 40px;
--login-submit-height: 40px;
--login-stat-chip-bg: rgba(8, 12, 24, 0.48);
--login-candle-up: #e07078;
--login-candle-down: #3db88a;
--login-trend-line: rgba(244, 247, 255, 0.88);
--login-mascot-red: #e8605a;
--login-mascot-red-shade: #c74a45;
--login-mascot-green: #46be93;
--login-mascot-green-shade: #37997a;
--login-mascot-eye: #f7f9fc;
--login-mascot-pupil: #1a2440;
--login-mascot-height: clamp(150px, 15vw, 260px);
--font-size-aux: 11.5px;
--font-size-caption: 12.5px;
@@ -516,6 +541,10 @@
--warning-line-strong: #66502d;
--control-shadow: 0 1px 3px rgba(0, 0, 0, .3);
--login-brand-gradient: linear-gradient(165deg, #080c18, #0e1730, #14224a);
--login-stat-chip-bg: rgba(6, 8, 16, 0.58);
--login-candle-up: #f06d73;
--login-candle-down: #43bc8a;
--login-trend-line: rgba(232, 236, 244, 0.9);
--dialog-backdrop: var(--backdrop);
--ladder-level-1: #2d2426;
--ladder-level-2: #2b2822;
+96
View File
@@ -3868,3 +3868,99 @@ test("desktop header keeps commands in view and tape text unclipped across works
await page.screenshot({ path: path.join(shotDir, "admin-390-night.png") });
fs.writeFileSync(path.join(shotDir, "measurements.json"), `${JSON.stringify(measurements, null, 2)}\n`);
});
test("HEL-183 heaven tools right-align and shell heights unify", async ({ page }) => {
const fs = require("node:fs");
const path = require("node:path");
const shotDir = path.join(__dirname, "../../runtime/hel183-shots");
fs.mkdirSync(shotDir, { recursive: true });
await page.setViewportSize({ width: 1440, height: 900 });
await mockApplication(page, session("admin", true));
await page.goto("/index.html");
const measure = () => page.evaluate(() => {
const box = (node) => {
if (!node) return null;
const r = node.getBoundingClientRect();
return { x: r.x, y: r.y, right: r.right, width: r.width, height: r.height };
};
const brand = document.querySelector(".module-nav .sidebar-brand") || document.querySelector(".sidebar-brand");
const header = document.querySelector(".app-header");
const actions = document.querySelector(".header-actions");
const collapse = document.querySelector(".sidebar-collapse-button");
const status = document.querySelector(".status-bar");
const overview = document.querySelector(".overview-strip");
const brandBox = box(brand);
const headerBox = box(header);
const actionsBox = box(actions);
const collapseBox = box(collapse);
const statusBox = box(status);
return {
brandHeight: brandBox ? Math.round(brandBox.height) : null,
headerHeight: headerBox ? Math.round(headerBox.height) : null,
brandBottom: brandBox ? Math.round(brandBox.y + brandBox.height) : null,
headerBottom: headerBox ? Math.round(headerBox.y + headerBox.height) : null,
collapseHeight: collapseBox ? Math.round(collapseBox.height) : null,
statusHeight: statusBox ? Math.round(statusBox.height) : null,
actionsNearRight: actionsBox && headerBox ? (headerBox.right - actionsBox.right) < 24 : false,
actionsMarginLeft: actions ? getComputedStyle(actions).marginLeft : null,
overviewDisplay: overview ? getComputedStyle(overview).display : null,
mentorPadTop: (() => {
const mentor = document.querySelector("#mentorView");
return mentor ? getComputedStyle(mentor).paddingTop : null;
})(),
};
});
await page.locator('[data-view="sentimentCycleView"]').first().click();
await expect(page.locator("#sentimentCycleView")).toHaveClass(/active-view/);
let geo = await measure();
expect(geo.brandHeight, "logo height").toBe(64);
expect(geo.headerHeight, "header height").toBe(64);
expect(geo.brandBottom, "logo/header bottom align").toBe(geo.headerBottom);
expect(geo.collapseHeight, "collapse height").toBe(28);
expect(geo.statusHeight, "status height").toBe(28);
expect(geo.actionsNearRight, "sentiment tools right").toBe(true);
await page.locator(".app-header").screenshot({ path: path.join(shotDir, "sentiment-header-day.png") });
await page.screenshot({ path: path.join(shotDir, "sentiment-page-day.png") });
await page.locator('[data-view="heavenView"]').first().click();
await expect(page.locator("#heavenView")).toHaveClass(/active-view/);
await expect(page.locator("#heavenView .heaven-page-head")).toBeVisible();
await expect(page.locator("#heavenView .heaven-page-head .segment")).toHaveCount(3);
geo = await measure();
expect(geo.overviewDisplay, "heaven hides overview").toBe("none");
expect(geo.actionsMarginLeft, "tools margin-left resolved").not.toBe("0px");
expect(Number.parseFloat(geo.actionsMarginLeft), "tools left auto gap").toBeGreaterThan(40);
expect(geo.actionsNearRight, "heaven tools right").toBe(true);
expect(geo.brandHeight).toBe(64);
expect(geo.collapseHeight).toBe(28);
await page.locator(".app-header").screenshot({ path: path.join(shotDir, "heaven-header-day.png") });
await page.screenshot({ path: path.join(shotDir, "heaven-page-day.png") });
await page.locator('[data-heaven-panel="fortune"]').click();
await expect(page.locator('[data-heaven-panel="fortune"]')).toHaveClass(/active/);
await expect(page.locator("#heavenFortunePanel")).toHaveClass(/active-heaven-panel/);
await page.locator('[data-view="mentorView"]').first().click();
await expect(page.locator("#mentorView")).toHaveClass(/active-view/);
geo = await measure();
expect(geo.mentorPadTop, "mentor top padding").toBe("14px");
await page.screenshot({ path: path.join(shotDir, "mentor-page-day.png") });
await page.locator("#themeToggle").click();
await page.locator('[data-view="heavenView"]').first().click();
await expect(page.locator("#heavenView")).toHaveClass(/active-view/);
geo = await measure();
expect(geo.actionsNearRight, "heaven night tools right").toBe(true);
expect(geo.brandHeight).toBe(64);
await page.locator(".app-header").screenshot({ path: path.join(shotDir, "heaven-header-night.png") });
await page.screenshot({ path: path.join(shotDir, "heaven-page-night.png") });
await page.locator('[data-view="sentimentCycleView"]').first().click();
await page.locator(".app-header").screenshot({ path: path.join(shotDir, "sentiment-header-night.png") });
await page.screenshot({ path: path.join(shotDir, "sentiment-page-night.png") });
await page.locator('[data-view="mentorView"]').first().click();
await page.screenshot({ path: path.join(shotDir, "mentor-page-night.png") });
});
+407 -2
View File
@@ -1,4 +1,9 @@
const { test, expect } = require("@playwright/test");
const fs = require("fs");
const path = require("path");
const SHOT_DIR = path.resolve(__dirname, "../../../verify-shots");
fs.mkdirSync(SHOT_DIR, { recursive: true });
function loginPayload(user) {
return {
@@ -54,6 +59,15 @@ async function mockLoginPortal(page, options = {}) {
return;
}
if ((url.pathname === "/api/auth/login" || url.pathname === "/api/auth/register") && method === "POST") {
if (options.loginDelay) await new Promise((resolve) => setTimeout(resolve, options.loginDelay));
if (options.loginFails) {
await route.fulfill({
status: 401,
contentType: "application/json",
body: JSON.stringify({ error: "账号名或密码不正确,请重新输入。" }),
});
return;
}
await route.fulfill({
status: 200,
contentType: "application/json",
@@ -67,14 +81,58 @@ async function mockLoginPortal(page, options = {}) {
return;
}
if (url.pathname === "/api/auth/me") {
const current = accounts.find((item) => Number(item.user_id) === Number(currentUserId)) || null;
const authenticated = Boolean(current) && !options.sessionExpired;
await route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({
ok: true,
authenticated: Boolean(currentUserId),
authenticated,
csrf_token: "portal-csrf",
user: accounts.find((item) => Number(item.user_id) === Number(currentUserId)) || null,
user: authenticated ? {
id: current.user_id,
username: current.username,
role: current.role,
membership: current.membership,
} : null,
}),
});
return;
}
if (url.pathname === "/api/dashboard") {
await route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({
ok: true,
meta: {
trade_date: "2026-07-22",
requested_date: "2026-07-22",
source: "tushare",
realtime: false,
cached: true,
market_status: "closed",
updated_at: "2026-07-22T15:00:00+08:00",
},
overview: {
up_count: 2100,
down_count: 2800,
limit_up_count: 42,
limit_down_count: 8,
broken_count: 17,
seal_rate: 71.2,
amount_billion: 12600,
sentiment_score: 48,
},
limits: [],
broken: [],
down_limits: [],
yesterday_limits: [],
limit_performance: [],
ladders: [],
sectors: [],
sector_rotation: [],
}),
});
return;
@@ -149,3 +207,350 @@ test("managing accounts removes a local record after inline confirmation", async
await expect(page.locator(".login-account-row")).toHaveCount(1);
await expect(page.locator(".login-account-row")).toContainText("alpha_user");
});
async function assertConfirmedSkeleton(page, { width, height }) {
await expect(page.locator(".login-brand-title")).toHaveText("看懂情绪周期,把复盘变成下一次的先手。");
await expect(page.locator(".login-brand-header")).toBeVisible();
await expect(page.locator(".login-brand-chart")).toBeVisible();
await expect(page.locator(".login-brand-stats")).toBeVisible();
await expect(page.locator(".login-brand-kicker")).toHaveText("收盘之后 · 复盘开始");
const brand = await page.locator(".login-brand").boundingBox();
const header = await page.locator(".login-brand-header").boundingBox();
const mark = await page.locator(".login-brand-mark").boundingBox();
const name = await page.locator(".login-brand-name").boundingBox();
const title = await page.locator(".login-brand-title").boundingBox();
const stats = await page.locator(".login-brand-stats").boundingBox();
const chart = await page.locator(".login-brand-chart").boundingBox();
const card = await page.locator(".login-card").boundingBox();
expect(brand).toBeTruthy();
expect(header.y - brand.y).toBeLessThan(48);
expect(Math.abs(mark.y - name.y)).toBeLessThan(16);
expect(title.y).toBeGreaterThan(height * 0.28);
expect(title.y).toBeLessThan(height * 0.72);
expect(stats.y).toBeGreaterThan(height * 0.55);
expect(chart.height).toBeGreaterThan(80);
await expect(page.locator("#loginMascots")).toBeVisible();
expect(card.width).toBeGreaterThan(380);
expect(card.width).toBeLessThan(450);
if (width === 1440) {
expect(brand.width).toBeGreaterThan(470);
expect(brand.width).toBeLessThan(520);
expect(brand.height).toBe(height);
const mascots = await page.locator("#loginMascots").boundingBox();
const kicker = await page.locator(".login-brand-kicker").boundingBox();
expect(mascots).toBeTruthy();
expect(kicker).toBeTruthy();
expect(mascots.y).toBeGreaterThan(header.y + header.height - 4);
expect(mascots.y + mascots.height).toBeLessThan(kicker.y + 8);
const gapTop = header.y + header.height;
const gapBottom = kicker.y;
const mid = (gapTop + gapBottom) / 2;
const mascotMid = mascots.y + mascots.height / 2;
expect(Math.abs(mascotMid - mid)).toBeLessThan(48);
expect(title.y).toBeGreaterThan(470);
expect(title.y).toBeLessThan(580);
} else if (width === 1920) {
expect(brand.width).toBeGreaterThan(540);
expect(brand.width).toBeLessThan(580);
} else {
expect(brand.width).toBeGreaterThan(560);
}
}
async function openPortal(page, { theme, width, height, accounts, currentUserId, loginFails, loginDelay }) {
await page.addInitScript((nextTheme) => {
localStorage.setItem("xiaobaiTheme", nextTheme);
}, theme);
await page.setViewportSize({ width, height });
await mockLoginPortal(page, { accounts, currentUserId, loginFails, loginDelay });
await page.goto("/login/");
}
for (const theme of ["light", "dark"]) {
for (const [width, height] of [[1440, 900], [1920, 1080]]) {
test(`confirmed skeleton ${theme} ${width}x${height}`, async ({ page }) => {
await openPortal(page, { theme, width, height, accounts: [] });
await assertConfirmedSkeleton(page, { width, height });
await expect(page.locator("#loginThemeToggle")).toHaveText(theme === "dark" ? "☀ 日间" : "🌙 夜间");
await page.screenshot({ path: path.join(SHOT_DIR, `first-${theme}-${width}.png`), fullPage: true });
});
}
}
test("ultrawide keeps the left brand from collapsing into a strip", async ({ page }) => {
await openPortal(page, { theme: "dark", width: 2560, height: 1080, accounts: [] });
await assertConfirmedSkeleton(page, { width: 2560, height: 1080 });
});
test("picker add remove error and loading share the same desktop skeleton", async ({ page }) => {
const accounts = SAVED_ACCOUNTS.map((item) => ({ ...item }));
await openPortal(page, {
theme: "dark",
width: 1440,
height: 900,
accounts,
currentUserId: 1,
});
await assertConfirmedSkeleton(page, { width: 1440, height: 900 });
await expect(page.locator(".login-card-title")).toHaveText("选择账号");
await expect(page.locator(".login-avatar")).toHaveCount(2);
await expect(page.locator(".login-add")).toBeVisible();
await page.screenshot({ path: path.join(SHOT_DIR, "picker-dark-1440.png"), fullPage: true });
await page.locator('[data-login-action="add"]').click();
await expect(page.locator(".login-card-title")).toHaveText("添加账号");
await assertConfirmedSkeleton(page, { width: 1440, height: 900 });
await page.screenshot({ path: path.join(SHOT_DIR, "add-dark-1440.png"), fullPage: true });
await page.locator('[data-login-action="picker"]').click();
await page.locator('[data-login-action="manage"]').click();
await expect(page.locator(".login-card-title")).toHaveText("管理账号记录");
await page.locator('[data-confirm-id="2"]').click();
await expect(page.locator(".login-confirm-copy")).toContainText("beta_user");
await page.screenshot({ path: path.join(SHOT_DIR, "remove-dark-1440.png"), fullPage: true });
});
test("login failure and loading keep the confirmed first-login skeleton", async ({ page }) => {
await openPortal(page, {
theme: "light",
width: 1440,
height: 900,
accounts: [],
loginFails: true,
});
await page.locator("#loginUsername").fill("baiqizhi");
await page.locator("#loginPassword").fill("wrong-password");
await page.locator(".login-submit").click();
await expect(page.locator(".login-error")).toContainText("账号名或密码不正确");
await expect(page.locator("#loginPassword")).toHaveClass(/is-invalid/);
await assertConfirmedSkeleton(page, { width: 1440, height: 900 });
await page.screenshot({ path: path.join(SHOT_DIR, "error-light-1440.png"), fullPage: true });
});
test("loading button appears on the confirmed first-login skeleton", async ({ page }) => {
await openPortal(page, {
theme: "dark",
width: 1440,
height: 900,
accounts: [],
loginDelay: 2500,
});
await page.evaluate(() => {
window.location.replace = () => {};
});
await page.locator("#loginUsername").fill("baiqizhi");
await page.locator("#loginPassword").fill("password12");
const submit = page.locator(".login-submit").click();
await expect(page.locator(".login-submit")).toContainText("正在登录...");
await expect(page.locator(".login-spinner")).toBeVisible();
await assertConfirmedSkeleton(page, { width: 1440, height: 900 });
await page.screenshot({ path: path.join(SHOT_DIR, "loading-dark-1440.png"), fullPage: true });
await submit;
});
async function openPicker(page, options = {}) {
const accounts = options.accounts || SAVED_ACCOUNTS.map((item) => ({ ...item }));
const currentUserId = options.currentUserId ?? 1;
const next = options.next || "/index.html?view=sentimentCycleView";
await page.unroute("**/api/**").catch(() => {});
await mockLoginPortal(page, {
accounts,
currentUserId,
sessionExpired: options.sessionExpired,
switchFails: options.switchFails,
});
await page.goto(`/login/?next=${encodeURIComponent(next)}`);
await expect(page.locator(".login-card-title")).toHaveText("选择账号");
}
test("clicking the current account from two workspace pages returns without switching", async ({ page }) => {
const views = ["sentimentCycleView", "ladderView"];
for (const viewId of views) {
const next = `/index.html?view=${viewId}`;
const switchCalls = [];
const onRequest = (request) => {
if (request.url().includes("/api/auth/switch") && request.method() === "POST") {
switchCalls.push(request);
}
};
page.on("request", onRequest);
await openPicker(page, { next });
await expect(page.locator('[data-resume-id="1"]')).toContainText("继续使用");
await expect(page.locator('[data-resume-id="1"]')).toContainText("当前");
await page.locator('[data-resume-id="1"]').click();
await expect(page).toHaveURL(new RegExp(`[?&]view=${viewId}\\b`));
expect(switchCalls).toEqual([]);
page.off("request", onRequest);
}
});
test("a lone current account can return from the picker instead of dead-ending", async ({ page }) => {
await openPicker(page, {
accounts: [SAVED_ACCOUNTS[0]],
currentUserId: 1,
next: "/index.html?view=reviewWorkspaceView",
});
await expect(page.locator(".login-account-row")).toHaveCount(1);
await expect(page.locator('[data-switch-id]')).toHaveCount(0);
await page.locator('[data-resume-id="1"]').click();
await expect(page).toHaveURL(/view=reviewWorkspaceView/);
});
test("the return control also restores the originating workspace page", async ({ page }) => {
await openPicker(page, { next: "/index.html?view=ladderView" });
await page.locator('[data-login-action="resume"]').click();
await expect(page).toHaveURL(/view=ladderView/);
});
test("refreshing the picker still returns to the originating page", async ({ page }) => {
await openPicker(page, { next: "/index.html?view=sentimentCycleView" });
await page.reload();
await expect(page.locator(".login-card-title")).toHaveText("选择账号");
await page.locator('[data-resume-id="1"]').click();
await expect(page).toHaveURL(/view=sentimentCycleView/);
});
test("an expired current session asks for login instead of pretending to return", async ({ page }) => {
await openPicker(page, {
next: "/index.html?view=sentimentCycleView",
sessionExpired: true,
});
await page.locator('[data-resume-id="1"]').click();
await expect(page.locator(".login-error")).toHaveText("当前会话已失效,请重新登录");
await expect(page).toHaveURL(/\/login\/?/);
});
test("other saved accounts still switch while the current row only resumes", async ({ page }) => {
await openPicker(page, { next: "/index.html?view=auctionView" });
const switched = page.waitForRequest((request) => (
request.url().includes("/api/auth/switch") && request.method() === "POST"
));
await page.locator('[data-switch-id="2"]').click();
const request = await switched;
expect(JSON.parse(request.postData() || "{}")).toEqual({ user_id: 2 });
});
test("workspace switch-account menu carries the current page back to the picker", async ({ page }) => {
await mockLoginPortal(page, {
accounts: SAVED_ACCOUNTS.map((item) => ({ ...item })),
currentUserId: 1,
});
await page.goto("/index.html?view=sentimentCycleView");
await expect(page.locator("#accountButton")).toBeVisible();
await page.locator("#accountButton").click();
await page.locator("#switchAccountMenuButton").click();
await expect(page).toHaveURL(/\/login\/\?next=/);
await expect(page.locator(".login-card-title")).toHaveText("选择账号");
await page.locator('[data-resume-id="1"]').click();
await expect(page).toHaveURL(/view=sentimentCycleView/);
});
test("workspace switch-account from a second page also returns to that page", async ({ page }) => {
await mockLoginPortal(page, {
accounts: SAVED_ACCOUNTS.map((item) => ({ ...item })),
currentUserId: 1,
});
await page.goto("/index.html?view=ladderView");
await expect(page.locator("#accountButton")).toBeVisible();
await page.locator("#accountButton").click();
await page.locator("#switchAccountMenuButton").click();
await expect(page.locator(".login-card-title")).toHaveText("选择账号");
await page.locator('[data-resume-id="1"]').click();
await expect(page).toHaveURL(/view=ladderView/);
});
test("desktop mascots react to account focus, password, toggle, loading, success and failure", async ({ page }) => {
await openPortal(page, { theme: "light", width: 1440, height: 900, accounts: [] });
const mascots = page.locator("#loginMascots");
await expect(mascots).toHaveAttribute("data-mood", "idle");
await page.locator("#loginUsername").focus();
await expect(mascots).toHaveAttribute("data-mood", "account");
await page.locator("#loginPassword").focus();
await expect(mascots).toHaveAttribute("data-mood", "password");
await expect.poll(async () => (
page.locator(".login-mascot.is-red .login-mascot-hand.is-left").evaluate((node) => getComputedStyle(node).opacity)
)).toBe("1");
await page.locator(".login-password-toggle").click();
await expect(page.locator("#loginPassword")).toHaveAttribute("type", "text");
await expect(mascots).toHaveAttribute("data-mood", "password");
await page.locator(".login-password-toggle").click();
await expect(page.locator("#loginPassword")).toHaveAttribute("type", "password");
await expect(mascots).toHaveAttribute("data-mood", "password");
await page.screenshot({ path: path.join(SHOT_DIR, "mascots-password-light-1440.png"), fullPage: true });
});
test("login loading and failure drive short mascot feedback", async ({ page }) => {
await openPortal(page, {
theme: "dark",
width: 1440,
height: 900,
accounts: [],
loginFails: true,
loginDelay: 800,
});
await page.locator("#loginUsername").fill("baiqizhi");
await page.locator("#loginPassword").fill("wrong-password");
const submit = page.locator(".login-submit").click();
await expect(page.locator("#loginMascots")).toHaveAttribute("data-mood", "busy");
await expect(page.locator(".login-error")).toContainText("账号名或密码不正确");
await expect(page.locator("#loginMascots")).toHaveAttribute("data-mood", "fail");
await page.screenshot({ path: path.join(SHOT_DIR, "mascots-fail-dark-1440.png"), fullPage: true });
await submit;
});
test("login success plays a hop before leaving the portal", async ({ page }) => {
await openPortal(page, { theme: "light", width: 1440, height: 900, accounts: [] });
await page.evaluate(() => {
window.location.replace = () => {};
});
await page.locator("#loginUsername").fill("baiqizhi");
await page.locator("#loginPassword").fill("password12");
const submit = page.locator(".login-submit").click();
await expect(page.locator("#loginMascots")).toHaveAttribute("data-mood", /busy|success/);
await expect(page.locator("#loginMascots")).toHaveAttribute("data-mood", "success", { timeout: 4000 });
await page.screenshot({ path: path.join(SHOT_DIR, "mascots-success-light-1440.png"), fullPage: true });
await submit;
});
test("reduced motion keeps static mascots without mouse tracking", async ({ page }) => {
await page.emulateMedia({ reducedMotion: "reduce" });
await openPortal(page, { theme: "dark", width: 1440, height: 900, accounts: [] });
const mascots = page.locator("#loginMascots");
await expect(mascots).toBeVisible();
await expect(mascots).toHaveAttribute("data-mood", "idle");
const before = await mascots.evaluate((node) => getComputedStyle(node.querySelector(".login-mascot.is-red")).getPropertyValue("--pupil-x"));
await page.mouse.move(1200, 120);
await page.waitForTimeout(120);
const after = await mascots.evaluate((node) => getComputedStyle(node.querySelector(".login-mascot.is-red")).getPropertyValue("--pupil-x"));
expect(after).toBe(before);
await page.locator("#loginPassword").focus();
await expect(mascots).toHaveAttribute("data-mood", "password");
await page.screenshot({ path: path.join(SHOT_DIR, "mascots-reduced-dark-1440.png"), fullPage: true });
});
test("mouse follow updates mascot pupils on a fine pointer", async ({ page }) => {
await openPortal(page, { theme: "light", width: 1440, height: 900, accounts: [] });
const mascots = page.locator("#loginMascots");
await page.mouse.move(80, 160);
await page.waitForTimeout(180);
const left = await mascots.evaluate((node) => getComputedStyle(node.querySelector(".login-mascot.is-red")).getPropertyValue("--pupil-x"));
await page.mouse.move(1280, 200);
await page.waitForTimeout(180);
const right = await mascots.evaluate((node) => getComputedStyle(node.querySelector(".login-mascot.is-red")).getPropertyValue("--pupil-x"));
expect(Number.parseFloat(right)).toBeGreaterThan(Number.parseFloat(left));
});
test("narrow screens hide mascots without moving the login card", async ({ page }) => {
await openPortal(page, { theme: "light", width: 800, height: 900, accounts: [] });
await expect(page.locator("#loginMascots")).toBeHidden();
await expect(page.locator(".login-card-title")).toHaveText("欢迎回来");
await expect(page.locator(".login-brand-title")).toBeHidden();
});
test("theme toggle keeps mascots in the brand gap", async ({ page }) => {
await openPortal(page, { theme: "light", width: 1440, height: 900, accounts: [] });
await page.locator("#loginThemeToggle").click();
await expect(page.locator("html")).toHaveAttribute("data-theme", "dark");
await assertConfirmedSkeleton(page, { width: 1440, height: 900 });
await page.screenshot({ path: path.join(SHOT_DIR, "mascots-idle-dark-1440.png"), fullPage: true });
});
+73 -6
View File
@@ -138,6 +138,11 @@ async function mockMobileApi(page, options = {}) {
payload = { items: [] };
} else if (path === "/api/search") {
payload = { groups: { stocks: [{ id: "002141", code: "002141", name: "贤丰控股", type: "stock", industry: "电子元件" }], sectors: [], themes: [], indices: [] } };
} else if (path === "/api/auth/accounts") {
payload = {
accounts: [{ user_id: auth.user.id, username: auth.user.username, role: auth.user.role, last_used_at: "2026-07-22T09:12:00+08:00" }],
current_user_id: auth.user.id,
};
} else if (path === "/api/account/status") {
payload = {
birth_profile_configured: true,
@@ -284,12 +289,16 @@ test("mobile login renders before authentication", async ({ page }) => {
test("four hub pages render their icon grids", async ({ page }) => {
await mockMobileApi(page);
await openMobile(page);
for (const hub of ["market", "tools", "review", "system"]) {
for (const hub of ["market", "tools", "review"]) {
await page.evaluate((h) => { window.MobileRouter.navigate("#/hub/" + h); }, hub);
await expect(page.locator(".m-hub-grid")).toBeVisible();
await expect(page.locator(".m-hub-grid .m-grid-item").first()).toBeVisible();
expect(await measureOverflow(page)).toBeLessThanOrEqual(1);
}
await page.evaluate(() => { window.MobileRouter.navigate("#/hub/system"); });
await expect(page.locator("[data-system-page='home']")).toBeVisible();
await expect(page.locator(".m-sys-row").first()).toBeVisible();
expect(await measureOverflow(page)).toBeLessThanOrEqual(1);
});
for (const theme of ["day", "night"]) {
@@ -344,7 +353,7 @@ test("system management pages render real content instead of placeholders", asyn
}
await navigateToFeature(page, "system/profile");
await expect(page.locator("#m-sys-birth-date")).toBeVisible();
await expect(page.locator("[data-system-switch]")).toBeVisible();
await expect(page.locator("[data-system-save-birth]")).toBeVisible();
await navigateToFeature(page, "system/password");
await expect(page.locator("#m-sys-password-current")).toBeVisible();
await navigateToFeature(page, "system/membership");
@@ -355,7 +364,64 @@ test("system management pages render real content instead of placeholders", asyn
await expect(page.locator("#m-sys-member-limit")).toBeVisible();
});
test("empty profile save click shows a toast instead of a dead button", async ({ page }) => {
test("system home groups entries and keeps admin-only items gated", async ({ page }) => {
await mockMobileApi(page);
await openMobile(page);
await page.evaluate(() => { window.MobileRouter.navigate("#/hub/system"); });
await expect(page.locator("[data-system-page='home']")).toBeVisible();
await expect(page.locator("#m-view")).toContainText("账号");
await expect(page.locator("#m-view")).toContainText("偏好");
await expect(page.locator("#m-view")).toContainText("管理员专区");
await expect(page.locator("[data-theme-toggle]")).toBeVisible();
await expect(page.locator("[data-system-switch]")).toBeVisible();
expect(await measureOverflow(page)).toBeLessThanOrEqual(1);
});
test("system settings tabs, model editor, delete confirm and theme toggle work", async ({ page }) => {
await mockMobileApi(page);
await openMobile(page);
await navigateToFeature(page, "system/admin");
await expect(page.locator("[data-system-admin-panel='market']")).toBeVisible();
await page.locator("[data-system-admin-tab='models']").click();
await expect(page.locator("[data-system-admin-panel='models']")).toBeVisible();
await page.locator("[data-system-edit-model]").first().click();
await expect(page.locator(".m-sheet-root.is-open")).toBeVisible();
await expect(page.locator(".m-sheet-head h2")).toHaveText("编辑模型");
await page.locator("[data-sheet-close]").click();
await page.locator("[data-system-admin-tab='market']").click();
await expect(page.locator("#m-sys-token")).toBeVisible();
await navigateToFeature(page, "system/profile");
await page.locator("[data-system-delete-birth]").click();
await expect(page.locator(".m-dialog")).toBeVisible();
await expect(page.locator(".m-dialog")).toContainText("删除命理资料");
await page.locator("[data-sheet-close]").click();
await page.evaluate(() => { window.MobileRouter.navigate("#/hub/system"); });
await expect(page.locator("[data-theme-toggle]")).toBeVisible();
const before = await page.locator("#m-app").getAttribute("data-theme");
await page.locator("[data-theme-toggle]").click();
await expect.poll(async () => page.locator("#m-app").getAttribute("data-theme")).not.toBe(before);
await navigateToFeature(page, "system/members");
await page.locator("[data-system-open-member]").click();
await expect(page.locator(".m-sheet-root.is-open")).toBeVisible();
await expect(page.locator(".m-sheet-head h2")).toContainText("管理会员");
});
test("password mismatch shows inline error instead of a silent submit", async ({ page }) => {
await mockMobileApi(page);
await openMobile(page);
await navigateToFeature(page, "system/password");
await page.locator("#m-sys-password-current").fill("OldPass12");
await page.locator("#m-sys-password-new").fill("NewPass123");
await page.locator("#m-sys-password-confirm").fill("OtherPass123");
await page.locator("[data-system-save-password]").click();
await expect(page.locator("[data-field-error='confirm']")).toBeVisible();
await expect(page.locator("[data-field-error='confirm']")).toContainText("两次输入的密码不一致");
});
test("empty birth profile save shows a validation toast", async ({ page }) => {
await mockMobileApi(page);
await openMobile(page);
await navigateToFeature(page, "system/profile");
@@ -371,9 +437,10 @@ test("non-admin cannot open system admin pages as placeholders", async ({ page }
await mockMobileApi(page, { auth: authSession("user", true) });
await openMobile(page);
await page.evaluate(() => { window.MobileRouter.navigate("#/hub/system"); });
await expect(page.locator(".m-hub-grid")).toBeVisible();
await expect(page.locator('.m-grid-item[data-route="#/feature/system/admin"]')).toHaveCount(0);
await expect(page.locator('.m-grid-item[data-route="#/feature/system/members"]')).toHaveCount(0);
await expect(page.locator("[data-system-page='home']")).toBeVisible();
await expect(page.locator('[data-route="#/feature/system/admin"]')).toHaveCount(0);
await expect(page.locator('[data-route="#/feature/system/members"]')).toHaveCount(0);
await expect(page.locator("[data-system-switch]")).toBeVisible();
await navigateToFeature(page, "system/admin");
await expect(page.locator("#m-view")).not.toContainText(PLACEHOLDER_COPY);
await expect(page.locator("[data-system-page='forbidden']")).toBeVisible();
+24
View File
@@ -0,0 +1,24 @@
from __future__ import annotations
import unittest
from backend.jobs.service import _verified_dashboard_result
class AdminRefreshStatusTests(unittest.TestCase):
def test_carried_snapshot_is_reported_as_failed_job(self):
result = _verified_dashboard_result(
{"meta": {"carried_forward": True, "notice": "官方涨跌停数据尚未返回"}}
)
self.assertEqual(result["status"], "failed")
self.assertEqual(result["error"], "官方涨跌停数据尚未返回")
def test_current_snapshot_is_reported_as_successful_job(self):
dashboard = {"meta": {"trade_date": "2026-08-28", "carried_forward": False}}
self.assertIs(_verified_dashboard_result(dashboard), dashboard)
if __name__ == "__main__":
unittest.main()
+124
View File
@@ -0,0 +1,124 @@
from __future__ import annotations
import os
import stat
import subprocess
import tempfile
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
CHECK = ROOT / "tools" / "check_deploy_baseline.sh"
BUILD = ROOT / "tools" / "build_image.sh"
def run_check(repo: Path, candidate: str, live: str) -> subprocess.CompletedProcess[str]:
env = os.environ.copy()
env["GIT_DIR"] = str(repo / ".git")
env["GIT_WORK_TREE"] = str(repo)
return subprocess.run(
["bash", str(CHECK), candidate, "--live-revision", live],
cwd=repo,
capture_output=True,
text=True,
env=env,
check=False,
)
def git(repo: Path, *args: str) -> str:
result = subprocess.run(
["git", *args],
cwd=repo,
capture_output=True,
text=True,
check=True,
)
return result.stdout.strip()
class DeployBaselineGateTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.tmpdir = tempfile.TemporaryDirectory()
cls.repo = Path(cls.tmpdir.name) / "repo"
cls.repo.mkdir()
git(cls.repo, "init")
git(cls.repo, "config", "user.email", "gate@example.com")
git(cls.repo, "config", "user.name", "Gate")
(cls.repo / "README").write_text("base\n", encoding="utf-8")
git(cls.repo, "add", "README")
git(cls.repo, "commit", "-m", "base")
cls.base = git(cls.repo, "rev-parse", "HEAD")
(cls.repo / "online.txt").write_text("live\n", encoding="utf-8")
git(cls.repo, "add", "online.txt")
git(cls.repo, "commit", "-m", "online")
cls.live = git(cls.repo, "rev-parse", "HEAD")
git(cls.repo, "checkout", "-b", "successor")
(cls.repo / "next.txt").write_text("next\n", encoding="utf-8")
git(cls.repo, "add", "next.txt")
git(cls.repo, "commit", "-m", "successor of live")
cls.successor = git(cls.repo, "rev-parse", "HEAD")
git(cls.repo, "checkout", "-B", "lagging-main", cls.base)
(cls.repo / "stale.txt").write_text("stale main\n", encoding="utf-8")
git(cls.repo, "add", "stale.txt")
git(cls.repo, "commit", "-m", "lagging main")
cls.lagging = git(cls.repo, "rev-parse", "HEAD")
git(cls.repo, "checkout", "-B", "side", cls.base)
(cls.repo / "side.txt").write_text("side branch\n", encoding="utf-8")
git(cls.repo, "add", "side.txt")
git(cls.repo, "commit", "-m", "unrelated side branch")
cls.side = git(cls.repo, "rev-parse", "HEAD")
git(cls.repo, "checkout", "-B", "successor", cls.successor)
@classmethod
def tearDownClass(cls) -> None:
cls.tmpdir.cleanup()
def test_check_script_is_executable(self) -> None:
self.assertTrue(CHECK.exists())
self.assertTrue(stat.S_IXUSR & CHECK.stat().st_mode)
def test_successor_of_live_passes(self) -> None:
result = run_check(self.repo, self.successor, self.live)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn(self.live, result.stdout)
self.assertIn(self.successor, result.stdout)
self.assertIn("next.txt", result.stdout)
self.assertIn("祖先关系通过", result.stdout)
def test_lagging_main_is_blocked(self) -> None:
result = run_check(self.repo, self.lagging, self.live)
self.assertNotEqual(result.returncode, 0)
self.assertIn("拒绝", result.stderr)
def test_side_branch_is_blocked(self) -> None:
result = run_check(self.repo, self.side, self.live)
self.assertNotEqual(result.returncode, 0)
self.assertIn("拒绝", result.stderr)
def test_unknown_commit_is_blocked(self) -> None:
result = run_check(self.repo, "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef", self.live)
self.assertNotEqual(result.returncode, 0)
self.assertIn("无法解析", result.stderr)
def test_build_image_calls_the_gate_and_rejects_latest(self) -> None:
source = BUILD.read_text(encoding="utf-8")
self.assertIn("check_deploy_baseline.sh", source)
self.assertIn("禁止构建 latest", source)
self.assertIn("org.opencontainers.image.revision", source)
gate = CHECK.read_text(encoding="utf-8")
self.assertIn("org.opencontainers.image.revision", gate)
self.assertIn("merge-base --is-ancestor", gate)
self.assertIn("候选将丢失的提交", gate)
self.assertIn("禁止人工填写", gate)
if __name__ == "__main__":
unittest.main()
+150 -1
View File
@@ -1,13 +1,162 @@
from __future__ import annotations
import json
import tempfile
import unittest
from http import HTTPStatus
from pathlib import Path
from unittest.mock import patch
from backend.features.heaven.engine import build_five_phase_field, hexagram_from_lines
from backend.features.heaven.knowledge import prepare_heaven_context
from backend.features.heaven.http import HeavenHttpMixin
from backend.features.heaven.knowledge import (
HeavenKnowledgeError,
clear_heaven_knowledge_cache,
prepare_heaven_context,
resolve_heaven_knowledge_path,
_knowledge_catalog,
)
from backend.features.heaven.six_yao import build_six_yao_chart
class HeavenKnowledgeTests(unittest.TestCase):
def tearDown(self) -> None:
clear_heaven_knowledge_cache()
def test_catalog_loads_from_trusted_repo_file(self):
clear_heaven_knowledge_cache()
path = resolve_heaven_knowledge_path()
catalog = _knowledge_catalog()
self.assertTrue(path.is_file())
self.assertEqual(path.name, "heaven_knowledge.json")
self.assertTrue(str(catalog.get("version") or "").startswith("2026."))
self.assertIn("zhouyi", catalog["sources"])
self.assertIn("neijing", catalog["sources"])
self.assertEqual(len(catalog["fortune"]["qi"]), 6)
self.assertEqual(len(catalog["fortune"]["personal_relations"]), 10)
def test_missing_knowledge_file_raises_chinese_structured_error(self):
clear_heaven_knowledge_cache()
missing = Path(tempfile.mkdtemp()) / "missing-heaven_knowledge.json"
with patch(
"backend.features.heaven.knowledge.KNOWLEDGE_FILE", missing
), patch(
"backend.features.heaven.knowledge.KNOWLEDGE_SEED_FILE",
missing.with_name("missing-seed.json"),
):
with self.assertRaises(HeavenKnowledgeError) as raised:
_knowledge_catalog()
self.assertEqual(raised.exception.error_code, "heaven_knowledge_missing")
self.assertIn("缺失", str(raised.exception))
def test_corrupt_knowledge_json_raises_chinese_structured_error(self):
clear_heaven_knowledge_cache()
with tempfile.TemporaryDirectory() as temp_dir:
broken = Path(temp_dir) / "heaven_knowledge.json"
broken.write_text("{not-json", encoding="utf-8")
with patch(
"backend.features.heaven.knowledge.KNOWLEDGE_FILE", broken
), patch(
"backend.features.heaven.knowledge.KNOWLEDGE_SEED_FILE",
Path(temp_dir) / "unused-seed.json",
):
with self.assertRaises(HeavenKnowledgeError) as raised:
_knowledge_catalog()
self.assertEqual(raised.exception.error_code, "heaven_knowledge_invalid")
self.assertIn("损坏", str(raised.exception))
def test_interpret_http_returns_structured_chinese_error_for_missing_file(self):
class FakeHandler(HeavenHttpMixin):
def __init__(self) -> None:
self.payload = None
self.status = None
self.application_service = type(
"Svc",
(),
{
"heaven_interpret": staticmethod(
lambda _body: (_ for _ in ()).throw(
HeavenKnowledgeError(
"问天知识文件缺失:未找到 heaven_knowledge.json。",
code="heaven_knowledge_missing",
)
)
)
},
)()
def read_json_body(self):
return {"mode": "trend", "trade_date": "2026-08-04"}
def send_json(self, payload, status=HTTPStatus.OK, headers=None):
self.payload = payload
self.status = status
handler = FakeHandler()
handler.heaven_interpret()
self.assertEqual(handler.status, HTTPStatus.BAD_REQUEST)
self.assertIn("缺失", handler.payload["error"])
self.assertEqual(handler.payload["code"], "heaven_knowledge_missing")
def test_interpret_http_returns_structured_chinese_error_for_corrupt_json(self):
class FakeHandler(HeavenHttpMixin):
def __init__(self) -> None:
self.payload = None
self.status = None
self.application_service = type(
"Svc",
(),
{
"heaven_interpret": staticmethod(
lambda _body: (_ for _ in ()).throw(
HeavenKnowledgeError(
"问天知识文件 JSON 损坏(heaven_knowledge.json),无法解析:第 1 行附近。",
code="heaven_knowledge_invalid",
)
)
)
},
)()
def read_json_body(self):
return {"mode": "trend", "trade_date": "2026-08-04"}
def send_json(self, payload, status=HTTPStatus.OK, headers=None):
self.payload = payload
self.status = status
handler = FakeHandler()
handler.heaven_interpret()
self.assertEqual(handler.status, HTTPStatus.BAD_REQUEST)
self.assertIn("损坏", handler.payload["error"])
self.assertEqual(handler.payload["code"], "heaven_knowledge_invalid")
def test_seed_fallback_when_data_file_missing(self):
clear_heaven_knowledge_cache()
with tempfile.TemporaryDirectory() as temp_dir:
seed = Path(temp_dir) / "seed.json"
seed.write_text(
json.dumps(
{
"version": "test-seed",
"sources": {"zhouyi": {"title": "周易"}},
"trend": {"method": "m", "rules": {"stable": "s", "single": "a", "multiple": "b"}},
"fortune": {},
"heart": {},
},
ensure_ascii=False,
),
encoding="utf-8",
)
missing_data = Path(temp_dir) / "data-heaven_knowledge.json"
with patch(
"backend.features.heaven.knowledge.KNOWLEDGE_FILE", missing_data
), patch(
"backend.features.heaven.knowledge.KNOWLEDGE_SEED_FILE", seed
):
catalog = _knowledge_catalog()
self.assertEqual(catalog["version"], "test-seed")
def test_fortune_context_excludes_weighted_summary_and_adds_bounded_industry_symbols(self):
field = build_five_phase_field("2026-08-04")
prepared = prepare_heaven_context(
+90
View File
@@ -0,0 +1,90 @@
from __future__ import annotations
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
LOGIN = (ROOT / "frontend" / "login" / "index.html").read_text(encoding="utf-8")
LOGIN_JS = (ROOT / "frontend" / "login" / "page.js").read_text(encoding="utf-8")
AUTH = (ROOT / "frontend" / "shared" / "auth.css").read_text(encoding="utf-8")
TOKENS = (ROOT / "frontend" / "shared" / "tokens.css").read_text(encoding="utf-8")
class LoginPortalContractTests(unittest.TestCase):
def test_confirmed_brand_skeleton_is_in_markup(self) -> None:
for needle in (
'class="login-brand-header"',
'class="login-brand-name"',
"A股个人复盘工作台",
'class="login-brand-kicker"',
"看懂情绪周期,把复盘变成下一次的先手。",
'class="login-brand-chart"',
'class="login-brand-stats"',
"股市有风险,投资需谨慎",
'id="loginThemeToggle"',
'id="loginCard"',
):
self.assertIn(needle, LOGIN)
def test_login_tokens_own_the_confirmed_layout_metrics(self) -> None:
for needle in (
"--login-brand-share: 34%;",
"--login-brand-cap: 560px;",
"--login-brand-wide-share: 29.2%;",
"--login-card-width: 408px;",
"--login-hero-size: 28px;",
"--login-account-row-min: 72px;",
):
self.assertIn(needle, TOKENS)
self.assertIn("width: var(--login-brand-share);", AUTH)
self.assertIn("width: var(--login-card-width);", AUTH)
self.assertIn("justify-content: flex-end;", AUTH)
self.assertIn("body.login-portal {", AUTH)
self.assertIn("padding-bottom: 0;", AUTH)
self.assertNotIn("padding: 0 0 var(--statusbar-height);", AUTH)
def test_confirmed_mascots_fill_the_brand_gap(self) -> None:
for needle in (
'id="loginMascots"',
'aria-hidden="true"',
'class="login-mascot is-red"',
'class="login-mascot is-green"',
'class="login-mascot-back"',
'class="login-mascot-slit"',
):
self.assertIn(needle, LOGIN)
for needle in (
"--login-mascot-red: #e8605a;",
"--login-mascot-green: #46be93;",
"--login-mascot-height: clamp(150px, 15vw, 260px);",
):
self.assertIn(needle, TOKENS)
self.assertIn("flex: 1 1 auto;", AUTH)
self.assertIn(".login-mascots {", AUTH)
self.assertIn("prefers-reduced-motion: reduce", AUTH)
self.assertIn('next === "password"', LOGIN_JS)
self.assertIn("toggle-password", LOGIN_JS)
self.assertIn("celebrateLogin", LOGIN_JS)
def test_portal_keeps_account_switch_and_theme_hooks(self) -> None:
self.assertIn("data-switch-id", LOGIN_JS)
self.assertIn("data-resume-id", LOGIN_JS)
self.assertIn('data-login-action="manage"', LOGIN_JS)
self.assertIn('data-login-action="add"', LOGIN_JS)
self.assertIn('data-login-action="resume"', LOGIN_JS)
self.assertIn("继续使用", LOGIN_JS)
self.assertIn("返回复盘", LOGIN_JS)
self.assertIn("/api/auth/me", LOGIN_JS)
self.assertIn("xiaobaiTheme", LOGIN_JS)
self.assertNotIn("内网个人版", LOGIN)
self.assertNotIn("192.168.200.11", LOGIN)
self.assertNotIn("/api/heaven", LOGIN_JS)
def test_current_account_row_stays_clickable_without_reswitching(self) -> None:
self.assertIn("resumeCurrentAccount", LOGIN_JS)
self.assertIn("当前会话已失效,请重新登录", LOGIN_JS)
self.assertNotIn("!managing && !current ? \"is-switchable\"", LOGIN_JS)
session = (ROOT / "frontend" / "shared" / "session.js").read_text(encoding="utf-8")
self.assertIn('params.set("next", next)', session)
self.assertIn(".login-account-action", AUTH)
+8
View File
@@ -43,6 +43,7 @@ class MobileSystemPagesRegressionTests(unittest.TestCase):
def test_system_pages_render_real_controls_not_stubs(self) -> None:
pages = PAGES.read_text(encoding="utf-8")
for marker in (
'data-system-page="home"',
'data-system-page="profile"',
'data-system-page="password"',
'data-system-page="membership"',
@@ -54,6 +55,11 @@ class MobileSystemPagesRegressionTests(unittest.TestCase):
"m-sys-token",
"m-sys-member-limit",
"data-system-switch",
"data-system-edit-model",
"data-system-open-member",
"管理员专区",
"保存密钥",
"保存分工",
'location.assign("/login/")',
):
self.assertIn(marker, pages)
@@ -73,6 +79,8 @@ class MobileSystemPagesRegressionTests(unittest.TestCase):
"data-system-save-models",
"data-system-save-market",
"data-system-refresh",
"data-system-toggle-refresh",
"data-system-save-model",
):
self.assertIn(name, pages)
self.assertNotIn(name + '">', pages)
+19
View File
@@ -111,6 +111,25 @@ class RealtimeDashboardTests(unittest.TestCase):
self.assertEqual(quote["amount_billion"], 3.0)
self.assertAlmostEqual(quote["turnover_rate"], 0.01)
def test_close_dashboard_marks_official_limit_data(self):
dashboard = self.client.dashboard("20260720")
self.assertEqual(dashboard["meta"]["limit_data_source"], "official")
def test_close_dashboard_marks_derived_limit_data_as_incomplete(self):
original_query = self.client.query
def query(api_name, params=None, fields=""):
if api_name == "limit_list_d":
return []
return original_query(api_name, params, fields)
self.client.query = query
dashboard = self.client.dashboard("20260720")
self.assertEqual(dashboard["meta"]["limit_data_source"], "derived")
self.assertIn("日线数据推算", dashboard["meta"]["notice"])
if __name__ == "__main__":
unittest.main()
+317
View File
@@ -0,0 +1,317 @@
from __future__ import annotations
import json
import tempfile
import threading
import unittest
from datetime import datetime
from pathlib import Path
from typing import Any
from unittest.mock import patch
from backend.features.market.backfill_history import (
build_backfill_audit,
classify_snapshot_coverage,
create_sqlite_backup,
select_open_trade_dates,
select_open_trade_dates_in_range,
)
from backend.features.market.service import MarketServiceMixin
from backend.features.sentiment.engine import (
build_sentiment_history,
latest_contiguous_history,
)
from backend.features.sentiment.service import SentimentServiceMixin
from database import ReviewDatabase
def _snapshot(trade_date: str, previous_trade_date: str) -> dict[str, Any]:
display = f"{trade_date[:4]}-{trade_date[4:6]}-{trade_date[6:8]}"
previous_display = (
f"{previous_trade_date[:4]}-{previous_trade_date[4:6]}-{previous_trade_date[6:8]}"
if previous_trade_date
else ""
)
return {
"meta": {
"trade_date": display,
"previous_trade_date": previous_display,
"source": "tushare",
},
"overview": {
"up_count": 2500,
"down_count": 2000,
"flat_count": 100,
"amount_billion": 12000,
"limit_up_count": 40,
"limit_down_count": 5,
"broken_count": 10,
"seal_rate": 70,
"max_height": 3,
"second_board_count": 8,
"three_plus_count": 4,
"previous_limit_count": 35,
"previous_positive_rate": 55,
"average_previous_change": 1.2,
"median_previous_change": 0.8,
"advance_rate": 20,
"severe_loss_rate": 5,
"previous_down_count": 3,
"ladder_completeness": 60,
"limit_amount_billion": 300,
},
"limits": [{"code": "000001"}],
"broken": [],
"down_limits": [],
"yesterday_limits": [],
}
class _BackfillHarness(MarketServiceMixin, SentimentServiceMixin):
def __init__(self, database: ReviewDatabase) -> None:
self.database = database
self.sync_lock = threading.Lock()
self.configured = True
self.token = "test-token"
self.current_user_id = 1
self._calendar_rows: list[dict[str, Any]] = []
self._fail_dates: set[str] = set()
self.sync_calls: list[str] = []
def _tushare_client(self): # type: ignore[override]
harness = self
class _Client:
def query(self, api_name, params, fields=""):
assert api_name == "trade_cal"
start = str(params["start_date"])
end = str(params["end_date"])
return [
row
for row in harness._calendar_rows
if start <= str(row["cal_date"]) <= end
]
return _Client()
def sync_dashboard(self, trade_date: str) -> dict[str, Any]: # type: ignore[override]
compact = trade_date.replace("-", "")
self.sync_calls.append(compact)
if compact in self._fail_dates:
raise ValueError(f"simulated failure for {compact}")
previous = ""
for row in self._calendar_rows:
if str(row["cal_date"]) == compact:
previous = str(row.get("pretrade_date") or "")
break
payload = _snapshot(compact, previous)
self.database.save_snapshot(compact, "tushare", payload)
return payload
def _apply_reason_overrides(self, dashboard: dict[str, Any]) -> dict[str, Any]:
return dashboard
def _with_storage(self, dashboard: dict[str, Any], cached: bool) -> dict[str, Any]:
return dashboard
class BackfillHistoryHelperTests(unittest.TestCase):
def test_select_open_trade_dates_skips_weekends_and_holidays(self) -> None:
rows = [
{"cal_date": "20260821", "is_open": 1, "pretrade_date": "20260820"},
{"cal_date": "20260822", "is_open": 0, "pretrade_date": "20260821"}, # Sat
{"cal_date": "20260823", "is_open": 0, "pretrade_date": "20260821"}, # Sun
{"cal_date": "20260824", "is_open": 1, "pretrade_date": "20260821"},
{"cal_date": "20260825", "is_open": 1, "pretrade_date": "20260824"},
{"cal_date": "20260826", "is_open": 1, "pretrade_date": "20260825"},
{"cal_date": "20260827", "is_open": 1, "pretrade_date": "20260826"},
]
selected = select_open_trade_dates(rows, "20260827", 4)
self.assertEqual(selected, ["20260824", "20260825", "20260826", "20260827"])
def test_range_mode_reports_non_trading_days_separately(self) -> None:
rows = [
{"cal_date": "20260821", "is_open": 1},
{"cal_date": "20260824", "is_open": 1},
]
open_dates, skipped = select_open_trade_dates_in_range(
rows, "20260821", "20260824"
)
self.assertEqual(open_dates, ["20260821", "20260824"])
self.assertEqual(skipped, ["20260822", "20260823"])
def test_classify_snapshot_coverage_finds_real_gaps(self) -> None:
coverage = classify_snapshot_coverage(
["20260824", "20260825", "20260826", "20260827"],
["20260824", "20260827"],
)
self.assertEqual(coverage["missing"], ["20260825", "20260826"])
self.assertEqual(coverage["present"], ["20260824", "20260827"])
class ContiguousHistoryGapTests(unittest.TestCase):
def test_missing_previous_trade_day_collapses_to_today(self) -> None:
payloads = [
_snapshot("20260824", "20260821"),
_snapshot("20260827", "20260826"), # gap: 20260826 missing
]
series = latest_contiguous_history(build_sentiment_history(payloads))
self.assertEqual([row["trade_date"] for row in series], ["20260827"])
def test_continuous_history_keeps_full_tail(self) -> None:
payloads = [
_snapshot("20260825", "20260824"),
_snapshot("20260826", "20260825"),
_snapshot("20260827", "20260826"),
]
series = latest_contiguous_history(build_sentiment_history(payloads))
self.assertEqual(
[row["trade_date"] for row in series],
["20260825", "20260826", "20260827"],
)
class SnapshotBackfillServiceTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.db_path = Path(self.temporary.name) / "review.db"
self.database = ReviewDatabase(self.db_path)
self.service = _BackfillHarness(self.database)
self.service._calendar_rows = [
{"cal_date": "20260820", "is_open": 1, "pretrade_date": "20260819"},
{"cal_date": "20260821", "is_open": 1, "pretrade_date": "20260820"},
{"cal_date": "20260822", "is_open": 0, "pretrade_date": "20260821"},
{"cal_date": "20260823", "is_open": 0, "pretrade_date": "20260821"},
{"cal_date": "20260824", "is_open": 1, "pretrade_date": "20260821"},
{"cal_date": "20260825", "is_open": 1, "pretrade_date": "20260824"},
{"cal_date": "20260826", "is_open": 1, "pretrade_date": "20260825"},
{"cal_date": "20260827", "is_open": 1, "pretrade_date": "20260826"},
]
# Sparse history mimicking .11: keep 0824 and today, miss 0825/0826.
self.database.save_snapshot("20260824", "tushare", _snapshot("20260824", "20260821"))
self.database.save_snapshot("20260827", "tushare", _snapshot("20260827", "20260826"))
def tearDown(self) -> None:
self.temporary.cleanup()
def test_recent_backfill_fills_gap_and_restores_history(self) -> None:
before = self.service.sentiment_history("20260827", 20)
self.assertEqual(before["available_days"], 1)
with patch(
"backend.features.market.service.create_sqlite_backup",
return_value=Path(self.temporary.name) / "fake-backup.db",
) as backup:
audit = self.service.backfill_recent_trading_days(
end_date="20260827",
lookback=4,
dry_run=False,
create_backup=True,
)
backup.assert_called_once()
self.assertEqual(sorted(self.service.sync_calls), ["20260825", "20260826"])
self.assertEqual(audit["missing"], ["2026-08-25", "2026-08-26"])
self.assertEqual(sorted(audit["created_dates"]), ["2026-08-25", "2026-08-26"])
after = self.service.sentiment_history("20260827", 20)
self.assertGreaterEqual(after["available_days"], 4)
self.assertEqual(
[row["trade_date"] for row in after["rows"]],
["20260824", "20260825", "20260826", "20260827"],
)
def test_dry_run_does_not_write_snapshots(self) -> None:
audit = self.service.backfill_recent_trading_days(
end_date="20260827",
lookback=4,
dry_run=True,
create_backup=True,
)
self.assertTrue(audit["dry_run"])
self.assertEqual(self.service.sync_calls, [])
self.assertIsNone(audit["backup_path"])
self.assertEqual(
self.database.list_snapshot_trade_dates("20260824", "20260827"),
["20260824", "20260827"],
)
def test_repeat_execution_skips_existing_days(self) -> None:
with patch(
"backend.features.market.service.create_sqlite_backup",
return_value=Path(self.temporary.name) / "fake-backup.db",
):
first = self.service.backfill_recent_trading_days(
end_date="20260827", lookback=4
)
self.service.sync_calls.clear()
second = self.service.backfill_recent_trading_days(
end_date="20260827", lookback=4
)
self.assertEqual(first["succeeded_count"], 2)
self.assertEqual(self.service.sync_calls, [])
self.assertEqual(second["missing_count"], 0)
self.assertEqual(second["skipped_count"], 4)
self.assertIsNone(second["backup_path"])
def test_partial_failure_continues_remaining_days(self) -> None:
self.service._fail_dates.add("20260825")
with patch(
"backend.features.market.service.create_sqlite_backup",
return_value=Path(self.temporary.name) / "fake-backup.db",
):
audit = self.service.backfill_recent_trading_days(
end_date="20260827", lookback=4
)
self.assertFalse(audit["ok"])
self.assertEqual(audit["failed_count"], 1)
self.assertEqual(audit["succeeded_count"], 1)
self.assertIn("20260826", self.database.list_snapshot_trade_dates())
self.assertNotIn("20260825", self.database.list_snapshot_trade_dates())
def test_range_backfill_skips_weekend_without_treating_as_error(self) -> None:
with patch(
"backend.features.market.service.create_sqlite_backup",
return_value=Path(self.temporary.name) / "fake-backup.db",
):
audit = self.service.backfill(
start_date="2026-08-21",
end_date="2026-08-24",
)
self.assertEqual(audit["mode"], "range")
self.assertEqual(audit["skipped_non_trading_days"], ["2026-08-22", "2026-08-23"])
self.assertEqual(sorted(self.service.sync_calls), ["20260821"])
self.assertTrue(audit["ok"])
def test_sqlite_backup_api_creates_restorable_copy(self) -> None:
backup_dir = Path(self.temporary.name) / "backups"
backup = create_sqlite_backup(
self.db_path,
backup_dir,
label="pre-recent-backfill",
stamped_at=datetime(2026, 8, 27, 15, 30, 0),
)
self.assertTrue(backup.exists())
self.assertIn("pre-recent-backfill-20260827-153000", backup.name)
restored = ReviewDatabase(backup)
self.assertEqual(
restored.list_snapshot_trade_dates(),
["20260824", "20260827"],
)
def test_audit_lists_only_snapshot_related_write_tables(self) -> None:
audit = build_backfill_audit(
mode="recent",
end_date="20260827",
lookback=60,
coverage={"trade_dates": [], "present": [], "missing": [], "present_count": 0, "missing_count": 0},
)
self.assertEqual(
audit["write_tables"],
["dashboard_snapshots", "data_snapshots", "sync_runs"],
)
self.assertNotIn("users", audit["write_tables"])
self.assertNotIn("system_settings", audit["write_tables"])
if __name__ == "__main__":
unittest.main()
+21 -5
View File
@@ -17,12 +17,28 @@ registry, and verification tools.
`backend/features/*/routes.py` owners.
- `python tools/build_architecture_inventory.py [--check]`: generate or verify
`config/architecture-inventory.json` from the current source tree.
- `bash tools/build_image.sh <commit> <tag>`: the only sanctioned way to build the
production Docker image. Streams `git archive <commit>` to the deploy host over SSH
(default `moxiaobai@192.168.200.11`), refuses tags that do not end with the commit
- `python tools/backfill_recent_snapshots.py --account <admin> [--lookback 60] [--dry-run]`:
auditable recent trading-day dashboard snapshot backfill. See
`docs/maintenance/行情历史补档.md`.
- `tools/update_from_main.sh` (deployed to the server as
`~/xiaobai-build/update-from-main.sh`): the server-side update-and-build entry for
the managed local worktree at `/opt/1panel/docker/compose/xiaobaifupan`. Fetches
Gitea `main`, enforces branch/clean/fast-forward checks, builds a
`main-<shortsha>` tagged image with the revision label, and verifies the label
after the build. `tools/xiaobai-git` is the matching git wrapper for that
worktree (`status`/`log`/`diff`).
- `bash tools/build_image.sh <commit> <tag>`: agent-grade entry that streams
`git archive <commit>` to the deploy host over SSH (default
`moxiaobai@192.168.200.11`), refuses tags that do not end with the commit
short SHA, verifies the revision label after the build, and appends a record to
`~/xiaobai-build/BUILD_LOG.tsv` on the host. Building from any server-side working
tree is forbidden; see `DOCKER_DEPLOY.md`.
`~/xiaobai-build/BUILD_LOG.tsv` on the host. Before building, it runs
`tools/check_deploy_baseline.sh` so the candidate commit must contain the currently
running container's Git revision as an ancestor.
- `bash tools/check_deploy_baseline.sh <commit> [--live-revision <sha>]`: deployment
ancestor gate. Reads the live `org.opencontainers.image.revision` from the running
`xiaobai-review` container (or `--live-revision` in tests), prints the live SHA,
candidate SHA, file diff, and commits the candidate would drop, then exits if the
live revision is not an ancestor of the candidate.
`verify_baseline.py` does not inspect a parent checkout or skip tests according to files outside
this application. Historical comparison scripts were retired after final standalone acceptance;
+112
View File
@@ -0,0 +1,112 @@
#!/usr/bin/env python3
"""Auditable recent trading-day dashboard snapshot backfill.
Examples:
python tools/backfill_recent_snapshots.py --account admin --dry-run
python tools/backfill_recent_snapshots.py --account admin --lookback 60
python tools/backfill_recent_snapshots.py --account admin --end-date 2026-08-27 --force
"""
from __future__ import annotations
import argparse
import json
import sys
from datetime import date
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
from backend.application import SERVICE
from backend.bootstrap.config import normalize_date
from backend.features.market.backfill_history import DEFAULT_RECENT_TRADING_DAYS
def main() -> None:
parser = argparse.ArgumentParser(
description="Backfill the latest N real trading-day dashboard snapshots"
)
parser.add_argument(
"--account",
required=True,
help="Account that can resolve the shared Tushare token",
)
parser.add_argument(
"--end-date",
default=date.today().isoformat(),
help="Inclusive end date YYYY-MM-DD (default: today)",
)
parser.add_argument(
"--lookback",
type=int,
default=DEFAULT_RECENT_TRADING_DAYS,
help=f"Number of open trading days to cover (default {DEFAULT_RECENT_TRADING_DAYS}, max 60)",
)
parser.add_argument(
"--dry-run",
action="store_true",
help="Plan only: classify missing gaps without writing",
)
parser.add_argument(
"--force",
action="store_true",
help="Re-sync days that already have snapshots",
)
parser.add_argument(
"--no-backup",
action="store_true",
help="Skip the SQLite backup API step (not recommended)",
)
parser.add_argument(
"--json",
action="store_true",
help="Print the full audit payload as JSON",
)
args = parser.parse_args()
user = SERVICE.database.user_by_username(args.account.strip())
if not user:
raise SystemExit("account not found")
SERVICE.bind_user(int(user["id"]))
end_date = normalize_date(args.end_date)
audit = SERVICE.backfill_recent_trading_days(
end_date=end_date,
lookback=args.lookback,
dry_run=args.dry_run,
force=args.force,
create_backup=not args.no_backup,
)
if args.json:
print(json.dumps(audit, ensure_ascii=False, indent=2))
raise SystemExit(0 if audit.get("ok") else 1)
print(
f"mode={audit['mode']} end={audit['end_date']} lookback={audit['lookback']} "
f"dry_run={audit['dry_run']}"
)
print(
f"present={audit['present_count']} missing={audit['missing_count']} "
f"succeeded={audit['succeeded_count']} skipped={audit['skipped_count']} "
f"failed={audit['failed_count']}"
)
if audit.get("backup_path"):
print(f"backup={audit['backup_path']}")
if audit.get("missing"):
print("missing_dates=" + ",".join(audit["missing"]))
if audit.get("created_dates"):
print("created_dates=" + ",".join(audit["created_dates"]))
failed = [row for row in audit.get("results") or [] if row.get("status") == "failed"]
for row in failed:
print(f"failed {row.get('requested_date')}: {row.get('error')}")
if not audit.get("ok"):
raise SystemExit(1)
print("backfill complete")
if __name__ == "__main__":
main()
+3
View File
@@ -59,6 +59,9 @@ if [[ "$TAG" != *-"$SHORT" ]]; then
exit 1
fi
echo "==> 部署基线门禁(线上提交必须是候选祖先)"
bash "$(git rev-parse --show-toplevel)/tools/check_deploy_baseline.sh" "$FULL_SHA"
echo "==> 构建计划"
echo " 提交: ${FULL_SHA} ${SUBJECT}"
echo " 镜像: ${REPO_NAME}:${TAG} @ ${HOST}"
+126
View File
@@ -0,0 +1,126 @@
#!/usr/bin/env bash
# 部署基线门禁(HEL-238):候选提交必须包含当前线上提交的全部历史。
# 线上提交号从运行中的容器镜像 label 读取,禁止人工填写“看起来正确”的基线。
set -euo pipefail
HOST_DEFAULT="moxiaobai@192.168.200.11"
CONTAINER_DEFAULT="xiaobai-review"
usage() {
cat <<'EOF'
用法: tools/check_deploy_baseline.sh <candidate_commit> [--live-revision <sha>]
<candidate_commit> 准备构建/部署的提交(完整或前缀)
--live-revision <sha> 仅测试用:直接指定线上提交,跳过 SSH 读取
环境变量:
XB_BUILD_HOST 部署机 SSH(默认 moxiaobai@192.168.200.11
XB_LIVE_CONTAINER 运行中容器名(默认 xiaobai-review
XB_LIVE_REVISION 若已设置则视为线上提交,不再 SSH
EOF
exit 2
}
[ $# -ge 1 ] || usage
CANDIDATE="$1"
shift
LIVE_OVERRIDE=""
while [ $# -gt 0 ]; do
case "$1" in
--live-revision)
[ $# -ge 2 ] || usage
LIVE_OVERRIDE="$2"
shift 2
;;
-h|--help)
usage
;;
*)
echo "拒绝:未知参数 $1" >&2
usage
;;
esac
done
HOST="${XB_BUILD_HOST:-$HOST_DEFAULT}"
CONTAINER="${XB_LIVE_CONTAINER:-$CONTAINER_DEFAULT}"
case "$HOST" in
*192.168.200.36*)
echo "拒绝:192.168.200.36 已永久废弃,严禁在其上构建或部署。" >&2
exit 1
;;
esac
cd "$(git rev-parse --show-toplevel)"
read_live_revision() {
if [ -n "${LIVE_OVERRIDE}" ]; then
printf '%s\n' "${LIVE_OVERRIDE}"
return
fi
if [ -n "${XB_LIVE_REVISION:-}" ]; then
printf '%s\n' "${XB_LIVE_REVISION}"
return
fi
ssh -o BatchMode=yes "$HOST" bash -s -- "$CONTAINER" <<'REMOTE'
set -euo pipefail
container="$1"
revision="$(docker inspect --format '{{index .Config.Labels "org.opencontainers.image.revision"}}' "$container" 2>/dev/null || true)"
if [ -z "$revision" ] || [ "$revision" = "<no value>" ]; then
image="$(docker inspect --format '{{.Image}}' "$container" 2>/dev/null || true)"
if [ -n "$image" ]; then
revision="$(docker inspect --format '{{index .Config.Labels "org.opencontainers.image.revision"}}' "$image" 2>/dev/null || true)"
fi
fi
if [ -z "$revision" ] || [ "$revision" = "<no value>" ]; then
echo "拒绝:无法从线上容器 ${container} 读取 org.opencontainers.image.revision。" >&2
exit 1
fi
printf '%s\n' "$revision"
REMOTE
}
LIVE_RAW="$(read_live_revision)"
LIVE_RAW="$(printf '%s' "$LIVE_RAW" | tr -d '[:space:]')"
if [ -z "$LIVE_RAW" ]; then
echo "拒绝:线上提交号为空,禁止继续构建或部署。" >&2
exit 1
fi
CANDIDATE_SHA="$(git rev-parse --verify --quiet "${CANDIDATE}^{commit}" || true)"
if [ -z "$CANDIDATE_SHA" ]; then
echo "拒绝:候选提交 ${CANDIDATE} 无法解析。" >&2
exit 1
fi
LIVE_SHA="$(git rev-parse --verify --quiet "${LIVE_RAW}^{commit}" || true)"
if [ -z "$LIVE_SHA" ]; then
echo "拒绝:线上提交 ${LIVE_RAW} 在本地仓库无法解析;请先 git fetch,禁止手工填写替代基线。" >&2
exit 1
fi
echo "==> 部署基线对照"
echo " 线上提交: ${LIVE_SHA}"
echo " 候选提交: ${CANDIDATE_SHA}"
echo "==> 候选相对线上的文件差异"
DIFF_FILES="$(git diff --name-only "$LIVE_SHA" "$CANDIDATE_SHA" || true)"
if [ -z "$DIFF_FILES" ]; then
echo " (无文件差异)"
else
printf '%s\n' "$DIFF_FILES" | sed 's/^/ /'
fi
echo "==> 候选将丢失的提交(线上有、候选没有)"
LOST="$(git log --oneline "$CANDIDATE_SHA".."$LIVE_SHA" || true)"
if [ -z "$LOST" ]; then
echo " (无)"
else
printf '%s\n' "$LOST" | sed 's/^/ /'
fi
if ! git merge-base --is-ancestor "$LIVE_SHA" "$CANDIDATE_SHA"; then
echo "拒绝:候选提交不是当前线上提交的后继,部署会丢失线上已有提交。禁止构建或部署。" >&2
exit 1
fi
echo "==> 祖先关系通过:线上 ${LIVE_SHA:0:7} 是候选 ${CANDIDATE_SHA:0:7} 的祖先"
+110
View File
@@ -0,0 +1,110 @@
#!/usr/bin/env bash
# 小白复盘服务器本地目录安全更新/构建入口(HEL-235B 固化)
# 作用:把 /opt/1panel/docker/compose/xiaobaifupan 的 Git 工作目录安全快进到 Gitea main,
# 校验“本地 HEAD = origin/main = 镜像 revision”后,从本地目录构建带提交号的镜像。
# 禁止:不从 main 构建;不使用不带提交短号的 tag;本地有改动/落后/分叉时一律停止。
# 说明:目录顶层归 root,本脚本用“截断写入”绕开 git 对顶层文件 unlink+重建的权限要求;
# 但 main 新增/删除顶层文件时无法自动处理,会列出需管理员执行的精确清单。
set -euo pipefail
GIT_DIR_PATH="$HOME/xiaobai-build/repos/xiaobai-review.git"
WORK_TREE="/opt/1panel/docker/compose/xiaobaifupan"
IMAGE_REPO="xiaobai-review"
LOG_FILE="$HOME/xiaobai-build/BUILD_LOG.tsv"
MODE="${1:-build}"
g() { git --git-dir="$GIT_DIR_PATH" --work-tree="$WORK_TREE" "$@"; }
refuse() { printf '拒绝:%s\n' "$*" >&2; exit 1; }
[ "$MODE" = "build" ] || [ "$MODE" = "verify-tag" ] || refuse "未知子命令「${MODE}」(可用:build / verify-tag <tag>"
[ -d "$GIT_DIR_PATH" ] || refuse "Git 目录不存在:$GIT_DIR_PATH"
echo "==> 拉取 Gitea origin/main"
g fetch --quiet origin main || refuse "无法连接 Gitea 拉取 origin/main"
echo "==> 检查分支与工作区"
BRANCH="$(g symbolic-ref --short HEAD 2>/dev/null || true)"
[ "$BRANCH" = "main" ] || refuse "当前不在 main 分支(${BRANCH:-detached}),停止"
DIRTY="$(g status --porcelain)"
[ -z "$DIRTY" ] || refuse "本地目录有未提交改动或多余文件,先处理再构建:
$DIRTY"
LOCAL_HEAD="$(g rev-parse HEAD)"
REMOTE_HEAD="$(g rev-parse origin/main)"
if [ "$LOCAL_HEAD" != "$REMOTE_HEAD" ]; then
g merge-base --is-ancestor "$LOCAL_HEAD" "$REMOTE_HEAD" \
|| refuse "本地 main 与 origin/main 历史分叉,停止(未改写工作目录)"
CHANGES="$(g diff --no-renames --name-status HEAD origin/main)"
TOP_AD="$(printf '%s\n' "$CHANGES" | grep -E "^[AD][[:space:]]+[^/]+$" || true)"
[ -z "$TOP_AD" ] || refuse "main 相比本地新增/删除了顶层文件,目录顶层归 root,需管理员执行:
$TOP_AD"
echo "==> 同步 origin/main 文件(顶层文件保留原 inode,避免目录权限限制)"
while IFS=$'\t' read -r status path; do
[ -n "$path" ] || continue
case "$status" in
D)
case "$path" in
*/*) rm -f -- "$WORK_TREE/$path" ;;
*) refuse "main 删除了顶层文件 $path,需管理员处理" ;;
esac
;;
A|M)
MODE_BITS="$(g ls-tree origin/main -- "$path" | awk '{print $1}')"
case "$MODE_BITS" in
100644|100755) ;;
*) refuse "文件 ${path} 的 Git 类型 ${MODE_BITS} 不支持自动同步,需管理员处理" ;;
esac
mkdir -p -- "$(dirname "$WORK_TREE/$path")"
g show "origin/main:$path" > "$WORK_TREE/$path"
[ "$MODE_BITS" = "100755" ] && chmod 755 "$WORK_TREE/$path" || chmod 644 "$WORK_TREE/$path"
;;
*) refuse "遇到未支持的 Git 变更类型 ${status}${path}" ;;
esac
done <<< "$CHANGES"
g read-tree origin/main
g update-ref refs/heads/main "$REMOTE_HEAD" "$LOCAL_HEAD"
DIRTY="$(g status --porcelain)"
[ -z "$DIRTY" ] || refuse "快进后工作区仍不一致,停止:
$DIRTY"
LOCAL_HEAD="$(g rev-parse HEAD)"
fi
[ "$LOCAL_HEAD" = "$REMOTE_HEAD" ] || refuse "本地 HEAD 与 origin/main 不一致,停止"
SHORT="${LOCAL_HEAD:0:7}"
echo "==> 校验通过:本地 HEAD = origin/main = ${LOCAL_HEAD}${SHORT}"
if [ "$MODE" = "verify-tag" ]; then
TAG="${2:?用法: update-from-main.sh verify-tag <tag>}"
[ "$TAG" = "main-${SHORT}" ] \
|| refuse "镜像标签必须是当前 main 对应的 main-${SHORT},收到:${TAG}"
LABEL="$(docker image inspect "${IMAGE_REPO}:${TAG}" \
--format '{{index .Config.Labels "org.opencontainers.image.revision"}}' 2>/dev/null)" \
|| refuse "镜像 ${IMAGE_REPO}:${TAG} 不存在"
[ "$LABEL" = "$LOCAL_HEAD" ] || refuse "镜像 revision${LABEL})与当前 main${LOCAL_HEAD})不一致,禁止部署"
echo "==> 通过:${IMAGE_REPO}:${TAG} 的 revision 与 main 一致,可以部署"
exit 0
fi
TAG="main-${SHORT}"
echo "==> 从本地目录构建 ${IMAGE_REPO}:${TAG}"
docker build --rm -t "${IMAGE_REPO}:${TAG}" \
--label "org.opencontainers.image.revision=${LOCAL_HEAD}" \
--label "org.opencontainers.image.created=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
"$WORK_TREE" 2>&1 | tail -5
echo "==> 回读校验镜像 revision"
GOT="$(docker image inspect "${IMAGE_REPO}:${TAG}" \
--format '{{index .Config.Labels "org.opencontainers.image.revision"}}')"
if [ "$GOT" != "$LOCAL_HEAD" ]; then
docker rmi "${IMAGE_REPO}:${TAG}" >/dev/null 2>&1 || true
refuse "镜像 revision${GOT})与 main${LOCAL_HEAD})不一致,已删除镜像"
fi
IMAGE_ID="$(docker image inspect "${IMAGE_REPO}:${TAG}" --format '{{.Id}}' | cut -c8-19)"
mkdir -p "$(dirname "$LOG_FILE")"
printf '%s\t%s\t%s\t%s\tlocal-worktree\n' \
"$(date '+%F %T')" "${IMAGE_REPO}:${TAG}" "${LOCAL_HEAD}" "${IMAGE_ID}" >> "$LOG_FILE"
cat <<EOF
==> 完成:${IMAGE_REPO}:${TAG}revision=${LOCAL_HEAD}
部署需人工确认,参考 ~/xiaobai-build/README.md 的换版与回滚步骤。
EOF
+6
View File
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
# 查看服务器本地目录 Git 状态的便捷入口:xiaobai-git status / log / diff 等
exec git \
--git-dir="$HOME/xiaobai-build/repos/xiaobai-review.git" \
--work-tree="/opt/1panel/docker/compose/xiaobaifupan" \
"$@"
+10
View File
@@ -0,0 +1,10 @@
.git
.gitignore
.env
.env.*
!.env.example
__pycache__/
*.py[cod]
*.log
data/
tests/
+13
View File
@@ -0,0 +1,13 @@
# Fernet key. Generate with: python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
DATAHUB_ENCRYPTION_KEY=
# Consumer API token for /v1 (32+ random bytes, shown once). Never log this value.
DATAHUB_TOKEN=
# Initial admin password for /admin. Forced change on first login.
DATAHUB_ADMIN_PASSWORD=
# Tushare Pro token. Stored encrypted after first launch; never returned by API or admin pages.
TUSHARE_TOKEN=
TZ=Asia/Shanghai
+36
View File
@@ -0,0 +1,36 @@
FROM python:3.12-slim-bookworm
ARG APP_UID=10002
ARG APP_GID=10002
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PYTHONUTF8=1 \
PIP_DISABLE_PIP_VERSION_CHECK=1 \
TZ=Asia/Shanghai
WORKDIR /app
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
ca-certificates \
tzdata \
&& groupadd --gid "${APP_GID}" datahub \
&& useradd --uid "${APP_UID}" --gid "${APP_GID}" --create-home --shell /usr/sbin/nologin datahub \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt ./
RUN python -m pip install --no-cache-dir -r requirements.txt
COPY --chown=datahub:datahub . .
RUN mkdir -p /app/data /app/data/backups && chown -R datahub:datahub /app/data
USER datahub
EXPOSE 8766
STOPSIGNAL SIGINT
HEALTHCHECK --interval=30s --timeout=5s --start-period=20s --retries=3 \
CMD ["python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8766/livez', timeout=4).read()"]
CMD ["python", "-u", "server.py", "--host", "0.0.0.0", "--port", "8766"]
+78
View File
@@ -0,0 +1,78 @@
# xiaobai-datahub
独立行情数据中枢(HEL-382 / P0)。与 `xiaobai-review` 同仓库、不同容器、不共享数据库文件。
本阶段不部署现网;只提供可本地运行、可自测的底座和盘后正式数据链路。
## 做什么
- SQLite WAL `datahub.db`,容器名 `xiaobai-datahub`,端口 `8766`
- Tushare 盘后正式数据:交易日历、股票主档、daily、daily_basic、adj_factor、index_daily、moneyflow、stk_auction
- 暂存 → 校验 → 整批原子发布 → 可回滚
- `/v1` 稳定接口(`X-Datahub-Token`
- `/admin/` 最小管理后台(总览 / 数据源 / 调度 / 发布 / 数据集 / 审计)
- 东财/腾讯/同花顺/选股宝/AKShare/iFinD 适配器位已预留,本阶段不拉实时源
## 单位口径(相对现站)
现站 `xiaobai-review` 按 Tushare 原始单位入库、展示时再换算。中枢在归一化层一次换算:
| 字段 | Tushare / 现站 | 中枢 canonical |
|---|---|---|
| `daily.amount` / `index_daily.amount` | 千元 | 元(×1000 |
| `daily.vol` / `index_daily.vol` | 手 | 股(×100 |
| `moneyflow.*_amount` | 万元 | 元(×1e4 |
| `daily_basic.total_mv` / `circ_mv` | 万元 | 元(×1e4 |
| `stk_auction.amount` | 元 | 元 |
差异为口径升级,golden 测试按上表对照,不为 0 的字段都有说明。
## 本地启动(不走 Docker
```bash
cd xiaobai-datahub
python -m venv .venv && .venv/bin/pip install -r requirements.txt
cp .env.example .env
# 填入 DATAHUB_ENCRYPTION_KEY / DATAHUB_TOKEN / DATAHUB_ADMIN_PASSWORD / TUSHARE_TOKEN
# 生成 Fernet 密钥:
# python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
.venv/bin/python server.py --host 127.0.0.1 --port 8766
```
- 管理后台:http://127.0.0.1:8766/admin/
- 存活检查:http://127.0.0.1:8766/livez (无需 token
- `/v1/*` 必须带请求头 `X-Datahub-Token`
## Docker(独立 compose,不改现网 review 服务)
```bash
cd xiaobai-datahub
cp .env.example .env # 填密钥
mkdir -p data
docker compose build
docker compose up -d
```
仓库根目录另有 `compose.datahub.yaml`,供总工以后与现有 `compose.yaml` 叠加部署,本卡不执行现网 `up`
## 自测
```bash
cd xiaobai-datahub
python -m unittest discover -s tests -v
```
不调用真实 Tushare;用内存/临时库和假适配器。
## 备份
每日 00:40 任务把 `datahub.db` 备份到 `data/backups/`(保留 14 份)。也可手动:
```bash
python -c "from pathlib import Path; from datahub.db import HubDB; HubDB(Path('data/datahub.db')).backup_to(Path('data/backups/manual.db'))"
```
## 安全
- 密钥只以 `configured / 末4位 / 更新时间` 出现在后台,不进日志、不进 `/v1`
- 回滚、补数需重新输入密码 + 确认词
- 容器非 rootuid 10002)、read_only、cap_drop ALL
+268
View File
@@ -0,0 +1,268 @@
const state = { csrf: "", page: "overview" };
function $(id) { return document.getElementById(id); }
async function api(path, options = {}) {
const headers = Object.assign({ "Content-Type": "application/json" }, options.headers || {});
if (state.csrf && (options.method || "GET") !== "GET") headers["X-CSRF-Token"] = state.csrf;
const res = await fetch(path, Object.assign({}, options, { headers, credentials: "same-origin" }));
const body = await res.json();
if (!res.ok) {
const msg = (body.error && body.error.message) || body.error || res.statusText;
throw new Error(msg);
}
return body;
}
function show(id) {
["login-view", "change-view", "shell"].forEach((key) => { $(key).hidden = key !== id; });
}
function esc(value) {
return String(value ?? "").replace(/[&<>"]/g, (ch) => ({ "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;" }[ch]));
}
function table(headers, rows) {
const thead = headers.map((h) => `<th>${esc(h)}</th>`).join("");
const body = rows.length
? rows.map((cols) => `<tr>${cols.map((c) => `<td>${c}</td>`).join("")}</tr>`).join("")
: `<tr><td colspan="${headers.length}">暂无数据</td></tr>`;
return `<table><thead><tr>${thead}</tr></thead><tbody>${body}</tbody></table>`;
}
async function boot() {
try {
const session = await api("/admin/api/session");
state.csrf = session.csrf;
$("who").textContent = session.username;
if (session.must_change) { show("change-view"); return; }
show("shell");
await render();
} catch {
show("login-view");
}
}
$("login-form").addEventListener("submit", async (event) => {
event.preventDefault();
const form = new FormData(event.target);
$("login-error").hidden = true;
try {
const result = await api("/admin/api/login", {
method: "POST",
body: JSON.stringify({ username: form.get("username"), password: form.get("password") }),
});
state.csrf = result.csrf;
if (result.must_change) show("change-view");
else { show("shell"); await render(); }
} catch (err) {
$("login-error").hidden = false;
$("login-error").textContent = err.message;
}
});
$("change-form").addEventListener("submit", async (event) => {
event.preventDefault();
const form = new FormData(event.target);
try {
await api("/admin/api/change-password", {
method: "POST",
body: JSON.stringify({ current: form.get("current"), new_password: form.get("new_password") }),
});
show("shell");
await render();
} catch (err) {
$("change-error").hidden = false;
$("change-error").textContent = err.message;
}
});
$("logout-btn").addEventListener("click", async () => {
await api("/admin/api/logout", { method: "POST", body: "{}" });
show("login-view");
});
$("theme-btn").addEventListener("click", () => {
const root = document.documentElement;
const next = root.getAttribute("data-theme") === "night" ? "" : "night";
if (next) root.setAttribute("data-theme", next);
else root.removeAttribute("data-theme");
$("theme-btn").textContent = next ? "日间" : "夜间";
});
document.querySelectorAll("nav button").forEach((btn) => {
btn.addEventListener("click", () => {
document.querySelectorAll("nav button").forEach((item) => item.classList.remove("active"));
btn.classList.add("active");
state.page = btn.dataset.page;
render();
});
});
async function render() {
const page = $("page");
if (state.page === "overview") {
const data = await api("/admin/api/overview");
$("phase").textContent = data.session_phase;
page.innerHTML = `
<div class="cards">
<div class="card"><div class="muted">交易日</div><strong>${esc(data.trade_date)}</strong></div>
<div class="card"><div class="muted">阶段</div><strong>${esc(data.session_phase)}</strong></div>
<div class="card"><div class="muted">今日发布</div><strong>${data.publications.length}</strong></div>
<div class="card"><div class="muted">异常批次</div><strong class="${data.anomalies.length ? "fail" : "ok"}">${data.anomalies.length}</strong></div>
</div>
<h2>最近调用</h2>
${table(["时间", "源", "端点", "结果", "耗时"], data.recent_calls.map((row) => [
esc(row.created_at), esc(row.provider), esc(row.endpoint),
row.ok ? '<span class="ok">成功</span>' : `<span class="fail">${esc(row.error)}</span>`,
`${row.latency_ms ?? "-"} ms`,
]))}
`;
return;
}
if (state.page === "sources") {
const data = await api("/admin/api/sources");
page.innerHTML = `<h2>数据源</h2>` + table(
["源", "角色", "状态", "凭据", "操作"],
data.items.map((item) => {
const cred = item.credential || {};
const credText = cred.configured ? `已配置 · ${esc(cred.last4 || "****")}` : "未配置";
return [
esc(item.provider),
esc(item.role),
esc((item.health && (item.health.state || item.health.status)) || "-"),
credText,
`<button data-probe="${esc(item.provider)}">探测一次</button>`,
];
}),
);
page.querySelectorAll("[data-probe]").forEach((btn) => {
btn.addEventListener("click", async () => {
const result = await api(`/admin/api/sources/${btn.dataset.probe}/probe`, { method: "POST", body: "{}" });
alert(JSON.stringify(result));
render();
});
});
return;
}
if (state.page === "jobs") {
const data = await api("/admin/api/jobs");
page.innerHTML = `
<h2>调度任务</h2>
${table(["任务", "时刻", "操作"], data.jobs.map((job) => [
`${esc(job.id)} · ${esc(job.title)}`, esc(job.at),
`<button data-run="${esc(job.id)}">手动触发</button>`,
]))}
<h3>最近运行</h3>
${table(["ID", "任务", "状态", "开始", "结束", "错误"], data.runs.map((row) => [
row.id, esc(row.job_id), esc(row.state), esc(row.started_at), esc(row.finished_at), esc(row.error),
]))}
`;
page.querySelectorAll("[data-run]").forEach((btn) => {
btn.addEventListener("click", async () => {
const date = prompt("交易日 YYYYMMDD(可留空=今天)", "") || "";
await api(`/admin/api/jobs/${btn.dataset.run}/run`, { method: "POST", body: JSON.stringify({ trade_date: date }) });
render();
});
});
return;
}
if (state.page === "release") {
const date = new Date().toISOString().slice(0, 10).replace(/-/g, "");
const data = await api(`/admin/api/batches?date=${date}`);
page.innerHTML = `
<h2>盘后发布 ${esc(data.trade_date)}</h2>
<div class="toolbar">
<label>日期 <input id="rel-date" value="${esc(data.trade_date)}" /></label>
<button type="button" id="rel-load">查看</button>
<button type="button" id="rel-backfill">补数</button>
</div>
<h3>当前映射</h3>
${table(["数据集", "活跃批次", "上一批次", "状态", "发布时间", "操作"], data.publications.map((row) => [
esc(row.dataset), esc(row.active_batch), esc(row.prev_batch), esc(row.state), esc(row.published_at),
row.prev_batch ? `<button class="danger" data-rollback="${esc(row.dataset)}">回滚</button>` : "-",
]))}
<h3>批次</h3>
${table(["batch_id", "数据集", "状态", "行数", "错误"], data.batches.map((row) => [
esc(row.batch_id), esc(row.dataset), esc(row.state), row.rows_out ?? "", esc(row.error),
]))}
`;
$bindRelease(page);
return;
}
if (state.page === "datasets") {
const data = await api("/admin/api/datasets?date=");
page.innerHTML = `
<h2>数据集 / 质量 ${esc(data.trade_date)}</h2>
${table(["数据集", "批次", "状态", "发布时间"], data.publications.map((row) => [
esc(row.dataset), esc(row.active_batch), esc(row.state), esc(row.published_at),
]))}
<h3>源间差异</h3>
${table(["指标", "左", "右", "偏差", "样本"], data.diff_reports.map((row) => [
esc(row.metric), esc(row.left_value), esc(row.right_value), esc(row.deviation), row.sample_count ?? "",
]))}
`;
return;
}
if (state.page === "audit") {
const data = await api("/admin/api/audit");
page.innerHTML = `<h2>审计</h2>` + table(
["时间", "操作者", "动作", "对象", "详情"],
data.items.map((row) => [esc(row.created_at), esc(row.actor), esc(row.action), esc(row.target), esc(row.detail)]),
);
}
}
function $bindRelease(page) {
page.querySelector("#rel-load").addEventListener("click", async () => {
const date = page.querySelector("#rel-date").value;
const data = await api(`/admin/api/batches?date=${encodeURIComponent(date)}`);
state.page = "release";
// re-render with fetched date by writing location hash
history.replaceState(null, "", `#release-${date}`);
$("page").innerHTML = renderRelease(data);
$bindRelease($("page"));
});
page.querySelector("#rel-backfill").addEventListener("click", () => dangerous("backfill"));
page.querySelectorAll("[data-rollback]").forEach((btn) => {
btn.addEventListener("click", () => dangerous("rollback", btn.dataset.rollback));
});
}
function renderRelease(data) {
return `
<h2>盘后发布 ${esc(data.trade_date)}</h2>
<div class="toolbar">
<label>日期 <input id="rel-date" value="${esc(data.trade_date)}" /></label>
<button type="button" id="rel-load">查看</button>
<button type="button" id="rel-backfill">补数</button>
</div>
<h3>当前映射</h3>
${table(["数据集", "活跃批次", "上一批次", "状态", "发布时间", "操作"], data.publications.map((row) => [
esc(row.dataset), esc(row.active_batch), esc(row.prev_batch), esc(row.state), esc(row.published_at),
row.prev_batch ? `<button class="danger" data-rollback="${esc(row.dataset)}">回滚</button>` : "-",
]))}
<h3>批次</h3>
${table(["batch_id", "数据集", "状态", "行数", "错误"], data.batches.map((row) => [
esc(row.batch_id), esc(row.dataset), esc(row.state), row.rows_out ?? "", esc(row.error),
]))}
`;
}
async function dangerous(kind, dataset) {
const date = ($("rel-date") && $("rel-date").value) || "";
const ds = dataset || prompt("数据集(daily / valuation / moneyflow / auction / index_daily / reference", "daily");
if (!ds) return;
const password = prompt("二次确认:输入管理密码");
if (!password) return;
const confirmWord = `${ds}:${date}`;
const typed = prompt(`请输入确认词:${confirmWord}`);
const path = kind === "rollback" ? "/admin/api/rollback" : "/admin/api/backfill";
await api(path, {
method: "POST",
body: JSON.stringify({ dataset: ds, trade_date: date, password, confirm: typed }),
});
render();
}
boot();
+53
View File
@@ -0,0 +1,53 @@
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>xiaobai-datahub 管理后台</title>
<link rel="stylesheet" href="/admin/styles.css" />
</head>
<body>
<div id="app">
<section id="login-view" class="panel auth-panel">
<h1>数据中枢</h1>
<p class="muted">内网管理后台,用于查看源状态、调度和盘后发布批次。</p>
<form id="login-form">
<label>账号 <input name="username" value="hub_admin" autocomplete="username" /></label>
<label>密码 <input name="password" type="password" autocomplete="current-password" /></label>
<button type="submit">登录</button>
<p id="login-error" class="error" hidden></p>
</form>
</section>
<section id="change-view" class="panel auth-panel" hidden>
<h1>修改初始密码</h1>
<form id="change-form">
<label>当前密码 <input name="current" type="password" /></label>
<label>新密码(至少 8 位) <input name="new_password" type="password" /></label>
<button type="submit">保存并继续</button>
<p id="change-error" class="error" hidden></p>
</form>
</section>
<section id="shell" hidden>
<header class="top">
<strong>xiaobai-datahub</strong>
<span id="phase" class="pill"></span>
<span id="who" class="muted"></span>
<button type="button" id="theme-btn" class="ghost">夜间</button>
<button type="button" id="logout-btn" class="ghost">退出</button>
</header>
<nav>
<button data-page="overview" class="active">总览</button>
<button data-page="sources">数据源</button>
<button data-page="jobs">调度任务</button>
<button data-page="release">盘后发布</button>
<button data-page="datasets">数据集</button>
<button data-page="audit">审计</button>
</nav>
<main id="page"></main>
</section>
</div>
<script src="/admin/app.js"></script>
</body>
</html>
+51
View File
@@ -0,0 +1,51 @@
:root {
color-scheme: light;
--bg: #f4f5f7;
--surface: #ffffff;
--text: #1f2329;
--muted: #646a73;
--line: #dee0e3;
--action: #3370ff;
--danger: #e04536;
--ok: #16a34a;
--warn: #b45309;
--radius: 8px;
--pad: 16px;
font-family: "Segoe UI", "PingFang SC", "Noto Sans SC", sans-serif;
}
:root[data-theme="night"] {
color-scheme: dark;
--bg: #111318;
--surface: #1b1e24;
--text: #e8eaed;
--muted: #9aa0a6;
--line: #2a2f38;
--action: #5b8cff;
}
* { box-sizing: border-box; }
body { margin: 0; background: var(--bg); color: var(--text); }
.panel, header.top, nav, main { background: var(--surface); }
.auth-panel { max-width: 420px; margin: 12vh auto; padding: 28px; border-radius: var(--radius); border: 1px solid var(--line); }
label { display: block; margin: 12px 0; }
input, select { width: 100%; padding: 8px 10px; border: 1px solid var(--line); border-radius: 4px; background: var(--bg); color: var(--text); }
button { background: var(--action); color: #fff; border: 0; border-radius: 4px; padding: 8px 14px; cursor: pointer; }
button.ghost { background: transparent; color: var(--text); border: 1px solid var(--line); }
button.danger { background: var(--danger); }
.muted { color: var(--muted); }
.error { color: var(--danger); }
.top { display: flex; gap: 12px; align-items: center; padding: 10px var(--pad); border-bottom: 1px solid var(--line); }
nav { display: flex; gap: 4px; padding: 8px var(--pad); border-bottom: 1px solid var(--line); }
nav button { background: transparent; color: var(--muted); }
nav button.active { color: var(--action); background: transparent; font-weight: 600; }
main { padding: var(--pad); min-height: calc(100vh - 96px); }
.cards { display: grid; grid-template-columns: repeat(auto-fit, minmax(180px, 1fr)); gap: 12px; margin-bottom: 16px; }
.card { border: 1px solid var(--line); border-radius: var(--radius); padding: 12px; }
table { width: 100%; border-collapse: collapse; font-size: 13px; }
th, td { text-align: left; padding: 8px; border-bottom: 1px solid var(--line); vertical-align: top; }
.pill { font-size: 12px; padding: 2px 8px; border-radius: 999px; border: 1px solid var(--line); }
.ok { color: var(--ok); }
.warn { color: var(--warn); }
.fail { color: var(--danger); }
.toolbar { display: flex; gap: 8px; flex-wrap: wrap; margin: 12px 0; align-items: end; }
.toolbar label { margin: 0; }
dialog { border: 1px solid var(--line); border-radius: var(--radius); background: var(--surface); color: var(--text); padding: 20px; }
+39
View File
@@ -0,0 +1,39 @@
services:
xiaobai-datahub:
build:
context: .
dockerfile: Dockerfile
image: xiaobai-datahub:local
container_name: xiaobai-datahub
ports:
- "0.0.0.0:8766:8766/tcp"
env_file:
- ./.env
environment:
DATAHUB_ENCRYPTION_KEY: "${DATAHUB_ENCRYPTION_KEY:?DATAHUB_ENCRYPTION_KEY must be set}"
DATAHUB_TOKEN: "${DATAHUB_TOKEN:?DATAHUB_TOKEN must be set}"
DATAHUB_ADMIN_PASSWORD: "${DATAHUB_ADMIN_PASSWORD:?DATAHUB_ADMIN_PASSWORD must be set}"
TUSHARE_TOKEN: "${TUSHARE_TOKEN:-}"
DATAHUB_DB_PATH: /app/data/datahub.db
DATAHUB_BACKUP_DIR: /app/data/backups
TZ: Asia/Shanghai
PYTHONUTF8: "1"
volumes:
- type: bind
source: ./data
target: /app/data
restart: unless-stopped
init: true
read_only: true
tmpfs:
- /tmp:size=64m,mode=1777
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
stop_grace_period: 30s
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
@@ -0,0 +1,15 @@
{
"daily_row_ratio": 0.98,
"null_rate_max": 0.01,
"cross_check_price_deviation": 0.03,
"cross_check_outlier_ratio": 0.05,
"index_price_deviation": 0.005,
"max_publish_attempts": 5,
"staging_retain_days": 14,
"job_run_retain_days": 90,
"backup_retain": 14,
"publication_generations": 3,
"tushare_rate_per_minute": 300,
"list_limit_default": 5000,
"list_limit_max": 5000
}
+4
View File
@@ -0,0 +1,4 @@
"""xiaobai-datahub: independent market-data service for xiaobai-review."""
__version__ = "0.1.0"
SCHEMA_VERSION = 1
@@ -0,0 +1,15 @@
from datahub.adapters.akshare import ADAPTER as akshare
from datahub.adapters.eastmoney import ADAPTER as eastmoney
from datahub.adapters.ifind import ADAPTER as ifind
from datahub.adapters.tencent import ADAPTER as tencent
from datahub.adapters.ths import ADAPTER as ths
from datahub.adapters.xgb import ADAPTER as xgb
RESERVED = {
"eastmoney": eastmoney,
"tencent": tencent,
"ths": ths,
"xgb": xgb,
"akshare": akshare,
"ifind": ifind,
}
@@ -0,0 +1,3 @@
from datahub.adapters.base import ReservedAdapter
ADAPTER = ReservedAdapter("akshare")
+47
View File
@@ -0,0 +1,47 @@
from __future__ import annotations
from abc import ABC, abstractmethod
from typing import Any
class AdapterError(RuntimeError):
pass
class MarketAdapter(ABC):
"""Uniform adapter: probe / fetch / normalize. Realtime adapters may be stubs in P0."""
name: str = "base"
@abstractmethod
def probe(self) -> dict[str, Any]:
"""Liveness check. Must not leak credentials."""
@abstractmethod
def fetch(self, dataset: str, params: dict[str, Any]) -> list[dict[str, Any]]:
"""Return provider-native rows (pre-canonical)."""
@abstractmethod
def normalize(self, dataset: str, rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
"""Map provider-native rows onto hub canonical fields."""
class ReservedAdapter(MarketAdapter):
"""Placeholder for a later free/licensed source. Does not pull data in P0."""
def __init__(self, name: str) -> None:
self.name = name
def probe(self) -> dict[str, Any]:
return {
"provider": self.name,
"configured": False,
"state": "reserved",
"message": "适配器位已预留,本阶段不接入",
}
def fetch(self, dataset: str, params: dict[str, Any]) -> list[dict[str, Any]]:
raise AdapterError(f"{self.name} 适配器本阶段未接入")
def normalize(self, dataset: str, rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
return []
@@ -0,0 +1,3 @@
from datahub.adapters.base import ReservedAdapter
ADAPTER = ReservedAdapter("eastmoney")
@@ -0,0 +1,3 @@
from datahub.adapters.base import ReservedAdapter
ADAPTER = ReservedAdapter("ifind")
@@ -0,0 +1,3 @@
from datahub.adapters.base import ReservedAdapter
ADAPTER = ReservedAdapter("tencent")
+3
View File
@@ -0,0 +1,3 @@
from datahub.adapters.base import ReservedAdapter
ADAPTER = ReservedAdapter("ths")
+154
View File
@@ -0,0 +1,154 @@
from __future__ import annotations
import json
import time
import urllib.error
import urllib.request
from typing import Any, Callable
from datahub.adapters.base import AdapterError, MarketAdapter
from datahub.normalize import (
normalize_auction,
normalize_calendar,
normalize_daily,
normalize_index_daily,
normalize_moneyflow,
normalize_stock,
normalize_valuation,
)
TUSHARE_URL = "http://api.tushare.pro"
TUSHARE_FIELDS = {
"trade_cal": "exchange,cal_date,is_open,pretrade_date",
"stock_basic": "ts_code,symbol,name,area,industry,market,list_status,list_date",
"daily": "ts_code,trade_date,open,high,low,close,pct_chg,vol,amount",
"daily_basic": "ts_code,trade_date,turnover_rate,volume_ratio,total_mv,circ_mv,pe_ttm,pb,ps_ttm,dv_ttm",
"adj_factor": "ts_code,trade_date,adj_factor",
"index_daily": "ts_code,trade_date,open,high,low,close,pct_chg,vol,amount",
"moneyflow": (
"ts_code,trade_date,buy_sm_amount,sell_sm_amount,buy_md_amount,sell_md_amount,"
"buy_lg_amount,sell_lg_amount,buy_elg_amount,sell_elg_amount,net_mf_amount"
),
"stk_auction": "ts_code,trade_date,vol,price,amount,pre_close,turnover_rate,volume_ratio,float_share",
}
DATASET_API = {
"calendar": "trade_cal",
"stocks": "stock_basic",
"daily": "daily",
"valuation": "daily_basic",
"adj_factor": "adj_factor",
"index_daily": "index_daily",
"moneyflow": "moneyflow",
"auction": "stk_auction",
}
DEFAULT_INDEX_CODES = ("000001.SH", "399001.SZ", "399006.SZ", "000300.SH")
class TushareAdapter(MarketAdapter):
name = "tushare"
def __init__(
self,
token: str,
timeout: int = 30,
transport: Callable[[str, dict[str, Any], str], list[dict[str, Any]]] | None = None,
) -> None:
self.token = token
self.timeout = timeout
self._transport = transport
def probe(self) -> dict[str, Any]:
if not self.token:
return {"provider": self.name, "configured": False, "state": "unconfigured"}
started = time.perf_counter()
try:
rows = self.fetch("calendar", {"exchange": "SSE", "start_date": "20200102", "end_date": "20200102"})
except AdapterError as exc:
return {
"provider": self.name,
"configured": True,
"state": "error",
"message": str(exc),
"latency_ms": round((time.perf_counter() - started) * 1000),
}
return {
"provider": self.name,
"configured": True,
"state": "ok" if rows else "empty",
"latency_ms": round((time.perf_counter() - started) * 1000),
}
def fetch(self, dataset: str, params: dict[str, Any]) -> list[dict[str, Any]]:
api_name = DATASET_API.get(dataset, dataset)
fields = TUSHARE_FIELDS.get(api_name, "")
query_params = dict(params)
if api_name == "stock_basic" and "list_status" not in query_params:
query_params["list_status"] = "L"
if api_name == "trade_cal" and "exchange" not in query_params:
query_params["exchange"] = "SSE"
if api_name == "index_daily" and "ts_code" not in query_params:
# Caller typically loops codes; a missing code would pull nothing useful.
query_params.setdefault("ts_code", DEFAULT_INDEX_CODES[0])
return self._query(api_name, query_params, fields)
def fetch_index_daily(self, trade_date: str, codes: tuple[str, ...] = DEFAULT_INDEX_CODES) -> list[dict[str, Any]]:
rows: list[dict[str, Any]] = []
for ts_code in codes:
rows.extend(self.fetch("index_daily", {"ts_code": ts_code, "trade_date": trade_date}))
return rows
def normalize(self, dataset: str, rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
mapping = {
"calendar": normalize_calendar,
"trade_cal": normalize_calendar,
"stocks": normalize_stock,
"stock_basic": normalize_stock,
"daily": normalize_daily,
"valuation": normalize_valuation,
"daily_basic": normalize_valuation,
"moneyflow": normalize_moneyflow,
"auction": normalize_auction,
"stk_auction": normalize_auction,
"index_daily": normalize_index_daily,
}
fn = mapping.get(dataset)
if fn is None:
if dataset == "adj_factor":
return [
{
"ts_code": str(row.get("ts_code") or "").upper(),
"trade_date": str(row.get("trade_date") or ""),
"adj_factor": row.get("adj_factor"),
}
for row in rows
]
raise AdapterError(f"unsupported dataset: {dataset}")
return [fn(row) for row in rows]
def _query(self, api_name: str, params: dict[str, Any], fields: str) -> list[dict[str, Any]]:
if self._transport is not None:
return self._transport(api_name, params, fields)
if not self.token:
raise AdapterError("Tushare token 未配置")
payload = json.dumps(
{"api_name": api_name, "token": self.token, "params": params, "fields": fields}
).encode("utf-8")
request = urllib.request.Request(
TUSHARE_URL,
data=payload,
headers={"Content-Type": "application/json", "User-Agent": "XiaobaiDatahub/0.1"},
method="POST",
)
try:
with urllib.request.urlopen(request, timeout=self.timeout) as response:
result = json.loads(response.read().decode("utf-8"))
except (urllib.error.URLError, TimeoutError, json.JSONDecodeError) as exc:
raise AdapterError(f"Tushare request failed: {exc}") from exc
if result.get("code") != 0:
raise AdapterError(result.get("msg") or "Tushare returned an unknown error")
data = result.get("data") or {}
columns = data.get("fields") or []
return [dict(zip(columns, item)) for item in data.get("items") or []]
+3
View File
@@ -0,0 +1,3 @@
from datahub.adapters.base import ReservedAdapter
ADAPTER = ReservedAdapter("xgb")
+162
View File
@@ -0,0 +1,162 @@
from __future__ import annotations
import json
from typing import Any
from datahub.adapters import RESERVED
from datahub.auth import AuthService
from datahub.db import HubDB
from datahub.pipeline import Pipeline
from datahub.scheduler import Scheduler
from datahub.serving import ApiError
from datahub.timeutil import isoformat, now_shanghai, session_phase, yyyymmdd
class AdminAPI:
def __init__(self, db: HubDB, pipeline: Pipeline, scheduler: Scheduler, auth: AuthService) -> None:
self.db = db
self.pipeline = pipeline
self.scheduler = scheduler
self.auth = auth
def overview(self) -> dict[str, Any]:
today = yyyymmdd(now_shanghai())
cal = self.db.fetchone(
"SELECT is_open FROM trade_calendar WHERE exchange = 'SSE' AND cal_date = ?",
(today,),
)
is_open = bool(cal and int(cal["is_open"]) == 1)
pubs = self.db.fetchall("SELECT * FROM publications WHERE trade_date = ?", (today,))
failed = self.db.fetchall(
"SELECT * FROM batches WHERE trade_date = ? AND state IN ('failed','staged')",
(today,),
)
calls = self.db.fetchall(
"SELECT * FROM src_calls ORDER BY id DESC LIMIT 20",
)
return {
"trade_date": today,
"session_phase": session_phase(now_shanghai(), is_open),
"is_open_day": is_open,
"publications": pubs,
"anomalies": failed,
"recent_calls": _public_calls(calls),
"source_count": len(self.db.fetchall("SELECT provider FROM src_health")),
}
def sources(self) -> dict[str, Any]:
health = {f"{row['provider']}:{row['endpoint_class']}": row for row in self.db.fetchall("SELECT * FROM src_health")}
items = [
{
"provider": "tushare",
"role": "official",
"health": health.get("tushare:pro") or {"state": "unknown"},
"credential": self.auth.credential_status("tushare_token") or {"configured": bool(self.pipeline.adapter.token)},
}
]
for name, adapter in RESERVED.items():
items.append(
{
"provider": name,
"role": "reserved",
"health": adapter.probe(),
"credential": {"configured": False, "last4": "", "updated_at": ""},
}
)
# Prefer encrypted last4 if stored
cred = self.auth.credential_status("tushare_token")
if cred.get("configured"):
items[0]["credential"] = cred
elif self.pipeline.adapter.token:
from datahub.crypto import mask_secret
items[0]["credential"] = {"configured": True, "last4": mask_secret(self.pipeline.adapter.token), "updated_at": ""}
return {"items": items}
def probe(self, provider: str) -> dict[str, Any]:
if provider == "tushare":
return self.pipeline.adapter.probe()
adapter = RESERVED.get(provider)
if adapter is None:
raise ApiError("INVALID_ARGUMENT", f"unknown provider: {provider}")
return adapter.probe()
def jobs(self) -> dict[str, Any]:
runs = self.db.fetchall("SELECT * FROM job_runs ORDER BY id DESC LIMIT 100")
return {
"jobs": [
{"id": "precheck", "at": "08:45", "title": "盘前预检"},
{"id": "eod_a", "at": "15:05", "title": "盘后批 A daily/valuation/moneyflow/auction"},
{"id": "eod_b", "at": "15:10", "title": "盘后批 B index_daily"},
{"id": "cleanup", "at": "00:30", "title": "清理 staging / 日志"},
{"id": "backup", "at": "00:40", "title": "SQLite 备份"},
],
"runs": runs,
}
def run_job(self, job_id: str, trade_date: str) -> dict[str, Any]:
return self.scheduler.run_job(job_id, yyyymmdd(trade_date or now_shanghai()))
def batches(self, date: str, dataset: str = "") -> dict[str, Any]:
trade_date = yyyymmdd(date or now_shanghai())
if dataset:
rows = self.db.fetchall(
"SELECT * FROM batches WHERE trade_date = ? AND dataset = ? ORDER BY started_at",
(trade_date, dataset),
)
else:
rows = self.db.fetchall(
"SELECT * FROM batches WHERE trade_date = ? ORDER BY started_at",
(trade_date,),
)
pubs = self.db.fetchall("SELECT * FROM publications WHERE trade_date = ?", (trade_date,))
return {"trade_date": trade_date, "batches": rows, "publications": pubs}
def datasets(self, date: str) -> dict[str, Any]:
trade_date = yyyymmdd(date or now_shanghai())
pubs = self.db.fetchall("SELECT * FROM publications WHERE trade_date = ?", (trade_date,))
diffs = self.db.fetchall(
"SELECT * FROM diff_reports WHERE trade_date = ? ORDER BY id",
(trade_date,),
)
return {"trade_date": trade_date, "publications": pubs, "diff_reports": diffs}
def audit(self) -> dict[str, Any]:
return {"items": self.db.fetchall("SELECT * FROM audit_log ORDER BY id DESC LIMIT 200")}
def rollback(self, dataset: str, trade_date: str, password: str, confirm: str, actor: str) -> dict[str, Any]:
self._dangerous(password, confirm, f"{dataset}:{trade_date}")
result = self.pipeline.rollback(dataset, trade_date, actor=actor)
return result
def backfill(self, dataset: str, trade_date: str, password: str, confirm: str, actor: str) -> dict[str, Any]:
self._dangerous(password, confirm, f"{dataset}:{trade_date}")
if dataset == "reference":
result = self.pipeline.ingest_reference(trade_date)
else:
result = self.pipeline.run_dataset(dataset, trade_date)
self.pipeline.audit(actor, "backfill", f"{dataset}:{trade_date}", json.dumps({"ok": True}))
return result
def _dangerous(self, password: str, confirm: str, expected: str) -> None:
if not self.auth.confirm_password(password):
raise ApiError("UNAUTHORIZED", "二次确认密码错误")
if confirm.strip() != expected:
raise ApiError("INVALID_ARGUMENT", f"确认词必须为 {expected}")
def _public_calls(rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
out = []
for row in rows:
out.append(
{
"id": row["id"],
"provider": row["provider"],
"endpoint": row["endpoint"],
"ok": bool(row["ok"]),
"latency_ms": row["latency_ms"],
"error": row["error"],
"created_at": row["created_at"],
}
)
return out
+190
View File
@@ -0,0 +1,190 @@
from __future__ import annotations
import base64
import hashlib
import hmac
import os
import secrets
from datetime import timedelta
from typing import Any
from datahub.crypto import SecretVault, mask_secret
from datahub.db import HubDB
from datahub.timeutil import isoformat, now_shanghai
PBKDF2_ROUNDS = 200_000
SESSION_HOURS = 12
LOGIN_FAIL_LIMIT = 5
LOCK_MINUTES = 10
def hash_password(password: str, salt: bytes | None = None) -> tuple[str, str]:
raw_salt = salt or os.urandom(16)
digest = hashlib.pbkdf2_hmac("sha256", password.encode("utf-8"), raw_salt, PBKDF2_ROUNDS, dklen=32)
return (
base64.urlsafe_b64encode(raw_salt).decode("ascii"),
base64.urlsafe_b64encode(digest).decode("ascii"),
)
def verify_password(password: str, salt_text: str, expected_hash: str) -> bool:
try:
salt = base64.urlsafe_b64decode(salt_text.encode("ascii"))
_, actual = hash_password(password, salt)
except (ValueError, TypeError):
return False
return hmac.compare_digest(actual, expected_hash)
def token_hash(token: str) -> str:
return hashlib.sha256(token.encode("utf-8")).hexdigest()
class AuthService:
def __init__(self, db: HubDB, vault: SecretVault, api_token: str, admin_password: str) -> None:
self.db = db
self.vault = vault
self._bootstrap(api_token, admin_password)
def _bootstrap(self, api_token: str, admin_password: str) -> None:
if api_token:
existing = self.db.fetchone("SELECT token_hash FROM api_tokens WHERE name = ?", ("review",))
hashed = token_hash(api_token)
last4 = mask_secret(api_token)
if existing is None:
self.db.execute(
"INSERT INTO api_tokens(token_hash, name, last4, created_at) VALUES (?,?,?,?)",
(hashed, "review", last4, isoformat()),
)
elif existing["token_hash"] != hashed:
self.db.execute(
"UPDATE api_tokens SET token_hash = ?, last4 = ? WHERE name = ?",
(hashed, last4, "review"),
)
admin = self.db.fetchone("SELECT id FROM hub_admin WHERE username = ?", ("hub_admin",))
if admin is None and admin_password:
salt, hashed = hash_password(admin_password)
now = isoformat()
self.db.execute(
"""
INSERT INTO hub_admin(username, password_salt, password_hash, password_must_change, created_at, updated_at)
VALUES (?, ?, ?, 1, ?, ?)
""",
("hub_admin", salt, hashed, now, now),
)
def check_api_token(self, supplied: str) -> bool:
if not supplied:
return False
row = self.db.fetchone(
"SELECT token_hash FROM api_tokens WHERE token_hash = ? AND revoked_at IS NULL",
(token_hash(supplied),),
)
return row is not None
def login(self, username: str, password: str) -> dict[str, Any]:
user = self.db.fetchone("SELECT * FROM hub_admin WHERE username = ?", (username,))
if not user:
raise PermissionError("账号或密码错误")
now = now_shanghai()
locked_until = user.get("locked_until")
if locked_until:
try:
from datetime import datetime
if datetime.fromisoformat(str(locked_until)) > now:
raise PermissionError("账号已锁定,请稍后再试")
except ValueError:
pass
if not verify_password(password, str(user["password_salt"]), str(user["password_hash"])):
fails = int(user["failed_attempts"] or 0) + 1
lock = isoformat(now + timedelta(minutes=LOCK_MINUTES)) if fails >= LOGIN_FAIL_LIMIT else None
self.db.execute(
"UPDATE hub_admin SET failed_attempts = ?, locked_until = ? WHERE id = ?",
(fails, lock, user["id"]),
)
raise PermissionError("账号或密码错误")
self.db.execute(
"UPDATE hub_admin SET failed_attempts = 0, locked_until = NULL WHERE id = ?",
(user["id"],),
)
session = secrets.token_urlsafe(32)
csrf = secrets.token_urlsafe(24)
expires = isoformat(now + timedelta(hours=SESSION_HOURS))
self.db.execute(
"INSERT INTO hub_sessions(token_hash, csrf_token, expires_at, created_at) VALUES (?,?,?,?)",
(token_hash(session), csrf, expires, isoformat(now)),
)
return {
"session": session,
"csrf": csrf,
"must_change": bool(user["password_must_change"]),
"expires_at": expires,
}
def session_user(self, raw_token: str) -> dict[str, Any] | None:
if not raw_token:
return None
row = self.db.fetchone(
"SELECT * FROM hub_sessions WHERE token_hash = ?",
(token_hash(raw_token),),
)
if not row:
return None
if str(row["expires_at"]) < isoformat():
self.db.execute("DELETE FROM hub_sessions WHERE token_hash = ?", (row["token_hash"],))
return None
admin = self.db.fetchone("SELECT username, password_must_change FROM hub_admin WHERE username = ?", ("hub_admin",))
return {
"username": (admin or {}).get("username") or "hub_admin",
"csrf_token": row["csrf_token"],
"must_change": bool((admin or {}).get("password_must_change")),
"token_hash": row["token_hash"],
}
def logout(self, raw_token: str) -> None:
if raw_token:
self.db.execute("DELETE FROM hub_sessions WHERE token_hash = ?", (token_hash(raw_token),))
def change_password(self, current: str, new_password: str) -> None:
if len(new_password) < 8:
raise ValueError("新密码至少 8 位")
user = self.db.fetchone("SELECT * FROM hub_admin WHERE username = ?", ("hub_admin",))
if not user or not verify_password(current, str(user["password_salt"]), str(user["password_hash"])):
raise PermissionError("当前密码错误")
salt, hashed = hash_password(new_password)
self.db.execute(
"UPDATE hub_admin SET password_salt=?, password_hash=?, password_must_change=0, updated_at=? WHERE id=?",
(salt, hashed, isoformat(), user["id"]),
)
def confirm_password(self, password: str) -> bool:
user = self.db.fetchone("SELECT * FROM hub_admin WHERE username = ?", ("hub_admin",))
if not user:
return False
return verify_password(password, str(user["password_salt"]), str(user["password_hash"]))
def credential_status(self, name: str) -> dict[str, Any]:
row = self.db.fetchone("SELECT last4, updated_at FROM credentials WHERE name = ?", (name,))
if not row:
return {"configured": False, "last4": "", "updated_at": ""}
return {"configured": True, "last4": row["last4"], "updated_at": row["updated_at"]}
def store_credential(self, name: str, secret: str) -> None:
payload = self.vault.encrypt_json({name: secret})
self.db.execute(
"""
INSERT INTO credentials(name, encrypted_payload, last4, updated_at)
VALUES (?, ?, ?, ?)
ON CONFLICT(name) DO UPDATE SET
encrypted_payload=excluded.encrypted_payload, last4=excluded.last4, updated_at=excluded.updated_at
""",
(name, payload, mask_secret(secret), isoformat()),
)
def load_credential(self, name: str) -> str:
row = self.db.fetchone("SELECT encrypted_payload FROM credentials WHERE name = ?", (name,))
if not row:
return ""
data = self.vault.decrypt_json(str(row["encrypted_payload"]))
return str(data.get(name) or "")
+26
View File
@@ -0,0 +1,26 @@
from __future__ import annotations
from datahub.db import HubDB
def resolve_code(db: HubDB, raw: str) -> str | None:
text = str(raw or "").strip().upper()
if not text:
return None
if "." in text:
row = db.fetchone("SELECT ts_code FROM stock_master WHERE ts_code = ?", (text,))
if row:
return row["ts_code"]
# indices are not always in stock_master
return text
matches = db.fetchall(
"SELECT ts_code FROM stock_master WHERE symbol = ? OR ts_code LIKE ?",
(text, f"{text}.%"),
)
if len(matches) == 1:
return matches[0]["ts_code"]
if len(matches) > 1:
return None
# unique exchange guess for 6-digit codes
suffix = "SH" if text.startswith("6") or text.startswith("9") else "SZ" if text.startswith(("0", "3")) else "BJ"
return f"{text}.{suffix}"
+42
View File
@@ -0,0 +1,42 @@
from __future__ import annotations
import json
from typing import Any
from cryptography.fernet import Fernet, InvalidToken
class SecretVault:
def __init__(self, key: str) -> None:
try:
self._fernet = Fernet(key.encode("ascii"))
except (ValueError, TypeError) as exc:
raise ValueError("DATAHUB_ENCRYPTION_KEY 格式无效。") from exc
@staticmethod
def generate_key() -> str:
return Fernet.generate_key().decode("ascii")
def encrypt_json(self, payload: dict[str, Any]) -> str:
raw = json.dumps(payload, ensure_ascii=False, separators=(",", ":")).encode("utf-8")
return self._fernet.encrypt(raw).decode("ascii")
def decrypt_json(self, token: str) -> dict[str, Any]:
if not token:
return {}
try:
payload = json.loads(self._fernet.decrypt(token.encode("ascii")).decode("utf-8"))
except (InvalidToken, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ValueError("凭据无法解密,请检查 DATAHUB_ENCRYPTION_KEY。") from exc
if not isinstance(payload, dict):
raise ValueError("凭据格式无效。")
return payload
def mask_secret(value: str, last_n: int = 4) -> str:
text = str(value or "")
if not text:
return ""
if len(text) <= last_n:
return "*" * len(text)
return ("*" * max(4, len(text) - last_n)) + text[-last_n:]
+340
View File
@@ -0,0 +1,340 @@
from __future__ import annotations
import sqlite3
import threading
from collections.abc import Iterator
from contextlib import contextmanager
from pathlib import Path
from typing import Any
from datahub.timeutil import isoformat
SCHEMA = """
CREATE TABLE IF NOT EXISTS schema_migrations (
version INTEGER PRIMARY KEY,
applied_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS credentials (
name TEXT PRIMARY KEY,
encrypted_payload TEXT NOT NULL,
last4 TEXT,
updated_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS hub_admin (
id INTEGER PRIMARY KEY,
username TEXT NOT NULL UNIQUE,
password_salt TEXT NOT NULL,
password_hash TEXT NOT NULL,
password_must_change INTEGER NOT NULL DEFAULT 1,
failed_attempts INTEGER NOT NULL DEFAULT 0,
locked_until TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS hub_sessions (
token_hash TEXT PRIMARY KEY,
csrf_token TEXT NOT NULL,
expires_at TEXT NOT NULL,
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS api_tokens (
token_hash TEXT PRIMARY KEY,
name TEXT NOT NULL,
last4 TEXT NOT NULL,
created_at TEXT NOT NULL,
revoked_at TEXT
);
CREATE TABLE IF NOT EXISTS trade_calendar (
exchange TEXT NOT NULL,
cal_date TEXT NOT NULL,
is_open INTEGER NOT NULL,
pretrade_date TEXT,
fetched_at TEXT NOT NULL,
PRIMARY KEY (exchange, cal_date)
);
CREATE TABLE IF NOT EXISTS stock_master (
ts_code TEXT PRIMARY KEY,
symbol TEXT,
name TEXT,
area TEXT,
industry TEXT,
market TEXT,
list_status TEXT,
list_date TEXT,
updated_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS eod_bars (
ts_code TEXT NOT NULL, trade_date TEXT NOT NULL,
open REAL, high REAL, low REAL, close REAL, pct_chg REAL,
volume REAL, amount REAL, adj_factor REAL,
batch_id TEXT NOT NULL,
PRIMARY KEY (ts_code, trade_date, batch_id)
) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS eod_valuation (
ts_code TEXT NOT NULL, trade_date TEXT NOT NULL,
turnover_rate REAL, volume_ratio REAL,
total_mv REAL, circ_mv REAL,
pe_ttm REAL, pb REAL, ps_ttm REAL, dv_ttm REAL,
batch_id TEXT NOT NULL,
PRIMARY KEY (ts_code, trade_date, batch_id)
) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS eod_moneyflow (
ts_code TEXT NOT NULL, trade_date TEXT NOT NULL,
buy_sm_amount REAL, sell_sm_amount REAL,
buy_md_amount REAL, sell_md_amount REAL,
buy_lg_amount REAL, sell_lg_amount REAL,
buy_elg_amount REAL, sell_elg_amount REAL,
net_mf_amount REAL,
batch_id TEXT NOT NULL,
PRIMARY KEY (ts_code, trade_date, batch_id)
) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS eod_auction (
ts_code TEXT NOT NULL, trade_date TEXT NOT NULL,
volume REAL, price REAL, amount REAL, pre_close REAL,
turnover_rate REAL, volume_ratio REAL, float_share REAL,
batch_id TEXT NOT NULL,
PRIMARY KEY (ts_code, trade_date, batch_id)
) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS eod_index_bars (
ts_code TEXT NOT NULL, trade_date TEXT NOT NULL,
open REAL, high REAL, low REAL, close REAL, pct_chg REAL,
volume REAL, amount REAL,
batch_id TEXT NOT NULL,
PRIMARY KEY (ts_code, trade_date, batch_id)
) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS staging_bars (
ts_code TEXT NOT NULL, trade_date TEXT NOT NULL, batch_id TEXT NOT NULL,
open REAL, high REAL, low REAL, close REAL, pct_chg REAL,
volume REAL, amount REAL, adj_factor REAL,
PRIMARY KEY (batch_id, ts_code, trade_date)
);
CREATE TABLE IF NOT EXISTS staging_valuation (
ts_code TEXT NOT NULL, trade_date TEXT NOT NULL, batch_id TEXT NOT NULL,
turnover_rate REAL, volume_ratio REAL,
total_mv REAL, circ_mv REAL, pe_ttm REAL, pb REAL, ps_ttm REAL, dv_ttm REAL,
PRIMARY KEY (batch_id, ts_code, trade_date)
);
CREATE TABLE IF NOT EXISTS staging_moneyflow (
ts_code TEXT NOT NULL, trade_date TEXT NOT NULL, batch_id TEXT NOT NULL,
buy_sm_amount REAL, sell_sm_amount REAL, buy_md_amount REAL, sell_md_amount REAL,
buy_lg_amount REAL, sell_lg_amount REAL, buy_elg_amount REAL, sell_elg_amount REAL,
net_mf_amount REAL,
PRIMARY KEY (batch_id, ts_code, trade_date)
);
CREATE TABLE IF NOT EXISTS staging_auction (
ts_code TEXT NOT NULL, trade_date TEXT NOT NULL, batch_id TEXT NOT NULL,
volume REAL, price REAL, amount REAL, pre_close REAL,
turnover_rate REAL, volume_ratio REAL, float_share REAL,
PRIMARY KEY (batch_id, ts_code, trade_date)
);
CREATE TABLE IF NOT EXISTS staging_index_bars (
ts_code TEXT NOT NULL, trade_date TEXT NOT NULL, batch_id TEXT NOT NULL,
open REAL, high REAL, low REAL, close REAL, pct_chg REAL,
volume REAL, amount REAL,
PRIMARY KEY (batch_id, ts_code, trade_date)
);
CREATE TABLE IF NOT EXISTS publications (
dataset TEXT NOT NULL, trade_date TEXT NOT NULL,
active_batch TEXT NOT NULL, prev_batch TEXT,
state TEXT NOT NULL,
published_at TEXT NOT NULL,
PRIMARY KEY (dataset, trade_date)
);
CREATE TABLE IF NOT EXISTS publication_history (
dataset TEXT NOT NULL, trade_date TEXT NOT NULL,
batch_id TEXT NOT NULL, published_at TEXT NOT NULL,
generation INTEGER NOT NULL,
PRIMARY KEY (dataset, trade_date, batch_id)
);
CREATE TABLE IF NOT EXISTS batches (
batch_id TEXT PRIMARY KEY,
dataset TEXT NOT NULL,
trade_date TEXT NOT NULL,
state TEXT NOT NULL,
attempt INTEGER DEFAULT 0,
rows_in INTEGER,
rows_out INTEGER,
quality_json TEXT,
started_at TEXT,
finished_at TEXT,
error TEXT
);
CREATE TABLE IF NOT EXISTS src_health (
provider TEXT NOT NULL, endpoint_class TEXT NOT NULL,
state TEXT NOT NULL,
last_ok_at TEXT, last_error TEXT,
consec_failures INTEGER DEFAULT 0,
opened_at TEXT,
cooldown_until TEXT,
PRIMARY KEY (provider, endpoint_class)
);
CREATE TABLE IF NOT EXISTS src_calls (
id INTEGER PRIMARY KEY AUTOINCREMENT,
provider TEXT NOT NULL,
endpoint TEXT NOT NULL,
ok INTEGER NOT NULL,
latency_ms INTEGER,
error TEXT,
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS job_runs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
job_id TEXT NOT NULL,
state TEXT NOT NULL,
started_at TEXT,
finished_at TEXT,
rows_in INTEGER,
rows_out INTEGER,
error TEXT,
attempt INTEGER DEFAULT 1,
detail TEXT
);
CREATE TABLE IF NOT EXISTS audit_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
actor TEXT NOT NULL,
action TEXT NOT NULL,
target TEXT,
detail TEXT,
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS rt_cache (
cache_key TEXT PRIMARY KEY,
payload TEXT NOT NULL,
source TEXT NOT NULL,
stored_at TEXT NOT NULL,
expires_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS last_known_good (
cache_key TEXT PRIMARY KEY,
payload TEXT NOT NULL,
source TEXT NOT NULL,
stored_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS diff_reports (
id INTEGER PRIMARY KEY AUTOINCREMENT,
trade_date TEXT NOT NULL,
metric TEXT NOT NULL,
left_source TEXT,
right_source TEXT,
left_value REAL,
right_value REAL,
deviation REAL,
sample_count INTEGER,
created_at TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_batches_date ON batches(trade_date, dataset);
CREATE INDEX IF NOT EXISTS idx_job_runs_job ON job_runs(job_id, started_at);
CREATE INDEX IF NOT EXISTS idx_src_calls_created ON src_calls(created_at);
CREATE INDEX IF NOT EXISTS idx_eod_bars_date ON eod_bars(trade_date, batch_id);
CREATE INDEX IF NOT EXISTS idx_calendar_open ON trade_calendar(is_open, cal_date);
"""
DATASET_TABLES = {
"daily": ("eod_bars", "staging_bars"),
"valuation": ("eod_valuation", "staging_valuation"),
"moneyflow": ("eod_moneyflow", "staging_moneyflow"),
"auction": ("eod_auction", "staging_auction"),
"index_daily": ("eod_index_bars", "staging_index_bars"),
}
class ManagedConnection(sqlite3.Connection):
def __exit__(self, exc_type, exc_value, traceback):
try:
return super().__exit__(exc_type, exc_value, traceback)
finally:
self.close()
class HubDB:
def __init__(self, path: Path, timeout_seconds: float = 20) -> None:
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
self.timeout_seconds = timeout_seconds
self._write_lock = threading.RLock()
self.initialize()
def connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(
self.path,
timeout=self.timeout_seconds,
factory=ManagedConnection,
)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA journal_mode=WAL")
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=20000")
connection.execute("PRAGMA synchronous=NORMAL")
return connection
def initialize(self) -> None:
with self.connect() as connection:
connection.executescript(SCHEMA)
row = connection.execute(
"SELECT version FROM schema_migrations ORDER BY version DESC LIMIT 1"
).fetchone()
if row is None:
connection.execute(
"INSERT INTO schema_migrations(version, applied_at) VALUES (1, ?)",
(isoformat(),),
)
@contextmanager
def write(self) -> Iterator[sqlite3.Connection]:
with self._write_lock:
with self.connect() as connection:
yield connection
def fetchall(self, sql: str, params: tuple[Any, ...] = ()) -> list[dict[str, Any]]:
with self.connect() as connection:
rows = connection.execute(sql, params).fetchall()
return [dict(row) for row in rows]
def fetchone(self, sql: str, params: tuple[Any, ...] = ()) -> dict[str, Any] | None:
with self.connect() as connection:
row = connection.execute(sql, params).fetchone()
return dict(row) if row else None
def execute(self, sql: str, params: tuple[Any, ...] = ()) -> None:
with self.write() as connection:
connection.execute(sql, params)
def executemany(self, sql: str, rows: list[tuple[Any, ...]]) -> None:
with self.write() as connection:
connection.executemany(sql, rows)
def backup_to(self, dest: Path) -> None:
dest.parent.mkdir(parents=True, exist_ok=True)
with self.connect() as source, sqlite3.connect(dest) as target:
source.backup(target)
def vacuum(self) -> None:
with self.connect() as connection:
connection.execute("VACUUM")
@@ -0,0 +1,13 @@
from datahub.governance.circuit import CircuitBreaker, CircuitState
from datahub.governance.lkg import LastKnownGood
from datahub.governance.ratelimit import TokenBucket
from datahub.governance.retry import RetryError, retry_call
__all__ = [
"CircuitBreaker",
"CircuitState",
"LastKnownGood",
"RetryError",
"TokenBucket",
"retry_call",
]
@@ -0,0 +1,107 @@
from __future__ import annotations
import threading
import time
from collections import deque
from dataclasses import dataclass
@dataclass
class CircuitState:
state: str = "closed" # closed | open | half_open
consec_failures: int = 0
opened_at: float | None = None
cooldown_until: float = 0.0
last_error: str = ""
last_ok_at: float | None = None
class CircuitBreaker:
"""Sliding-window breaker: 5 consecutive failures or >50% of 60s window → open."""
def __init__(
self,
failure_threshold: int = 5,
window_seconds: float = 60.0,
open_seconds: float = 120.0,
max_open_seconds: float = 600.0,
clock=time.monotonic,
) -> None:
self.failure_threshold = failure_threshold
self.window_seconds = window_seconds
self.open_seconds = open_seconds
self.max_open_seconds = max_open_seconds
self._clock = clock
self._lock = threading.Lock()
self._events: deque[tuple[float, bool]] = deque()
self.status = CircuitState()
self._open_stretch = open_seconds
def allow(self) -> bool:
with self._lock:
self._refresh_locked()
if self.status.state == "open":
return False
if self.status.state == "half_open":
# single probe in flight: caller must record success/failure
return True
return True
def record_success(self) -> CircuitState:
with self._lock:
now = self._clock()
self._events.append((now, True))
self.status.last_ok_at = now
self.status.consec_failures = 0
self.status.last_error = ""
self._open_stretch = self.open_seconds
self.status.state = "closed"
self.status.opened_at = None
self.status.cooldown_until = 0.0
return self._copy()
def record_failure(self, error: str = "") -> CircuitState:
with self._lock:
now = self._clock()
self._events.append((now, False))
self.status.consec_failures += 1
self.status.last_error = error
self._prune_locked(now)
failures = sum(1 for _, ok in self._events if not ok)
total = len(self._events)
rate = (failures / total) if total else 0.0
trip = self.status.consec_failures >= self.failure_threshold or (
total >= self.failure_threshold and rate > 0.5
)
if trip:
self.status.state = "open"
self.status.opened_at = now
self.status.cooldown_until = now + self._open_stretch
self._open_stretch = min(self.max_open_seconds, self._open_stretch * 2)
return self._copy()
def snapshot(self) -> CircuitState:
with self._lock:
self._refresh_locked()
return self._copy()
def _refresh_locked(self) -> None:
now = self._clock()
self._prune_locked(now)
if self.status.state == "open" and now >= self.status.cooldown_until:
self.status.state = "half_open"
def _prune_locked(self, now: float) -> None:
cutoff = now - self.window_seconds
while self._events and self._events[0][0] < cutoff:
self._events.popleft()
def _copy(self) -> CircuitState:
return CircuitState(
state=self.status.state,
consec_failures=self.status.consec_failures,
opened_at=self.status.opened_at,
cooldown_until=self.status.cooldown_until,
last_error=self.status.last_error,
last_ok_at=self.status.last_ok_at,
)
+84
View File
@@ -0,0 +1,84 @@
from __future__ import annotations
import json
from typing import Any
from datahub.db import HubDB
from datahub.timeutil import isoformat, now_shanghai
class LastKnownGood:
def __init__(self, db: HubDB) -> None:
self.db = db
def store(self, cache_key: str, payload: Any, source: str) -> None:
self.db.execute(
"""
INSERT INTO last_known_good(cache_key, payload, source, stored_at)
VALUES (?, ?, ?, ?)
ON CONFLICT(cache_key) DO UPDATE SET
payload=excluded.payload, source=excluded.source, stored_at=excluded.stored_at
""",
(cache_key, json.dumps(payload, ensure_ascii=False), source, isoformat()),
)
def load(self, cache_key: str) -> dict[str, Any] | None:
row = self.db.fetchone("SELECT * FROM last_known_good WHERE cache_key = ?", (cache_key,))
if not row:
return None
return {
"payload": json.loads(row["payload"]),
"source": row["source"],
"stored_at": row["stored_at"],
}
def put_rt(self, cache_key: str, payload: Any, source: str, ttl_seconds: int) -> None:
now = now_shanghai()
expires = isoformat(now.replace(microsecond=0))
# expires_at stored as iso; compute by adding ttl via timestamp
from datetime import timedelta
self.db.execute(
"""
INSERT INTO rt_cache(cache_key, payload, source, stored_at, expires_at)
VALUES (?, ?, ?, ?, ?)
ON CONFLICT(cache_key) DO UPDATE SET
payload=excluded.payload, source=excluded.source,
stored_at=excluded.stored_at, expires_at=excluded.expires_at
""",
(
cache_key,
json.dumps(payload, ensure_ascii=False),
source,
isoformat(now),
isoformat(now + timedelta(seconds=ttl_seconds)),
),
)
self.store(cache_key, payload, source)
def get_rt(self, cache_key: str, max_stale_seconds: int | None = None) -> dict[str, Any] | None:
row = self.db.fetchone("SELECT * FROM rt_cache WHERE cache_key = ?", (cache_key,))
if not row:
lkg = self.load(cache_key)
if not lkg:
return None
return {**lkg, "stale": True}
stored_at = row["stored_at"]
expired = row["expires_at"] < isoformat()
result = {
"payload": json.loads(row["payload"]),
"source": row["source"],
"stored_at": stored_at,
"stale": expired,
}
if expired and max_stale_seconds is not None:
from datetime import datetime
try:
stored = datetime.fromisoformat(stored_at)
age = (now_shanghai() - stored).total_seconds()
except ValueError:
age = max_stale_seconds + 1
if age > max_stale_seconds:
return None
return result
@@ -0,0 +1,36 @@
from __future__ import annotations
import threading
import time
class TokenBucket:
def __init__(self, rate_per_minute: float, capacity: float | None = None, clock=time.monotonic) -> None:
self.rate_per_second = max(0.001, rate_per_minute / 60.0)
self.capacity = float(capacity if capacity is not None else rate_per_minute)
self._tokens = self.capacity
self._updated = clock()
self._clock = clock
self._lock = threading.Lock()
def acquire(self, tokens: float = 1.0, block: bool = True) -> bool:
while True:
with self._lock:
now = self._clock()
elapsed = max(0.0, now - self._updated)
self._tokens = min(self.capacity, self._tokens + elapsed * self.rate_per_second)
self._updated = now
if self._tokens >= tokens:
self._tokens -= tokens
return True
wait = (tokens - self._tokens) / self.rate_per_second
if not block:
return False
time.sleep(min(wait, 0.05))
@property
def remaining(self) -> float:
with self._lock:
now = self._clock()
elapsed = max(0.0, now - self._updated)
return min(self.capacity, self._tokens + elapsed * self.rate_per_second)
@@ -0,0 +1,35 @@
from __future__ import annotations
import time
from collections.abc import Callable
from typing import TypeVar
T = TypeVar("T")
class RetryError(RuntimeError):
def __init__(self, message: str, attempts: int, last_error: BaseException | None = None) -> None:
super().__init__(message)
self.attempts = attempts
self.last_error = last_error
def retry_call(
fn: Callable[[], T],
attempts: int = 5,
base_delay: float = 0.2,
max_delay: float = 8.0,
sleeper: Callable[[float], None] = time.sleep,
retry_on: tuple[type[BaseException], ...] = (Exception,),
) -> T:
last: BaseException | None = None
for attempt in range(1, max(1, attempts) + 1):
try:
return fn()
except retry_on as exc:
last = exc
if attempt >= attempts:
break
delay = min(max_delay, base_delay * (2 ** (attempt - 1)))
sleeper(delay)
raise RetryError(f"retry exhausted after {attempts} attempts: {last}", attempts, last)
+242
View File
@@ -0,0 +1,242 @@
from __future__ import annotations
import json
import mimetypes
import secrets
from http import HTTPStatus
from http.cookies import SimpleCookie
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from typing import Any
from urllib.parse import unquote, urlparse
from datahub.hub import Hub
from datahub.logutil import configure_logging, get_logger
from datahub.serving import ApiError, parse_query
LOGGER = get_logger()
SESSION_COOKIE = "datahub_session"
class HubRequestHandler(BaseHTTPRequestHandler):
hub: Hub
def log_message(self, format: str, *args: Any) -> None:
LOGGER.info(format % args)
def do_GET(self) -> None: # noqa: N802
self._dispatch("GET")
def do_POST(self) -> None: # noqa: N802
self._dispatch("POST")
def do_OPTIONS(self) -> None: # noqa: N802
self.send_response(HTTPStatus.NO_CONTENT)
self.send_header("Allow", "GET, POST, OPTIONS")
self.end_headers()
def _dispatch(self, method: str) -> None:
parsed = urlparse(self.path)
path = unquote(parsed.path)
try:
if path in {"/livez", "/healthz"}:
self._json({"status": "ok"}, HTTPStatus.OK)
return
if path.startswith("/v1/"):
self._v1(path, parsed.query)
return
if path.startswith("/admin/api/"):
self._admin_api(method, path)
return
if path.startswith("/admin"):
self._admin_static(path)
return
if path == "/":
self.send_response(HTTPStatus.FOUND)
self.send_header("Location", "/admin/")
self.end_headers()
return
self._json({"error": {"code": "INVALID_ARGUMENT", "message": "Not found"}}, HTTPStatus.NOT_FOUND)
except ApiError as exc:
self._json(exc.payload(), exc.status)
except PermissionError as exc:
self._json({"error": {"code": "UNAUTHORIZED", "message": str(exc)}}, HTTPStatus.UNAUTHORIZED)
except ValueError as exc:
self._json({"error": {"code": "INVALID_ARGUMENT", "message": str(exc)}}, HTTPStatus.BAD_REQUEST)
except Exception:
LOGGER.exception("internal error")
self._json({"error": {"code": "INTERNAL", "message": "internal error"}}, HTTPStatus.INTERNAL_SERVER_ERROR)
def _v1(self, path: str, query: str) -> None:
token = self.headers.get("X-Datahub-Token", "")
if not self.hub.auth.check_api_token(token):
self.hub.pipeline.audit("anonymous", "unauthorized", path, "")
raise ApiError("UNAUTHORIZED", "missing or invalid X-Datahub-Token")
payload = self.hub.api.handle(path, parse_query(query))
self._json(payload, HTTPStatus.OK)
def _admin_api(self, method: str, path: str) -> None:
if path == "/admin/api/login" and method == "POST":
body = self._read_json()
result = self.hub.auth.login(str(body.get("username") or "hub_admin"), str(body.get("password") or ""))
self._json(
{"ok": True, "must_change": result["must_change"], "csrf": result["csrf"]},
HTTPStatus.OK,
extra_headers=[self._cookie(result["session"])],
)
return
user = self.hub.auth.session_user(self._cookie_value(SESSION_COOKIE))
if not user:
raise ApiError("UNAUTHORIZED", "请先登录")
if method == "POST" and path != "/admin/api/login":
csrf = self.headers.get("X-CSRF-Token", "")
if not csrf or not secrets.compare_digest(csrf, str(user["csrf_token"])):
raise ApiError("UNAUTHORIZED", "CSRF 校验失败")
if path == "/admin/api/logout" and method == "POST":
self.hub.auth.logout(self._cookie_value(SESSION_COOKIE))
self._json({"ok": True}, HTTPStatus.OK, extra_headers=[self._cookie("", clear=True)])
return
if path == "/admin/api/session" and method == "GET":
self._json({"username": user["username"], "must_change": user["must_change"], "csrf": user["csrf_token"]}, HTTPStatus.OK)
return
if path == "/admin/api/change-password" and method == "POST":
body = self._read_json()
self.hub.auth.change_password(str(body.get("current") or ""), str(body.get("new_password") or ""))
self.hub.pipeline.audit(user["username"], "change_password", "hub_admin", "")
self._json({"ok": True}, HTTPStatus.OK)
return
if user["must_change"] and path not in {"/admin/api/change-password", "/admin/api/session"}:
raise ApiError("UNAUTHORIZED", "请先修改初始密码")
if path == "/admin/api/overview" and method == "GET":
self._json(self.hub.admin.overview(), HTTPStatus.OK)
return
if path == "/admin/api/sources" and method == "GET":
self._json(self.hub.admin.sources(), HTTPStatus.OK)
return
if path.startswith("/admin/api/sources/") and path.endswith("/probe") and method == "POST":
provider = path.split("/")[4]
self._json(self.hub.admin.probe(provider), HTTPStatus.OK)
return
if path == "/admin/api/jobs" and method == "GET":
self._json(self.hub.admin.jobs(), HTTPStatus.OK)
return
if path.startswith("/admin/api/jobs/") and path.endswith("/run") and method == "POST":
job_id = path.split("/")[4]
body = self._read_json(allow_empty=True)
self._json(self.hub.admin.run_job(job_id, str(body.get("trade_date") or "")), HTTPStatus.OK)
return
if path == "/admin/api/batches" and method == "GET":
query = parse_query(urlparse(self.path).query)
date = (query.get("date") or [""])[0]
dataset = (query.get("dataset") or [""])[0]
self._json(self.hub.admin.batches(date, dataset), HTTPStatus.OK)
return
if path == "/admin/api/datasets" and method == "GET":
query = parse_query(urlparse(self.path).query)
self._json(self.hub.admin.datasets((query.get("date") or [""])[0]), HTTPStatus.OK)
return
if path == "/admin/api/audit" and method == "GET":
self._json(self.hub.admin.audit(), HTTPStatus.OK)
return
if path == "/admin/api/rollback" and method == "POST":
body = self._read_json()
result = self.hub.admin.rollback(
str(body.get("dataset") or ""),
str(body.get("trade_date") or ""),
str(body.get("password") or ""),
str(body.get("confirm") or ""),
user["username"],
)
self._json(result, HTTPStatus.OK)
return
if path == "/admin/api/backfill" and method == "POST":
body = self._read_json()
result = self.hub.admin.backfill(
str(body.get("dataset") or ""),
str(body.get("trade_date") or ""),
str(body.get("password") or ""),
str(body.get("confirm") or ""),
user["username"],
)
self._json(result, HTTPStatus.OK)
return
raise ApiError("INVALID_ARGUMENT", f"unknown admin endpoint: {path}")
def _admin_static(self, path: str) -> None:
relative = path[len("/admin"):].lstrip("/") or "index.html"
candidate = (self.hub.static_dir / relative).resolve()
try:
candidate.relative_to(self.hub.static_dir.resolve())
except ValueError:
self.send_error(HTTPStatus.FORBIDDEN)
return
if candidate.is_dir():
candidate = candidate / "index.html"
if not candidate.is_file():
candidate = self.hub.static_dir / "index.html"
content = candidate.read_bytes()
content_type = mimetypes.guess_type(candidate.name)[0] or "application/octet-stream"
if content_type.startswith("text/") or content_type in {"application/javascript", "application/json"}:
content_type += "; charset=utf-8"
self.send_response(HTTPStatus.OK)
self.send_header("Content-Type", content_type)
self.send_header("Content-Length", str(len(content)))
self.send_header("Cache-Control", "no-cache")
self.end_headers()
self.wfile.write(content)
def _read_json(self, allow_empty: bool = False) -> dict[str, Any]:
length = int(self.headers.get("Content-Length", "0") or 0)
if length == 0 and allow_empty:
return {}
if length <= 0 or length > 65536:
raise ValueError("请求内容为空或过大")
return json.loads(self.rfile.read(length).decode("utf-8"))
def _cookie_value(self, name: str) -> str:
cookie = SimpleCookie()
try:
cookie.load(self.headers.get("Cookie", ""))
except Exception:
return ""
morsel = cookie.get(name)
return morsel.value if morsel else ""
def _cookie(self, value: str, clear: bool = False) -> str:
max_age = 0 if clear else 12 * 3600
return f"{SESSION_COOKIE}={value}; Path=/; HttpOnly; SameSite=Strict; Max-Age={max_age}"
def _json(self, payload: dict[str, Any], status: HTTPStatus, extra_headers: list[str] | None = None) -> None:
raw = json.dumps(payload, ensure_ascii=False).encode("utf-8")
self.send_response(status)
self.send_header("Content-Type", "application/json; charset=utf-8")
self.send_header("Content-Length", str(len(raw)))
self.send_header("Cache-Control", "no-store")
for header in extra_headers or []:
self.send_header("Set-Cookie", header)
self.end_headers()
self.wfile.write(raw)
def make_handler(hub: Hub) -> type[HubRequestHandler]:
class BoundHandler(HubRequestHandler):
pass
BoundHandler.hub = hub
BoundHandler.protocol_version = "HTTP/1.1"
return BoundHandler
def serve(hub: Hub, host: str, port: int) -> None:
configure_logging(hub.settings.log_level)
handler = make_handler(hub)
server = ThreadingHTTPServer((host, port), handler)
hub.start()
LOGGER.info("xiaobai-datahub listening", extra={"hub": {"host": host, "port": port}})
print(f"xiaobai-datahub is running at http://{host}:{port}/admin/")
try:
server.serve_forever()
except KeyboardInterrupt:
pass
finally:
hub.stop()
server.server_close()
+54
View File
@@ -0,0 +1,54 @@
from __future__ import annotations
from pathlib import Path
from datahub.adapters.tushare import TushareAdapter
from datahub.admin_api import AdminAPI
from datahub.auth import AuthService
from datahub.crypto import SecretVault
from datahub.db import HubDB
from datahub.governance.circuit import CircuitBreaker
from datahub.governance.lkg import LastKnownGood
from datahub.governance.ratelimit import TokenBucket
from datahub.pipeline import Pipeline
from datahub.scheduler import Scheduler
from datahub.serving import V1API
from datahub.settings import Settings, load_settings
class Hub:
def __init__(self, settings: Settings, adapter: TushareAdapter | None = None) -> None:
if not settings.encryption_key:
raise SystemExit("DATAHUB_ENCRYPTION_KEY 未配置")
self.settings = settings
self.db = HubDB(settings.db_path)
self.vault = SecretVault(settings.encryption_key)
self.auth = AuthService(self.db, self.vault, settings.api_token, settings.admin_password)
token = settings.tushare_token or self.auth.load_credential("tushare_token")
if settings.tushare_token:
self.auth.store_credential("tushare_token", settings.tushare_token)
token = settings.tushare_token
self.adapter = adapter or TushareAdapter(token)
self.pipeline = Pipeline(
self.db,
self.adapter,
settings,
bucket=TokenBucket(settings.tushare_rate_per_minute),
breaker=CircuitBreaker(),
)
self.lkg = LastKnownGood(self.db)
self.scheduler = Scheduler(self.db, self.pipeline)
self.api = V1API(self.db, self.pipeline, settings)
self.admin = AdminAPI(self.db, self.pipeline, self.scheduler, self.auth)
self.static_dir = Path(__file__).resolve().parents[1] / "admin"
def start(self) -> None:
if self.settings.scheduler_enabled:
self.scheduler.start()
def stop(self) -> None:
self.scheduler.stop()
def build_hub(settings: Settings | None = None) -> Hub:
return Hub(settings or load_settings())
+56
View File
@@ -0,0 +1,56 @@
from __future__ import annotations
import json
import logging
import sys
from typing import Any
from datahub.timeutil import isoformat
_SECRET_KEYS = (
"token", "password", "secret", "key", "authorization", "credential",
"tushare_token", "datahub_token", "encryption_key", "cookie",
)
def _redact(value: Any, key: str = "") -> Any:
lowered = key.lower()
if any(part in lowered for part in _SECRET_KEYS):
return "***"
if isinstance(value, dict):
return {str(item_key): _redact(item_value, str(item_key)) for item_key, item_value in value.items()}
if isinstance(value, list):
return [_redact(item) for item in value]
return value
class JsonFormatter(logging.Formatter):
def format(self, record: logging.LogRecord) -> str:
payload: dict[str, Any] = {
"ts": isoformat(),
"level": record.levelname,
"logger": record.name,
"message": record.getMessage(),
}
extra = getattr(record, "hub", None)
if isinstance(extra, dict):
payload.update(_redact(extra))
if record.exc_info:
payload["exc"] = self.formatException(record.exc_info)
return json.dumps(payload, ensure_ascii=False, default=str)
def configure_logging(level: str = "INFO") -> logging.Logger:
logger = logging.getLogger("datahub")
if logger.handlers:
return logger
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(JsonFormatter())
logger.addHandler(handler)
logger.setLevel(getattr(logging, level.upper(), logging.INFO))
logger.propagate = False
return logger
def get_logger() -> logging.Logger:
return logging.getLogger("datahub")
+201
View File
@@ -0,0 +1,201 @@
"""Canonical field normalization for Tushare-native rows.
Units (architecture §7.1):
- price: 4 decimal REAL
- pct_chg: percent, 4 decimal REAL
- volume: shares (Tushare daily/index vol is ×100)
- amount: yuan (Tushare daily/index amount is 千元 ×1000)
- moneyflow amounts: yuan (Tushare is 万元 ×1e4)
- daily_basic total_mv / circ_mv: yuan (Tushare is 万元 ×1e4)
- stk_auction.amount is already yuan in Tushare; volume ×100
Existing xiaobai-review stores Tushare native units and converts at display time.
Hub converts once at ingest. Golden tests compare hub output against applying
these same factors to review-native rows.
"""
from __future__ import annotations
from typing import Any
from datahub.numbers import finite_number, round4
AMOUNT_THOUSAND_YUAN = 1000.0
AMOUNT_WAN_YUAN = 10000.0
VOLUME_LOT = 100.0
DAILY_FIELDS = ("ts_code", "trade_date", "open", "high", "low", "close", "pct_chg", "vol", "amount")
VALUATION_FIELDS = (
"ts_code", "trade_date", "turnover_rate", "volume_ratio",
"total_mv", "circ_mv", "pe_ttm", "pb", "ps_ttm", "dv_ttm",
)
MONEYFLOW_FIELDS = (
"ts_code", "trade_date",
"buy_sm_amount", "sell_sm_amount", "buy_md_amount", "sell_md_amount",
"buy_lg_amount", "sell_lg_amount", "buy_elg_amount", "sell_elg_amount",
"net_mf_amount",
)
AUCTION_FIELDS = (
"ts_code", "trade_date", "vol", "price", "amount", "pre_close",
"turnover_rate", "volume_ratio", "float_share",
)
INDEX_FIELDS = ("ts_code", "trade_date", "open", "high", "low", "close", "pct_chg", "vol", "amount")
CALENDAR_FIELDS = ("exchange", "cal_date", "is_open", "pretrade_date")
STOCK_FIELDS = ("ts_code", "symbol", "name", "area", "industry", "market", "list_status", "list_date")
def _code(value: Any) -> str:
return str(value or "").strip().upper()
def _date(value: Any) -> str:
return str(value or "").replace("-", "")[:8]
def review_daily_to_canonical(row: dict[str, Any]) -> dict[str, Any]:
"""Convert a review-stored daily row (Tushare native units) to hub canonical."""
return normalize_daily(row)
def normalize_daily(row: dict[str, Any], adj_factor: float | None = None) -> dict[str, Any]:
return {
"ts_code": _code(row.get("ts_code")),
"trade_date": _date(row.get("trade_date")),
"open": round4(finite_number(row.get("open"))),
"high": round4(finite_number(row.get("high"))),
"low": round4(finite_number(row.get("low"))),
"close": round4(finite_number(row.get("close"))),
"pct_chg": round4(finite_number(row.get("pct_chg"))),
"volume": round4(_scale(row.get("vol"), VOLUME_LOT)),
"amount": round4(_scale(row.get("amount"), AMOUNT_THOUSAND_YUAN)),
"adj_factor": round4(finite_number(adj_factor if adj_factor is not None else row.get("adj_factor"))),
}
def normalize_valuation(row: dict[str, Any]) -> dict[str, Any]:
return {
"ts_code": _code(row.get("ts_code")),
"trade_date": _date(row.get("trade_date")),
"turnover_rate": round4(finite_number(row.get("turnover_rate"))),
"volume_ratio": round4(finite_number(row.get("volume_ratio"))),
"total_mv": round4(_scale(row.get("total_mv"), AMOUNT_WAN_YUAN)),
"circ_mv": round4(_scale(row.get("circ_mv"), AMOUNT_WAN_YUAN)),
"pe_ttm": round4(finite_number(row.get("pe_ttm"))),
"pb": round4(finite_number(row.get("pb"))),
"ps_ttm": round4(finite_number(row.get("ps_ttm"))),
"dv_ttm": round4(finite_number(row.get("dv_ttm"))),
}
def normalize_moneyflow(row: dict[str, Any]) -> dict[str, Any]:
converted = {
"ts_code": _code(row.get("ts_code")),
"trade_date": _date(row.get("trade_date")),
}
for field in MONEYFLOW_FIELDS[2:]:
converted[field] = round4(_scale(row.get(field), AMOUNT_WAN_YUAN))
return converted
def normalize_auction(row: dict[str, Any]) -> dict[str, Any]:
return {
"ts_code": _code(row.get("ts_code")),
"trade_date": _date(row.get("trade_date")),
"volume": round4(_scale(row.get("vol") if row.get("vol") is not None else row.get("volume"), VOLUME_LOT)),
"price": round4(finite_number(row.get("price"))),
"amount": round4(finite_number(row.get("amount"))),
"pre_close": round4(finite_number(row.get("pre_close"))),
"turnover_rate": round4(finite_number(row.get("turnover_rate"))),
"volume_ratio": round4(finite_number(row.get("volume_ratio"))),
"float_share": round4(_scale(row.get("float_share"), AMOUNT_WAN_YUAN) if row.get("float_share") is not None else None),
}
def normalize_index_daily(row: dict[str, Any]) -> dict[str, Any]:
return {
"ts_code": _code(row.get("ts_code")),
"trade_date": _date(row.get("trade_date")),
"open": round4(finite_number(row.get("open"))),
"high": round4(finite_number(row.get("high"))),
"low": round4(finite_number(row.get("low"))),
"close": round4(finite_number(row.get("close"))),
"pct_chg": round4(finite_number(row.get("pct_chg"))),
"volume": round4(_scale(row.get("vol"), VOLUME_LOT)),
"amount": round4(_scale(row.get("amount"), AMOUNT_THOUSAND_YUAN)),
}
def normalize_calendar(row: dict[str, Any]) -> dict[str, Any]:
is_open = row.get("is_open")
if is_open in (True, "1", 1, "Y", "y"):
open_flag = 1
elif is_open in (False, "0", 0, "N", "n", None, ""):
open_flag = 0
else:
open_flag = int(is_open)
return {
"exchange": str(row.get("exchange") or "SSE"),
"cal_date": _date(row.get("cal_date") or row.get("calDate")),
"is_open": open_flag,
"pretrade_date": _date(row.get("pretrade_date")) or None,
}
def normalize_stock(row: dict[str, Any]) -> dict[str, Any]:
ts_code = _code(row.get("ts_code"))
symbol = str(row.get("symbol") or "").strip() or (ts_code.split(".")[0] if ts_code else "")
return {
"ts_code": ts_code,
"symbol": symbol,
"name": str(row.get("name") or "").strip(),
"area": str(row.get("area") or "").strip() or None,
"industry": str(row.get("industry") or "").strip() or None,
"market": str(row.get("market") or "").strip() or None,
"list_status": str(row.get("list_status") or "L").strip() or "L",
"list_date": _date(row.get("list_date")) or None,
}
def apply_qfq(price: float | None, factor: float | None, latest_factor: float | None) -> float | None:
if price is None:
return None
current = factor if factor not in (None, 0) else 1.0
latest = latest_factor if latest_factor not in (None, 0) else current
return round4(price * current / latest)
def qfq_bar(row: dict[str, Any], latest_factor: float | None) -> dict[str, Any]:
factor = finite_number(row.get("adj_factor"), 1.0) or 1.0
out = dict(row)
for field in ("open", "high", "low", "close"):
out[field] = apply_qfq(finite_number(row.get(field)), factor, latest_factor)
return out
NORMALIZERS = {
"daily": normalize_daily,
"valuation": normalize_valuation,
"daily_basic": normalize_valuation,
"moneyflow": normalize_moneyflow,
"auction": normalize_auction,
"stk_auction": normalize_auction,
"index_daily": normalize_index_daily,
"trade_cal": normalize_calendar,
"calendar": normalize_calendar,
"stock_basic": normalize_stock,
"stocks": normalize_stock,
}
def normalize_rows(dataset: str, rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
fn = NORMALIZERS.get(dataset)
if fn is None:
raise ValueError(f"unknown dataset: {dataset}")
return [fn(row) for row in rows]
def _scale(value: Any, factor: float) -> float | None:
number = finite_number(value)
if number is None:
return None
return number * factor
+23
View File
@@ -0,0 +1,23 @@
from __future__ import annotations
import math
from typing import Any
def finite_number(value: Any, default: float | None = None) -> float | None:
"""Return a finite float, or default (None means JSON null)."""
if value is None or value == "":
return default
try:
number = float(value)
except (TypeError, ValueError):
return default
if not math.isfinite(number):
return default
return number
def round4(value: float | None) -> float | None:
if value is None:
return None
return round(float(value), 4)
+480
View File
@@ -0,0 +1,480 @@
from __future__ import annotations
import json
import time
from collections.abc import Callable
from datetime import timedelta
from typing import Any
from datahub.adapters.base import AdapterError
from datahub.adapters.tushare import DEFAULT_INDEX_CODES, TushareAdapter
from datahub.db import DATASET_TABLES, HubDB
from datahub.governance.circuit import CircuitBreaker
from datahub.governance.ratelimit import TokenBucket
from datahub.governance.retry import RetryError, retry_call
from datahub.logutil import get_logger
from datahub.normalize import finite_number, normalize_daily
from datahub.settings import Settings
from datahub.timeutil import add_days, isoformat, now_shanghai, yyyymmdd
LOGGER = get_logger()
HARD_DATASETS = {"daily", "valuation", "index_daily"}
SOFT_DATASETS = {"moneyflow", "auction"}
STAGING_INSERT = {
"daily": (
"INSERT INTO staging_bars(ts_code,trade_date,batch_id,open,high,low,close,pct_chg,volume,amount,adj_factor) "
"VALUES (?,?,?,?,?,?,?,?,?,?,?)",
lambda r, b: (
r["ts_code"], r["trade_date"], b, r.get("open"), r.get("high"), r.get("low"),
r.get("close"), r.get("pct_chg"), r.get("volume"), r.get("amount"), r.get("adj_factor"),
),
),
"valuation": (
"INSERT INTO staging_valuation(ts_code,trade_date,batch_id,turnover_rate,volume_ratio,total_mv,circ_mv,pe_ttm,pb,ps_ttm,dv_ttm) "
"VALUES (?,?,?,?,?,?,?,?,?,?,?)",
lambda r, b: (
r["ts_code"], r["trade_date"], b, r.get("turnover_rate"), r.get("volume_ratio"),
r.get("total_mv"), r.get("circ_mv"), r.get("pe_ttm"), r.get("pb"), r.get("ps_ttm"), r.get("dv_ttm"),
),
),
"moneyflow": (
"INSERT INTO staging_moneyflow(ts_code,trade_date,batch_id,buy_sm_amount,sell_sm_amount,buy_md_amount,sell_md_amount,buy_lg_amount,sell_lg_amount,buy_elg_amount,sell_elg_amount,net_mf_amount) "
"VALUES (?,?,?,?,?,?,?,?,?,?,?,?)",
lambda r, b: (
r["ts_code"], r["trade_date"], b,
r.get("buy_sm_amount"), r.get("sell_sm_amount"), r.get("buy_md_amount"), r.get("sell_md_amount"),
r.get("buy_lg_amount"), r.get("sell_lg_amount"), r.get("buy_elg_amount"), r.get("sell_elg_amount"),
r.get("net_mf_amount"),
),
),
"auction": (
"INSERT INTO staging_auction(ts_code,trade_date,batch_id,volume,price,amount,pre_close,turnover_rate,volume_ratio,float_share) "
"VALUES (?,?,?,?,?,?,?,?,?,?)",
lambda r, b: (
r["ts_code"], r["trade_date"], b, r.get("volume"), r.get("price"), r.get("amount"),
r.get("pre_close"), r.get("turnover_rate"), r.get("volume_ratio"), r.get("float_share"),
),
),
"index_daily": (
"INSERT INTO staging_index_bars(ts_code,trade_date,batch_id,open,high,low,close,pct_chg,volume,amount) "
"VALUES (?,?,?,?,?,?,?,?,?,?)",
lambda r, b: (
r["ts_code"], r["trade_date"], b, r.get("open"), r.get("high"), r.get("low"),
r.get("close"), r.get("pct_chg"), r.get("volume"), r.get("amount"),
),
),
}
EOD_COPY = {
"daily": (
"INSERT OR REPLACE INTO eod_bars "
"SELECT ts_code,trade_date,open,high,low,close,pct_chg,volume,amount,adj_factor,batch_id "
"FROM staging_bars WHERE batch_id = ?"
),
"valuation": (
"INSERT OR REPLACE INTO eod_valuation "
"SELECT ts_code,trade_date,turnover_rate,volume_ratio,total_mv,circ_mv,pe_ttm,pb,ps_ttm,dv_ttm,batch_id "
"FROM staging_valuation WHERE batch_id = ?"
),
"moneyflow": (
"INSERT OR REPLACE INTO eod_moneyflow "
"SELECT ts_code,trade_date,buy_sm_amount,sell_sm_amount,buy_md_amount,sell_md_amount,"
"buy_lg_amount,sell_lg_amount,buy_elg_amount,sell_elg_amount,net_mf_amount,batch_id "
"FROM staging_moneyflow WHERE batch_id = ?"
),
"auction": (
"INSERT OR REPLACE INTO eod_auction "
"SELECT ts_code,trade_date,volume,price,amount,pre_close,turnover_rate,volume_ratio,float_share,batch_id "
"FROM staging_auction WHERE batch_id = ?"
),
"index_daily": (
"INSERT OR REPLACE INTO eod_index_bars "
"SELECT ts_code,trade_date,open,high,low,close,pct_chg,volume,amount,batch_id "
"FROM staging_index_bars WHERE batch_id = ?"
),
}
class QualityError(RuntimeError):
def __init__(self, message: str, report: dict[str, Any]) -> None:
super().__init__(message)
self.report = report
class Pipeline:
def __init__(
self,
db: HubDB,
adapter: TushareAdapter,
settings: Settings,
bucket: TokenBucket | None = None,
breaker: CircuitBreaker | None = None,
before_commit: Callable[[], None] | None = None,
clock=None,
) -> None:
self.db = db
self.adapter = adapter
self.settings = settings
self.bucket = bucket or TokenBucket(settings.tushare_rate_per_minute)
self.breaker = breaker or CircuitBreaker()
self.before_commit = before_commit
self.clock = clock or now_shanghai
def next_batch_id(self, dataset: str, trade_date: str) -> str:
row = self.db.fetchone(
"SELECT COUNT(*) AS n FROM batches WHERE dataset = ? AND trade_date = ?",
(dataset, trade_date),
)
seq = int((row or {}).get("n") or 0) + 1
return f"{trade_date}-{dataset}-{seq:03d}"
def ingest_reference(self, trade_date: str | None = None) -> dict[str, Any]:
"""Refresh trade calendar (window) and stock master. Not versioned by batch."""
day = yyyymmdd(trade_date or self.clock())
start = add_days(day, -400)
end = add_days(day, 30)
calendar = self.adapter.normalize(
"calendar",
self._guarded_fetch("calendar", {"exchange": "SSE", "start_date": start, "end_date": end}),
)
stocks = self.adapter.normalize("stocks", self._guarded_fetch("stocks", {"list_status": "L"}))
fetched_at = isoformat(self.clock())
with self.db.write() as connection:
for row in calendar:
connection.execute(
"""
INSERT INTO trade_calendar(exchange, cal_date, is_open, pretrade_date, fetched_at)
VALUES (?, ?, ?, ?, ?)
ON CONFLICT(exchange, cal_date) DO UPDATE SET
is_open=excluded.is_open, pretrade_date=excluded.pretrade_date, fetched_at=excluded.fetched_at
""",
(row["exchange"], row["cal_date"], row["is_open"], row.get("pretrade_date"), fetched_at),
)
for row in stocks:
connection.execute(
"""
INSERT INTO stock_master(ts_code,symbol,name,area,industry,market,list_status,list_date,updated_at)
VALUES (?,?,?,?,?,?,?,?,?)
ON CONFLICT(ts_code) DO UPDATE SET
symbol=excluded.symbol, name=excluded.name, area=excluded.area,
industry=excluded.industry, market=excluded.market,
list_status=excluded.list_status, list_date=excluded.list_date,
updated_at=excluded.updated_at
""",
(
row["ts_code"], row.get("symbol"), row.get("name"), row.get("area"),
row.get("industry"), row.get("market"), row.get("list_status"),
row.get("list_date"), fetched_at,
),
)
return {"calendar": len(calendar), "stocks": len(stocks), "trade_date": day}
def run_dataset(self, dataset: str, trade_date: str, attempts: int | None = None) -> dict[str, Any]:
trade_date = yyyymmdd(trade_date)
batch_id = self.next_batch_id(dataset, trade_date)
max_attempts = attempts or self.settings.max_publish_attempts
self._set_batch(batch_id, dataset, trade_date, "scheduled", 0)
try:
self._set_batch(batch_id, dataset, trade_date, "fetching", 1)
rows = retry_call(
lambda: self._fetch_dataset(dataset, trade_date),
attempts=max_attempts,
base_delay=0.05,
sleeper=lambda _d: None if attempts == 1 else time.sleep(_d),
)
self._stage(dataset, batch_id, rows)
self._set_batch(batch_id, dataset, trade_date, "staged", 1, rows_in=len(rows), rows_out=len(rows))
self._set_batch(batch_id, dataset, trade_date, "validating", 1)
report = self.validate(dataset, batch_id, trade_date, rows)
if report["hard_fail"]:
self._set_batch(
batch_id, dataset, trade_date, "staged", 1,
rows_in=len(rows), rows_out=len(rows),
quality=report, error="; ".join(report["errors"]),
)
raise QualityError("integrity gate failed", report)
self._set_batch(batch_id, dataset, trade_date, "deriving", 1, rows_in=len(rows), rows_out=len(rows), quality=report)
self._set_batch(batch_id, dataset, trade_date, "publishing", 1, rows_in=len(rows), rows_out=len(rows), quality=report)
state = "degraded" if report["soft_fail"] else "published"
self.publish(dataset, trade_date, batch_id, state=state)
self._set_batch(
batch_id, dataset, trade_date, "published", 1,
rows_in=len(rows), rows_out=len(rows), quality=report, finished=True,
)
return {"batch_id": batch_id, "dataset": dataset, "trade_date": trade_date, "rows": len(rows), "state": state, "quality": report}
except RetryError as exc:
self._set_batch(batch_id, dataset, trade_date, "failed", max_attempts, error=str(exc), finished=True)
raise
except QualityError:
raise
except Exception as exc:
self._set_batch(batch_id, dataset, trade_date, "failed", 1, error=str(exc), finished=True)
raise
def run_eod_batch_a(self, trade_date: str) -> dict[str, Any]:
results = {}
for dataset in ("daily", "valuation", "moneyflow", "auction"):
results[dataset] = self.run_dataset(dataset, trade_date)
return results
def run_eod_batch_b(self, trade_date: str) -> dict[str, Any]:
return {"index_daily": self.run_dataset("index_daily", trade_date)}
def validate(self, dataset: str, batch_id: str, trade_date: str, rows: list[dict[str, Any]]) -> dict[str, Any]:
quality = self.settings.quality
errors: list[str] = []
warnings: list[str] = []
listed = self.db.fetchone(
"SELECT COUNT(*) AS n FROM stock_master WHERE list_status = 'L'",
)
listed_n = int((listed or {}).get("n") or 0)
row_n = len(rows)
keys = [(row.get("ts_code"), row.get("trade_date")) for row in rows]
dup = row_n - len(set(keys))
if dup:
errors.append(f"duplicate keys: {dup}")
bad_date = sum(1 for row in rows if str(row.get("trade_date")) != trade_date)
if bad_date:
errors.append(f"date mismatch rows: {bad_date}")
ratio = (row_n / listed_n) if listed_n else 1.0
if dataset == "daily" and listed_n and ratio < float(quality.get("daily_row_ratio") or 0.98):
errors.append(f"row ratio {ratio:.4f} < {quality.get('daily_row_ratio')}")
null_fields = ("open", "high", "low", "close", "amount") if dataset in {"daily", "index_daily"} else ()
if null_fields and rows:
nulls = sum(1 for row in rows if any(row.get(field) is None for field in null_fields))
null_rate = nulls / row_n
if null_rate >= float(quality.get("null_rate_max") or 0.01):
errors.append(f"null rate {null_rate:.4f}")
if dataset in SOFT_DATASETS and row_n == 0:
warnings.append("empty soft dataset")
hard_fail = bool(errors) and dataset in HARD_DATASETS.union({"daily", "valuation", "index_daily"})
if dataset in SOFT_DATASETS:
hard_fail = bool(dup or bad_date)
return {
"rows": row_n,
"listed": listed_n,
"ratio": round(ratio, 4),
"errors": errors,
"warnings": warnings,
"hard_fail": hard_fail,
"soft_fail": bool(warnings) and not hard_fail,
"batch_id": batch_id,
}
def publish(self, dataset: str, trade_date: str, batch_id: str, state: str = "published") -> None:
copy_sql = EOD_COPY[dataset]
published_at = isoformat(self.clock())
with self.db.write() as connection:
current = connection.execute(
"SELECT active_batch FROM publications WHERE dataset = ? AND trade_date = ?",
(dataset, trade_date),
).fetchone()
prev = str(current["active_batch"]) if current else None
connection.execute(copy_sql, (batch_id,))
if self.before_commit:
self.before_commit()
connection.execute(
"""
INSERT INTO publications(dataset, trade_date, active_batch, prev_batch, state, published_at)
VALUES (?, ?, ?, ?, ?, ?)
ON CONFLICT(dataset, trade_date) DO UPDATE SET
prev_batch=excluded.prev_batch,
active_batch=excluded.active_batch,
state=excluded.state,
published_at=excluded.published_at
""",
(dataset, trade_date, batch_id, prev, state, published_at),
)
max_gen = connection.execute(
"SELECT COALESCE(MAX(generation), 0) AS g FROM publication_history WHERE dataset = ? AND trade_date = ?",
(dataset, trade_date),
).fetchone()
generation = int(max_gen["g"]) + 1
connection.execute(
"INSERT OR REPLACE INTO publication_history(dataset, trade_date, batch_id, published_at, generation) VALUES (?,?,?,?,?)",
(dataset, trade_date, batch_id, published_at, generation),
)
keep = int(self.settings.quality.get("publication_generations") or 3)
stale = connection.execute(
"""
SELECT batch_id FROM publication_history
WHERE dataset = ? AND trade_date = ?
ORDER BY generation DESC
""",
(dataset, trade_date),
).fetchall()
for row in stale[keep:]:
connection.execute(
"DELETE FROM publication_history WHERE dataset = ? AND trade_date = ? AND batch_id = ?",
(dataset, trade_date, row["batch_id"]),
)
def rollback(self, dataset: str, trade_date: str, actor: str = "admin") -> dict[str, Any]:
trade_date = yyyymmdd(trade_date)
pub = self.db.fetchone(
"SELECT * FROM publications WHERE dataset = ? AND trade_date = ?",
(dataset, trade_date),
)
if not pub or not pub.get("prev_batch"):
raise ValueError("没有可回滚的上一批次")
target = pub["prev_batch"]
published_at = isoformat(self.clock())
with self.db.write() as connection:
connection.execute(
"""
UPDATE publications
SET prev_batch = active_batch, active_batch = ?, published_at = ?, state = 'published'
WHERE dataset = ? AND trade_date = ?
""",
(target, published_at, dataset, trade_date),
)
self.audit(actor, "rollback", f"{dataset}:{trade_date}", json.dumps({"to": target, "from": pub["active_batch"]}))
return {"dataset": dataset, "trade_date": trade_date, "active_batch": target, "prev_batch": pub["active_batch"]}
def active_batch(self, dataset: str, trade_date: str) -> str | None:
row = self.db.fetchone(
"SELECT active_batch FROM publications WHERE dataset = ? AND trade_date = ?",
(dataset, trade_date),
)
return str(row["active_batch"]) if row else None
def cleanup(self) -> dict[str, int]:
staging_days = int(self.settings.quality.get("staging_retain_days") or 14)
job_days = int(self.settings.quality.get("job_run_retain_days") or 90)
now = now_shanghai(self.clock())
cutoff_staging = add_days(yyyymmdd(now), -staging_days)
cutoff_jobs = isoformat(now - timedelta(days=job_days))
deleted = 0
with self.db.write() as connection:
for dataset, (_eod, staging) in DATASET_TABLES.items():
cur = connection.execute(
f"DELETE FROM {staging} WHERE trade_date < ?",
(cutoff_staging,),
)
deleted += cur.rowcount
connection.execute("DELETE FROM job_runs WHERE started_at < ?", (cutoff_jobs,))
connection.execute("DELETE FROM src_calls WHERE created_at < ?", (cutoff_jobs,))
return {"staging_deleted": deleted}
def audit(self, actor: str, action: str, target: str = "", detail: str = "") -> None:
self.db.execute(
"INSERT INTO audit_log(actor, action, target, detail, created_at) VALUES (?,?,?,?,?)",
(actor, action, target, detail, isoformat(self.clock())),
)
def _fetch_dataset(self, dataset: str, trade_date: str) -> list[dict[str, Any]]:
if dataset == "daily":
raw = self._guarded_fetch("daily", {"trade_date": trade_date})
factors = {
(row["ts_code"], row["trade_date"]): finite_number(row.get("adj_factor"))
for row in self._guarded_fetch("adj_factor", {"trade_date": trade_date})
}
return [
normalize_daily(row, adj_factor=factors.get((str(row.get("ts_code") or "").upper(), str(row.get("trade_date") or ""))))
for row in raw
]
if dataset == "index_daily":
rows: list[dict[str, Any]] = []
for ts_code in DEFAULT_INDEX_CODES:
raw = self._guarded_fetch("index_daily", {"ts_code": ts_code, "trade_date": trade_date})
rows.extend(self.adapter.normalize("index_daily", raw))
return rows
api_dataset = dataset
raw = self._guarded_fetch(api_dataset, {"trade_date": trade_date})
return self.adapter.normalize(api_dataset, raw)
def _guarded_fetch(self, dataset: str, params: dict[str, Any]) -> list[dict[str, Any]]:
if not self.breaker.allow():
raise AdapterError("Tushare circuit open")
self.bucket.acquire()
started = time.perf_counter()
try:
# For daily we want RAW tushare rows so adj_factor can be merged later.
rows = self.adapter.fetch(dataset, params)
latency = round((time.perf_counter() - started) * 1000)
self.breaker.record_success()
self._log_call(dataset, True, latency, "")
self._persist_health("ok")
return rows
except Exception as exc:
latency = round((time.perf_counter() - started) * 1000)
self.breaker.record_failure(str(exc))
self._log_call(dataset, False, latency, str(exc))
self._persist_health("error", str(exc))
raise
def _stage(self, dataset: str, batch_id: str, rows: list[dict[str, Any]]) -> None:
sql, mapper = STAGING_INSERT[dataset]
with self.db.write() as connection:
connection.execute(
f"DELETE FROM {DATASET_TABLES[dataset][1]} WHERE batch_id = ?",
(batch_id,),
)
connection.executemany(sql, [mapper(row, batch_id) for row in rows])
def _set_batch(
self,
batch_id: str,
dataset: str,
trade_date: str,
state: str,
attempt: int,
rows_in: int | None = None,
rows_out: int | None = None,
quality: dict[str, Any] | None = None,
error: str | None = None,
finished: bool = False,
) -> None:
now = isoformat(self.clock())
existing = self.db.fetchone("SELECT batch_id FROM batches WHERE batch_id = ?", (batch_id,))
payload = json.dumps(quality, ensure_ascii=False) if quality else None
with self.db.write() as connection:
if existing is None:
connection.execute(
"""
INSERT INTO batches(batch_id, dataset, trade_date, state, attempt, rows_in, rows_out, quality_json, started_at, finished_at, error)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(batch_id, dataset, trade_date, state, attempt, rows_in, rows_out, payload, now, now if finished else None, error),
)
else:
connection.execute(
"""
UPDATE batches SET state=?, attempt=?,
rows_in=COALESCE(?, rows_in), rows_out=COALESCE(?, rows_out),
quality_json=COALESCE(?, quality_json),
finished_at=CASE WHEN ? THEN ? ELSE finished_at END,
error=COALESCE(?, error)
WHERE batch_id = ?
""",
(state, attempt, rows_in, rows_out, payload, 1 if finished else 0, now, error, batch_id),
)
def _log_call(self, endpoint: str, ok: bool, latency_ms: int, error: str) -> None:
self.db.execute(
"INSERT INTO src_calls(provider, endpoint, ok, latency_ms, error, created_at) VALUES (?,?,?,?,?,?)",
("tushare", endpoint, 1 if ok else 0, latency_ms, error, isoformat(self.clock())),
)
def _persist_health(self, state: str, error: str = "") -> None:
snap = self.breaker.snapshot()
self.db.execute(
"""
INSERT INTO src_health(provider, endpoint_class, state, last_ok_at, last_error, consec_failures, opened_at, cooldown_until)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(provider, endpoint_class) DO UPDATE SET
state=excluded.state, last_ok_at=excluded.last_ok_at, last_error=excluded.last_error,
consec_failures=excluded.consec_failures, opened_at=excluded.opened_at, cooldown_until=excluded.cooldown_until
""",
(
"tushare", "pro",
snap.state,
isoformat(self.clock()) if state == "ok" else None,
error or snap.last_error,
snap.consec_failures,
isoformat(self.clock()) if snap.state == "open" else None,
None,
),
)
+145
View File
@@ -0,0 +1,145 @@
from __future__ import annotations
import threading
from collections.abc import Callable
from datetime import datetime, time
from typing import Any
from datahub.db import HubDB
from datahub.logutil import get_logger
from datahub.pipeline import Pipeline
from datahub.timeutil import isoformat, now_shanghai, yyyymmdd
LOGGER = get_logger()
JobFn = Callable[[str], Any]
def is_open_day(db: HubDB, day: str) -> bool:
row = db.fetchone(
"SELECT is_open FROM trade_calendar WHERE exchange = 'SSE' AND cal_date = ?",
(day,),
)
if row is None:
return True # unknown calendar: do not skip reference refresh
return int(row["is_open"]) == 1
class Scheduler:
"""Calendar-driven in-process scheduler. Non-trading days skip EOD fetches."""
def __init__(self, db: HubDB, pipeline: Pipeline, jobs: dict[str, JobFn] | None = None) -> None:
self.db = db
self.pipeline = pipeline
self.jobs = jobs or {
"precheck": self._precheck,
"eod_a": self._eod_a,
"eod_b": self._eod_b,
"cleanup": self._cleanup,
"backup": self._backup,
}
self._stop = threading.Event()
self._thread: threading.Thread | None = None
self._fired: set[tuple[str, str, str]] = set()
def start(self, interval_seconds: float = 30.0) -> None:
if self._thread and self._thread.is_alive():
return
def loop() -> None:
while not self._stop.wait(interval_seconds):
try:
self.tick()
except Exception:
LOGGER.exception("scheduler tick failed")
self._thread = threading.Thread(target=loop, name="datahub-scheduler", daemon=True)
self._thread.start()
def stop(self, timeout: float = 5.0) -> None:
self._stop.set()
if self._thread and self._thread is not threading.current_thread():
self._thread.join(timeout)
def tick(self, clock: datetime | None = None) -> list[str]:
now = clock or now_shanghai()
day = yyyymmdd(now)
current = now.timetz() if False else now.time()
ran: list[str] = []
plan = [
("precheck", time(8, 45)),
("eod_a", time(15, 5)),
("eod_b", time(15, 10)),
("cleanup", time(0, 30)),
("backup", time(0, 40)),
]
open_day = is_open_day(self.db, day)
for job_id, at in plan:
if current < at:
continue
key = (job_id, day, at.strftime("%H%M"))
if key in self._fired:
continue
if job_id in {"eod_a", "eod_b"} and not open_day:
self._fired.add(key)
continue
self._fired.add(key)
self.run_job(job_id, day)
ran.append(job_id)
return ran
def run_job(self, job_id: str, trade_date: str) -> dict[str, Any]:
fn = self.jobs.get(job_id)
if fn is None:
raise KeyError(job_id)
started = isoformat()
run_id = None
with self.db.write() as connection:
cur = connection.execute(
"INSERT INTO job_runs(job_id, state, started_at, attempt) VALUES (?,?,?,1)",
(job_id, "running", started),
)
run_id = cur.lastrowid
try:
result = fn(trade_date) or {}
with self.db.write() as connection:
connection.execute(
"UPDATE job_runs SET state=?, finished_at=?, rows_out=?, detail=? WHERE id=?",
("ok", isoformat(), result.get("rows") if isinstance(result, dict) else None, str(result)[:2000], run_id),
)
return {"job_id": job_id, "result": result, "state": "ok"}
except Exception as exc:
with self.db.write() as connection:
connection.execute(
"UPDATE job_runs SET state=?, finished_at=?, error=? WHERE id=?",
("failed", isoformat(), str(exc), run_id),
)
raise
def _precheck(self, trade_date: str) -> dict[str, Any]:
return self.pipeline.ingest_reference(trade_date)
def _eod_a(self, trade_date: str) -> dict[str, Any]:
return self.pipeline.run_eod_batch_a(trade_date)
def _eod_b(self, trade_date: str) -> dict[str, Any]:
return self.pipeline.run_eod_batch_b(trade_date)
def _cleanup(self, trade_date: str) -> dict[str, Any]:
result = self.pipeline.cleanup()
if now_shanghai().weekday() == 6:
self.pipeline.db.vacuum()
result["vacuum"] = True
return result
def _backup(self, trade_date: str) -> dict[str, Any]:
from pathlib import Path
dest_dir = Path(self.pipeline.settings.backup_dir)
dest = dest_dir / f"datahub-{trade_date}.db"
self.pipeline.db.backup_to(dest)
keep = int(self.pipeline.settings.quality.get("backup_retain") or 14)
backups = sorted(dest_dir.glob("datahub-*.db"))
for old in backups[:-keep]:
old.unlink(missing_ok=True)
return {"path": str(dest.name), "kept": min(len(backups), keep)}
+376
View File
@@ -0,0 +1,376 @@
from __future__ import annotations
from http import HTTPStatus
from typing import Any
from urllib.parse import parse_qs
from datahub import SCHEMA_VERSION
from datahub.codes import resolve_code
from datahub.db import HubDB
from datahub.normalize import qfq_bar
from datahub.numbers import finite_number
from datahub.pipeline import Pipeline
from datahub.settings import Settings
from datahub.timeutil import isoformat, now_shanghai, session_phase, yyyymmdd
ERROR_STATUS = {
"UNAUTHORIZED": HTTPStatus.UNAUTHORIZED,
"INVALID_ARGUMENT": HTTPStatus.BAD_REQUEST,
"RATE_LIMITED": HTTPStatus.TOO_MANY_REQUESTS,
"SOURCE_UNAVAILABLE": HTTPStatus.SERVICE_UNAVAILABLE,
"DATASET_NOT_PUBLISHED": HTTPStatus.NOT_FOUND,
"STALE_DATA": HTTPStatus.OK,
"INTERNAL": HTTPStatus.INTERNAL_SERVER_ERROR,
}
class ApiError(Exception):
def __init__(self, code: str, message: str, retry_after: int | None = None, extra: dict[str, Any] | None = None) -> None:
super().__init__(message)
self.code = code
self.message = message
self.retry_after = retry_after
self.extra = extra or {}
def payload(self) -> dict[str, Any]:
body: dict[str, Any] = {"code": self.code, "message": self.message}
if self.retry_after is not None:
body["retry_after"] = self.retry_after
body.update(self.extra)
return {"error": body}
@property
def status(self) -> HTTPStatus:
return ERROR_STATUS.get(self.code, HTTPStatus.INTERNAL_SERVER_ERROR)
def envelope(data: Any, meta: dict[str, Any]) -> dict[str, Any]:
return {"schema_version": SCHEMA_VERSION, "data": data, "meta": meta}
class V1API:
def __init__(self, db: HubDB, pipeline: Pipeline, settings: Settings) -> None:
self.db = db
self.pipeline = pipeline
self.settings = settings
def handle(self, path: str, query: dict[str, list[str]]) -> dict[str, Any]:
q = {key: values[-1] if values else "" for key, values in query.items()}
if path == "/v1/health":
return self.health()
if path == "/v1/calendar":
return self.calendar(q.get("from") or "", q.get("to") or "")
if path == "/v1/stocks":
return self.stocks(q.get("updated_since") or "", q)
if path == "/v1/bars/daily":
return self.daily_bars(q)
if path == "/v1/indexes/bars":
return self.index_bars(q)
if path == "/v1/valuation":
return self.valuation(q)
if path == "/v1/moneyflow":
return self.moneyflow(q)
if path == "/v1/auction":
return self.auction(q)
if path == "/v1/datasets/status":
return self.dataset_status(q.get("date") or "")
if path == "/v1/batches":
return self.batches(q.get("date") or "", q.get("dataset") or "")
raise ApiError("INVALID_ARGUMENT", f"unknown endpoint: {path}")
def health(self) -> dict[str, Any]:
today = yyyymmdd(now_shanghai())
cal = self.db.fetchone(
"SELECT is_open FROM trade_calendar WHERE exchange = 'SSE' AND cal_date = ?",
(today,),
)
is_open = bool(cal and cal["is_open"] == 1)
sources = self.db.fetchall("SELECT * FROM src_health")
return envelope(
{
"status": "ok",
"session_phase": session_phase(now_shanghai(), is_open),
"trade_date": today,
"is_open_day": is_open,
"sources": [
{
"provider": row["provider"],
"endpoint_class": row["endpoint_class"],
"state": row["state"],
"last_ok_at": row["last_ok_at"],
"consec_failures": row["consec_failures"],
}
for row in sources
],
},
{"tier": "official", "trade_date": today, "source": "datahub", "stale": False, "staleness_seconds": 0},
)
def calendar(self, start: str, end: str) -> dict[str, Any]:
start = yyyymmdd(start or add_default(-30))
end = yyyymmdd(end or add_default(5))
rows = self.db.fetchall(
"""
SELECT cal_date, is_open, pretrade_date,
(SELECT MAX(cal_date) FROM trade_calendar t2
WHERE t2.exchange = 'SSE' AND t2.is_open = 1 AND t2.cal_date < t1.cal_date) AS prev_open
FROM trade_calendar t1
WHERE exchange = 'SSE' AND cal_date >= ? AND cal_date <= ?
ORDER BY cal_date
""",
(start, end),
)
items = [
{
"cal_date": row["cal_date"],
"is_open": bool(row["is_open"]),
"pretrade_date": row["pretrade_date"],
"prev_open": row["prev_open"],
}
for row in rows
]
return envelope(items, self._official_meta("calendar", end if items else start, source="tushare:trade_cal"))
def stocks(self, updated_since: str, q: dict[str, str]) -> dict[str, Any]:
limit, offset = self._page(q)
if updated_since:
rows = self.db.fetchall(
"SELECT * FROM stock_master WHERE updated_at >= ? ORDER BY ts_code LIMIT ? OFFSET ?",
(updated_since, limit, offset),
)
else:
rows = self.db.fetchall(
"SELECT * FROM stock_master ORDER BY ts_code LIMIT ? OFFSET ?",
(limit, offset),
)
return envelope(rows, self._official_meta("stocks", yyyymmdd(), source="tushare:stock_basic"))
def daily_bars(self, q: dict[str, str]) -> dict[str, Any]:
return self._published_rows(
dataset="daily",
table="eod_bars",
q=q,
source="tushare:daily",
adjust=q.get("adjust") or "none",
)
def index_bars(self, q: dict[str, str]) -> dict[str, Any]:
return self._published_rows(
dataset="index_daily",
table="eod_index_bars",
q=q,
source="tushare:index_daily",
default_code="000001.SH",
)
def valuation(self, q: dict[str, str]) -> dict[str, Any]:
return self._published_rows(dataset="valuation", table="eod_valuation", q=q, source="tushare:daily_basic")
def moneyflow(self, q: dict[str, str]) -> dict[str, Any]:
return self._published_rows(dataset="moneyflow", table="eod_moneyflow", q=q, source="tushare:moneyflow")
def auction(self, q: dict[str, str]) -> dict[str, Any]:
return self._published_rows(dataset="auction", table="eod_auction", q=q, source="tushare:stk_auction")
def dataset_status(self, date: str) -> dict[str, Any]:
trade_date = yyyymmdd(date or now_shanghai())
datasets = ("daily", "valuation", "moneyflow", "auction", "index_daily")
items = []
for dataset in datasets:
pub = self.db.fetchone(
"SELECT * FROM publications WHERE dataset = ? AND trade_date = ?",
(dataset, trade_date),
)
batch = None
if pub:
batch = self.db.fetchone("SELECT * FROM batches WHERE batch_id = ?", (pub["active_batch"],))
items.append(
{
"dataset": dataset,
"trade_date": trade_date,
"state": (pub or {}).get("state") or "unpublished",
"batch_id": (pub or {}).get("active_batch"),
"published_at": (pub or {}).get("published_at"),
"rows_out": (batch or {}).get("rows_out"),
"quality": _parse_json((batch or {}).get("quality_json")),
}
)
return envelope(items, self._official_meta("status", trade_date, source="datahub"))
def batches(self, date: str, dataset: str) -> dict[str, Any]:
trade_date = yyyymmdd(date or now_shanghai())
if dataset:
rows = self.db.fetchall(
"SELECT * FROM batches WHERE trade_date = ? AND dataset = ? ORDER BY started_at",
(trade_date, dataset),
)
else:
rows = self.db.fetchall(
"SELECT * FROM batches WHERE trade_date = ? ORDER BY started_at",
(trade_date,),
)
return envelope(rows, self._official_meta("batches", trade_date, source="datahub"))
def _published_rows(
self,
dataset: str,
table: str,
q: dict[str, str],
source: str,
adjust: str = "none",
default_code: str = "",
) -> dict[str, Any]:
trade_date = q.get("date") or q.get("trade_date") or ""
code = q.get("code") or default_code
start = q.get("from") or ""
end = q.get("to") or ""
if trade_date:
trade_date = yyyymmdd(trade_date)
start = end = trade_date
if not start or not end:
if not trade_date:
raise ApiError("INVALID_ARGUMENT", "date or from/to is required")
else:
start = yyyymmdd(start)
end = yyyymmdd(end)
ts_code = ""
if code:
resolved = resolve_code(self.db, code)
if resolved is None:
raise ApiError("INVALID_ARGUMENT", f"ambiguous code: {code}")
ts_code = resolved
# For a range, use per-date published batch. Single-date is the common path.
if start == end:
pub = self.db.fetchone(
"SELECT * FROM publications WHERE dataset = ? AND trade_date = ?",
(dataset, start),
)
if not pub:
raise ApiError(
"DATASET_NOT_PUBLISHED",
f"{dataset} {start} 尚未发布",
extra={"expected_at": "15:05+08:00"},
)
limit, offset = self._page(q)
sql = f"SELECT * FROM {table} WHERE trade_date = ? AND batch_id = ?"
params: list[Any] = [start, pub["active_batch"]]
if ts_code:
sql += " AND ts_code = ?"
params.append(ts_code)
sql += " ORDER BY ts_code LIMIT ? OFFSET ?"
params.extend([limit, offset])
rows = [dict(row) for row in self.db.fetchall(sql, tuple(params))]
if adjust == "qfq" and dataset == "daily":
rows = self._apply_qfq(rows)
meta = {
"tier": "official",
"trade_date": start,
"published_at": pub["published_at"],
"source": source,
"batch_id": pub["active_batch"],
"stale": False,
"staleness_seconds": 0,
"state": pub["state"],
}
return envelope(rows, meta)
# multi-day: walk published dates
pubs = self.db.fetchall(
"SELECT * FROM publications WHERE dataset = ? AND trade_date >= ? AND trade_date <= ? ORDER BY trade_date",
(dataset, start, end),
)
if not pubs:
raise ApiError("DATASET_NOT_PUBLISHED", f"{dataset} {start}-{end} 尚未发布")
rows: list[dict[str, Any]] = []
limit, offset = self._page(q)
for pub in pubs:
sql = f"SELECT * FROM {table} WHERE trade_date = ? AND batch_id = ?"
params = [pub["trade_date"], pub["active_batch"]]
if ts_code:
sql += " AND ts_code = ?"
params.append(ts_code)
sql += " ORDER BY ts_code"
rows.extend(self.db.fetchall(sql, tuple(params)))
sliced = rows[offset: offset + limit]
if adjust == "qfq" and dataset == "daily":
sliced = self._apply_qfq(sliced)
last = pubs[-1]
return envelope(
sliced,
{
"tier": "official",
"trade_date": last["trade_date"],
"published_at": last["published_at"],
"source": source,
"batch_id": last["active_batch"],
"stale": False,
"staleness_seconds": 0,
},
)
def _apply_qfq(self, rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
by_code: dict[str, list[dict[str, Any]]] = {}
for row in rows:
by_code.setdefault(str(row["ts_code"]), []).append(row)
out: list[dict[str, Any]] = []
for code, group in by_code.items():
latest = None
factors = [finite_number(item.get("adj_factor")) for item in group]
factors = [item for item in factors if item]
if factors:
latest = max(factors)
else:
extra = self.db.fetchone(
"SELECT MAX(adj_factor) AS f FROM eod_bars WHERE ts_code = ?",
(code,),
)
latest = finite_number((extra or {}).get("f"), 1.0)
out.extend(qfq_bar(item, latest) for item in group)
return out
def _page(self, q: dict[str, str]) -> tuple[int, int]:
try:
limit = int(q.get("limit") or self.settings.list_limit_default)
offset = int(q.get("offset") or 0)
except ValueError as exc:
raise ApiError("INVALID_ARGUMENT", "limit/offset must be integers") from exc
limit = max(1, min(limit, self.settings.list_limit_max))
offset = max(0, offset)
return limit, offset
def _official_meta(self, dataset: str, trade_date: str, source: str) -> dict[str, Any]:
pub = self.db.fetchone(
"SELECT * FROM publications WHERE dataset = ? AND trade_date = ?",
(dataset, trade_date),
)
return {
"tier": "official",
"trade_date": trade_date,
"published_at": (pub or {}).get("published_at"),
"source": source,
"batch_id": (pub or {}).get("active_batch"),
"stale": False,
"staleness_seconds": 0,
}
def add_default(days: int) -> str:
from datetime import timedelta
return (now_shanghai() + timedelta(days=days)).strftime("%Y%m%d")
def parse_query(raw: str) -> dict[str, list[str]]:
return parse_qs(raw, keep_blank_values=True)
def _parse_json(raw: Any) -> Any:
if not raw:
return None
if isinstance(raw, dict):
return raw
import json
try:
return json.loads(str(raw))
except json.JSONDecodeError:
return None
+72
View File
@@ -0,0 +1,72 @@
from __future__ import annotations
import json
import os
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[1]
DEFAULT_DB_PATH = Path(os.environ.get("DATAHUB_DB_PATH") or (ROOT / "data" / "datahub.db"))
DEFAULT_BACKUP_DIR = Path(os.environ.get("DATAHUB_BACKUP_DIR") or (ROOT / "data" / "backups"))
DEFAULT_CONFIG_PATH = ROOT / "config" / "hub-quality.config.json"
def _load_quality(path: Path) -> dict[str, Any]:
if not path.is_file():
return {}
return json.loads(path.read_text(encoding="utf-8"))
@dataclass
class Settings:
host: str = "127.0.0.1"
port: int = 8766
encryption_key: str = ""
api_token: str = ""
admin_password: str = ""
tushare_token: str = ""
db_path: Path = DEFAULT_DB_PATH
backup_dir: Path = DEFAULT_BACKUP_DIR
quality: dict[str, Any] = field(default_factory=dict)
log_level: str = "INFO"
scheduler_enabled: bool = True
@property
def tushare_rate_per_minute(self) -> int:
return int(self.quality.get("tushare_rate_per_minute") or 300)
@property
def max_publish_attempts(self) -> int:
return int(self.quality.get("max_publish_attempts") or 5)
@property
def list_limit_default(self) -> int:
return int(self.quality.get("list_limit_default") or 5000)
@property
def list_limit_max(self) -> int:
return int(self.quality.get("list_limit_max") or 5000)
def load_settings(
env: dict[str, str] | None = None,
config_path: Path | None = None,
) -> Settings:
environ = env if env is not None else dict(os.environ)
quality_path = config_path or DEFAULT_CONFIG_PATH
db_path = Path(environ.get("DATAHUB_DB_PATH") or DEFAULT_DB_PATH)
backup_dir = Path(environ.get("DATAHUB_BACKUP_DIR") or DEFAULT_BACKUP_DIR)
return Settings(
host=environ.get("DATAHUB_HOST") or "127.0.0.1",
port=int(environ.get("DATAHUB_PORT") or 8766),
encryption_key=str(environ.get("DATAHUB_ENCRYPTION_KEY") or "").strip(),
api_token=str(environ.get("DATAHUB_TOKEN") or "").strip(),
admin_password=str(environ.get("DATAHUB_ADMIN_PASSWORD") or "").strip(),
tushare_token=str(environ.get("TUSHARE_TOKEN") or "").strip(),
db_path=db_path,
backup_dir=backup_dir,
quality=_load_quality(quality_path),
log_level=environ.get("DATAHUB_LOG_LEVEL") or "INFO",
scheduler_enabled=str(environ.get("DATAHUB_SCHEDULER") or "1") not in {"0", "false", "False"},
)
+64
View File
@@ -0,0 +1,64 @@
from __future__ import annotations
from datetime import date, datetime, time, timedelta, timezone
from typing import Any
from zoneinfo import ZoneInfo
SHANGHAI = ZoneInfo("Asia/Shanghai")
def now_shanghai(clock: datetime | None = None) -> datetime:
if clock is not None:
if clock.tzinfo is None:
return clock.replace(tzinfo=SHANGHAI)
return clock.astimezone(SHANGHAI)
return datetime.now(SHANGHAI)
def isoformat(value: datetime | None = None) -> str:
current = now_shanghai(value)
return current.isoformat(timespec="seconds")
def yyyymmdd(value: date | datetime | str | None = None) -> str:
if value is None:
return now_shanghai().strftime("%Y%m%d")
if isinstance(value, str):
digits = value.replace("-", "")[:8]
if len(digits) != 8 or not digits.isdigit():
raise ValueError(f"invalid trade_date: {value}")
return digits
if isinstance(value, datetime):
return value.astimezone(SHANGHAI).strftime("%Y%m%d")
return value.strftime("%Y%m%d")
def parse_trade_date(value: str) -> date:
text = yyyymmdd(value)
return date(int(text[:4]), int(text[4:6]), int(text[6:8]))
def session_phase(clock: datetime | None, is_open_day: bool) -> str:
"""pre | intradaily | lunch | eod | closed"""
if not is_open_day:
return "closed"
current = now_shanghai(clock).time()
if current < time(9, 15):
return "pre"
if current < time(11, 30) or (time(13, 0) <= current <= time(15, 5)):
return "intraday"
if current < time(13, 0):
return "lunch"
if current <= time(23, 40):
return "eod"
return "closed"
def add_days(trade_date: str, days: int) -> str:
return (parse_trade_date(trade_date) + timedelta(days=days)).strftime("%Y%m%d")
def utc_timestamp(value: Any) -> str:
if isinstance(value, datetime):
return isoformat(value)
return isoformat()
+1
View File
@@ -0,0 +1 @@
cryptography==49.0.0
+27
View File
@@ -0,0 +1,27 @@
"""xiaobai-datahub process entry."""
from __future__ import annotations
import argparse
from datahub.hub import build_hub
from datahub.httpapp import serve
from datahub.settings import load_settings
def main() -> None:
parser = argparse.ArgumentParser(description="xiaobai-datahub")
parser.add_argument("--host", default=None)
parser.add_argument("--port", type=int, default=None)
args = parser.parse_args()
settings = load_settings()
if args.host:
settings.host = args.host
if args.port:
settings.port = args.port
hub = build_hub(settings)
serve(hub, settings.host, settings.port)
if __name__ == "__main__":
main()
View File
+59
View File
@@ -0,0 +1,59 @@
from __future__ import annotations
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
TRADE_DATE = "20240902"
RAW = {
"trade_cal": [
{"exchange": "SSE", "cal_date": "20240902", "is_open": 1, "pretrade_date": "20240830"},
{"exchange": "SSE", "cal_date": "20240903", "is_open": 1, "pretrade_date": "20240902"},
{"exchange": "SSE", "cal_date": "20240907", "is_open": 0, "pretrade_date": "20240906"},
],
"stock_basic": [
{"ts_code": "600000.SH", "symbol": "600000", "name": "浦发银行", "area": "上海", "industry": "银行", "market": "主板", "list_status": "L", "list_date": "19991110"},
{"ts_code": "000001.SZ", "symbol": "000001", "name": "平安银行", "area": "深圳", "industry": "银行", "market": "主板", "list_status": "L", "list_date": "19910403"},
],
"daily": [
{"ts_code": "600000.SH", "trade_date": "20240902", "open": 10.11, "high": 10.25, "low": 10.01, "close": 10.20, "pct_chg": 1.2345, "vol": 1000.0, "amount": 2000.0},
{"ts_code": "000001.SZ", "trade_date": "20240902", "open": 11.00, "high": 11.20, "low": 10.90, "close": 11.10, "pct_chg": -0.5, "vol": 2000.0, "amount": 4000.0},
],
"daily_basic": [
{"ts_code": "600000.SH", "trade_date": "20240902", "turnover_rate": 1.2, "volume_ratio": 0.8, "total_mv": 1000.0, "circ_mv": 800.0, "pe_ttm": 5.1, "pb": 0.6, "ps_ttm": 1.1, "dv_ttm": 4.0},
{"ts_code": "000001.SZ", "trade_date": "20240902", "turnover_rate": 2.2, "volume_ratio": 1.1, "total_mv": 2000.0, "circ_mv": 1500.0, "pe_ttm": 6.2, "pb": 0.7, "ps_ttm": 1.2, "dv_ttm": 3.0},
],
"adj_factor": [
{"ts_code": "600000.SH", "trade_date": "20240902", "adj_factor": 1.1},
{"ts_code": "000001.SZ", "trade_date": "20240902", "adj_factor": 2.0},
],
"index_daily": [
{"ts_code": "000001.SH", "trade_date": "20240902", "open": 2700, "high": 2750, "low": 2690, "close": 2740, "pct_chg": 0.5, "vol": 3000.0, "amount": 500000.0},
{"ts_code": "399001.SZ", "trade_date": "20240902", "open": 8000, "high": 8100, "low": 7900, "close": 8050, "pct_chg": 0.4, "vol": 2000.0, "amount": 300000.0},
{"ts_code": "399006.SZ", "trade_date": "20240902", "open": 1600, "high": 1620, "low": 1580, "close": 1610, "pct_chg": 0.3, "vol": 1000.0, "amount": 100000.0},
{"ts_code": "000300.SH", "trade_date": "20240902", "open": 3500, "high": 3550, "low": 3480, "close": 3520, "pct_chg": 0.2, "vol": 1500.0, "amount": 200000.0},
],
"moneyflow": [
{"ts_code": "600000.SH", "trade_date": "20240902", "buy_sm_amount": 10, "sell_sm_amount": 8, "buy_md_amount": 20, "sell_md_amount": 15, "buy_lg_amount": 30, "sell_lg_amount": 25, "buy_elg_amount": 40, "sell_elg_amount": 35, "net_mf_amount": 17},
{"ts_code": "000001.SZ", "trade_date": "20240902", "buy_sm_amount": 11, "sell_sm_amount": 9, "buy_md_amount": 21, "sell_md_amount": 16, "buy_lg_amount": 31, "sell_lg_amount": 26, "buy_elg_amount": 41, "sell_elg_amount": 36, "net_mf_amount": 18},
],
"stk_auction": [
{"ts_code": "600000.SH", "trade_date": "20240902", "vol": 100, "price": 10.15, "amount": 1500000, "pre_close": 10.00, "turnover_rate": 0.1, "volume_ratio": 1.2, "float_share": 2000},
{"ts_code": "000001.SZ", "trade_date": "20240902", "vol": 80, "price": 11.05, "amount": 1200000, "pre_close": 11.10, "turnover_rate": 0.2, "volume_ratio": 0.9, "float_share": 1800},
],
}
def fake_transport(api_name: str, params: dict, fields: str):
if api_name == "index_daily":
code = params.get("ts_code")
return [row for row in RAW["index_daily"] if row["ts_code"] == code]
if api_name == "trade_cal":
start = str(params.get("start_date") or "")
end = str(params.get("end_date") or "99999999")
return [row for row in RAW["trade_cal"] if start <= row["cal_date"] <= end]
return list(RAW.get(api_name) or [])
+96
View File
@@ -0,0 +1,96 @@
from __future__ import annotations
import json
import tempfile
import threading
import unittest
from http.server import ThreadingHTTPServer
from pathlib import Path
from urllib.request import Request, urlopen
from datahub.adapters.tushare import TushareAdapter
from datahub.crypto import SecretVault
from datahub.httpapp import make_handler
from datahub.hub import Hub
from datahub.settings import Settings
from tests.fixtures import fake_transport
class AdminTests(unittest.TestCase):
def setUp(self) -> None:
self.tmp = tempfile.TemporaryDirectory()
settings = Settings(
encryption_key=SecretVault.generate_key(),
api_token="z" * 32,
admin_password="StartPass1",
tushare_token="real-tushare-token-abcdef",
db_path=Path(self.tmp.name) / "hub.db",
scheduler_enabled=False,
)
self.hub = Hub(settings, adapter=TushareAdapter("real-tushare-token-abcdef", transport=fake_transport))
handler = make_handler(self.hub)
self.server = ThreadingHTTPServer(("127.0.0.1", 0), handler)
threading.Thread(target=self.server.serve_forever, daemon=True).start()
self.base = f"http://127.0.0.1:{self.server.server_address[1]}"
def tearDown(self) -> None:
self.server.shutdown()
self.server.server_close()
self.tmp.cleanup()
def _json(self, path, method="GET", body=None, cookie="", csrf=""):
data = None if body is None else json.dumps(body).encode()
headers = {"Content-Type": "application/json"}
if cookie:
headers["Cookie"] = cookie
if csrf:
headers["X-CSRF-Token"] = csrf
req = Request(self.base + path, data=data, headers=headers, method=method)
with urlopen(req, timeout=5) as resp:
set_cookie = resp.headers.get("Set-Cookie", "")
return resp.status, json.loads(resp.read().decode()), set_cookie
def test_login_change_password_and_secret_masking(self) -> None:
status, body, cookie_header = self._json(
"/admin/api/login", "POST", {"username": "hub_admin", "password": "StartPass1"}
)
self.assertEqual(status, 200)
self.assertTrue(body["must_change"])
cookie = cookie_header.split(";")[0]
csrf = body["csrf"]
status, _, _ = self._json(
"/admin/api/change-password",
"POST",
{"current": "StartPass1", "new_password": "NewPass123"},
cookie=cookie,
csrf=csrf,
)
self.assertEqual(status, 200)
_, sources, _ = self._json("/admin/api/sources", cookie=cookie, csrf=csrf)
blob = json.dumps(sources)
self.assertNotIn("real-tushare-token-abcdef", blob)
self.assertTrue(sources["items"][0]["credential"]["configured"])
self.assertTrue(str(sources["items"][0]["credential"]["last4"]).endswith("cdef") or "****" in str(sources["items"][0]["credential"]["last4"]))
def test_rollback_requires_password_and_confirm(self) -> None:
_, body, cookie_header = self._json(
"/admin/api/login", "POST", {"username": "hub_admin", "password": "StartPass1"}
)
cookie = cookie_header.split(";")[0]
csrf = body["csrf"]
self._json("/admin/api/change-password", "POST", {"current": "StartPass1", "new_password": "NewPass123"}, cookie, csrf)
from urllib.error import HTTPError
with self.assertRaises(HTTPError) as ctx:
self._json(
"/admin/api/rollback",
"POST",
{"dataset": "daily", "trade_date": "20240902", "password": "wrong", "confirm": "daily:20240902"},
cookie,
csrf,
)
self.assertEqual(ctx.exception.code, 401)
if __name__ == "__main__":
unittest.main()
+147
View File
@@ -0,0 +1,147 @@
from __future__ import annotations
import json
import tempfile
import threading
import unittest
from http.server import ThreadingHTTPServer
from pathlib import Path
from urllib.error import HTTPError
from urllib.request import Request, urlopen
from datahub.adapters.tushare import TushareAdapter
from datahub.crypto import SecretVault
from datahub.httpapp import make_handler
from datahub.hub import Hub
from datahub.settings import Settings
from tests.fixtures import TRADE_DATE, fake_transport
ERROR_CODES = {
"UNAUTHORIZED",
"INVALID_ARGUMENT",
"RATE_LIMITED",
"SOURCE_UNAVAILABLE",
"DATASET_NOT_PUBLISHED",
"STALE_DATA",
"INTERNAL",
}
class ApiContractTests(unittest.TestCase):
def setUp(self) -> None:
self.tmp = tempfile.TemporaryDirectory()
key = SecretVault.generate_key()
self.token = "k" * 32
settings = Settings(
host="127.0.0.1",
port=0,
encryption_key=key,
api_token=self.token,
admin_password="StartPass1",
tushare_token="tushare-secret-token-xyz",
db_path=Path(self.tmp.name) / "hub.db",
backup_dir=Path(self.tmp.name) / "backups",
scheduler_enabled=False,
quality={"daily_row_ratio": 0.5, "null_rate_max": 0.5, "list_limit_default": 5000, "list_limit_max": 5000},
)
adapter = TushareAdapter("tushare-secret-token-xyz", transport=fake_transport)
self.hub = Hub(settings, adapter=adapter)
self.hub.pipeline.ingest_reference(TRADE_DATE)
for dataset in ("daily", "valuation", "moneyflow", "auction", "index_daily"):
self.hub.pipeline.run_dataset(dataset, TRADE_DATE)
handler = make_handler(self.hub)
self.server = ThreadingHTTPServer(("127.0.0.1", 0), handler)
self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
self.thread.start()
self.base = f"http://127.0.0.1:{self.server.server_address[1]}"
def tearDown(self) -> None:
self.server.shutdown()
self.server.server_close()
self.hub.stop()
self.tmp.cleanup()
def _get(self, path: str, token: str | None = None) -> tuple[int, dict]:
headers = {}
if token is not None:
headers["X-Datahub-Token"] = token
req = Request(self.base + path, headers=headers)
try:
with urlopen(req, timeout=5) as resp:
return resp.status, json.loads(resp.read().decode())
except HTTPError as exc:
return exc.code, json.loads(exc.read().decode())
def test_livez_no_token(self) -> None:
status, body = self._get("/livez", token=None)
self.assertEqual(status, 200)
self.assertEqual(body["status"], "ok")
def test_missing_and_bad_token_401(self) -> None:
status, body = self._get("/v1/health", token=None)
self.assertEqual(status, 401)
self.assertEqual(body["error"]["code"], "UNAUTHORIZED")
status, body = self._get("/v1/health", token="wrong")
self.assertEqual(status, 401)
self.assertNotIn("tushare-secret-token-xyz", json.dumps(body))
self.assertNotIn(self.token, json.dumps(body))
def test_core_endpoints_schema(self) -> None:
paths = [
"/v1/health",
f"/v1/calendar?from=20240901&to=20240907",
"/v1/stocks",
f"/v1/bars/daily?date={TRADE_DATE}&code=600000.SH&adjust=none",
f"/v1/bars/daily?date={TRADE_DATE}&code=600000.SH&adjust=qfq",
f"/v1/indexes/bars?date={TRADE_DATE}&code=000001.SH",
f"/v1/valuation?date={TRADE_DATE}&code=600000.SH",
f"/v1/moneyflow?date={TRADE_DATE}&code=600000.SH",
f"/v1/auction?date={TRADE_DATE}",
f"/v1/datasets/status?date={TRADE_DATE}",
f"/v1/batches?date={TRADE_DATE}",
]
for path in paths:
status, body = self._get(path, token=self.token)
self.assertEqual(status, 200, path)
self.assertEqual(body["schema_version"], 1)
self.assertIn("data", body)
self.assertIn("meta", body)
self.assertIn("tier", body["meta"])
def test_qfq_matches_formula(self) -> None:
_, none = self._get(f"/v1/bars/daily?date={TRADE_DATE}&code=600000.SH&adjust=none", token=self.token)
_, qfq = self._get(f"/v1/bars/daily?date={TRADE_DATE}&code=600000.SH&adjust=qfq", token=self.token)
raw = none["data"][0]
adj = qfq["data"][0]
expected = round(raw["close"] * raw["adj_factor"] / raw["adj_factor"], 4)
self.assertEqual(adj["close"], expected)
def test_unpublished_code(self) -> None:
status, body = self._get("/v1/bars/daily?date=19990101", token=self.token)
self.assertEqual(status, 404)
self.assertEqual(body["error"]["code"], "DATASET_NOT_PUBLISHED")
self.assertIn("expected_at", body["error"])
def test_error_code_set_documented(self) -> None:
self.assertGreaterEqual(ERROR_CODES, {"UNAUTHORIZED", "DATASET_NOT_PUBLISHED", "INVALID_ARGUMENT"})
def test_six_digit_code(self) -> None:
status, body = self._get(f"/v1/bars/daily?date={TRADE_DATE}&code=600000", token=self.token)
self.assertEqual(status, 200)
self.assertEqual(body["data"][0]["ts_code"], "600000.SH")
def test_amount_unit_is_yuan(self) -> None:
_, body = self._get(f"/v1/bars/daily?date={TRADE_DATE}&code=600000.SH", token=self.token)
self.assertEqual(body["data"][0]["amount"], 2_000_000.0)
_, flow = self._get(f"/v1/moneyflow?date={TRADE_DATE}&code=600000.SH", token=self.token)
self.assertEqual(flow["data"][0]["net_mf_amount"], 170000.0)
def test_token_never_in_health_or_admin_sources(self) -> None:
_, health = self._get("/v1/health", token=self.token)
blob = json.dumps(health)
self.assertNotIn("tushare-secret-token-xyz", blob)
self.assertNotIn(self.token, blob)
if __name__ == "__main__":
unittest.main()
+56
View File
@@ -0,0 +1,56 @@
from __future__ import annotations
import unittest
from datahub.governance.circuit import CircuitBreaker
from datahub.governance.ratelimit import TokenBucket
from datahub.governance.retry import RetryError, retry_call
class FakeClock:
def __init__(self) -> None:
self.value = 0.0
def __call__(self) -> float:
return self.value
class GovernanceTests(unittest.TestCase):
def test_token_bucket_caps_burst_at_capacity(self) -> None:
clock = FakeClock()
bucket = TokenBucket(rate_per_minute=300, capacity=300, clock=clock)
ok = 0
for _ in range(400):
if bucket.acquire(block=False):
ok += 1
self.assertEqual(ok, 300)
clock.value = 60
self.assertTrue(bucket.acquire(block=False))
def test_circuit_opens_after_five_failures_and_half_opens(self) -> None:
clock = FakeClock()
breaker = CircuitBreaker(clock=clock, open_seconds=120)
for _ in range(5):
breaker.record_failure("boom")
self.assertEqual(breaker.snapshot().state, "open")
self.assertFalse(breaker.allow())
clock.value = 120
self.assertEqual(breaker.snapshot().state, "half_open")
self.assertTrue(breaker.allow())
breaker.record_success()
self.assertEqual(breaker.snapshot().state, "closed")
def test_retry_exhausts(self) -> None:
calls = {"n": 0}
def fail():
calls["n"] += 1
raise RuntimeError("no")
with self.assertRaises(RetryError):
retry_call(fail, attempts=3, sleeper=lambda _d: None)
self.assertEqual(calls["n"], 3)
if __name__ == "__main__":
unittest.main()

Some files were not shown because too many files have changed in this diff Show More