Compare commits

..
Author SHA1 Message Date
总管andmultica-agent b950ea4344 fix(HEL-494): keep filtered reference lookups inside datahub
Co-authored-by: multica-agent <github@multica.ai>
2026-09-09 00:50:33 +08:00
总管andmultica-agent 41f8509a98 fix(HEL-494): retry incomplete sector publications
Co-authored-by: multica-agent <github@multica.ai>
2026-09-09 00:46:18 +08:00
总管andmultica-agent c9e2d30780 fix(HEL-494): enforce datahub as sole website market boundary
Co-authored-by: multica-agent <github@multica.ai>
2026-09-09 00:22:27 +08:00
8a7d1f3698 fix(HEL-494): 网站市场客户端改为纯中枢 Facade,并迁移 iFinD 凭据到中枢
生产 gateway 不再读取 Tushare token 或实例化 TushareProvider/TushareClient;问财凭据经带鉴权的中枢接口加密入库,避免发版后 iFinD 未配置。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-08 23:37:53 +08:00
100752f43c fix(HEL-494): 切断网站生产装配外源直连,iFinD 与实时观察改走中枢
生产 gateway 不再实例化 iFinD、东财图和免费实时聚合器;问财与竞价快照作为中枢内部数据源。全站阻断外源测试覆盖日K、报价、图表、问财和竞价快照。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-08 22:41:51 +08:00
0b8419abca fix(HEL-494): 数据中枢独占调度,主网站不再回退旧接口
主网站只向中枢要业务数据;来源选择、切源、补数全部在中枢内部完成,失败不再走东财/腾讯/Tushare 保底。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-08 21:43:31 +08:00
ef13d6feb5 fix(HEL-494): 盘中双免费源故障切换并禁止问天假0覆盖
主源东财失败后自动改走腾讯行情,成功结果写入缓存;两源都失败时返回最近真实快照并标明延迟,不再显示假0。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-08 17:38:38 +08:00
b5d65ecb41 fix(HEL-494): 日K默认45根并修复问天行业0/205覆盖
悬浮窗和详情页只画最近45个交易日,中枢仍保留250日历史。盘后缺sw_daily时保留成分日线内核,外显走免费申万;成分行情改为全市场快照+分页,不再截成前60只。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-08 17:00:01 +08:00
3e828b346c fix(HEL-494): 修复个股缺失指标、问天遮罩、四爻外显并回补250日K
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-08 16:26:13 +08:00
c8a9376adb fix(HEL-490): 真实装配接通中枢并收编估值晚间复核
把 query/行情钩子绑到内层 TushareClient,图表接受不完整日K窗口;收编现网 HEL-423 未提交的估值复核,避免换版丢掉。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-08 15:03:34 +08:00
1c2f2ac057 feat(HEL-490): 剩余行情改由数据中枢主线路提供
正式页面以 8766 为主线路,旧接口只作故障备用;compose 钉死全部 DATAHUB_READ_*,避免现网残留 0 造成假完成。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-08 12:03:30 +08:00
5d3465987d fix(HEL-488): 盘中日K补上今天实时变化的一根
悬浮窗和详情页在 Tushare rt_k / iFinD 不可用时,改用免费实时行情或当日分时生成今日K,收盘后正式日K就绪再无缝替换。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-08 11:18:21 +08:00
dd89a09643 fix(HEL-487): 盘中当天看板在 rt_k 无权限时降级到免费实时源
rt_k 失败、无权限、超时或空结果时改用东财全市场快照,再失败则用腾讯批量行情;两者都失败仍不退回昨天。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-08 10:52:10 +08:00
a043bc9eb1 fix(HEL-485): 盘中选择当天不再整页退回昨天
交易时段缺少盘后正式数据时继续展示当天盘中行情,只有开盘前、周末和历史日期才沿用最近收盘结果。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-08 10:16:52 +08:00
acde4de40d fix(HEL-484): 中枢分时接口空 date 按当天查询
缺少或为空的 date 不再 400,按当天处理;显式历史日期保持原行为。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-08 10:07:44 +08:00
3d2c1252f1 fix(HEL-482): 开盘前分时回退最近交易日,并接通中枢失败回旧通道
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-08 09:44:00 +08:00
605f97e5df feat(HEL-463): 接入剩余行情数据到 datahub
扩展盘后正式集(涨跌停/人气/龙虎榜/板块日线)与盘中观察 API(报价/指数/分时),网站 bridge 按开关接入并回退旧链路;问天改为按数据依赖跟随开关,不再整栈强制旧路径。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-05 17:30:58 +08:00
16ba83ec01 fix(HEL-461): 切换事务失败写入 release-group 审计日志
整组切换中断时除回滚与废弃批次外,同步记录
action=release-group 的失败审计,便于后台追踪。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-05 16:02:11 +08:00
1c740a9d48 fix(HEL-461): 后台整组切换异常统一为 FAILED_PRECONDITION
管理后台补数在切换事务中断时不再抛出原始异常,
统一映射为 ApiError FAILED_PRECONDITION,并保留旧完整版本。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-05 15:56:28 +08:00
75c2e33b68 fix(HEL-461): CLI/后台强制重发改为整组边界切换
eod-refresh --force 与管理后台补数不再单数据集发布,
统一走 force_republish_boundary,避免绕过 A/B 完整边界。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-05 11:26:27 +08:00
32f565ecb9 fix(HEL-461): 整批发布按完整边界重暂存,主档与快照同事务
边界内任有缺失则整组重暂存后统一切换,避免旧新批次混发;
refresh_stocks 失败时主档保持旧值,并补齐回归测试。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-05 08:52:50 +08:00
16841e9ae3 fix(HEL-459): 影子比较按请求字段投影,盘后整批原子发布
比较侧只对网站本次请求字段计业务差异,忽略数据中枢额外列;
盘后 A/B/重发改为先整批暂存与交叉校验,再单事务切换公开版本。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-05 08:39:31 +08:00
multica-agentandmultica-agent bed6450992 feat(HEL-457): 估值字段级质量门、股票主档每日发布和资金流历史回补
- field_gates 按数据集配置关键字段非空率下限/非有限比例/相对上一批次的塌陷保护,
  字段大面积为空的批次拒发并保留上一正式批次,可读失败原因入 batches.error
- 股票主档交易日 20:00/23:10 自动刷新并发布版本化快照(eod_stocks + publications),
  覆盖新上市/简称变化/N前缀摘除;/v1/stocks 携带 batch_id/published_at,无变化跳过
- moneyflow 历史回补(默认 60 交易日,跳过已发布日期);未发布点查返回
  available_from/available_to 与 history_not_backfilled 标记,缺失不再静默
- eod-refresh 新增 --force --dataset 安全重发(仍走全部质量门,上一批次可回滚)
- 保持 HEL-435 盘后重试机制;新增 22 项测试覆盖字段拒发/正常通过/旧批保留/
  主档新增改名/资金流覆盖/重复执行幂等

Co-authored-by: multica-agent <github@multica.ai>
2026-09-04 21:36:20 +08:00
总工andmultica-agent c9892050c3 feat(HEL-435): 盘后未出数时晚间自动重试并提供安全补跑
Co-authored-by: multica-agent <github@multica.ai>
2026-09-03 22:30:15 +08:00
a836cda1b2 feat(HEL-421): 回补历史日历和指数并标记区间不完整
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-02 22:16:05 +08:00
总工 25ff6bbe06 fix(HEL-417): 配置根日志让影子对比报告落入容器日志 2026-09-02 21:53:19 +08:00
5085cacf0d fix(HEL-412): 刷新降级不再整次失败,并补齐准备中提示
手动刷新与自动补跑共用可用数据判定:日线推算或上一交易日快照记为部分/准备中成功,避免前端误报刷新失败。HTTP JSON 解析错误不再把请求正文写入日志。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-02 18:08:40 +08:00
0d13066386 feat(HEL-402): 接通网站首批只读 datahub 并建立双路对比
默认全部读取/影子开关关闭,网站继续走旧 Tushare 链路;开启单项时只替换该类原料并在失败时回旧,问天保持旧路径。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-02 17:09:28 +08:00
f5dc0f8076 fix(HEL-396): 禁止空数据批次冒充正式发布
统一发布入口在有效行数为 0 时不再写成 published,也不推进正式批次指针。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-02 15:56:29 +08:00
031eefab4d fix(HEL-386): 清理任务按 ISO 截止时间删除 job_runs/src_calls
YYYYMMDD 与 ISO 字符串比较会把同年保留期内记录全部误删。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-02 12:26:09 +08:00
3498dd7a4b feat(HEL-382): 搭建 datahub 底座和盘后正式数据链路
新增独立 xiaobai-datahub 服务(SQLite WAL、Tushare 盘后发布、/v1 契约和管理后台),不改现站页面与数据链路。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-02 12:05:26 +08:00
c2ebc0ab91 docs(HEL-353): 恢复普通 README 并移除多媒体展示
放弃宣传稿与 docs/images 素材,按当前仓库真实结构重写常规说明。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-01 21:58:35 +08:00
71a6d68ed7 docs(HEL-277): 用 25 组真实素材替换 README 占位
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-09-01 21:39:08 +08:00
cda13a787f docs(HEL-272): 用终版项目介绍更新仓库 README
将总工审核通过的项目介绍写入根目录 README,保留本机启动、Docker 与系统配置等必要说明,技术栈保持公开。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-30 20:58:16 +08:00
总管andmultica-agent 6b688fd473 修复(HEL-235B): 可靠同步受限权限工作目录
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 23:29:11 +08:00
总管andmultica-agent 1e8da5fee2 部署(HEL-235B): 补严分叉与镜像标签校验
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 23:27:18 +08:00
总工andmultica-agent 51f410d942 部署(HEL-235B): 服务器本地目录纳入 Git 管理,固化 main 校验构建流程
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 23:22:59 +08:00
总工 a8732f51be 部署集成(HEL-252): 合入登录页动态小人 4a63ccd 2026-08-29 21:59:09 +08:00
4a63ccd10f feat(HEL-251): 按确认稿实现登录页动态小K线人物
在桌面登录门户品牌空白带加入红绿小K线角色,支持焦点、密码遮挡、登录反馈和减少动态效果,且不移动原有文案与行情图。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 21:46:05 +08:00
总工 865d8b0516 部署集成(HEL-244): 合入切换账号返回修复 d2a165a 2026-08-29 21:01:12 +08:00
d2a165ada8 fix(HEL-243): 当前账号可点击返回,避免切换死路
当前账号行原先只显示对号、不可点击,单账号时只能关网页。现改为整行继续使用且不重新签发会话,并带回切换前的业务页。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 20:44:18 +08:00
总工 e29d5115fa 部署集成(HEL-241): 合入桌面登录门户返工 aef8a05(含 28px 底部缺口修复) 2026-08-29 20:09:41 +08:00
aef8a059f2 fix(HEL-240): 去掉登录页底部状态栏缺口
登录页覆盖全局 body 底边距,让左栏品牌面板铺满视口高度。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 20:04:22 +08:00
f905b44675 feat(HEL-240): 按确认样图还原桌面登录页左右骨架
把登录门户改回确认稿的品牌行、中下部主标题、底部行情标签和 K 线背景,并让超宽屏按比例展开左栏。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 19:21:13 +08:00
541fb48c1c feat(HEL-238): 按确认样图重做手机端系统管理,并加入部署基线门禁
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 17:34:50 +08:00
总工andmultica-agent 2a2d205a38 恢复集成(HEL-237): 并入 HEL-235 安全构建入口工具链(仅构建工具与文档,无运行时影响)
Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 16:01:54 +08:00
总工andmultica-agent 34cb32d78f 恢复集成(HEL-237): 以完整基线 89b8d33 为底合入登录线 cefc869
事故根因:HEL-214/221/226/233 均直接基于 main(8a5e78f) 构建部署,
绕过了 .11 正式线(HEL-183/188/190/191/192/193/199/207/208 + HEL-164
共 14 个已验收提交),导致问天工具行右对齐、侧栏等高、管理员刷新
结果、快照补档、收盘日线修复等整体丢失。

本合并以 deploy 前最后完整基线 89b8d33(镜像 official-limit-guard-d9ee725)
为底,合入 cefc869(HEL-221 情绪周期等高、HEL-226 登录门户与免密切换、
HEL-233 移动端系统管理五页),三处 CSS 缓存版本统一刷新为 20260829-hel237,
architecture-inventory 按合并后源码重新生成。

Co-authored-by: multica-agent <github@multica.ai>
2026-08-29 15:51:48 +08:00
总管andmultica-agent 89b8d33de7 fix(HEL-164): 排除构建中的数据库备份
Co-authored-by: multica-agent <github@multica.ai>
2026-08-28 11:32:21 +00:00
总管andmultica-agent d9ee725744 fix(HEL-164): 拒绝缓存不完整涨停快照
Co-authored-by: multica-agent <github@multica.ai>
2026-08-28 09:44:51 +00:00
总管andmultica-agent 1cb2745867 feat(HEL-164): 显示管理员刷新实际结果
Co-authored-by: multica-agent <github@multica.ai>
2026-08-28 09:29:06 +00:00
总管andmultica-agent f27471238a fix(HEL-164): 撤回盘后刷新改动并恢复原逻辑
Co-authored-by: multica-agent <github@multica.ai>
2026-08-28 09:05:42 +00:00
总工andmultica-agent 6d7a839202 chore(HEL-208): 刷新 architecture-inventory 以对齐 HEL-207 行情改动
Co-authored-by: multica-agent <github@multica.ai>
2026-08-28 08:09:49 +00:00
总工 fc1e5b89e4 merge(HEL-208): 集成收盘行情修复 cf206c7 到 .11 正式线(基于 09a935a) 2026-08-28 08:09:28 +00:00
cf206c7de9 fix(HEL-207): 收盘后改走日线,禁止误调 rt_k,回退旧快照记失败
将实时窗口与调度窗口统一到 15:05;盘后优先日线并补关闭盘后同步;沿用旧快照时 sync/job 记 failed。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-28 08:00:08 +00:00
总工andmultica-agent 09a935aac4 merge(HEL-193): 以 .11 线上基线 013ed29 整合行情历史补档 8e94c7b(HEL-190/HEL-199)
- 合入 agent/agent/ebb4e3e0638a:真实交易日历补最近60日快照 + sys.path 引导返工
- architecture-inventory 于合并后重新生成

Co-authored-by: multica-agent <github@multica.ai>
2026-08-27 16:13:34 +00:00
8e94c7b429 fix(HEL-199): 为补档工具补上仓库根 sys.path 引导
使 python3 tools/backfill_recent_snapshots.py --help 在干净环境下可直接运行,并同步文档运行示例为容器内执行。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-27 15:12:42 +00:00
总工 013ed29ffb merge(HEL-192): 集成问天解势修复到 .11 正式线(含 HEL-183 顶栏修复) 2026-08-27 14:58:11 +00:00
7ad445bc9f fix(HEL-190): 按真实交易日历补齐最近60日快照,修复断档后只显示当天
保留连续性过滤,新增可审计补档工具与备份步骤;周末/节假日与真缺档分开处理,支持重复执行与部分失败续跑。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-27 14:50:32 +00:00
ef86b31f6b fix(HEL-191): 补回问天解势知识文件兜底并消除 Failed to fetch
宿主机 data 挂载会遮盖镜像内 heaven_knowledge.json;增加不受挂载影响的 seed,
并将缺文件/坏 JSON 转为结构化中文错误,前端展示可读提示而非 Failed to fetch。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-27 14:47:27 +00:00
94e6f618c8 施工(HEL-188): 刷新 architecture-inventory 以对齐 HEL-183 运行时资源指纹
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-27 14:05:18 +00:00
f68f950106 施工(HEL-183): 问天工具行右对齐并统一 logo/收起/问师顶距与页头签
恢复顶栏工具行 margin-left:auto,问天隐藏行情条后仍靠右;logo 区与收起按钮对齐顶栏/状态条高度;问师恢复页顶 14px;问天三签改用全站 segmented 页头。

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: multica-agent <github@multica.ai>
2026-08-27 13:41:23 +00:00
183 changed files with 24386 additions and 1236 deletions
+1
View File
@@ -9,6 +9,7 @@ __pycache__/
*.log
runtime/
data/cache/
data/backups/
data/private-mentor-skills/
data/*.db
data/*.db-shm
+12 -5
View File
@@ -1,13 +1,20 @@
# Generated automatically when omitted. Back it up together with the database.
APP_ENCRYPTION_KEY=
# Initial shared market-data credential. After first launch it is encrypted into
# the system settings; all accounts use the same backend market snapshot.
# Market-source credentials are consumed and encrypted only by xiaobai-datahub.
# compose.yaml masks them from the xiaobai-review website process.
TUSHARE_TOKEN=your_tushare_token_here
# Optional iFinD HTTP credential. The backend exchanges it for a short-lived
# access token and never exposes either token to browsers.
IFIND_REFRESH_TOKEN=your_ifind_refresh_token_here
# Official xiaobai-datahub client. Read flags default on in config/datahub.config.json.
# compose.yaml pins every DATAHUB_READ_* to 1. The website has no provider
# fallback; source selection and failover happen inside xiaobai-datahub.
# DATAHUB_SHADOW_* can still override a single dataset.
DATAHUB_BASE_URL=http://127.0.0.1:8766
DATAHUB_TOKEN=
# iFinD credentials live on xiaobai-datahub, not the website process.
# IFIND_REFRESH_TOKEN=your_ifind_refresh_token_here
# IFIND_ACCESS_TOKEN=
# Initial platform member models (OpenAI-compatible). After first launch these
# are encrypted into system settings and used only by admins and active members.
+3
View File
@@ -8,6 +8,9 @@ data/*.db
data/*.db-shm
data/*.db-wal
data/backups/
datahub-data/
xiaobai-datahub/data/
xiaobai-datahub/.venv/
data/*.bak
data/*.backup
*.log
+11 -4
View File
@@ -38,10 +38,15 @@ background scheduler
fields, and feature-specific exceptions belong to `backend/features/<feature>/routes.py`.
- `backend/features/<feature>/` owns the mechanically moved service, repository, HTTP, agent,
or deterministic calculation code for that product area.
- `backend/data/` owns provider construction, source policy, provenance, units, freshness,
coverage, display-versus-calculation eligibility, and shared numeric normalization policies.
- `backend/data/` owns the website-side DataHub client, stable dataset contracts, provenance,
units, freshness, coverage, display-versus-calculation eligibility, and shared numeric
normalization policies. The website process does not construct or configure external market
providers; provider credentials, source selection, retries, fallbacks, caching, and backfill
belong exclusively to the `xiaobai-datahub` service.
- `backend/data/providers/tushare_client.py` is the stable public `TushareClient` facade and
owns only its dataclass fields and shared cache state. Tushare HTTP transport belongs to
is retained as the dataset-contract compatibility surface and isolated test facade. Production
website services never instantiate it directly: its query methods are served by the DataHub
proxy. Its split modules document the stable contract: Tushare HTTP transport belongs to
`tushare_transport.py`; market overview and realtime breadth belong to
`tushare_dashboard.py`; indices belong to `tushare_indices.py`; Shenwan membership and
industry snapshots belong to `tushare_industries.py`; generic sector snapshots belong to
@@ -54,7 +59,9 @@ background scheduler
feature repository mixins; do not add feature queries to it.
- `backend/jobs/` owns job definitions, locks, retries, idempotency, and persisted run state.
`backend/jobs/service.py` is the application-facing owner of scheduler start/stop, manual
refresh submission, and periodic refresh coordination.
refresh submission, and periodic refresh coordination. `backend/jobs/refresh.py` owns
whether a dashboard payload is a usable refresh result versus a failed job, and whether
after-hours official catch-up is due.
- `backend/llm/` owns model selection, membership/quota checks, fallback, provider transport,
streaming rules, and call audit. Feature agents only prepare messages and interpret
feature-specific results.
+59 -69
View File
@@ -14,18 +14,37 @@
v
xiaobai-review 容器 :8765
|-- /app 只读应用代码
| `-- backend/features/heaven/assets/heaven_knowledge.json
| 镜像内 seed(不受 data 挂载遮盖)
`-- /app/data 宿主机 ./data 持久化挂载
|-- review.db
|-- iching_zh.json
`-- heaven_knowledge.json 优先读取;缺失时回退到上方 seed
```
账号、加密后的公共数据 Token、平台模型 API Key、生辰资料、行情快照和复盘数据均在
`data/review.db`。解密密钥来自 `.env` 中的 `APP_ENCRYPTION_KEY`。数据库与
密钥必须成对备份,任意一个丢失都无法恢复账号内的加密资料。
账号、平台模型 API Key、生辰资料、行情快照和复盘数据均在 `data/review.db`。外部行情源
Token 只允许保存在 `xiaobai-datahub` 的环境或凭据库,网站进程不读取、不保存,也不向
提供方直接发请求。解密密钥来自 `.env` 中的 `APP_ENCRYPTION_KEY`。数据库与密钥必须成对
备份,任意一个丢失都无法恢复账号内的加密资料。
问天静态知识文件:
- `data/iching_zh.json``data/heaven_knowledge.json` 纳入 Git 与镜像构建;
`.dockerignore` 不排除这两个文件(只排除 `data/*.db``data/cache/` 等运行时产物)。
- Compose 把宿主机 `./data` 整目录挂到 `/app/data`,会遮盖镜像里同路径文件。
因此宿主机 `data/` 应保留上述两个 JSON;若只缺 `heaven_knowledge.json`
服务会回退读取镜像内
`backend/features/heaven/assets/heaven_knowledge.json`,解势仍可用。
- 持久化位置:正式环境以宿主机项目目录下的 `./data/heaven_knowledge.json` 为准;
补文件后无需改代码,重启容器即可加载。
管理员私有的问师 Skill 保存在宿主机 `data/private-mentor-skills/`。该目录随 `data`
挂载进入容器,但被 Git 与 Docker 构建上下文排除,不会进入 Gitea 或镜像。私有 Skill
只对管理员账号返回和开放调用,也会随本指南的 `data` 备份一起保存。
首个注册账号自动成为管理员。管理员在“系统管理”中配置全站共享行情、后台刷新、平台会员模型及手动会员;普通用户的“账号设置”用于个人资料、会员状态、修改密码和切换账号。后台行情更新不会主动刷新任何浏览器页面。
首个注册账号自动成为管理员。管理员在网站“系统管理”中查看数据中枢状态并配置后台刷新、
平台会员模型及手动会员;行情源凭据和调度策略在数据中枢后台统一管理。普通用户的“账号设置”
用于个人资料、会员状态、修改密码和切换账号。后台行情更新不会主动刷新任何浏览器页面。
## 2. 服务器要求
@@ -149,87 +168,58 @@ docker compose restart xiaobai-review
docker compose down
```
### 镜像构建的唯一安全入口(2026-08 HEL-235 起
### 服务器本地目录更新与构建(日常推荐
生产机 `192.168.200.11` `/opt/1panel/docker/compose/xiaobaifupan` 只是历史文件树:
不是 Git 仓库、内容停在旧提交、与线上镜像不一致,且其 `compose.yaml` 会把构建结果打进
`xiaobai-review:latest`。**禁止在该目录(或任何服务器工作树)里 `docker build` /
`docker compose build`**,否则会把已上线功能悄悄打回旧版。
唯一安全构建方式是在有仓库检出、能免密 SSH 到部署机的机器上运行:
```bash
tools/build_image.sh <提交号> <镜像tag>
# 示例:tools/build_image.sh cefc86917d89 verify-hel235-cefc869
```
该脚本的行为约束:
-`git fetch`,再把提交号解析为完整 SHA,解析失败立即中止,绝不使用本地脏状态或服务器旧目录;
- 镜像 tag 必须以 `-<提交短号7位>` 结尾(如 `hel234-cefc869`),禁止 `latest``rollback-*`
- 通过 `git archive <提交> | ssh 部署机 docker build -` 流式构建,服务器上不存在构建用工作树;
- 构建后回读镜像 label 里的 `org.opencontainers.image.revision`,与预期提交不一致则删除镜像并中止;
- 每次构建在部署机 `~/xiaobai-build/BUILD_LOG.tsv` 留痕,可追溯每个镜像的来源提交。
构建只产出镜像,不启动、不替换任何容器;换版用新 tag 起新容器,回滚用既有镜像 tag 重跑。
### 使用 Gitea 更新程序(旧方式,生产机禁用)
代码仓库为:
生产机 `192.168.200.11``/opt/1panel/docker/compose/xiaobaifupan` 自 2026-08-29HEL-235B
起已是受 Git 管理的工作目录,只跟踪 Gitea `main`(仓库
`http://192.168.200.36:3200/leefer/xiaobai-review.git`)。由于目录顶层归 root
`.git` 存放在部署账号家目录(外部 Git 目录方案):
```text
http://192.168.200.36:3200/leefer/xiaobaifupan.git
~/xiaobai-build/repos/xiaobai-review.git Git 元数据(分支/历史/索引)
/opt/1panel/docker/compose/xiaobaifupan 工作目录(程序文件本体)
~/xiaobai-build/update-from-main.sh 一键更新+构建入口
~/xiaobai-git 便捷查看(status/log/diff
```
首次在服务器部署代码时,可以直接克隆到目标目录
日常更新只需要在服务器上执行一条命令
```bash
sudo mkdir -p /opt/xiaobai-review
sudo chown "$USER":"$USER" /opt/xiaobai-review
git clone http://192.168.200.36:3200/leefer/xiaobaifupan.git /opt/xiaobai-review
cd /opt/xiaobai-review
~/xiaobai-build/update-from-main.sh # 更新到 main 并构建 main-<短号> 镜像
~/xiaobai-build/update-from-main.sh verify-tag main-a8732f5 # 部署前复核镜像与 main 一致
```
私有仓库会提示输入 Gitea 用户名和密码或访问令牌。不要把密码写入仓库 URL、
`compose.yaml` 或脚本。然后把原 `.env``data/` 放回该目录;这两项已被 Git
忽略,后续拉取代码不会覆盖数据库与密钥。
脚本在构建前强制完成五道校验,任一不符立即停止、不产出镜像:
如需部署管理员私有问师,通过 NAS 文件管理器将本地
`data/private-mentor-skills/` 复制到服务器项目的同名 `data` 目录,并保持目录仅由
部署账号和容器运行用户读取。该内容不会通过 Gitea 同步。
1. `git fetch` 成功(连不上 Gitea 即停);
2. 必须在 `main` 分支(智能体不得用功能分支直接当正式线);
3. 工作区无未提交改动、无多余文件;
4. 只允许快进合并到 `origin/main`(分叉即停);main 新增/删除顶层文件时会给出
需管理员执行的精确清单(目录顶层归 root);
5. 构建后回读镜像 `org.opencontainers.image.revision`,与 `main` 提交不一致则删除镜像。
每次更新前先创建 SQLite 一致性备份,再拉取并重建容器(注意:`docker compose up -d --build`
从服务器本地工作树构建,仅适用于来源可信的全新环境;生产机 `192.168.200.11` 禁用,
请用 `tools/build_image.sh` 构建后换容器):
镜像 tag 固定为 `main-<提交短号7位>`(不带提交号的模糊 tag 一律禁止);每次构建在
`~/xiaobai-build/BUILD_LOG.tsv` 留痕。构建只产出镜像,不启动、不替换容器;换版与
回滚步骤见 `~/xiaobai-build/README.md`
```bash
cd /opt/xiaobai-review
docker compose exec -T xiaobai-review python -c "import sqlite3; s=sqlite3.connect('/app/data/review.db'); d=sqlite3.connect('/app/data/review-before-update.db'); s.backup(d); d.close(); s.close()"
git pull --ff-only origin main
docker compose up -d --build
docker compose ps
curl --fail http://127.0.0.1:8765/api/health
```
`compose.yaml` 的镜像名与 revision 标签同样做了强校验:直接 `docker compose up -d --build`
会因缺少 `XIAOBAI_GIT_REV` / `XIAOBAI_GIT_SHORT` 变量而拒绝执行,避免再出现构建进
`latest` 的模糊版本。需要用 compose 时先 `export` 这两个变量(值以
`~/xiaobai-build/xiaobai-git rev-parse HEAD` 为准),或直接用上面的脚本。
`docker compose up -d --build` 会原地替换应用容器,不删除宿主机的 `data` 目录。
数据库迁移会在新容器启动时自动执行。若 `git pull --ff-only` 提示本地代码有修改,
先用 `git status` 查明原因,不要用强制重置覆盖 `.env``data`
### 智能体高级入口:Git 归档流式构建
### 不使用 Git 时更新(生产机禁用)
有仓库检出、能免密 SSH 到部署机的智能体可以用 `tools/build_image.sh <提交号> <镜像tag>`
从任意明确提交流式构建(`git archive | ssh docker build`),tag 同样必须以
`-<提交短号7位>` 结尾,构建后回读 revision 校验并留痕。用于在服务器不便拉取时的
应急构建;日常正式线仍应走 `main`
`docker compose build` 会从服务器本地目录构建,来源提交不可追溯。生产机
`192.168.200.11` 上禁止使用本节方式,一律改用上一节的 `tools/build_image.sh`
### 历史方式(已废弃)
重新上传代码后执行:
```bash
docker compose down
docker compose build --pull
docker compose up -d
```
`docker compose down` 不会删除宿主机的 `data` 目录。不要使用带有手工删除
`data` 目录的清理命令。
早期文档建议在服务器重新 `git clone` 一份或手工上传代码后 `docker compose up --build`
这两条路径已废弃:服务器上**只允许存在一个受管工作目录**(上述
`/opt/1panel/docker/compose/xiaobaifupan`),任何脱离 Git 校验的本地构建都会把
来源提交变成不可追溯状态,禁止使用。
## 7. 备份与恢复
+4 -1
View File
@@ -23,7 +23,10 @@ COPY requirements.txt ./
RUN python -m pip install --no-cache-dir -r requirements.txt
COPY --chown=xiaobai:xiaobai . .
RUN mkdir -p /app/data && chown -R xiaobai:xiaobai /app/data
RUN mkdir -p /app/data && chown -R xiaobai:xiaobai /app/data \
&& test -f /app/data/heaven_knowledge.json \
&& test -f /app/data/iching_zh.json \
&& test -f /app/backend/features/heaven/assets/heaven_knowledge.json
USER xiaobai
+104 -42
View File
@@ -1,76 +1,138 @@
# 小白复盘 Web
# 小白复盘
一个面向 A 股盘后复盘的本地 Web 工作台。后端使用 Python 访问 Tushare Pro,前端不依赖构建工具
面向 A 股盘后复盘的本地 Web 工作台。收盘后把涨停、炸板、连板梯队、板块轮动、集合竞价、龙虎榜等数据整理成可浏览的复盘界面;不接券商、不代为下单,也不提供个股推荐
本目录是经过保真迁移、结构治理和用户人工验收的唯一正式源码,不依赖父目录旧程序或失败版本
目录职责见[ARCHITECTURE.md](ARCHITECTURE.md),产品与维护文档见[docs/README.md](docs/README.md)。
本目录是唯一正式源码。模块边界见 [ARCHITECTURE.md](ARCHITECTURE.md),产品与维护文档见 [docs/README.md](docs/README.md)
当前包含集合竞价、涨停池、炸板池、跌停板、昨日涨停、涨停表现、市场天梯、板块轮动、题材库、人气热榜、龙虎榜和个人复盘工作区。交易日快照与同步记录保存在本地 SQLite 数据库 `data/review.db`
## 主要功能
集合竞价中心采用盘前生命周期:9:15 前显示预告,9:15–9:25 明确等待最终竞价,9:25–9:30 自动读取并重试最终竞价筛选,9:30 后停止更新并冻结为复盘归档。当前 Tushare 只提供 9:25 最终竞价快照,不将其表述为动态虚拟撮合行情
登录后左侧共 16 个页面,另有一个内嵌页「策略持续跟踪」。交易日快照保存在本地 SQLite 数据库 `data/review.db`
第三阶段加入了机构席位、席位别名、个股复权日 K、资金流、自选股、涨停原因修订、个股笔记、每日复盘和历史数据回补。
- **情绪周期**:0–100 情绪温度与阶段判定(默认首页)
- **涨停池 / 炸板池 / 跌停板 / 昨日涨停 / 涨停表现**:封板结构、炸板、跌停与昨日反馈
- **市场天梯**:按连板高度排列的市场梯队
- **板块轮动**:近若干交易日板块热力与成分下钻
- **集合竞价**:盘前生命周期;9:30 后停止更新并冻结为复盘归档。当前数据源提供 9:25 最终竞价快照,不是动态虚拟撮合行情
- **题材库 / 人气热榜 / 龙虎榜**:题材成分、双榜人气、席位与游资档案
- **智能选股**(会员):六阶段策略、精选策略库、自然语言编译为受控公式后的确定性筛选与滚动回测;候选需手动加入后才进入五交易日跟踪
- **问师**(会员):按选定的游资思维 Skill 单师对话;新增公开角色时在 `游资skills` 下增加含 `SKILL.md` 的目录,并在 `游资skills/mentor_catalog.json` 登记。管理员私有角色放在 `data/private-mentor-skills`(不进 Git / 镜像)
- **问天**(会员):观势 / 观气 / 观心。卦象、干支、节气与气机由本地程序确定性计算,大模型只负责文字解释。此前仅冻结过界面视觉方案,现已解冻;问天可纳入后续数据与功能迁移,本阶段不主动重做视觉。
- **我的复盘**:手工交易日志、每日复盘、提醒中心与复盘助手;不接券商、不自动下单
股票代码在桌面端悬停后会显示分时与日 K 快速预览,默认优先展示日 K;移动端点击代码后从底部打开预览面板。股票详情以及板块、题材、指数详情均可在日 K 与最新分时之间切换。日 K 复用个股详情缓存;分时优先使用 iFinD,东方财富仅作隔离的展示兜底,并使用短时内存缓存。图表数据不写入主行情不参与情绪、选股或问天计算;不可用时明确显示“分时不可用”,不会用日 K 模拟分时走势
全局能力:日间 / 夜间主题、股票代码悬停预览日 K 与分时、`Ctrl + K` 全局搜索。图表数据不写入主行情,也不参与情绪、选股或问天计算。
智能选股包含六阶段盘后候选、29 套精选策略、自定义公式 DSL、自然语言公式编译、候选排名和滚动回测。阶段与精选策略在当日行情更新后由后台确定性计算;自定义选股由用户手动执行,LLM 只负责编译自然语言条件,不参与候选筛选。竞价、估值、财务、资金、人气和席位等字段按已登记的数据可用性进入因子库,缺失时明确显示覆盖问题。
## 技术栈
候选只有经用户手动加入后才进入五交易日持续跟踪,展示 T+1 开盘/收盘、T+3、T+5、最大涨幅与最大回撤。提醒中心支持手工日期提醒,并在策略首日反馈和五日跟踪完成时生成账号私有的站内提醒。
| 层面 | 说明 |
| --- | --- |
| 运行时 | Python 3.12;标准库 `ThreadingHTTPServer`,无独立 Web 框架 |
| 依赖 | `requirements.txt` 仅含 `cryptography`;问天历法使用仓库内 `vendor/lunar_python` |
| 数据库 | SQLiteWAL),默认文件 `data/review.db` |
| 前端 | 原生 HTML / CSS / JavaScript,无打包、无构建步骤 |
| 部署 | Docker / Docker Compose,或本机直接运行 `server.py` |
| 安全 | 账号密码 scrypt 哈希;行情 Token 与模型密钥用 `APP_ENCRYPTION_KEY` 加密后存库 |
问师模块会读取当前复盘、近十日市场情绪、涨跌停、昨日反馈、板块轮动、市场阶段、龙虎榜和指定个股数据,再按选中的游资思维 Skill 进行单师对话。对话记录按账号、老师和交易日期保存在服务端;主模型不可用时自动切换辅助模型。
## 环境要求
新增公开问师角色时,在 `游资skills` 下增加一个包含 `SKILL.md` 的独立目录,并在 `游资skills/mentor_catalog.json` 中登记素材等级与结构质检。管理员私有角色放在 `data/private-mentor-skills`,该目录不进入 Git 或 Docker 镜像,且只会出现在管理员的问师列表中。系统会从 Skill 的 frontmatter、一级标题、核心模型和引用语中自动生成角色信息,无需修改注册代码。
- Python 3.12(与 `Dockerfile` 一致)
- 本机启动:能执行 `python` / `pip`
- Docker 部署:Docker Engine 24+Compose v2`docker compose`
- 行情:部署者自行申请并配置 Tushare Pro Token;部分分时优先使用同花顺 iFinD(可选)
- 问师、问天解释、复盘助手、自然语言编译公式:需配置 OpenAI 兼容接口;未配置时市场数据页仍可用
问天模块包含三个相互独立的部分:观势以市场数据生成三才六爻,用于观察“势”,行情缺失或自动取象明显偏差时可显式手动校准六爻,人工结果与自动来源严格区分;观气依据干支、精确节气、五运六气及客主加临关系观察“运”,行业五行仅作传统取象归类;观心先准备1秒,再完成5轮“吸3秒、顿2秒、呼4秒”,随后以六次三枚铜钱起卦、察念和解卦完成一次不输入问题的问心仪式。卦象、干支、节气与气机关系均由本地确定性程序计算,LLM只负责解释,不参与起卦或改动结果。
## 安装与启动
问天模块使用项目本地的 `lunar-python` 计算历法,并使用 `data/iching_zh.json` 中的固定六十四卦、卦辞和爻辞。第三方授权见 `THIRD_PARTY_NOTICES.md`
仓库根目录即为运行目录(`server.py``requirements.txt` 都在根目录)
“我的复盘”包含结构化手工交易日志,可记录方向、价格、数量、仓位、盈亏、逻辑、执行、情绪和标签,不接券商也不自动下单。顶部“复盘助手”以流式方式读取市场统计、策略跟踪、提醒、个人复盘和交易日志;对话按账号保存,只提供分析和条件化计划。
## 启动
```powershell
cd app
```bash
python -m pip install -r requirements.txt
python server.py
```
浏览器打开 `http://127.0.0.1:8765`,首次使用先注册账号。首个账号自动成为管理员,后续账号默认为普通用户。主行情不再回退演示数据:盘前、非交易日或临时取数失败时沿用最近真实收盘快照;没有任何真实快照时提示等待管理员完成首次同步。
默认监听 `127.0.0.1:8765`(仅本机可访问)。浏览器打开该地址,首次使用先注册账号;第一个账号自动成为管理员,之后注册的默认为普通用户。
需要后台启动本地验收端口时,使用`tools/start_local.ps1`。该工具把日志、进程号和Python缓存
统一写入`runtime/`,不在源码根目录产生运行文件:
主行情不再回退演示数据:盘前、非交易日或临时取数失败时沿用最近真实收盘快照;没有任何真实快照时,页面会提示等待管理员完成首次同步。
可选参数:
```bash
python server.py --host 127.0.0.1 --port 8765
```
Windows 下若需要后台启动,并把日志、进程号和 Python 缓存写入 `runtime/`(不在源码根目录产生运行文件):
```powershell
powershell -ExecutionPolicy Bypass -File tools/start_local.ps1 -Port 8797
powershell -ExecutionPolicy Bypass -File tools/start_local.ps1
```
局域网 Docker 部署使用 `Dockerfile``compose.yaml`,完整的迁移、持久化、
防火墙、备份和恢复步骤见 [DOCKER_DEPLOY.md](DOCKER_DEPLOY.md)。
该脚本默认端口为 `8797`。统一验收:
账号密码使用 scrypt 哈希;公共 Tushare Token、平台模型密钥以及原始生辰资料均使用 `APP_ENCRYPTION_KEY` 加密后保存在 SQLite。公共数据和平台模型归系统所有,生辰资料仍按账号隔离。普通用户不配置 LLM,只有管理员授权的有效会员可以使用平台模型。请将 `.env` 与数据库一起备份,丢失加密密钥后无法恢复这些资料。
```bash
python tools/verify_baseline.py
```
## 系统与账号配置
涉及运行时或前端时再加 `--e2e`Playwright)。
管理员通过页面右上角“系统管理”保存公共 Tushare Token、平台主/辅助模型、会员每日额度和后台刷新开关。所有用户读取同一份 SQLite 行情快照,不再分别配置行情 Token。已有个人凭据中的 Tushare Token 会在升级时迁移到系统配置并从个人凭据移除。
## Docker 使用
局域网或服务器部署使用仓库根目录的 `Dockerfile``compose.yaml`。容器监听 `8765`,默认以非 root 用户运行,并把宿主机 `./data` 挂到容器内 `/app/data`
1. 复制 `.env.example``.env`,填入 `APP_ENCRYPTION_KEY` 以及行情 / 模型等初始化配置。密钥不会返回到浏览器。
2. `compose.yaml` 构建时要求带上当前 Git 提交号,避免打出无版本标签的镜像:
```bash
export XIAOBAI_GIT_REV="$(git rev-parse HEAD)"
export XIAOBAI_GIT_SHORT="$(git rev-parse --short=7 HEAD)"
docker compose build
docker compose up -d
```
3. 检查健康接口:
```bash
docker compose ps
curl http://127.0.0.1:8765/api/health
```
健康响应类似 `{"ok": true, "storage": "sqlite", "account_required": true}`
完整的迁移、持久化、防火墙、备份、恢复与正式线构建入口见 [DOCKER_DEPLOY.md](DOCKER_DEPLOY.md)。`.env` 必须与数据库成对备份;丢失 `APP_ENCRYPTION_KEY` 后无法恢复加密资料。
## 目录与配置
```text
TUSHARE_TOKEN=你的Token
server.py 进程入口
backend/ 服务、路由、数据、任务、LLM
frontend/ 无构建前端(shared + pages
config/ 页面 / API / 任务等注册表
data/ SQLite 与私有数据(数据库文件不进 Git)
runtime/ 本地日志、PID、缓存(不进 Git)
tools/ 启动、验收与构建辅助脚本
游资skills/ 公开问师角色
vendor/ 本地第三方库(含 lunar-python
Dockerfile
compose.yaml
.env.example 环境变量模板(复制为 .env 后填写)
```
`.env` 中的 Tushare 和平台 LLM 配置只用于初始化系统配置,密钥不会返回到浏览器。后台刷新只在交易时段更新 SQLite 快照,不会主动刷新或重绘用户页面;用户点击页面“刷新”时读取最新快照。管理员也可点“后台刷新”立即启动一次后台同步,当前页面仍保持不变
管理员通过页面右上角「系统管理」保存公共 Tushare Token、平台主/辅助模型、会员每日额度和后台刷新开关。所有用户读取同一份 SQLite 行情快照。`.env` 中的 Tushare 和平台 LLM 配置只用于初始化系统配置。
普通用户在账号设置中维护个人资料、查看会员状态和修改密码,不配置个人 LLM。有效会员自动使用平台模型;管理员可在“系统管理”中手动开通、续期、停用会员。平台模型受管理员设置的每日调用次数限制,管理员账号始终可用。
普通用户在账号设置中维护个人资料、查看会员状态和修改密码,不配置个人 LLM。有效会员使用平台模型;管理员可开通、续期、停用会员。平台模型受每日调用次数限制,管理员账号始终可用。
Tushare 各接口有独立积分权限。程序优先使用 `limit_list_d` 获取涨跌停明细;该接口不可用时,会尝试通过日线和每日涨跌停价格推算。
相关文档:
## 隔离实时聚合验证
- [ARCHITECTURE.md](ARCHITECTURE.md) — 模块边界
- [docs/README.md](docs/README.md) — 交接手册入口
- [DOCKER_DEPLOY.md](DOCKER_DEPLOY.md) — Docker 部署、备份与恢复
- [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) — 第三方授权(含问天历法库)
- [AGENTS.md](AGENTS.md) — 维护约束
`backend/data/realtime.py`用于验证东方财富、同花顺和选股宝网页数据源。它不写入 SQLite 主行情快照,也不参与情绪评分或智能选股;当 Tushare 实时指数权限不可用时,观势会使用东方财富三大指数和板块外显,并继续使用 Tushare 的板块成分内核与个股数据。
## 注意事项与免责声明
登录后可调用:
```text
GET /api/realtime-aggregate/health?sector=元器件
```
返回内容包括东方财富三大指数及板块快照、指数时间差、同花顺和选股宝可用性、每个来源的耗时与错误。盘中指数时间差不超过15秒,收盘后不超过120秒。`ready=true` 仅表示本次验证满足聚合层约束,不代表这些网页内部接口具有长期稳定性或商业使用授权
- 本项目是个人研究与复盘工具,全部数据、指标、候选与文字分析均不构成投资建议、证券推荐或买卖要约。
- 不接券商、不代为下单。交易日志只做手工记录与统计,不代表实际成交。
- 情绪温度、阶段判定、连板梯队、策略筛选等均为基于公开数据的统计与规则计算,不预测走势,不保证收益。
- 「问天」属于传统文化视角的观察工具,不具备预测功能,不得作为投资依据。问天不是永久冻结区:此前只冻结过界面视觉方案,现已解冻,后续数据与功能迁移可以纳入。
- 行情来自第三方接口,可能延迟、缺失或口径调整;不可用时页面会明确提示,请以交易所与券商正式披露为准。
- 不要把服务端口直接暴露到公网。不要把 Token、密码、密钥、数据库或 `.env` 提交进 Git。
- 股市有风险,入市需谨慎。投资决策及其后果由使用者本人承担
-17
View File
@@ -60,22 +60,6 @@ from backend.llm.service import LLMServiceMixin
from database import ReviewDatabase
LEGACY_SECRET_KEYS = {
"TUSHARE_TOKEN",
"IFIND_REFRESH_TOKEN",
"IFIND_ACCESS_TOKEN",
"LLM_API_KEY",
"LLM_BASE_URL",
"LLM_MODEL",
"LLM_PRIMARY_API_KEY",
"LLM_PRIMARY_BASE_URL",
"LLM_PRIMARY_MODEL",
"LLM_FALLBACK_API_KEY",
"LLM_FALLBACK_BASE_URL",
"LLM_FALLBACK_MODEL",
}
class DashboardService(
SystemServiceMixin,
AccountApplicationMixin,
@@ -121,7 +105,6 @@ class DashboardService(
self._system_credentials,
MENTOR_SKILLS_DIR,
PRIVATE_MENTOR_SKILLS_DIR,
lambda: self.token,
)
self.data_gateway = self.container.data_gateway
self.ifind = self.container.ifind
+5 -7
View File
@@ -2,7 +2,6 @@ from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from collections.abc import Callable
from backend.data import DataGateway, build_data_gateway
from backend.database.repositories import RepositoryBundle, build_repository_bundle
@@ -13,8 +12,8 @@ from backend.features.screener.engine import ScreenerEngine
from backend.features.screener.tracking import StrategyTrackingService
from backend.jobs import InProcessJobRunner, JobRegistry, SQLiteJobRunRepository
from database import ReviewDatabase
from backend.data.providers.ifind_client import IfindHttpClient
from backend.data.realtime import WebRealtimeAggregator
from backend.data.datahub.ifind_proxy import HubIfindProxy
from backend.data.datahub.realtime_proxy import HubRealtimeProxy
from backend.features.market.charts import MarketChartClient
@@ -23,13 +22,13 @@ class ApplicationContainer:
database: ReviewDatabase
repositories: RepositoryBundle
data_gateway: DataGateway
ifind: IfindHttpClient
ifind: HubIfindProxy
screener: ScreenerEngine
strategy_tracking: StrategyTrackingService
alert_service: AlertService
trade_journal: TradeJournalService
mentor_skills: MentorSkillRegistry
realtime_aggregator: WebRealtimeAggregator
realtime_aggregator: HubRealtimeProxy
chart_data: MarketChartClient
jobs: InProcessJobRunner
@@ -39,9 +38,8 @@ def build_application_container(
credentials: dict[str, object],
mentor_skills_dir: Path,
private_mentor_skills_dir: Path,
tushare_token_supplier: Callable[[], str] | None = None,
) -> ApplicationContainer:
data_gateway = build_data_gateway(credentials, tushare_token_supplier)
data_gateway = build_data_gateway(credentials)
repositories = build_repository_bundle(database)
jobs = InProcessJobRunner(JobRegistry.load(), SQLiteJobRunRepository(database))
return ApplicationContainer(
+12
View File
@@ -1,11 +1,23 @@
from __future__ import annotations
import argparse
import logging
from http.server import ThreadingHTTPServer
from typing import Any
def configure_logging() -> None:
"""让 INFO 级结构化日志(含 datahub 影子对比报告)落到容器日志。"""
if logging.getLogger().handlers:
return
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(name)s %(message)s",
)
def main(handler_class: type[Any] | None = None, service: Any | None = None) -> None:
configure_logging()
if handler_class is None or service is None:
from backend.application import RequestHandler, SERVICE
+2 -3
View File
@@ -10,9 +10,8 @@ from backend.features.accounts.security import SecretVault
def environment_credentials(environment: Mapping[str, str]) -> dict[str, str]:
return {
"tushare_token": str(environment.get("TUSHARE_TOKEN") or "").strip(),
"ifind_refresh_token": str(environment.get("IFIND_REFRESH_TOKEN") or "").strip(),
"ifind_access_token": str(environment.get("IFIND_ACCESS_TOKEN") or "").strip(),
"datahub_token": str(environment.get("DATAHUB_TOKEN") or "").strip(),
"datahub_base_url": str(environment.get("DATAHUB_BASE_URL") or "").strip(),
"platform_llm_primary_api_key": str(
environment.get("LLM_PRIMARY_API_KEY") or environment.get("LLM_API_KEY") or ""
).strip(),
+15
View File
@@ -0,0 +1,15 @@
from backend.data.datahub.bridge import DatahubAwareTushareClient, DatahubBridge
from backend.data.datahub.client import DatahubClient, DatahubResponse
from backend.data.datahub.errors import DatahubError
from backend.data.datahub.settings import DATASETS, DatahubSettings, DatasetFlags
__all__ = [
"DATASETS",
"DatahubAwareTushareClient",
"DatahubBridge",
"DatahubClient",
"DatahubError",
"DatahubResponse",
"DatahubSettings",
"DatasetFlags",
]
+605
View File
@@ -0,0 +1,605 @@
from __future__ import annotations
import logging
import sys
from threading import Lock
from typing import Any, Callable, ClassVar
from backend.data.datahub.client import DatahubClient, DatahubResponse
from backend.data.datahub.compare import compare_rows
from backend.data.datahub.errors import DatahubError
from backend.data.datahub.native import (
API_TO_DATASET,
filter_calendar_rows,
filter_stock_rows,
project_fields,
to_native_rows,
yyyymmdd,
)
from backend.data.datahub.redact import redact_text, redact_value
from backend.data.datahub.route_state import LEDGER
from backend.data.datahub.settings import DatahubSettings
from backend.data.providers.tushare_daily import DailyMarketMixin
from backend.data.providers.tushare_dashboard import DashboardMixin
from backend.data.providers.tushare_dragon_tiger import DragonTigerMixin
from backend.data.providers.tushare_indices import IndexMixin
from backend.data.providers.tushare_industries import ShenwanIndustryMixin
from backend.data.providers.tushare_sectors import SectorMixin
from backend.data.providers.tushare_stocks import StockMixin
from backend.data.providers.tushare_transport import TushareError
LOGGER = logging.getLogger("xiaobai.datahub")
ShadowSink = Callable[[dict[str, Any]], None]
def _usable_intraday_points(rows: list[Any]) -> list[dict[str, Any]]:
points: list[dict[str, Any]] = []
for row in rows:
if not isinstance(row, dict):
continue
try:
close = float(row.get("close") or 0)
except (TypeError, ValueError):
close = 0.0
if close <= 0:
continue
point = dict(row)
if "average" not in point and point.get("avg_price") is not None:
point["average"] = point.get("avg_price")
points.append(point)
return points
EMPTY_FAIL_DATASETS = {
"stocks", "daily", "index_daily", "valuation", "moneyflow", "auction",
"limit_events", "sector_daily",
}
def looks_like_heaven(module_name: str, filename: str = "") -> bool:
"""问天调用栈识别(诊断用)。问天按数据集依赖接入,不再整栈强制旧链路。"""
path = filename.replace("\\", "/")
return module_name.startswith("backend.features.heaven") or "/features/heaven/" in path
def caller_is_heaven(depth: int = 24) -> bool:
frame = sys._getframe(1)
for _ in range(depth):
frame = frame.f_back if frame is not None else None
if frame is None:
return False
name = str(frame.f_globals.get("__name__") or "")
filename = str(frame.f_code.co_filename or "")
if looks_like_heaven(name, filename):
return True
return False
class DatahubBridge:
def __init__(
self,
settings: DatahubSettings,
client: DatahubClient,
shadow_sink: ShadowSink | None = None,
heaven_guard: Callable[[], bool] | None = None,
) -> None:
self.settings = settings
self.client = client
self.shadow_sink = shadow_sink
self.heaven_guard = heaven_guard or caller_is_heaven
def dataset_status(self, trade_date: str) -> list[dict[str, Any]] | None:
flags = self.settings.flags("status")
if not flags.read and not flags.shadow:
return None
try:
response = self._require_fresh(self.client.dataset_status(yyyymmdd(trade_date)), "status")
rows = list(response.data or [])
if flags.shadow:
self._emit_shadow(compare_rows("status", [], rows, response.meta))
if flags.read:
return rows
return None
except Exception as exc:
self._log_failure("status", exc)
if flags.shadow:
self._emit_shadow(compare_rows("status", [], [], {}, self._error_text(exc)))
return None
def batches(self, trade_date: str, dataset: str = "") -> list[dict[str, Any]] | None:
flags = self.settings.flags("status")
if not flags.read:
return None
try:
response = self._require_fresh(
self.client.batches(yyyymmdd(trade_date), dataset),
"status",
)
return list(response.data or [])
except Exception as exc:
self._log_failure("status", exc)
return None
def try_intraday(self, code: str) -> dict[str, Any] | None:
flags = self.settings.flags("intraday")
if not flags.read:
return None
try:
response = self.client.intraday_points(code=code)
data = response.data
if not isinstance(data, dict):
raise DatahubError("EMPTY", "datahub intraday payload invalid")
points = _usable_intraday_points(data.get("points") or [])
if not points:
raise DatahubError("EMPTY", "datahub intraday empty")
if (response.meta or {}).get("stale"):
raise DatahubError("STALE", "datahub intraday stale")
self._record_route("intraday", "datahub", str((response.meta or {}).get("source") or "datahub"))
return {
"entity_type": str(data.get("entity_type") or "stock"),
"identifier": str(data.get("identifier") or code),
"name": str(data.get("name") or ""),
"code": str(data.get("code") or code),
"trade_date": str(data.get("trade_date") or points[-1].get("date") or ""),
"previous_close": float(data.get("previous_close") or 0),
"points": points,
"source": "datahub",
}
except Exception as exc:
self._log_failure("intraday", exc)
return None
def try_market_quotes(self, trade_date: str = "") -> list[dict[str, Any]] | None:
return self._try_quote_rows("quotes", {}, expected_date=trade_date, minimum=200)
def try_quotes(self, codes: list[str]) -> list[dict[str, Any]] | None:
cleaned = [str(item or "").strip() for item in codes if str(item or "").strip()]
if not cleaned:
return None
return self._try_quote_rows("quotes", {"codes": ",".join(cleaned)}, minimum=1)
def try_index_quotes(self) -> list[dict[str, Any]] | None:
flags = self.settings.flags("index_quotes")
if not flags.read:
return None
try:
response = self.client.index_quotes()
rows = [dict(item) for item in (response.data or []) if isinstance(item, dict)]
if len(rows) < 3:
raise DatahubError("EMPTY", "datahub index quotes incomplete")
if (response.meta or {}).get("stale"):
raise DatahubError("STALE", "datahub index quotes stale")
self._record_route(
"index_quotes",
"datahub",
str((response.meta or {}).get("source") or "datahub"),
)
return rows
except Exception as exc:
self._log_failure("index_quotes", exc)
return None
def try_sector_quote(self, code: str, trade_date: str = "") -> dict[str, Any] | None:
flags = self.settings.flags("quotes")
if not flags.read:
return None
try:
response = self.client.sector_quote(code, trade_date)
data = response.data
if not isinstance(data, dict) or not data:
raise DatahubError("EMPTY", "datahub sector quote empty")
row = dict(data)
if (response.meta or {}).get("stale"):
row["delayed"] = True
row["delay_seconds"] = int((response.meta or {}).get("staleness_seconds") or 0)
row["delay_notice"] = str((response.meta or {}).get("delay_notice") or "")
self._record_route("quotes", "datahub", str((response.meta or {}).get("source") or "datahub"))
return row
except Exception as exc:
self._log_failure("quotes", exc)
return None
def try_limit_pool(self, trade_date: str = "") -> list[dict[str, Any]] | None:
flags = self.settings.flags("limit_events")
if not flags.read:
return None
try:
response = self.client.limit_pool(trade_date)
rows = [dict(item) for item in (response.data or []) if isinstance(item, dict)]
if not rows:
raise DatahubError("EMPTY", "datahub limit pool empty")
self._record_route(
"limit_events",
"datahub",
str((response.meta or {}).get("source") or "datahub"),
)
return rows
except Exception as exc:
self._log_failure("limit_events", exc)
return None
def try_daily_chart(
self,
code: str,
end_date: str,
limit: int = 90,
dataset: str = "daily",
) -> list[dict[str, Any]] | None:
flags = self.settings.flags(dataset)
if not flags.read:
return None
compact_end = yyyymmdd(end_date)
if not compact_end:
return None
try:
start = _shift_yyyymmdd(compact_end, -max(190, int(limit) * 3))
if dataset == "index_daily":
response = self._paginate(
self.client.index_bars,
{"code": code, "from": start, "to": compact_end},
)
elif dataset == "sector_daily":
response = self._paginate(
self.client.sectors,
{"code": code, "from": start, "to": compact_end},
)
else:
response = self._paginate(
self.client.daily_bars,
{"code": code, "from": start, "to": compact_end, "adjust": "none"},
)
# Charts can use a partial history window; do not discard usable bars
# just because the requested lookback is not fully covered.
self._validate_usable(
dataset,
list(response.data or []),
response,
require_complete=False,
)
rows = _chart_bars(list(response.data or []))
if not rows:
raise DatahubError("EMPTY", f"{dataset} chart empty")
self._record_route(dataset, "datahub", str((response.meta or {}).get("source") or "datahub"))
return rows[-max(1, int(limit)):]
except Exception as exc:
self._log_failure(dataset, exc)
return None
def record_legacy(self, dataset: str, source: str = "", error: str = "") -> None:
self._record_route(dataset, "legacy", source, error)
def route_snapshot(self) -> list[dict[str, Any]]:
return LEDGER.snapshot()
def _try_quote_rows(
self,
dataset: str,
params: dict[str, Any],
expected_date: str = "",
minimum: int = 1,
) -> list[dict[str, Any]] | None:
flags = self.settings.flags(dataset)
if not flags.read:
return None
try:
response = self.client.quotes_latest(**params)
rows = [_native_quote(item) for item in (response.data or []) if isinstance(item, dict)]
rows = [item for item in rows if item]
want = yyyymmdd(expected_date)
if want:
dated = [item for item in rows if not item.get("quote_date") or item.get("quote_date") == want]
if dated:
rows = dated
if len(rows) < minimum:
raise DatahubError("EMPTY", f"datahub {dataset} empty")
stale = bool((response.meta or {}).get("stale"))
delay = int((response.meta or {}).get("staleness_seconds") or 0)
notice = str((response.meta or {}).get("delay_notice") or "")
source = str((response.meta or {}).get("source") or "datahub")
if stale:
for item in rows:
item["delayed"] = True
item["delay_seconds"] = delay
item["delay_notice"] = notice
item["source"] = source
self._record_route(dataset, "datahub", source)
return rows
except Exception as exc:
self._log_failure(dataset, exc)
return None
def query(
self,
api_name: str,
params: dict[str, Any] | None = None,
fields: str = "",
) -> list[dict[str, Any]]:
if api_name == "rt_sw_k":
raise TushareError("rt_sw_k is disabled; use published sw_daily or free Shenwan realtime")
dataset = API_TO_DATASET.get(api_name)
if dataset:
flags = self.settings.flags(dataset)
if flags.read:
try:
response = self._fetch_dataset(dataset, params or {}, api_name=api_name)
hub_canonical = self._extract_rows(dataset, response, params or {})
hub_rows = to_native_rows(dataset, hub_canonical)
self._validate_usable(dataset, hub_rows, response)
self._record_route(dataset, "datahub", str(response.meta.get("source") or "datahub"))
return project_fields(hub_rows, fields)
except Exception as exc:
self._log_failure(dataset, exc)
try:
response = self.client.query_api(api_name, params or {}, fields)
rows = [dict(item) for item in (response.data or []) if isinstance(item, dict)]
if dataset:
self._record_route(dataset, "datahub", str((response.meta or {}).get("source") or "datahub"))
else:
self._record_route(api_name, "datahub", str((response.meta or {}).get("source") or "datahub"))
return rows if not fields else project_fields(rows, fields)
except Exception as exc:
self._log_failure(dataset or api_name, exc)
raise TushareError(self._error_text(exc)) from exc
def _fetch_dataset(self, dataset: str, params: dict[str, Any], api_name: str = "") -> DatahubResponse:
date = yyyymmdd(params.get("trade_date") or params.get("date"))
start = yyyymmdd(params.get("start_date") or params.get("from") or date)
end = yyyymmdd(params.get("end_date") or params.get("to") or date)
code = str(params.get("ts_code") or params.get("code") or "").strip()
if dataset == "calendar":
if not start or not end:
raise DatahubError("INVALID_ARGUMENT", "calendar requires start_date and end_date")
return self.client.calendar(start, end)
if dataset == "stocks":
return self._paginate(self.client.stocks, {})
fetchers = {
"daily": self.client.daily_bars,
"index_daily": self.client.index_bars,
"valuation": self.client.valuation,
"moneyflow": self.client.moneyflow,
"auction": self.client.auction,
"limit_events": self.client.limit_events,
"popularity": self.client.popularity,
"dragon_tiger": self.client.dragon_tiger,
"sector_daily": self.client.sectors,
}
fetcher = fetchers[dataset]
query: dict[str, Any] = {}
if code:
query["code"] = code
if date and not (params.get("start_date") or params.get("end_date")):
query["date"] = date
else:
if start:
query["from"] = start
if end:
query["to"] = end
if dataset == "daily":
query["adjust"] = "none"
if dataset == "limit_events":
limit_type = str(params.get("limit_type") or "").strip().upper()
if limit_type:
query["limit_type"] = limit_type
if dataset == "popularity":
if api_name == "ths_hot":
query["source"] = "ths"
elif api_name == "dc_hot":
query["source"] = "dc"
if dataset == "sector_daily":
family = {
"ths_daily": "ths",
"dc_index": "dc",
"sw_daily": "sw",
}.get(api_name, "")
if family:
query["family"] = family
return self._paginate(fetcher, query)
def _paginate(self, fetcher: Callable[..., DatahubResponse], params: dict[str, Any]) -> DatahubResponse:
limit = self.settings.page_limit
offset = 0
rows: list[Any] = []
meta: dict[str, Any] = {}
schema_version = 1
while True:
page = fetcher(**{**params, "limit": limit, "offset": offset})
meta = dict(page.meta)
schema_version = page.schema_version
data = page.data or []
if not isinstance(data, list):
raise DatahubError("INTERNAL", "datahub returned a non-list payload")
rows.extend(data)
if len(data) < limit:
break
offset += limit
if offset > 200_000:
break
return DatahubResponse(data=rows, meta=meta, schema_version=schema_version)
def _extract_rows(
self,
dataset: str,
response: DatahubResponse,
params: dict[str, Any],
) -> list[dict[str, Any]]:
rows = [dict(item) for item in (response.data or [])]
if dataset == "calendar":
return filter_calendar_rows(rows, params)
if dataset == "stocks":
return filter_stock_rows(rows, params)
return rows
def _validate_usable(
self,
dataset: str,
rows: list[dict[str, Any]],
response: DatahubResponse,
require_complete: bool = True,
) -> None:
meta = response.meta or {}
stale_seconds = int(meta.get("staleness_seconds") or 0)
if meta.get("stale") or stale_seconds > self.settings.stale_seconds_max:
raise DatahubError("STALE", f"{dataset} data is stale")
if dataset in EMPTY_FAIL_DATASETS and not rows:
raise DatahubError("EMPTY", f"{dataset} returned no rows")
coverage = meta.get("coverage") if isinstance(meta.get("coverage"), dict) else {}
if require_complete and (meta.get("incomplete") is True or coverage.get("complete") is False):
missing = coverage.get("missing_count")
raise DatahubError("INCOMPLETE", f"{dataset} range is incomplete missing={missing}")
def _require_fresh(self, response: DatahubResponse, dataset: str) -> DatahubResponse:
self._validate_usable(dataset, list(response.data or []) if isinstance(response.data, list) else [], response)
return response
def _emit_shadow(self, report: dict[str, Any]) -> None:
safe = redact_value(report, secrets=self.settings.secrets())
LOGGER.info("datahub shadow %s", safe)
if self.shadow_sink is not None:
self.shadow_sink(report)
def _log_failure(self, dataset: str, exc: Exception) -> None:
error = redact_text(self._error_text(exc), self.settings.secrets())
LOGGER.warning("datahub unavailable dataset=%s error=%s", dataset, error)
self._record_route(dataset, "datahub", "unavailable", error)
def _record_route(self, dataset: str, route: str, source: str = "", error: str = "") -> None:
LEDGER.record(dataset, route, source, redact_text(error, self.settings.secrets()))
def _error_text(self, exc: Exception) -> str:
if isinstance(exc, DatahubError):
text = f"{exc.code}: {exc.message}"
else:
text = str(exc)
return redact_text(text, self.settings.secrets())
def _native_quote(row: dict[str, Any]) -> dict[str, Any] | None:
ts_code = str(row.get("ts_code") or "").strip()
close = _finite(row.get("close") if row.get("close") not in (None, "") else row.get("price"))
previous = _finite(
row.get("pre_close") if row.get("pre_close") not in (None, "") else row.get("previous_close")
)
if not ts_code or close <= 0 or previous <= 0:
return None
volume = _finite(row.get("vol") if row.get("vol") not in (None, "") else row.get("volume"))
payload = {
"ts_code": ts_code,
"name": str(row.get("name") or ts_code).strip(),
"pre_close": previous,
"open": _finite(row.get("open")),
"high": _finite(row.get("high")),
"low": _finite(row.get("low")),
"close": close,
"vol": volume,
"amount": _finite(row.get("amount")),
"num": 0,
"quote_date": yyyymmdd(row.get("quote_date") or row.get("trade_date")),
"source": str(row.get("source") or "datahub"),
}
if row.get("delayed"):
payload["delayed"] = True
payload["delay_seconds"] = int(row.get("delay_seconds") or 0)
payload["delay_notice"] = str(row.get("delay_notice") or "")
return payload
def _chart_bars(rows: list[Any]) -> list[dict[str, Any]]:
normalized: list[dict[str, Any]] = []
for row in rows:
if not isinstance(row, dict):
continue
compact = yyyymmdd(row.get("trade_date"))
close = _finite(row.get("close"))
if len(compact) != 8 or close <= 0:
continue
volume = _finite(row.get("volume") if row.get("volume") not in (None, "") else row.get("vol"))
amount = _finite(row.get("amount"))
if volume and volume < close * 10 and amount > 1000:
volume = volume * 100
trade_date = f"{compact[:4]}-{compact[4:6]}-{compact[6:8]}"
previous = normalized[-1]["close"] if normalized else 0.0
normalized.append(
{
"trade_date": trade_date,
"open": _finite(row.get("open")),
"high": _finite(row.get("high")),
"low": _finite(row.get("low")),
"close": close,
"change": round((close / previous - 1) * 100, 4) if previous else _finite(row.get("pct_chg")),
"volume": volume,
"amount_billion": amount / 100_000_000,
}
)
return normalized
def _shift_yyyymmdd(value: str, days: int) -> str:
from datetime import datetime, timedelta
stamp = datetime.strptime(value, "%Y%m%d")
return (stamp + timedelta(days=days)).strftime("%Y%m%d")
def _finite(value: Any) -> float:
try:
return float(value or 0)
except (TypeError, ValueError):
return 0.0
class DatahubAwareTushareClient(
DashboardMixin,
IndexMixin,
ShenwanIndustryMixin,
SectorMixin,
DragonTigerMixin,
StockMixin,
DailyMarketMixin,
):
"""Website market facade. Mixins call query(); query talks only to the hub."""
_realtime_reference_cache: ClassVar[dict[str, dict[str, Any]]] = {}
_realtime_reference_lock: ClassVar[Lock] = Lock()
_capital_cache: ClassVar[dict[str, dict[str, Any]]] = {}
_latest_realtime_market: ClassVar[dict[str, dict[str, Any]]] = {}
_stock_activity_cache: ClassVar[dict[str, dict[str, Any]]] = {}
_stock_listing_cache: ClassVar[dict[str, Any]] = {}
_stock_listing_lock: ClassVar[Lock] = Lock()
_suspension_cache: ClassVar[dict[str, dict[str, str] | None]] = {}
_suspension_lock: ClassVar[Lock] = Lock()
_sw_member_cache: ClassVar[dict[str, Any]] = {}
_sw_member_lock: ClassVar[Lock] = Lock()
def __init__(self, first: Any, second: Any | None = None) -> None:
# Production: DatahubAwareTushareClient(bridge)
# Older tests: DatahubAwareTushareClient(unused_legacy, bridge)
self._bridge = second if second is not None else first
self.token = "datahub"
self.timeout = 30
self.realtime_aggregator = None
def query(
self,
api_name: str,
params: dict[str, Any] | None = None,
fields: str = "",
) -> list[dict[str, Any]]:
return self._bridge.query(api_name, params, fields)
def try_market_quotes(self, trade_date: str = "") -> list[dict[str, Any]] | None:
return self._bridge.try_market_quotes(trade_date)
def try_quotes(self, codes: list[str]) -> list[dict[str, Any]] | None:
return self._bridge.try_quotes(codes)
def try_index_quotes(self) -> list[dict[str, Any]] | None:
return self._bridge.try_index_quotes()
def try_sector_quote(self, code: str, trade_date: str = "") -> dict[str, Any] | None:
return self._bridge.try_sector_quote(code, trade_date)
def try_limit_pool(self, trade_date: str = "") -> list[dict[str, Any]] | None:
return self._bridge.try_limit_pool(trade_date)
def record_datahub_legacy(self, dataset: str, source: str = "", error: str = "") -> None:
self._bridge.record_legacy(dataset, source, error)
+249
View File
@@ -0,0 +1,249 @@
from __future__ import annotations
import json
import logging
import urllib.error
import urllib.parse
import urllib.request
from dataclasses import dataclass, field
from typing import Any, Callable
from backend.data.datahub.errors import DatahubError
from backend.data.datahub.redact import redact_text
from backend.data.datahub.settings import DatahubSettings
LOGGER = logging.getLogger("xiaobai.datahub")
UrlOpen = Callable[..., Any]
@dataclass(frozen=True)
class DatahubResponse:
data: Any
meta: dict[str, Any] = field(default_factory=dict)
schema_version: int = 1
status: int = 200
class DatahubClient:
def __init__(
self,
settings: DatahubSettings,
urlopen: UrlOpen = urllib.request.urlopen,
) -> None:
self.settings = settings
self._urlopen = urlopen
def health(self) -> DatahubResponse:
return self.get("/v1/health")
def calendar(self, start: str, end: str) -> DatahubResponse:
return self.get("/v1/calendar", {"from": start, "to": end})
def stocks(self, updated_since: str = "", limit: int | None = None, offset: int = 0) -> DatahubResponse:
params: dict[str, Any] = {"offset": offset, "limit": limit or self.settings.page_limit}
if updated_since:
params["updated_since"] = updated_since
return self.get("/v1/stocks", params)
def daily_bars(self, **params: Any) -> DatahubResponse:
return self.get("/v1/bars/daily", params)
def index_bars(self, **params: Any) -> DatahubResponse:
return self.get("/v1/indexes/bars", params)
def valuation(self, **params: Any) -> DatahubResponse:
return self.get("/v1/valuation", params)
def moneyflow(self, **params: Any) -> DatahubResponse:
return self.get("/v1/moneyflow", params)
def auction(self, **params: Any) -> DatahubResponse:
return self.get("/v1/auction", params)
def limit_events(self, **params: Any) -> DatahubResponse:
return self.get("/v1/limit-events", params)
def popularity(self, **params: Any) -> DatahubResponse:
return self.get("/v1/popularity", params)
def dragon_tiger(self, **params: Any) -> DatahubResponse:
return self.get("/v1/dragon-tiger", params)
def sectors(self, **params: Any) -> DatahubResponse:
return self.get("/v1/sectors", params)
def quotes_latest(self, **params: Any) -> DatahubResponse:
return self.get("/v1/quotes/latest", params)
def index_quotes(self, **params: Any) -> DatahubResponse:
return self.get("/v1/indexes/quotes", params)
def intraday_points(self, **params: Any) -> DatahubResponse:
return self.get("/v1/intraday/points", params)
def dataset_status(self, date: str) -> DatahubResponse:
return self.get("/v1/datasets/status", {"date": date})
def batches(self, date: str, dataset: str = "") -> DatahubResponse:
params: dict[str, Any] = {"date": date}
if dataset:
params["dataset"] = dataset
return self.get("/v1/batches", params)
def query_api(self, api_name: str, params: dict[str, Any] | None = None, fields: str = "") -> DatahubResponse:
return self.post(
"/v1/query",
{"api_name": api_name, "params": params or {}, "fields": fields},
)
def sector_quote(self, code: str, date: str = "") -> DatahubResponse:
payload: dict[str, Any] = {"code": code}
if date:
payload["date"] = date
return self.get("/v1/sectors/quote", payload)
def limit_pool(self, trade_date: str = "") -> DatahubResponse:
params: dict[str, Any] = {}
if trade_date:
params["date"] = trade_date
return self.get("/v1/limit-pool", params)
def get(self, path: str, params: dict[str, Any] | None = None) -> DatahubResponse:
if not self.settings.token:
raise DatahubError("NOT_CONFIGURED", "DATAHUB_TOKEN is not configured")
query = {
key: value
for key, value in (params or {}).items()
if value is not None and value != ""
}
url = self.settings.base_url + path
if query:
url = f"{url}?{urllib.parse.urlencode(query)}"
attempts = 1 + max(0, self.settings.retries)
last_error: DatahubError | None = None
for attempt in range(attempts):
try:
return self._request(url)
except DatahubError as exc:
last_error = exc
if exc.code not in {"TIMEOUT", "UNAVAILABLE"} or attempt + 1 >= attempts:
raise
LOGGER.warning(
"datahub retry %s/%s %s",
attempt + 1,
attempts,
redact_text(str(exc), self.settings.secrets()),
)
raise last_error or DatahubError("INTERNAL", "datahub request failed")
def post(self, path: str, body: dict[str, Any] | None = None) -> DatahubResponse:
if not self.settings.token:
raise DatahubError("NOT_CONFIGURED", "DATAHUB_TOKEN is not configured")
url = self.settings.base_url + path
attempts = 1 + max(0, self.settings.retries)
last_error: DatahubError | None = None
payload = json.dumps(body or {}, ensure_ascii=False).encode("utf-8")
for attempt in range(attempts):
try:
return self._request(url, method="POST", data=payload)
except DatahubError as exc:
last_error = exc
if exc.code not in {"TIMEOUT", "UNAVAILABLE"} or attempt + 1 >= attempts:
raise
LOGGER.warning(
"datahub retry %s/%s %s",
attempt + 1,
attempts,
redact_text(str(exc), self.settings.secrets()),
)
raise last_error or DatahubError("INTERNAL", "datahub request failed")
def _request(self, url: str, method: str = "GET", data: bytes | None = None) -> DatahubResponse:
headers = {
"Accept": "application/json",
"X-Datahub-Token": self.settings.token,
"User-Agent": "XiaobaiReviewDatahub/1.0",
}
if data is not None:
headers["Content-Type"] = "application/json"
request = urllib.request.Request(
url,
data=data,
headers=headers,
method=method,
)
try:
with self._urlopen(request, timeout=self.settings.timeout_seconds) as response:
status = int(getattr(response, "status", 200) or 200)
raw = response.read().decode("utf-8")
except TimeoutError as exc:
raise DatahubError("TIMEOUT", "datahub request timed out") from exc
except urllib.error.HTTPError as exc:
body = _read_error_body(exc)
raise _http_error(exc.code, body, self.settings.secrets()) from exc
except urllib.error.URLError as exc:
reason = redact_text(str(getattr(exc, "reason", exc)), self.settings.secrets())
if "timed out" in reason.lower():
raise DatahubError("TIMEOUT", "datahub request timed out") from exc
raise DatahubError("UNAVAILABLE", f"datahub unavailable: {reason}") from exc
payload = _parse_json(raw, self.settings.secrets())
return _as_response(payload, status, self.settings.secrets())
def _parse_json(raw: str, secrets: tuple[str, ...]) -> dict[str, Any]:
try:
payload = json.loads(raw)
except json.JSONDecodeError as exc:
raise DatahubError("INTERNAL", "datahub returned invalid json") from exc
if not isinstance(payload, dict):
raise DatahubError("INTERNAL", "datahub returned a non-object payload")
return payload
def _as_response(payload: dict[str, Any], status: int, secrets: tuple[str, ...]) -> DatahubResponse:
error = payload.get("error")
if isinstance(error, dict):
raise _mapped_error(str(error.get("code") or "INTERNAL"), str(error.get("message") or "datahub error"), status)
if status >= 400:
raise DatahubError("UNAVAILABLE", f"datahub http {status}", status)
return DatahubResponse(
data=payload.get("data"),
meta=dict(payload.get("meta") or {}),
schema_version=int(payload.get("schema_version") or 1),
status=status,
)
def _http_error(status: int, payload: dict[str, Any], secrets: tuple[str, ...]) -> DatahubError:
error = payload.get("error") if isinstance(payload.get("error"), dict) else {}
code = str((error or {}).get("code") or "")
message = str((error or {}).get("message") or payload.get("message") or f"datahub http {status}")
message = redact_text(message, secrets)
if status == 401 or code == "UNAUTHORIZED":
return DatahubError("UNAUTHORIZED", message, status)
if status == 404 or code == "DATASET_NOT_PUBLISHED":
return DatahubError("DATASET_NOT_PUBLISHED", message, status)
if status == 400 or code == "INVALID_ARGUMENT":
return DatahubError("INVALID_ARGUMENT", message, status)
if status in {429, 503} or code in {"RATE_LIMITED", "SOURCE_UNAVAILABLE"}:
return DatahubError("UNAVAILABLE", message, status)
return DatahubError(code or "INTERNAL", message, status)
def _mapped_error(code: str, message: str, status: int) -> DatahubError:
if code == "STALE_DATA":
return DatahubError("STALE", message, status)
if code in {"UNAUTHORIZED", "DATASET_NOT_PUBLISHED", "INVALID_ARGUMENT"}:
return DatahubError(code, message, status)
if code in {"RATE_LIMITED", "SOURCE_UNAVAILABLE"}:
return DatahubError("UNAVAILABLE", message, status)
return DatahubError(code or "INTERNAL", message, status)
def _read_error_body(exc: urllib.error.HTTPError) -> dict[str, Any]:
try:
raw = exc.read().decode("utf-8")
payload = json.loads(raw)
return payload if isinstance(payload, dict) else {"message": raw}
except Exception:
return {"message": str(exc)}
+161
View File
@@ -0,0 +1,161 @@
from __future__ import annotations
from typing import Any
from backend.data.datahub.native import SCALE_FIELDS, row_key, to_canonical_row, yyyymmdd
NUMERIC_TOLERANCE = 1e-4
CANONICAL_ALIASES = {"volume": "vol"}
def compare_rows(
dataset: str,
legacy_rows: list[dict[str, Any]],
hub_rows: list[dict[str, Any]] | None,
hub_meta: dict[str, Any] | None = None,
hub_error: str | None = None,
fields: str = "",
) -> dict[str, Any]:
hub = hub_rows or []
requested = _requested_fields(fields)
legacy_map = {row_key(dataset, row): row for row in legacy_rows}
hub_map = {row_key(dataset, _align_hub_row(row)): row for row in hub}
missing_hub = sorted(key for key in legacy_map if key not in hub_map)
missing_legacy = sorted(key for key in hub_map if key not in legacy_map)
value_diffs: list[dict[str, Any]] = []
unit_conversion: list[dict[str, Any]] = []
matched = 0
for key, legacy in legacy_map.items():
hub_row = hub_map.get(key)
if hub_row is None:
continue
field_report = _compare_fields(dataset, legacy, hub_row, requested)
if field_report["unit_conversion"]:
unit_conversion.append({"key": list(key), "fields": field_report["unit_conversion"]})
if field_report["value_diff"]:
value_diffs.append({"key": list(key), "fields": field_report["value_diff"]})
if not field_report["unit_conversion"] and not field_report["value_diff"]:
matched += 1
stale_seconds = int((hub_meta or {}).get("staleness_seconds") or 0)
time_skew = bool((hub_meta or {}).get("stale")) or stale_seconds > 0
return {
"dataset": dataset,
"legacy_rows": len(legacy_rows),
"hub_rows": len(hub),
"matched": matched,
"missing_hub": [list(item) for item in missing_hub[:20]],
"missing_legacy": [list(item) for item in missing_legacy[:20]],
"missing_hub_count": len(missing_hub),
"missing_legacy_count": len(missing_legacy),
"value_diff_count": len(value_diffs),
"unit_conversion_count": len(unit_conversion),
"value_diffs": value_diffs[:20],
"unit_conversion": unit_conversion[:20],
"time_skew": time_skew,
"staleness_seconds": stale_seconds,
"published_at": (hub_meta or {}).get("published_at"),
"trade_date": yyyymmdd((hub_meta or {}).get("trade_date")),
"hub_error": hub_error,
"fields_compared": sorted(requested) if requested is not None else None,
"equal": (
not hub_error
and not missing_hub
and not missing_legacy
and not value_diffs
and not unit_conversion
and not time_skew
),
}
def _align_hub_row(row: dict[str, Any]) -> dict[str, Any]:
aligned = dict(row)
if "volume" in aligned and "vol" not in aligned:
aligned["vol"] = aligned.get("volume")
return aligned
def _requested_fields(fields: str) -> list[str] | None:
"""Fields the website actually asked for; None means "no projection"."""
keys = [item.strip() for item in str(fields or "").split(",") if item.strip()]
if not keys:
return None
seen: list[str] = []
for key in keys:
canonical = CANONICAL_ALIASES.get(key, key)
if canonical not in seen:
seen.append(canonical)
return seen
def _compare_fields(
dataset: str,
legacy: dict[str, Any],
hub: dict[str, Any],
requested: list[str] | None = None,
) -> dict[str, list[dict[str, Any]]]:
canonical_legacy = to_canonical_row(dataset, legacy)
hub_canonical = _hub_canonical(dataset, hub)
native_hub = _align_hub_row(hub)
value_diff: list[dict[str, Any]] = []
unit_conversion: list[dict[str, Any]] = []
keys = (set(canonical_legacy) | set(hub_canonical)) - {"batch_id", "updated_at", "volume"}
if requested is not None:
# Compare only what the website asked for. Extra hub columns are
# transport detail, not business differences; a requested field still
# alarms when it is missing or holds a different value.
keys = set(requested) - {"batch_id", "updated_at", "volume"}
scales = SCALE_FIELDS.get(dataset) or {}
for field in sorted(keys):
left = canonical_legacy.get(field)
right = hub_canonical.get(field)
if _same(left, right):
continue
native_left = legacy.get(field)
hub_raw = native_hub.get(field)
if field in scales and _near(_optional(native_left), _optional(hub_raw)):
unit_conversion.append(
{"field": field, "legacy": native_left, "hub": hub_raw, "reason": "unit_conversion"}
)
continue
value_diff.append({"field": field, "legacy": left, "hub": right, "reason": "value_diff"})
return {"value_diff": value_diff, "unit_conversion": unit_conversion}
def _hub_canonical(dataset: str, row: dict[str, Any]) -> dict[str, Any]:
"""Hub API rows are already canonical; only align field names."""
aligned = dict(row)
if "volume" in aligned and "vol" not in aligned:
aligned["vol"] = aligned.get("volume")
if dataset == "calendar":
is_open = aligned.get("is_open")
aligned["is_open"] = 1 if is_open in (True, 1, "1", "Y", "y") else 0
aligned["cal_date"] = yyyymmdd(aligned.get("cal_date"))
aligned["pretrade_date"] = yyyymmdd(aligned.get("pretrade_date")) or None
aligned["exchange"] = str(aligned.get("exchange") or "SSE")
return aligned
def _same(left: Any, right: Any) -> bool:
if left in (None, "") and right in (None, ""):
return True
if isinstance(left, (int, float)) or isinstance(right, (int, float)):
return _near(_optional(left), _optional(right))
return str(left or "") == str(right or "")
def _near(left: float | None, right: float | None) -> bool:
if left is None and right is None:
return True
if left is None or right is None:
return False
return abs(left - right) <= max(NUMERIC_TOLERANCE, abs(left) * 1e-9, abs(right) * 1e-9)
def _optional(value: Any) -> float | None:
if value in (None, ""):
return None
try:
return float(value)
except (TypeError, ValueError):
return None
+12
View File
@@ -0,0 +1,12 @@
from __future__ import annotations
class DatahubError(RuntimeError):
def __init__(self, code: str, message: str, status: int | None = None) -> None:
super().__init__(message)
self.code = code
self.message = message
self.status = status
def __str__(self) -> str:
return f"{self.code}: {self.message}"
+140
View File
@@ -0,0 +1,140 @@
from __future__ import annotations
import time
from typing import Any
from backend.data.datahub.bridge import DatahubBridge
from backend.data.datahub.errors import DatahubError
from backend.data.providers.ifind_client import IfindError
class HubIfindProxy:
"""Website-facing iFinD facade. Talks only to xiaobai-datahub."""
def __init__(self, datahub: DatahubBridge) -> None:
self._datahub = datahub
self._status: dict[str, Any] | None = None
self._status_at = 0.0
@property
def configured(self) -> bool:
return bool(self.status().get("configured"))
def status(self) -> dict[str, Any]:
now = time.monotonic()
if self._status is not None and now - self._status_at < 30:
return dict(self._status)
fallback = {"configured": False, "access_ready": False, "access_expires_at": ""}
if not self._datahub.settings.token:
self._status = fallback
self._status_at = now
return dict(fallback)
try:
rows = self._rows("ifind_status", {})
except IfindError:
self._status = fallback
self._status_at = now
return dict(fallback)
row = rows[0] if rows else {}
status = {
"configured": bool(row.get("configured")),
"access_ready": bool(row.get("access_ready")),
"access_expires_at": str(row.get("access_expires_at") or ""),
}
self._status = status
self._status_at = now
return dict(status)
def wencai(self, query: str, search_type: str = "stock", cache_ttl: int = 300) -> list[dict[str, Any]]:
return self._rows(
"ifind_wencai",
{"query": query, "search_type": search_type, "cache_ttl": cache_ttl},
)
def snapshots(
self,
codes: str | list[str],
indicators: list[str],
start_time: str,
end_time: str,
cache_ttl: int = 8,
) -> list[dict[str, Any]]:
return self._rows(
"ifind_snapshots",
{
"codes": codes,
"indicators": indicators,
"start_time": start_time,
"end_time": end_time,
"cache_ttl": cache_ttl,
},
)
def history(
self,
codes: str | list[str],
indicators: list[str],
start_date: str,
end_date: str,
cache_ttl: int = 300,
) -> list[dict[str, Any]]:
return self._rows(
"ifind_history",
{
"codes": codes,
"indicators": indicators,
"start_date": start_date,
"end_date": end_date,
"cache_ttl": cache_ttl,
},
)
def real_time(
self,
codes: str | list[str],
indicators: list[str],
cache_ttl: int = 10,
) -> list[dict[str, Any]]:
return self._rows(
"ifind_realtime",
{"codes": codes, "indicators": indicators, "cache_ttl": cache_ttl},
)
def intraday(
self,
code: str,
start_time: str,
end_time: str,
cache_ttl: int = 20,
) -> list[dict[str, Any]]:
return self._rows(
"ifind_intraday",
{
"code": code,
"start_time": start_time,
"end_time": end_time,
"cache_ttl": cache_ttl,
},
)
def test_connection(self) -> dict[str, Any]:
payload = self.real_time(
"000001.SH",
["open", "high", "low", "latest", "preClose"],
cache_ttl=0,
)
return {
"ok": bool(payload),
"sample_time": str(payload[0].get("time") or "") if payload else "",
}
def _rows(self, api_name: str, params: dict[str, Any]) -> list[dict[str, Any]]:
try:
response = self._datahub.client.query_api(api_name, params)
except DatahubError as exc:
raise IfindError(str(exc) or "iFinD 数据中枢暂不可用") from exc
data = response.data
if isinstance(data, list):
return [dict(item) for item in data if isinstance(item, dict)]
if isinstance(data, dict):
return [dict(data)]
return []
+204
View File
@@ -0,0 +1,204 @@
from __future__ import annotations
from typing import Any
from backend.data.numbers import finite_number
AMOUNT_THOUSAND_YUAN = 1000.0
AMOUNT_WAN_YUAN = 10000.0
VOLUME_LOT = 100.0
API_TO_DATASET = {
"trade_cal": "calendar",
"stock_basic": "stocks",
"daily": "daily",
"daily_basic": "valuation",
"index_daily": "index_daily",
"moneyflow": "moneyflow",
"stk_auction": "auction",
"limit_list_d": "limit_events",
"ths_hot": "popularity",
"dc_hot": "popularity",
"hm_detail": "dragon_tiger",
"ths_daily": "sector_daily",
"dc_index": "sector_daily",
"sw_daily": "sector_daily",
}
SCALE_FIELDS = {
"daily": {"vol": VOLUME_LOT, "amount": AMOUNT_THOUSAND_YUAN},
"index_daily": {"vol": VOLUME_LOT, "amount": AMOUNT_THOUSAND_YUAN},
"valuation": {"total_mv": AMOUNT_WAN_YUAN, "circ_mv": AMOUNT_WAN_YUAN},
"moneyflow": {
"buy_sm_amount": AMOUNT_WAN_YUAN,
"sell_sm_amount": AMOUNT_WAN_YUAN,
"buy_md_amount": AMOUNT_WAN_YUAN,
"sell_md_amount": AMOUNT_WAN_YUAN,
"buy_lg_amount": AMOUNT_WAN_YUAN,
"sell_lg_amount": AMOUNT_WAN_YUAN,
"buy_elg_amount": AMOUNT_WAN_YUAN,
"sell_elg_amount": AMOUNT_WAN_YUAN,
"net_mf_amount": AMOUNT_WAN_YUAN,
},
"auction": {"vol": VOLUME_LOT, "float_share": AMOUNT_WAN_YUAN},
"limit_events": {
"limit_amount": AMOUNT_WAN_YUAN,
"float_mv": AMOUNT_WAN_YUAN,
"total_mv": AMOUNT_WAN_YUAN,
},
"dragon_tiger": {
"buy_amount": AMOUNT_WAN_YUAN,
"sell_amount": AMOUNT_WAN_YUAN,
"net_amount": AMOUNT_WAN_YUAN,
},
}
def yyyymmdd(value: Any) -> str:
return str(value or "").replace("-", "")[:8]
def to_native_rows(dataset: str, rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
return [to_native_row(dataset, row) for row in rows]
def to_native_row(dataset: str, row: dict[str, Any]) -> dict[str, Any]:
if dataset == "calendar":
is_open = row.get("is_open")
return {
"exchange": str(row.get("exchange") or "SSE"),
"cal_date": yyyymmdd(row.get("cal_date")),
"is_open": 1 if is_open in (True, 1, "1", "Y", "y") else 0,
"pretrade_date": yyyymmdd(row.get("pretrade_date")) or None,
}
converted = dict(row)
converted.pop("batch_id", None)
if "volume" in converted and "vol" not in converted:
converted["vol"] = converted.pop("volume")
elif "volume" in converted:
converted.pop("volume", None)
scales = SCALE_FIELDS.get(dataset) or {}
for field, factor in scales.items():
if field in converted:
converted[field] = _unscale(converted.get(field), factor)
if dataset == "stocks":
converted.pop("updated_at", None)
if dataset == "popularity":
# keep hub source; callers filter ths/dc themselves when needed
if converted.get("ts_name") and not converted.get("name"):
converted["name"] = converted.get("ts_name")
if dataset == "dragon_tiger":
if converted.get("ts_name") and not converted.get("name"):
converted["name"] = converted.get("ts_name")
if dataset == "sector_daily":
if converted.get("pct_change") is not None and converted.get("pct_chg") is None:
converted["pct_chg"] = converted.get("pct_change")
return converted
def to_canonical_row(dataset: str, row: dict[str, Any]) -> dict[str, Any]:
if dataset == "calendar":
is_open = row.get("is_open")
return {
"exchange": str(row.get("exchange") or "SSE"),
"cal_date": yyyymmdd(row.get("cal_date")),
"is_open": 1 if is_open in (True, 1, "1", "Y", "y") else 0,
"pretrade_date": yyyymmdd(row.get("pretrade_date")) or None,
}
converted = dict(row)
if "volume" in converted and "vol" not in converted:
converted["vol"] = converted.pop("volume")
scales = SCALE_FIELDS.get(dataset) or {}
for field, factor in scales.items():
if field in converted:
converted[field] = _scale(converted.get(field), factor)
return converted
def row_key(dataset: str, row: dict[str, Any]) -> tuple[str, ...]:
if dataset == "calendar":
return (yyyymmdd(row.get("cal_date")),)
if dataset == "stocks":
return (str(row.get("ts_code") or "").upper(),)
if dataset == "status":
return (str(row.get("dataset") or ""), yyyymmdd(row.get("trade_date")))
if dataset == "limit_events":
return (
str(row.get("ts_code") or "").upper(),
yyyymmdd(row.get("trade_date")),
str(row.get("limit_type") or ""),
)
if dataset == "popularity":
return (
str(row.get("ts_code") or "").upper(),
yyyymmdd(row.get("trade_date")),
str(row.get("source") or ""),
)
if dataset == "dragon_tiger":
return (
str(row.get("ts_code") or "").upper(),
yyyymmdd(row.get("trade_date")),
str(row.get("hm_name") or ""),
)
if dataset == "sector_daily":
return (
str(row.get("ts_code") or "").upper(),
yyyymmdd(row.get("trade_date")),
str(row.get("family") or ""),
)
return (str(row.get("ts_code") or "").upper(), yyyymmdd(row.get("trade_date")))
def project_fields(rows: list[dict[str, Any]], fields: str) -> list[dict[str, Any]]:
keys = [item.strip() for item in str(fields or "").split(",") if item.strip()]
if not keys:
return rows
return [{key: row.get(key) for key in keys} for row in rows]
def filter_stock_rows(rows: list[dict[str, Any]], params: dict[str, Any] | None) -> list[dict[str, Any]]:
payload = params or {}
ts_code = str(payload.get("ts_code") or "").strip().upper()
status = str(payload.get("list_status") or "").strip()
name = str(payload.get("name") or "").strip()
filtered = rows
if ts_code:
filtered = [row for row in filtered if str(row.get("ts_code") or "").upper() == ts_code]
if status:
filtered = [row for row in filtered if str(row.get("list_status") or status) == status]
if name:
filtered = [row for row in filtered if name.casefold() in str(row.get("name") or "").casefold()]
return filtered
def filter_calendar_rows(rows: list[dict[str, Any]], params: dict[str, Any] | None) -> list[dict[str, Any]]:
payload = params or {}
if payload.get("is_open") in (1, "1", True):
return [row for row in rows if int(row.get("is_open") or 0) == 1]
if payload.get("is_open") in (0, "0", False):
return [row for row in rows if int(row.get("is_open") or 0) == 0]
return rows
def _scale(value: Any, factor: float) -> float | None:
number = _optional_number(value)
if number is None:
return None
return number * factor
def _unscale(value: Any, factor: float) -> float | None:
number = _optional_number(value)
if number is None or factor == 0:
return None
return number / factor
def _optional_number(value: Any) -> float | None:
if value in (None, ""):
return None
number = finite_number(value, default=float("nan"))
if number != number:
return None
return number
+180
View File
@@ -0,0 +1,180 @@
from __future__ import annotations
from datetime import datetime
from typing import Any
from backend.data.datahub.bridge import DatahubBridge
from backend.data.realtime import RealtimeAggregateError
class HubRealtimeProxy:
"""Realtime observation facade. Talks only to xiaobai-datahub."""
def __init__(self, datahub: DatahubBridge) -> None:
self._datahub = datahub
def health_snapshot(self, sector: str = "") -> dict[str, Any]:
started = datetime.now().astimezone()
indices: list[dict[str, Any]] = []
error = ""
try:
indices = self.tencent_indices()
except RealtimeAggregateError as exc:
error = str(exc)
epochs = [int(item.get("quote_time_epoch") or 0) for item in indices]
now = datetime.now().astimezone()
max_skew = 120 if now.hour >= 15 else 15
index_consistent = bool(epochs) and max(epochs) - min(epochs) <= max_skew
ready = len(indices) == 3 and index_consistent
return {
"ready": ready,
"isolated": True,
"generated_at": started.isoformat(timespec="seconds"),
"elapsed_ms": 0,
"indices": indices,
"index_consistent": index_consistent,
"sector": None,
"sources": {
"datahub_indices": {
"ok": ready,
"error": error,
"source": "datahub",
}
},
"observations": {},
"policy": {
"integration": "datahub_exclusive",
"max_index_time_skew_seconds": max_skew,
"notice": "实时观察只走数据中枢,主网站不再直连东财/腾讯。",
},
}
def tencent_indices(self) -> list[dict[str, Any]]:
rows = self._datahub.try_index_quotes() or []
result = [_as_index(item) for item in rows if _as_index(item)]
wanted = {"000001", "399001", "399006"}
result = [item for item in result if item.get("code") in wanted]
result.sort(key=lambda item: str(item.get("code") or ""))
if len(result) != 3:
raise RealtimeAggregateError(f"datahub returned {len(result)}/3 indices")
return result
def eastmoney_indices(self) -> list[dict[str, Any]]:
return self.tencent_indices()
def tencent_stock_quote(self, code: str, expected_date: str = "") -> dict[str, Any]:
return self._stock_quote(code, expected_date)
def eastmoney_stock_quote(self, code: str, expected_date: str = "") -> dict[str, Any]:
return self._stock_quote(code, expected_date)
def tencent_stock_quotes(
self,
codes: list[str],
expected_date: str = "",
minimum: int | None = None,
) -> list[dict[str, Any]]:
return self._stock_quotes(codes, expected_date, minimum)
def eastmoney_stock_quotes(
self,
codes: list[str],
expected_date: str = "",
) -> list[dict[str, Any]]:
return self._stock_quotes(codes, expected_date, None)
def eastmoney_shenwan_quote(self, ts_code: str, expected_date: str = "") -> dict[str, Any]:
quote = self._datahub.try_sector_quote(ts_code, expected_date)
if not quote:
raise RealtimeAggregateError(f"datahub shenwan quote unavailable for {ts_code}")
return quote
def _stock_quote(self, code: str, expected_date: str) -> dict[str, Any]:
rows = self._stock_quotes([code], expected_date, 1)
if not rows:
raise RealtimeAggregateError(f"datahub stock quote unavailable for {code}")
return rows[0]
def _stock_quotes(
self,
codes: list[str],
expected_date: str,
minimum: int | None,
) -> list[dict[str, Any]]:
cleaned = [str(item or "").strip() for item in codes if str(item or "").strip()]
rows = self._datahub.try_quotes(cleaned) if cleaned else (self._datahub.try_market_quotes(expected_date) or [])
quotes = [_as_stock(item) for item in (rows or []) if _as_stock(item)]
if expected_date:
compact = str(expected_date).replace("-", "")
quotes = [
item
for item in quotes
if not item.get("quote_date") or str(item.get("quote_date") or "").replace("-", "") == compact
]
if minimum is not None and len(quotes) < minimum:
raise RealtimeAggregateError(f"datahub returned {len(quotes)} quotes, need {minimum}")
return quotes
def _as_index(row: dict[str, Any]) -> dict[str, Any] | None:
code = str(row.get("code") or str(row.get("ts_code") or "").split(".")[0] or "")
price = _number(row.get("price") if row.get("price") not in (None, "") else row.get("close"))
if not code or price <= 0:
return None
epoch = int(_number(row.get("quote_time_epoch")))
amount = _number(row.get("amount_billion"))
if amount <= 0:
amount = round(_number(row.get("amount")) / 100_000_000, 2)
return {
"code": code,
"name": row.get("name") or code,
"price": price,
"change": _number(row.get("change") if row.get("change") not in (None, "") else row.get("pct_chg")),
"change_amount": _number(row.get("change_amount")),
"open": _number(row.get("open")),
"high": _number(row.get("high")),
"low": _number(row.get("low")),
"previous_close": _number(
row.get("previous_close") if row.get("previous_close") not in (None, "") else row.get("pre_close")
),
"amount_billion": amount,
"quote_time_epoch": epoch,
"quote_time": str(row.get("quote_time") or ""),
"source": str(row.get("source") or "datahub"),
"cache_age_seconds": 0,
}
def _as_stock(row: dict[str, Any]) -> dict[str, Any] | None:
close = _number(row.get("close") if row.get("close") not in (None, "") else row.get("price"))
if close <= 0:
return None
ts_code = str(row.get("ts_code") or "")
code = str(row.get("code") or ts_code.split(".")[0] or "")
return {
"ts_code": ts_code or code,
"code": code,
"name": row.get("name") or "",
"close": close,
"pre_close": _number(
row.get("pre_close") if row.get("pre_close") not in (None, "") else row.get("previous_close")
),
"open": _number(row.get("open")),
"high": _number(row.get("high")),
"low": _number(row.get("low")),
"volume": _number(row.get("volume") if row.get("volume") not in (None, "") else row.get("vol")),
"vol": _number(row.get("vol") if row.get("vol") not in (None, "") else row.get("volume")),
"amount": _number(row.get("amount")),
"quote_time_epoch": int(_number(row.get("quote_time_epoch"))),
"quote_time": str(row.get("quote_time") or ""),
"quote_date": str(row.get("quote_date") or ""),
"source": str(row.get("source") or "datahub"),
"delayed": bool(row.get("delayed")),
}
def _number(value: Any) -> float:
try:
return float(value or 0)
except (TypeError, ValueError):
return 0.0
+39
View File
@@ -0,0 +1,39 @@
from __future__ import annotations
from typing import Any
SECRET_HINTS = (
"token",
"password",
"secret",
"key",
"authorization",
"credential",
"cookie",
)
def redact_value(value: Any, key: str = "", secrets: tuple[str, ...] = ()) -> Any:
lowered = key.lower()
if any(part in lowered for part in SECRET_HINTS):
return "***"
if isinstance(value, dict):
return {
str(item_key): redact_value(item_value, str(item_key), secrets)
for item_key, item_value in value.items()
}
if isinstance(value, list):
return [redact_value(item, key, secrets) for item in value]
text = str(value) if value is not None and not isinstance(value, (int, float, bool)) else value
if isinstance(text, str):
return redact_text(text, secrets)
return value
def redact_text(text: str, secrets: tuple[str, ...] = ()) -> str:
redacted = text
for secret in secrets:
if secret:
redacted = redacted.replace(secret, "***")
return redacted
+57
View File
@@ -0,0 +1,57 @@
from __future__ import annotations
from datetime import datetime
from threading import Lock
from typing import Any
from backend.data.datahub.settings import DATASETS
DATASET_LABELS = {
"calendar": "交易日历",
"stocks": "股票主档",
"daily": "个股日K",
"index_daily": "指数日K",
"valuation": "估值",
"moneyflow": "资金流",
"auction": "竞价",
"limit_events": "涨停池",
"popularity": "人气榜",
"dragon_tiger": "龙虎榜",
"sector_daily": "题材板块",
"quotes": "全市场实时行情",
"index_quotes": "指数实时行情",
"intraday": "分时",
"status": "数据集状态",
}
class DatahubRouteLedger:
def __init__(self) -> None:
self._lock = Lock()
self._rows: dict[str, dict[str, Any]] = {}
def record(self, dataset: str, route: str, source: str = "", error: str = "") -> None:
name = str(dataset or "").strip() or "unknown"
with self._lock:
self._rows[name] = {
"dataset": name,
"label": DATASET_LABELS.get(name, name),
"route": "legacy" if route == "legacy" else "datahub",
"source": str(source or "").strip(),
"error": str(error or "").strip(),
"at": datetime.now().astimezone().isoformat(timespec="seconds"),
}
def snapshot(self) -> list[dict[str, Any]]:
with self._lock:
rows = [dict(item) for item in self._rows.values()]
order = {name: index for index, name in enumerate(DATASETS)}
rows.sort(key=lambda item: (order.get(str(item.get("dataset")), 99), str(item.get("dataset"))))
return rows
def clear(self) -> None:
with self._lock:
self._rows.clear()
LEDGER = DatahubRouteLedger()
+134
View File
@@ -0,0 +1,134 @@
from __future__ import annotations
import json
import os
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Mapping
from backend.bootstrap.config import APP_DIR
DATASETS = (
"calendar",
"stocks",
"daily",
"index_daily",
"valuation",
"moneyflow",
"auction",
"limit_events",
"popularity",
"dragon_tiger",
"sector_daily",
"quotes",
"index_quotes",
"intraday",
"status",
)
ENV_DATASET = {
"calendar": "CALENDAR",
"stocks": "STOCKS",
"daily": "DAILY",
"index_daily": "INDEX_DAILY",
"valuation": "VALUATION",
"moneyflow": "MONEYFLOW",
"auction": "AUCTION",
"limit_events": "LIMIT_EVENTS",
"popularity": "POPULARITY",
"dragon_tiger": "DRAGON_TIGER",
"sector_daily": "SECTOR_DAILY",
"quotes": "QUOTES",
"index_quotes": "INDEX_QUOTES",
"intraday": "INTRADAY",
"status": "STATUS",
}
DEFAULT_CONFIG_PATH = APP_DIR / "config" / "datahub.config.json"
def _truthy(value: Any) -> bool:
return str(value or "").strip().lower() in {"1", "true", "yes", "on"}
def _int(value: Any, default: int) -> int:
try:
return int(value)
except (TypeError, ValueError):
return default
@dataclass(frozen=True)
class DatasetFlags:
name: str
read: bool = False
shadow: bool = False
@dataclass(frozen=True)
class DatahubSettings:
base_url: str
token: str
timeout_seconds: int = 8
retries: int = 1
page_limit: int = 5000
stale_seconds_max: int = 86400
datasets: dict[str, DatasetFlags] | None = None
def flags(self, dataset: str) -> DatasetFlags:
mapped = self.datasets or {}
return mapped.get(dataset) or DatasetFlags(dataset)
def any_enabled(self) -> bool:
return any(item.read or item.shadow for item in (self.datasets or {}).values())
def secrets(self) -> tuple[str, ...]:
return tuple(item for item in (self.token,) if item)
@classmethod
def load(
cls,
path: Path | None = None,
environ: Mapping[str, str] | None = None,
credentials: Mapping[str, object] | None = None,
) -> "DatahubSettings":
config_path = path or DEFAULT_CONFIG_PATH
payload: dict[str, Any] = {}
if config_path.is_file():
payload = json.loads(config_path.read_text(encoding="utf-8"))
env = dict(os.environ if environ is None else environ)
creds = dict(credentials or {})
dataset_flags: dict[str, DatasetFlags] = {}
raw_datasets = payload.get("datasets") or {}
for name in DATASETS:
item = raw_datasets.get(name) or {}
env_key = ENV_DATASET[name]
read = _truthy(env.get(f"DATAHUB_READ_{env_key}")) if f"DATAHUB_READ_{env_key}" in env else bool(item.get("read"))
shadow = (
_truthy(env.get(f"DATAHUB_SHADOW_{env_key}"))
if f"DATAHUB_SHADOW_{env_key}" in env
else bool(item.get("shadow"))
)
dataset_flags[name] = DatasetFlags(name, read=read, shadow=shadow)
token = str(
env.get("DATAHUB_TOKEN")
or creds.get("datahub_token")
or payload.get("token")
or ""
).strip()
base_url = str(
env.get("DATAHUB_BASE_URL")
or creds.get("datahub_base_url")
or payload.get("base_url")
or "http://127.0.0.1:8766"
).strip().rstrip("/")
return cls(
base_url=base_url,
token=token,
timeout_seconds=_int(env.get("DATAHUB_TIMEOUT") or payload.get("timeout_seconds"), 8),
retries=max(0, _int(env.get("DATAHUB_RETRIES") or payload.get("retries"), 1)),
page_limit=max(1, _int(payload.get("page_limit"), 5000)),
stale_seconds_max=max(0, _int(payload.get("stale_seconds_max"), 86400)),
datasets=dataset_flags,
)
+50 -22
View File
@@ -1,40 +1,71 @@
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
from datetime import datetime
from typing import Any
from backend.data.contracts import DataUsage
from backend.data.datahub import DatahubAwareTushareClient, DatahubBridge, DatahubClient, DatahubSettings
from backend.data.datahub.ifind_proxy import HubIfindProxy
from backend.data.datahub.realtime_proxy import HubRealtimeProxy
from backend.data.policy import DataSourcePolicy
from backend.data.providers import IfindProvider, TushareProvider
from backend.data.providers import IfindProvider
from backend.data.quality import DataQualityGate, QualityEvidence, QualityReport
from backend.data.providers.ifind_client import IfindHttpClient
from backend.data.providers.tushare_client import TushareClient
from backend.data.realtime import WebRealtimeAggregator
from backend.features.market.charts import EastmoneyChartClient, MarketChartClient
from backend.features.market.charts import MarketChartClient
@dataclass(frozen=True)
class DataGateway:
policy: DataSourcePolicy
quality: DataQualityGate
tushare_provider: TushareProvider
ifind_provider: IfindProvider
chart_data: MarketChartClient
realtime_observer: WebRealtimeAggregator
realtime_observer: HubRealtimeProxy
datahub: DatahubBridge
@property
def ifind(self) -> IfindHttpClient:
def ifind(self) -> HubIfindProxy:
return self.ifind_provider.client
def tushare(
self,
dataset_id: str = "",
usage: DataUsage = "calculation",
) -> TushareClient:
) -> DatahubAwareTushareClient:
if dataset_id:
self.policy.assert_allowed(dataset_id, "tushare", usage)
return self.tushare_provider.client()
return DatahubAwareTushareClient(self.datahub)
def dataset_status(self, trade_date: str) -> list[dict[str, Any]] | None:
return self.datahub.dataset_status(trade_date)
def batches(self, trade_date: str, dataset: str = "") -> list[dict[str, Any]] | None:
return self.datahub.batches(trade_date, dataset)
def datahub_status(self) -> dict[str, Any]:
from backend.data.datahub.route_state import DATASET_LABELS, LEDGER
from backend.data.datahub.settings import DATASETS
settings = self.datahub.settings
flags = []
enabled = 0
for name in DATASETS:
read = bool(settings.flags(name).read)
if read:
enabled += 1
flags.append({"dataset": name, "label": DATASET_LABELS.get(name, name), "read": read})
routes = LEDGER.snapshot()
fallbacks = [item for item in routes if item.get("route") == "legacy"]
return {
"configured": bool(settings.token and settings.base_url),
"base_url": settings.base_url,
"enabled_reads": enabled,
"total_reads": len(DATASETS),
"flags": flags,
"routes": routes,
"fallback_count": len(fallbacks),
"fallback_labels": [str(item.get("label") or item.get("dataset")) for item in fallbacks],
}
def assert_source(self, dataset_id: str, provider_id: str, usage: DataUsage) -> None:
self.policy.assert_allowed(dataset_id, provider_id, usage)
@@ -63,21 +94,18 @@ class DataGateway:
def build_data_gateway(
credentials: dict[str, object],
tushare_token_supplier: Callable[[], str] | None = None,
datahub_settings: DatahubSettings | None = None,
) -> DataGateway:
ifind = IfindHttpClient(
str(credentials.get("ifind_refresh_token") or ""),
str(credentials.get("ifind_access_token") or ""),
)
token_supplier = tushare_token_supplier or (
lambda: str(credentials.get("tushare_token") or "")
)
policy = DataSourcePolicy.load()
settings = datahub_settings or DatahubSettings.load(credentials=credentials)
datahub_client = DatahubClient(settings)
datahub = DatahubBridge(settings, datahub_client)
ifind = HubIfindProxy(datahub)
return DataGateway(
policy=policy,
quality=DataQualityGate.load(policy),
tushare_provider=TushareProvider(token_supplier),
ifind_provider=IfindProvider(ifind),
chart_data=MarketChartClient(ifind, EastmoneyChartClient()),
realtime_observer=WebRealtimeAggregator(),
chart_data=MarketChartClient(datahub),
realtime_observer=HubRealtimeProxy(datahub),
datahub=datahub,
)
+5 -3
View File
@@ -1,11 +1,13 @@
from __future__ import annotations
from backend.data.providers.ifind_client import IfindHttpClient
from typing import Any
class IfindProvider:
def __init__(self, client: IfindHttpClient) -> None:
def __init__(self, client: Any) -> None:
self.client = client
def set_credentials(self, refresh_token: str, access_token: str = "") -> None:
self.client.set_credentials(refresh_token, access_token)
setter = getattr(self.client, "set_credentials", None)
if callable(setter):
setter(refresh_token, access_token)
+2
View File
@@ -66,3 +66,5 @@ class TushareClient(
_stock_listing_lock: ClassVar[Lock] = Lock()
_suspension_cache: ClassVar[dict[str, dict[str, str] | None]] = {}
_suspension_lock: ClassVar[Lock] = Lock()
_sw_member_cache: ClassVar[dict[str, Any]] = {}
_sw_member_lock: ClassVar[Lock] = Lock()
+76 -4
View File
@@ -3,7 +3,12 @@ from __future__ import annotations
from typing import Any
from backend.data.numbers import finite_number as _number
from backend.data.providers.tushare_helpers import _display_time, _prices_equal
from backend.data.providers.tushare_helpers import (
_display_time,
_optional_number,
_prices_equal,
calendar_is_open,
)
class DailyMarketMixin:
@@ -17,7 +22,11 @@ class DailyMarketMixin:
trade_date = requested
else:
row = requested_rows[0]
trade_date = row["cal_date"] if row.get("is_open") == 1 else row.get("pretrade_date", requested)
trade_date = (
row["cal_date"]
if calendar_is_open(row.get("is_open"))
else row.get("pretrade_date", requested)
)
resolved_rows = self.query(
"trade_cal",
@@ -129,7 +138,66 @@ class DailyMarketMixin:
)
item["capital_trade_date"] = str(capital.get("trade_date") or "")
result.append(item)
return result
return self._overlay_board_fields(result, trade_date)
def _overlay_board_fields(
self,
rows: list[dict[str, Any]],
trade_date: str,
) -> list[dict[str, Any]]:
if not rows:
return rows
official = self._official_board_map(trade_date)
free = self._free_board_map(trade_date) if not official else {}
merged: list[dict[str, Any]] = []
for row in rows:
code = str(row.get("ts_code") or "")
extra = official.get(code) or free.get(code) or {}
if not extra:
merged.append(row)
continue
item = dict(row)
for key in (
"first_time",
"last_time",
"fd_amount",
"open_times",
"limit_times",
"turnover_ratio",
):
incoming = extra.get(key)
current = item.get(key)
if incoming in (None, "", "--"):
continue
if current in (None, "", "--", 0, 0.0):
item[key] = incoming
merged.append(item)
return merged
def _official_board_map(self, trade_date: str) -> dict[str, dict[str, Any]]:
mapped: dict[str, dict[str, Any]] = {}
try:
for row in self._load_limit_lists(trade_date):
code = str(row.get("ts_code") or "")
if code:
mapped[code] = row
except Exception:
return {}
return mapped
def _free_board_map(self, trade_date: str) -> dict[str, dict[str, Any]]:
loader = getattr(self, "try_limit_pool", None)
if not callable(loader):
return {}
try:
rows = loader(trade_date) or []
except Exception:
return {}
return {
str(row.get("ts_code") or ""): row
for row in rows
if row.get("ts_code")
}
@staticmethod
def _normalize_limit(row: dict[str, Any], status: str) -> dict[str, Any]:
@@ -154,7 +222,11 @@ class DailyMarketMixin:
"turnover_source": row.get("turnover_source") or "provider",
"capital_trade_date": row.get("capital_trade_date") or "",
"amount_billion": round(amount_billion, 2),
"seal_amount_million": round(_number(row.get("fd_amount")) / 10000, 0),
"seal_amount_million": (
round(fd / 10000, 0)
if (fd := _optional_number(row.get("fd_amount"))) is not None
else None
),
"float_mv_billion": round(_number(row.get("float_mv")) / 100000000, 1),
"status": status,
}
+143 -17
View File
@@ -16,6 +16,12 @@ from backend.data.providers.tushare_transport import TushareError
class DashboardMixin:
def _now(self) -> datetime:
clock = getattr(self, "clock", None)
if callable(clock):
return clock()
return datetime.now().astimezone()
def dashboard(self, requested_date: str) -> dict[str, Any]:
trade_date, previous_trade_date = self.resolve_trade_context(requested_date)
if self.should_use_realtime(requested_date, trade_date):
@@ -26,11 +32,12 @@ class DashboardMixin:
)
daily = self._load_daily(trade_date)
now = self._now()
if (
not daily
and requested_date == datetime.now().astimezone().strftime("%Y%m%d")
and requested_date == now.strftime("%Y%m%d")
and trade_date == requested_date
and datetime.now().astimezone().time().replace(tzinfo=None) >= dt_time(9, 15)
and now.time().replace(tzinfo=None) >= dt_time(9, 15)
):
return self._realtime_dashboard(
requested_date,
@@ -41,13 +48,16 @@ class DashboardMixin:
raise TushareError(f"No daily data returned for {trade_date}")
notices: list[str] = []
limit_data_source = "official"
try:
limit_rows = self._load_limit_lists(trade_date)
previous_limit_rows = self._load_limit_type(previous_trade_date, "U")
if not limit_rows:
limit_data_source = "derived"
notices.append("涨跌停高级接口当日数据尚未更新,已使用日线数据推算。")
limit_rows = self._derive_limits(trade_date, daily)
except TushareError as exc:
limit_data_source = "derived"
notices.append(f"涨跌停高级接口不可用,已使用日线数据推算:{exc}")
limit_rows = self._derive_limits(trade_date, daily)
previous_daily = self._load_daily(previous_trade_date)
@@ -79,6 +89,7 @@ class DashboardMixin:
"trade_date": _display_date(trade_date),
"previous_trade_date": _display_date(previous_trade_date),
"source": "tushare",
"limit_data_source": limit_data_source,
"updated_at": datetime.now().astimezone().isoformat(timespec="seconds"),
"notice": "".join(notices),
},
@@ -94,15 +105,14 @@ class DashboardMixin:
}
return apply_sentiment_to_dashboard(dashboard)
@staticmethod
def should_use_realtime(requested_date: str, trade_date: str) -> bool:
"""Use rt_k for today's open market until end-of-day datasets settle."""
now = datetime.now().astimezone()
def should_use_realtime(self, requested_date: str, trade_date: str) -> bool:
"""Use live quotes for today's open session until official daily settles."""
now = self._now()
today = now.strftime("%Y%m%d")
return (
requested_date == today
and trade_date == today
and dt_time(9, 15) <= now.time().replace(tzinfo=None) < dt_time(16, 30)
and dt_time(9, 15) <= now.time().replace(tzinfo=None) < dt_time(15, 5)
)
def _realtime_dashboard(
@@ -118,7 +128,7 @@ class DashboardMixin:
)
if not codes:
raise TushareError("No active stock codes available for rt_k")
quotes = self.query("rt_k", {"ts_code": codes})
quotes, quote_source = self._load_realtime_quotes(codes, trade_date)
if not quotes:
raise TushareError(f"No realtime data returned for {trade_date}")
@@ -174,14 +184,35 @@ class DashboardMixin:
)
sectors = _build_sectors(limits)
previous_sectors = _build_sectors(previous_limits)
now = datetime.now().astimezone()
now = self._now()
market_status = _realtime_market_status(now.time().replace(tzinfo=None))
if quote_source == "datahub":
notice = (
"盘中行情由数据中枢统一提供;涨停原因、封板时间和开板次数以盘后榜单校正为准。"
)
source_name = "datahub"
elif quote_source == "eastmoney_clist":
notice = (
"盘中行情由东财免费实时快照计算;涨停原因、封板时间和开板次数以盘后榜单校正为准。"
)
source_name = "eastmoney"
elif quote_source == "tencent_qt":
notice = (
"盘中行情由腾讯免费实时行情计算;涨停原因、封板时间和开板次数以盘后榜单校正为准。"
)
source_name = "tencent"
else:
notice = (
"盘中行情由 Tushare rt_k 实时计算;涨停原因、封板时间和开板次数以盘后榜单校正为准。"
)
source_name = "tushare"
dashboard = {
"meta": {
"requested_date": _display_date(requested_date),
"trade_date": _display_date(trade_date),
"previous_trade_date": _display_date(previous_trade_date),
"source": "tushare",
"source": source_name,
"quote_source": quote_source,
"mode": "realtime",
"realtime": True,
"market_status": market_status,
@@ -189,7 +220,8 @@ class DashboardMixin:
"auto_refresh": False,
"quote_count": len(daily),
"updated_at": now.isoformat(timespec="seconds"),
"notice": "盘中行情由 Tushare rt_k 实时计算;涨停原因、封板时间和开板次数以盘后榜单校正为准。",
"notice": notice,
"indices": self._free_realtime_indices() if quote_source != "tushare_rt_k" else [],
},
"overview": _build_overview(daily, up_rows, down_rows, broken_rows),
"limits": limits,
@@ -203,6 +235,62 @@ class DashboardMixin:
}
return apply_sentiment_to_dashboard(dashboard)
def _realtime_aggregator(self):
aggregator = getattr(self, "realtime_aggregator", None)
if aggregator is None:
raise TushareError("免费实时源未配置")
return aggregator
def _load_realtime_quotes(
self,
codes: str,
trade_date: str,
) -> tuple[list[dict[str, Any]], str]:
hub = getattr(self, "try_market_quotes", None)
if callable(hub):
quotes = hub(trade_date)
if quotes:
return list(quotes), "datahub"
named = getattr(self, "try_quotes", None)
code_list = [item for item in str(codes or "").split(",") if item]
if callable(named) and code_list:
collected: list[dict[str, Any]] = []
for index in range(0, len(code_list), 60):
collected.extend(named(code_list[index:index + 60]) or [])
if collected:
delayed = any(item.get("delayed") for item in collected)
return collected, "datahub_delayed" if delayed else "datahub"
try:
quotes = self.query("rt_k", {"ts_code": codes})
if quotes:
delayed = any(item.get("delayed") for item in quotes)
return list(quotes), "datahub_delayed" if delayed else "datahub"
except TushareError as exc:
raise TushareError(f"当天盘中实时行情不可用:{exc}") from exc
raise TushareError("当天盘中实时行情不可用:数据中枢未返回可用行情")
def _mark_quote_legacy(self, source: str, error: str = "") -> None:
marker = getattr(self, "record_datahub_legacy", None)
if callable(marker):
marker("quotes", source, error)
def _free_realtime_quotes(
self,
trade_date: str,
codes: str = "",
) -> tuple[list[dict[str, Any]], str]:
del trade_date, codes
raise TushareError("主网站不再直连免费行情源,请走数据中枢")
def _free_realtime_indices(self) -> list[dict[str, Any]]:
hub = getattr(self, "try_index_quotes", None)
if callable(hub):
rows = hub()
converted = [item for item in (_hub_index_quote(row) for row in rows or []) if item]
if converted:
return converted
return []
def _load_realtime_reference(
self,
trade_date: str,
@@ -230,7 +318,7 @@ class DashboardMixin:
{"trade_date": previous_trade_date},
"ts_code,trade_date,total_share,float_share,free_share,total_mv,circ_mv",
)
if not basic_rows or not price_limits:
if not basic_rows:
raise TushareError(f"Realtime reference data is incomplete for {trade_date}")
result = {
"basic_rows": basic_rows,
@@ -250,10 +338,9 @@ class DashboardMixin:
ts_code: str,
reference_date: str = "",
) -> dict[str, Any]:
rows = self.query("rt_k", {"ts_code": ts_code})
if not rows:
row = self._realtime_quote_row(ts_code, reference_date)
if not row:
raise TushareError(f"No realtime quote returned for {ts_code}")
row = rows[0]
close = _number(row.get("close"))
previous_close = _number(row.get("pre_close"))
if close <= 0 or previous_close <= 0:
@@ -337,10 +424,24 @@ class DashboardMixin:
"float_share_10k": float_share,
"capital_trade_date": str(capital.get("trade_date") or ""),
"turnover_source": "rt_volume/latest_float_share" if float_share else "unavailable",
"data_source": "tushare",
"data_source": str(row.get("source") or "tushare"),
"realtime": True,
}
def _realtime_quote_row(self, ts_code: str, reference_date: str = "") -> dict[str, Any]:
hub = getattr(self, "try_quotes", None)
if callable(hub):
rows = hub([ts_code]) or []
if rows:
return dict(rows[0])
try:
rows = self.query("rt_k", {"ts_code": ts_code})
if rows:
return dict(rows[0])
except TushareError:
pass
return {}
def _stock_activity_metrics(
self,
ts_code: str,
@@ -458,7 +559,7 @@ class DashboardMixin:
for row in reference.get("basic_rows") or []
if row.get("ts_code")
]
quotes = self.query("rt_k", {"ts_code": ",".join(codes)}, "")
quotes, quote_source = self._load_realtime_quotes(",".join(codes), trade_date)
rows = [
row for row in quotes
if _number(row.get("close")) > 0 and _number(row.get("pre_close")) > 0
@@ -604,6 +705,31 @@ def _build_yesterday_performance(
return result
def _hub_index_quote(row: dict[str, Any]) -> dict[str, Any] | None:
ts_code = str(row.get("ts_code") or "")
code = str(row.get("code") or ts_code.split(".")[0])
close = _number(row.get("price") if row.get("price") not in (None, "") else row.get("close"))
previous = _number(
row.get("previous_close") if row.get("previous_close") not in (None, "") else row.get("pre_close")
)
if close <= 0 or previous <= 0:
return None
amount = _number(row.get("amount"))
amount_billion = _number(row.get("amount_billion"))
if not amount_billion and amount:
amount_billion = round(amount / 100_000_000, 2)
return {
"code": code,
"name": str(row.get("name") or code),
"price": close,
"change": _number(row.get("pct_chg") if row.get("pct_chg") not in (None, "") else row.get("change")),
"previous_close": previous,
"amount_billion": amount_billion,
"quote_time": str(row.get("quote_time") or ""),
"source": "datahub",
}
def _build_limit_performance(rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
result = []
for level in sorted({int(row.get("prior_streak") or 1) for row in rows}, reverse=True):
+60
View File
@@ -6,12 +6,72 @@ from typing import Any
from backend.data.numbers import finite_number as _number
def calendar_is_open(value: Any) -> bool:
if value in (True, 1, "1", "Y", "y"):
return True
if value in (False, 0, "0", "N", "n", None, ""):
return False
try:
return int(value) == 1
except (TypeError, ValueError):
return False
def _text(value: Any) -> str:
if isinstance(value, (list, tuple, set)):
return "".join(str(item).strip() for item in value if str(item).strip())
return str(value or "").strip()
def _optional_number(value: Any) -> float | None:
if value in (None, "", "-"):
return None
number = _number(value, default=float("nan"))
if number != number:
return None
return number
def _moneyflow_payload(flow: dict[str, Any] | None) -> dict[str, Any]:
if not flow:
return {
"available": False,
"net_million": None,
"large_million": None,
"medium_million": None,
"small_million": None,
}
net = _optional_number(flow.get("net_mf_amount"))
buy_lg = _optional_number(flow.get("buy_lg_amount"))
sell_lg = _optional_number(flow.get("sell_lg_amount"))
buy_elg = _optional_number(flow.get("buy_elg_amount"))
sell_elg = _optional_number(flow.get("sell_elg_amount"))
buy_md = _optional_number(flow.get("buy_md_amount"))
sell_md = _optional_number(flow.get("sell_md_amount"))
buy_sm = _optional_number(flow.get("buy_sm_amount"))
sell_sm = _optional_number(flow.get("sell_sm_amount"))
large = None
if None not in (buy_lg, sell_lg, buy_elg, sell_elg):
large = (buy_lg + buy_elg - sell_lg - sell_elg)
elif _optional_number(flow.get("large_amount")) is not None:
large = _optional_number(flow.get("large_amount"))
medium = None if None in (buy_md, sell_md) else (buy_md - sell_md)
if medium is None:
medium = _optional_number(flow.get("medium_amount"))
small = None if None in (buy_sm, sell_sm) else (buy_sm - sell_sm)
if small is None:
small = _optional_number(flow.get("small_amount"))
if net is None and large is None and medium is None and small is None:
return _moneyflow_payload(None)
return {
"available": True,
"net_million": None if net is None else round(net / 100, 2),
"large_million": None if large is None else round(large / 100, 2),
"medium_million": None if medium is None else round(medium / 100, 2),
"small_million": None if small is None else round(small / 100, 2),
}
def _prices_equal(left: Any, right: Any) -> bool:
if left is None or right is None:
return False
+71
View File
@@ -59,6 +59,73 @@ class IndexMixin:
}
def realtime_market_indices(self, requested_date: str) -> dict[str, Any]:
hub = getattr(self, "try_index_quotes", None)
if callable(hub):
rows = hub()
if rows:
return self._hub_realtime_market_indices(requested_date, rows)
raise TushareError("Realtime index quotes are incomplete")
def _hub_realtime_market_indices(
self,
requested_date: str,
rows: list[dict[str, Any]],
) -> dict[str, Any]:
trade_date, _ = self.resolve_trade_context(requested_date)
index_names = {
"000001.SH": "上证指数",
"399001.SZ": "深证成指",
"399006.SZ": "创业板指",
}
by_code = {str(row.get("ts_code") or ""): row for row in rows}
by_symbol = {str(row.get("code") or ""): row for row in rows}
indices = []
for ts_code, name in index_names.items():
row = by_code.get(ts_code) or by_symbol.get(ts_code.split(".")[0])
if not row:
continue
close = _number(row.get("price") if row.get("price") not in (None, "") else row.get("close"))
previous_close = _number(
row.get("previous_close") if row.get("previous_close") not in (None, "") else row.get("pre_close")
)
if close <= 0 or previous_close <= 0:
continue
amount = _number(row.get("amount"))
amount_billion = _number(row.get("amount_billion"))
if not amount_billion and amount:
amount_billion = round(amount / 100_000_000, 2)
indices.append(
{
"ts_code": ts_code,
"name": str(row.get("name") or name).strip(),
"trade_date": trade_date,
"close": close,
"pct_chg": round(
_number(row.get("pct_chg")) or (close / previous_close - 1) * 100,
3,
),
"return_5d": 0,
"amount_billion": amount_billion,
"quote_time": str(row.get("quote_time") or ""),
"source": "datahub",
}
)
if len(indices) != 3:
raise TushareError("Realtime index quotes are incomplete")
return {
"trade_date": trade_date,
"source": "datahub",
"realtime": True,
"precise": True,
"indices": indices,
"aggregate": {
"average_pct_chg": round(sum(item["pct_chg"] for item in indices) / len(indices), 3),
"average_return_5d": 0,
"average_return_20d": 0,
},
}
def _tushare_realtime_market_indices(self, requested_date: str) -> dict[str, Any]:
trade_date, _ = self.resolve_trade_context(requested_date)
index_names = {
"000001.SH": "上证指数",
@@ -116,3 +183,7 @@ class IndexMixin:
"average_return_20d": 0,
},
}
def _free_realtime_market_indices(self, requested_date: str) -> dict[str, Any]:
del requested_date
raise TushareError("主网站不再直连免费行情源,请走数据中枢")
+318 -75
View File
@@ -1,11 +1,16 @@
from __future__ import annotations
import json
from datetime import datetime, timedelta
from pathlib import Path
from typing import Any
from backend.data.numbers import finite_number as _number
from backend.data.providers.tushare_transport import TushareError
_SW_MEMBER_TTL = timedelta(hours=24)
_SW_MEMBER_DIR = Path(__file__).resolve().parents[3] / "data" / "cache" / "sw_members"
class ShenwanIndustryMixin:
def sw_stock_industry(self, ts_code: str, trade_date: str) -> dict[str, Any]:
@@ -132,21 +137,63 @@ class ShenwanIndustryMixin:
actual_trade_date = str(daily.get("trade_date") or "")
outer_precise = actual_trade_date == trade_date
outer_error = "" if outer_precise else (
f"No Shenwan daily returned for {sector_code} on {trade_date}"
f"申万行业 {sector_code} 当日盘后正式数据尚未入库"
)
outer_source = "tushare_sw_daily" if outer_precise else "unavailable"
if not outer_precise and allow_realtime_close:
try:
return self._sw_realtime_sector_snapshot(
industry,
members,
inner_ok = bool(member_rows) and not coverage_issue
if inner_ok:
sw_row, rt_source, rt_error = self._sw_outer_realtime(
sector_code,
str(industry.get("l2_name") or ""),
trade_date,
previous_trade_date,
finalized=True,
)
except TushareError as exc:
outer_error = f"{outer_error}; realtime close fallback failed: {exc}"
if sw_row:
daily = sw_row
actual_trade_date = str(
sw_row.get("quote_date") or sw_row.get("trade_date") or ""
)
trade_time = str(sw_row.get("trade_time") or sw_row.get("quote_time") or "")
quote_clock = (
trade_time[11:19]
if len(trade_time) >= 19
else str(sw_row.get("quote_clock") or "")
)
outer_precise = actual_trade_date == trade_date
if quote_clock and quote_clock < "15:00:00":
outer_precise = False
outer_source = rt_source or "eastmoney_sw"
outer_error = "" if outer_precise else (
rt_error or f"申万行业 {sector_code} 免费实时尚未形成收盘快照"
)
else:
outer_error = rt_error or outer_error
else:
try:
snapshot = self._sw_realtime_sector_snapshot(
industry,
members,
trade_date,
previous_trade_date,
finalized=True,
)
snapshot.update({
"raw_member_count": raw_member_count,
"excluded_member_count": len(excluded_members),
"excluded_members": excluded_members,
})
return snapshot
except TushareError:
outer_error = f"{outer_error}; 免费实时成分暂不可用"
official_change = _number(daily.get("pct_change")) if outer_precise else None
official_change = None
if outer_precise:
official_change = _number(
daily.get("pct_change")
if daily.get("pct_change") not in (None, "")
else daily.get("change")
)
return {
"code": sector_code,
"name": industry.get("l2_name") or daily.get("name") or sector_code,
@@ -173,9 +220,9 @@ class ShenwanIndustryMixin:
"amount_billion": round(amount_billion, 2),
"count": 0,
"max_streak": 0,
"source": "tushare_sw_daily+member_daily" if outer_precise else "tushare_member_daily",
"source": f"{outer_source}+tushare_member_daily" if outer_precise else "tushare_member_daily",
"inner_source": "tushare_member_daily",
"outer_source": "tushare_sw_daily" if outer_precise else "unavailable",
"outer_source": outer_source,
"taxonomy": "sw_l2",
"industry": industry,
"trade_date": trade_date,
@@ -189,7 +236,7 @@ class ShenwanIndustryMixin:
"inner_error": inner_error,
"outer_error": outer_error,
"schema_version": 6,
"methodology": "外显使用申万二级行业官方日线;内核独立使用当日成分日线宽度与等权涨跌聚合",
"methodology": "外显使用已发布 sw_daily 或免费申万实时;内核优先使用当日成分日线,不调用 rt_sw_k",
}
def _sw_sector_members(
@@ -197,23 +244,100 @@ class ShenwanIndustryMixin:
sector_code: str,
trade_date: str,
) -> list[dict[str, Any]]:
rows = []
for is_new in ("Y", "N"):
rows.extend(
self.query(
"index_member_all",
{"l2_code": sector_code, "is_new": is_new},
"l2_code,l2_name,ts_code,name,in_date,out_date,is_new",
cached_rows = self._read_local_sw_members(sector_code)
if cached_rows is not None:
return _active_members(cached_rows, trade_date)
rows: list[dict[str, Any]] = []
try:
for is_new in ("Y", "N"):
rows.extend(
self.query(
"index_member_all",
{"l2_code": sector_code, "is_new": is_new},
"l2_code,l2_name,ts_code,name,in_date,out_date,is_new",
)
)
except TushareError:
stale = self._read_local_sw_members(sector_code, allow_stale=True) or []
if stale:
return _active_members(stale, trade_date)
raise
reconciled = _reconcile_membership_rows(rows)
self._write_local_sw_members(sector_code, reconciled)
return _active_members(reconciled, trade_date)
def _read_local_sw_members(
self,
sector_code: str,
allow_stale: bool = False,
) -> list[dict[str, Any]] | None:
now = datetime.now().astimezone()
cache = getattr(self, "_sw_member_cache", None)
lock = getattr(self, "_sw_member_lock", None)
if isinstance(cache, dict) and lock is not None:
with lock:
packed = cache.get(sector_code)
if isinstance(packed, dict):
loaded_at = packed.get("loaded_at")
rows = packed.get("rows")
fresh = (
isinstance(loaded_at, datetime)
and now - loaded_at < _SW_MEMBER_TTL
)
if isinstance(rows, list) and (fresh or allow_stale):
return [dict(item) for item in rows]
path = _sw_member_path(sector_code)
if not path.exists():
return None
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
rows = list(payload.get("rows") or [])
updated = str(payload.get("updated_at") or "")
fresh = False
try:
stamped = datetime.fromisoformat(updated)
if stamped.tzinfo is None:
stamped = stamped.replace(tzinfo=now.tzinfo)
fresh = now - stamped.astimezone(now.tzinfo) < _SW_MEMBER_TTL
except ValueError:
fresh = False
if rows and (fresh or allow_stale):
self._remember_sw_members(sector_code, rows)
return rows
return None
def _write_local_sw_members(self, sector_code: str, rows: list[dict[str, Any]]) -> None:
packed = [dict(item) for item in rows]
self._remember_sw_members(sector_code, packed)
path = _sw_member_path(sector_code)
try:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(
json.dumps(
{
"sector_code": sector_code,
"updated_at": datetime.now().astimezone().isoformat(timespec="seconds"),
"rows": packed,
},
ensure_ascii=False,
),
encoding="utf-8",
)
deduped: dict[str, dict[str, Any]] = {}
for row in _reconcile_membership_rows(rows):
code = str(row.get("ts_code") or "")
if code and _membership_active_on(row, trade_date):
current = deduped.get(code)
if current is None or str(row.get("in_date") or "") > str(current.get("in_date") or ""):
deduped[code] = row
return list(deduped.values())
except OSError:
pass
def _remember_sw_members(self, sector_code: str, rows: list[dict[str, Any]]) -> None:
cache = getattr(self, "_sw_member_cache", None)
lock = getattr(self, "_sw_member_lock", None)
if not isinstance(cache, dict) or lock is None:
return
with lock:
cache[sector_code] = {
"loaded_at": datetime.now().astimezone(),
"rows": [dict(item) for item in rows],
}
def sw_sector_members(self, sector_code: str, trade_date: str) -> list[dict[str, Any]]:
"""Return constituents active in a Shenwan L2 industry on the target date."""
@@ -311,37 +435,37 @@ class ShenwanIndustryMixin:
finalized: bool = False,
) -> dict[str, Any]:
sector_code = str(industry.get("l2_code") or "")
sw_rows = self.query(
"rt_sw_k",
{"ts_code": sector_code},
"ts_code,name,trade_time,close,pre_close,high,open,low,vol,amount,pct_change",
sw_row, outer_source, outer_error = self._sw_outer_realtime(
sector_code,
str(industry.get("l2_name") or ""),
trade_date,
finalized=finalized,
)
sw_row = sw_rows[0] if sw_rows else {}
trade_time = str(sw_row.get("trade_time") or "")
quote_date = trade_time[:10].replace("-", "")
quote_clock = trade_time[11:19] if len(trade_time) >= 19 else ""
trade_time = str(sw_row.get("trade_time") or sw_row.get("quote_time") or "")
quote_date = str(sw_row.get("quote_date") or trade_time[:10].replace("-", ""))
quote_clock = trade_time[11:19] if len(trade_time) >= 19 else str(sw_row.get("quote_clock") or "")
outer_precise = bool(sw_row and quote_date == trade_date)
if finalized and (not quote_clock or quote_clock < "15:00:00"):
if finalized and quote_clock and quote_clock < "15:00:00":
outer_precise = False
official_change = _number(sw_row.get("pct_change"))
official_change = _number(sw_row.get("pct_change") if sw_row.get("pct_change") not in (None, "") else sw_row.get("change"))
if not official_change:
close = _number(sw_row.get("close"))
pre_close = _number(sw_row.get("pre_close"))
close = _number(sw_row.get("close") if sw_row.get("close") not in (None, "") else sw_row.get("price"))
pre_close = _number(sw_row.get("pre_close") if sw_row.get("pre_close") not in (None, "") else sw_row.get("previous_close"))
official_change = (close / pre_close - 1) * 100 if close and pre_close else 0
if not outer_precise:
official_change = None
outer_error = ""
if not sw_row:
outer_error = f"No Shenwan realtime index returned for {sector_code}"
elif quote_date != trade_date:
outer_error = f"Shenwan realtime index date is {quote_date or 'unknown'}, expected {trade_date}"
elif finalized and (not quote_clock or quote_clock < "15:00:00"):
outer_error = f"Shenwan realtime index is not a close snapshot ({trade_time})"
if not sw_row and not outer_error:
outer_error = f"申万行业 {sector_code} 当日外显待盘后正式数据或免费实时源"
elif quote_date and quote_date != trade_date:
outer_error = f"申万实时行业日期是 {quote_date},期望 {trade_date}"
elif finalized and quote_clock and quote_clock < "15:00:00":
outer_error = f"申万行业尚未形成收盘快照({trade_time}"
valid: list[dict[str, Any]] = []
codes: list[str] = []
reference: dict[str, Any] = {}
inner_error = ""
inner_source = "unavailable"
try:
reference = self._load_realtime_reference(trade_date, previous_trade_date)
active_codes = {
@@ -352,20 +476,27 @@ class ShenwanIndustryMixin:
codes = [
str(row.get("ts_code") or "")
for row in members
if str(row.get("ts_code") or "") in active_codes
if str(row.get("ts_code") or "")
]
if codes:
quotes = self.query("rt_k", {"ts_code": ",".join(codes)}, "")
for row in quotes:
close = _number(row.get("close"))
previous_close = _number(row.get("pre_close"))
if close <= 0 or previous_close <= 0:
continue
valid.append({**row, "change": (close / previous_close - 1) * 100})
else:
if active_codes:
listed = [code for code in codes if code in active_codes]
if listed:
codes = listed
quotes, inner_source = self._load_member_realtime_quotes(codes, trade_date)
for row in quotes:
close = _number(row.get("close"))
previous_close = _number(row.get("pre_close"))
if close <= 0 or previous_close <= 0:
continue
valid.append({**row, "change": (close / previous_close - 1) * 100})
if not codes:
inner_error = f"No active Shenwan members returned for {sector_code}"
elif not quotes:
inner_error = f"申万成分实时行情暂不可用:{sector_code}"
except TushareError as exc:
inner_error = str(exc)
if "rt_k" in inner_error or "权限" in inner_error:
inner_error = "申万成分实时行情暂不可用,已避开无权限接口"
coverage = len(valid) / max(len(codes), 1) * 100
valid_codes = {str(item.get("ts_code") or "") for item in valid}
@@ -390,16 +521,18 @@ class ShenwanIndustryMixin:
}
equal_change = sum(item["change"] for item in valid) / len(valid) if valid else 0
amount_billion = sum(_number(item.get("amount")) for item in valid) / 100000000
market_rows: list[dict[str, Any]] = []
try:
self._ensure_realtime_market_cache(trade_date)
with self._realtime_reference_lock:
market_rows = list(
(self._latest_realtime_market.get(trade_date) or {}).get("rows") or []
)
market_rows = self._ensure_realtime_market_cache(trade_date)
except TushareError as exc:
market_rows = []
inner_precise = False
inner_error = inner_error or str(exc)
message = str(exc)
if "rt_k" in message or "权限" in message:
market_error = "全市场实时行情暂不可用,已避开无权限接口"
else:
market_error = message
if not valid:
inner_precise = False
inner_error = inner_error or market_error
capital_map = {
str(item.get("ts_code") or ""): item
for item in reference.get("capital_rows") or []
@@ -408,20 +541,28 @@ class ShenwanIndustryMixin:
for item in valid:
capital = capital_map.get(str(item.get("ts_code") or ""), {})
float_share = _number(capital.get("float_share"))
if float_share:
sector_turnovers.append(_number(item.get("vol")) / float_share / 100)
volume = _number(item.get("vol"))
if float_share and volume:
# 免费源成交量为股;daily_basic.float_share 为万股。
sector_turnovers.append(volume / float_share / 100)
market_turnovers = []
for item in market_rows:
capital = capital_map.get(str(item.get("ts_code") or ""), {})
float_share = _number(capital.get("float_share"))
if float_share:
market_turnovers.append(_number(item.get("vol")) / float_share / 100)
volume = _number(item.get("vol"))
if float_share and volume:
market_turnovers.append(volume / float_share / 100)
average_turnover = sum(sector_turnovers) / len(sector_turnovers) if sector_turnovers else 0
market_turnover = sum(market_turnovers) / len(market_turnovers) if market_turnovers else 0
relative_turnover = average_turnover / market_turnover if market_turnover else 0
if not relative_turnover:
inner_precise = False
inner_error = inner_error or "Shenwan member relative turnover is unavailable"
delayed = "delayed" in str(inner_source) or any(item.get("delayed") for item in valid)
delay_seconds = max((int(item.get("delay_seconds") or 0) for item in valid), default=0)
delay_notice = ""
if delayed:
delay_notice = next(
(str(item.get("delay_notice") or "") for item in valid if item.get("delay_notice")),
"",
) or f"主备免费行情均暂不可用,显示最近一次真实快照(延迟 {delay_seconds} 秒)"
return {
"code": sector_code,
"name": str(industry.get("l2_name") or sw_row.get("name") or ""),
@@ -447,9 +588,9 @@ class ShenwanIndustryMixin:
"amount_billion": round(amount_billion, 2),
"count": sum(item["change"] >= 9.5 for item in valid),
"max_streak": 0,
"source": "tushare_rt_sw_k+sw_members_rt_k",
"inner_source": "tushare_sw_members+rt_k",
"outer_source": "tushare_rt_sw_k",
"source": f"{outer_source or 'unavailable'}+{inner_source}",
"inner_source": inner_source,
"outer_source": outer_source or "unavailable",
"taxonomy": "sw_l2",
"industry": industry,
"trade_date": trade_date,
@@ -463,10 +604,96 @@ class ShenwanIndustryMixin:
"precise": inner_precise and outer_precise,
"inner_error": inner_error,
"outer_error": outer_error,
"delayed": delayed,
"delay_seconds": delay_seconds,
"delay_notice": delay_notice,
"schema_version": 6,
"methodology": "外显使用申万官方 rt_sw_k;内核独立使用申万成分 rt_k 宽度与相对换手聚合",
"methodology": "外显使用已发布 sw_daily 或免费申万实时;内核使用数据中枢/免费实时成分,不调用 rt_sw_k",
}
def _sw_outer_realtime(
self,
sector_code: str,
sector_name: str,
trade_date: str,
finalized: bool = False,
) -> tuple[dict[str, Any], str, str]:
hub = getattr(self, "try_sector_quote", None)
if callable(hub):
try:
row = hub(sector_code, "" if finalized else trade_date)
except Exception as exc:
message = str(exc)
if finalized:
return {}, "", f"申万行业 {sector_code} 盘后正式数据待入库"
return {}, "", f"数据中枢申万实时暂不可用:{message[:180]}"
if row:
return dict(row), str(row.get("source") or "datahub"), ""
if finalized:
return {}, "", f"申万行业 {sector_code} 当日盘后正式数据尚未入库"
return {}, "", f"申万行业 {sector_code} 当日外显待补充"
def _load_member_realtime_quotes(
self,
codes: list[str],
trade_date: str,
) -> tuple[list[dict[str, Any]], str]:
wanted = [str(code).strip() for code in codes if str(code or "").strip()]
if not wanted:
return [], "unavailable"
best_rows: list[dict[str, Any]] = []
best_source = "unavailable"
def consider(rows: list[dict[str, Any]] | None, source: str) -> list[dict[str, Any]]:
nonlocal best_rows, best_source
filtered = _filter_quotes_for_codes(rows, wanted)
if len(filtered) > len(best_rows):
best_rows = filtered
best_source = source
return filtered
hub_market = getattr(self, "try_market_quotes", None)
if callable(hub_market):
filtered = consider(hub_market(trade_date) or [], "datahub")
if len(filtered) >= max(1, int(len(wanted) * 0.9)):
delayed = any(item.get("delayed") for item in filtered)
return filtered, "datahub_delayed" if delayed else "datahub"
hub = getattr(self, "try_quotes", None)
if callable(hub):
collected: list[dict[str, Any]] = []
for index in range(0, len(wanted), _QUOTE_BATCH):
collected.extend(hub(wanted[index:index + _QUOTE_BATCH]) or [])
filtered = consider(collected, "datahub")
if len(filtered) >= max(1, int(len(wanted) * 0.9)):
delayed = any(item.get("delayed") for item in filtered)
return filtered, "datahub_delayed" if delayed else "datahub"
if best_rows:
delayed = any(item.get("delayed") for item in best_rows)
if delayed and not str(best_source).endswith("_delayed"):
return best_rows, f"{best_source}_delayed"
return best_rows, best_source
return [], "unavailable"
_QUOTE_BATCH = 60
def _filter_quotes_for_codes(
rows: list[dict[str, Any]] | None,
codes: list[str],
) -> list[dict[str, Any]]:
wanted = {str(code) for code in codes if code}
filtered: list[dict[str, Any]] = []
seen: set[str] = set()
for row in rows or []:
ts_code = str(row.get("ts_code") or "")
if ts_code in wanted and ts_code not in seen:
seen.add(ts_code)
filtered.append(row)
return filtered
def _filter_members_by_listing(
members: list[dict[str, Any]],
@@ -568,6 +795,22 @@ def _reconcile_membership_rows(rows: list[dict[str, Any]]) -> list[dict[str, Any
return list(reconciled.values())
def _sw_member_path(sector_code: str) -> Path:
safe = "".join(ch if ch.isalnum() or ch in "._-" else "_" for ch in str(sector_code or ""))
return _SW_MEMBER_DIR / f"{safe or 'unknown'}.json"
def _active_members(rows: list[dict[str, Any]], trade_date: str) -> list[dict[str, Any]]:
deduped: dict[str, dict[str, Any]] = {}
for row in rows:
code = str(row.get("ts_code") or "")
if code and _membership_active_on(row, trade_date):
current = deduped.get(code)
if current is None or str(row.get("in_date") or "") > str(current.get("in_date") or ""):
deduped[code] = dict(row)
return list(deduped.values())
def _match_sector_row(rows: list[dict[str, Any]], identifier: str) -> dict[str, Any] | None:
if not rows:
return None
+5 -19
View File
@@ -5,13 +5,14 @@ from typing import Any
from backend.bootstrap.config import display_compact_date as _display_date
from backend.data.numbers import finite_number as _number
from backend.data.providers.tushare_helpers import _moneyflow_payload
class StockMixin:
def stock_detail(self, ts_code: str, requested_date: str) -> dict[str, Any]:
trade_date, _ = self.resolve_trade_context(requested_date)
end = datetime.strptime(trade_date, "%Y%m%d")
start_date = (end - timedelta(days=190)).strftime("%Y%m%d")
start_date = (end - timedelta(days=400)).strftime("%Y%m%d")
daily = self.query(
"daily",
{"ts_code": ts_code, "start_date": start_date, "end_date": trade_date},
@@ -41,7 +42,7 @@ class StockMixin:
factor_map = {row["trade_date"]: _number(row.get("adj_factor"), 1) for row in factors}
latest_factor = max(factor_map.values(), default=1) or 1
prices = []
for row in sorted(daily, key=lambda item: item.get("trade_date", ""))[-90:]:
for row in sorted(daily, key=lambda item: item.get("trade_date", ""))[-250:]:
factor = factor_map.get(row.get("trade_date"), latest_factor)
ratio = factor / latest_factor
prices.append(
@@ -56,7 +57,7 @@ class StockMixin:
"amount_billion": round(_number(row.get("amount")) / 100000, 2),
}
)
flow = moneyflow[0] if moneyflow else {}
flow = moneyflow[0] if moneyflow else None
basic = basics[0] if basics else {}
daily_basic = daily_basics[0] if daily_basics else {}
latest = prices[-1] if prices else {}
@@ -87,22 +88,7 @@ class StockMixin:
"amount_billion": latest.get("amount_billion", 0),
},
"prices": prices,
"moneyflow": {
"net_million": round(_number(flow.get("net_mf_amount")) / 100, 2),
"large_million": round(
(_number(flow.get("buy_lg_amount")) + _number(flow.get("buy_elg_amount"))
- _number(flow.get("sell_lg_amount")) - _number(flow.get("sell_elg_amount"))) / 100,
2,
),
"medium_million": round(
(_number(flow.get("buy_md_amount")) - _number(flow.get("sell_md_amount"))) / 100,
2,
),
"small_million": round(
(_number(flow.get("buy_sm_amount")) - _number(flow.get("sell_sm_amount"))) / 100,
2,
),
},
"moneyflow": _moneyflow_payload(flow),
}
def stock_intraday(self, ts_code: str, requested_date: str) -> dict[str, Any]:
@@ -20,6 +20,8 @@ class TushareTransportMixin:
params: dict[str, Any] | None = None,
fields: str = "",
) -> list[dict[str, Any]]:
if api_name == "rt_sw_k":
raise TushareError("rt_sw_k is disabled; use published sw_daily or free Shenwan realtime")
payload = json.dumps(
{
"api_name": api_name,
+521 -4
View File
@@ -19,8 +19,22 @@ class RealtimeAggregateError(RuntimeError):
EASTMONEY_INDEX_URL = "https://push2.eastmoney.com/api/qt/ulist.np/get"
EASTMONEY_STOCK_URL = "https://push2.eastmoney.com/api/qt/stock/get"
EASTMONEY_STOCK_FIELDS = "f43,f44,f45,f46,f47,f48,f57,f58,f60,f86,f168,f62,f66,f72,f78,f84"
EASTMONEY_SECTOR_URL = "https://push2.eastmoney.com/api/qt/clist/get"
EASTMONEY_ZT_POOL_URL = "https://push2ex.eastmoney.com/getTopicZTPool"
EASTMONEY_ZB_POOL_URL = "https://push2ex.eastmoney.com/getTopicZBPool"
EASTMONEY_A_SHARE_BOARDS = (
"m:0+t:6",
"m:0+t:80",
"m:1+t:2",
"m:1+t:23",
"m:0+t:81",
)
EASTMONEY_QUOTE_FIELDS = "f12,f13,f14,f2,f3,f4,f5,f6,f15,f16,f17,f18,f8,f124"
EASTMONEY_MARKET_PAGE_SIZE = 100
TENCENT_INDEX_URL = "https://qt.gtimg.cn/q=sh000001,sz399001,sz399006"
TENCENT_QUOTE_URL = "https://qt.gtimg.cn/q="
THS_LIMIT_URL = "https://data.10jqka.com.cn/dataapi/limit_up/limit_up_pool"
XGB_POOL_URL = "https://flash-api.xuangubao.cn/api/pool/detail"
BROWSER_USER_AGENT = (
@@ -134,6 +148,312 @@ class WebRealtimeAggregator:
raise RealtimeAggregateError(f"Eastmoney returned {len(result)}/3 indices")
return result
def eastmoney_market_quotes(self, expected_date: str = "") -> list[dict[str, Any]]:
"""Full A-share snapshot via Eastmoney clist, used when Tushare rt_k is unavailable."""
now = time.time()
cache_key = "assembled:eastmoney_market"
with self._response_cache_lock:
cached = self._response_cache.get(cache_key)
cache_age = now - float((cached or {}).get("created_at") or 0)
if cached and cache_age <= min(20, self.response_cache_ttl_seconds):
quotes = list(cached.get("payload") or [])
return self._filter_quotes_by_date(quotes, expected_date)
rows: list[dict[str, Any]] = []
board_errors: list[str] = []
for board in EASTMONEY_A_SHARE_BOARDS:
try:
rows.extend(self._eastmoney_board_quotes(board))
except Exception as exc:
board_errors.append(f"{board}:{exc}")
quotes = []
seen: set[str] = set()
for row in rows:
quote = _normalize_eastmoney_quote(row)
ts_code = str((quote or {}).get("ts_code") or "")
if not quote or ts_code in seen:
continue
seen.add(ts_code)
quotes.append(quote)
if len(quotes) < 200:
detail = f"{'; '.join(board_errors)}" if board_errors else ""
raise RealtimeAggregateError(
f"Eastmoney market snapshot too small: {len(quotes)}{detail}"
)
quotes = self._filter_quotes_by_date(quotes, expected_date)
with self._response_cache_lock:
self._response_cache[cache_key] = {"created_at": now, "payload": quotes}
return quotes
def _eastmoney_board_quotes(self, board: str) -> list[dict[str, Any]]:
first = self._eastmoney_market_page(board, 1)
data = first.get("data") or {}
rows = _diff_rows(data)
total = int(_number(data.get("total")))
page_count = 1
if total > 0:
page_count = max(1, (total + EASTMONEY_MARKET_PAGE_SIZE - 1) // EASTMONEY_MARKET_PAGE_SIZE)
for page in range(2, min(page_count, 40) + 1):
payload = self._eastmoney_market_page(board, page)
rows.extend(_diff_rows(payload.get("data") or {}))
return rows
def _eastmoney_market_page(self, board: str, page: int) -> dict[str, Any]:
return self._get_json(
EASTMONEY_SECTOR_URL,
{
"pn": str(page),
"pz": str(EASTMONEY_MARKET_PAGE_SIZE),
"po": "1",
"np": "1",
"fltt": "2",
"invt": "2",
"fid": "f12",
"fs": board,
"fields": EASTMONEY_QUOTE_FIELDS,
},
referer="https://quote.eastmoney.com/center/gridlist.html",
)
def _filter_quotes_by_date(
self,
quotes: list[dict[str, Any]],
expected_date: str,
) -> list[dict[str, Any]]:
want = str(expected_date or "").replace("-", "")
if not want or not quotes:
return quotes
dated = [item for item in quotes if str(item.get("quote_date") or "") == want]
if dated and len(dated) >= max(100, int(len(quotes) * 0.2)):
return dated
if dated:
return dated
if all(not item.get("quote_date") for item in quotes):
return quotes
raise RealtimeAggregateError(f"Eastmoney quotes are not for {want}")
def tencent_market_quotes(
self,
codes: list[str],
expected_date: str = "",
) -> list[dict[str, Any]]:
quotes = self.tencent_stock_quotes(codes, expected_date="", minimum=200)
return self._filter_quotes_by_date(quotes, expected_date)
def tencent_stock_quotes(
self,
codes: list[str],
expected_date: str = "",
minimum: int | None = None,
) -> list[dict[str, Any]]:
symbols: list[str] = []
seen: set[str] = set()
for raw in codes:
ts = str(raw or "").strip().upper()
if not ts:
continue
symbol = ts.split(".")[0]
if not symbol.isdigit() or len(symbol) != 6 or symbol in seen:
continue
seen.add(symbol)
if ts.endswith(".SH") or symbol.startswith(("5", "6", "9")):
symbols.append(f"sh{symbol}")
elif ts.endswith(".BJ") or symbol.startswith(("4", "8")):
symbols.append(f"bj{symbol}")
else:
symbols.append(f"sz{symbol}")
if not symbols:
raise RealtimeAggregateError("No stock codes available for Tencent quotes")
quotes: list[dict[str, Any]] = []
batch_size = 80
def load_batch(batch: list[str]) -> list[dict[str, Any]]:
raw, _cache_age = self._get_text(
f"{TENCENT_QUOTE_URL}{','.join(batch)}",
referer="https://gu.qq.com/",
encoding="gb18030",
)
return [
quote
for line in raw.splitlines()
if (quote := _parse_tencent_stock_quote(line))
]
batches = [symbols[index:index + batch_size] for index in range(0, len(symbols), batch_size)]
errors: list[str] = []
with ThreadPoolExecutor(max_workers=4) as executor:
for result in executor.map(self._capture, [lambda batch=batch: load_batch(batch) for batch in batches]):
rows, status = result
if status.get("ok") and rows:
quotes.extend(rows)
elif not status.get("ok"):
errors.append(str(status.get("error") or "batch failed"))
floor = minimum if minimum is not None else max(1, int(len(symbols) * 0.5))
if len(quotes) < floor:
detail = f"{'; '.join(errors[:3])}" if errors else ""
raise RealtimeAggregateError(
f"Tencent quotes too small: {len(quotes)}/{len(symbols)}{detail}"
)
return self._filter_quotes_by_date(quotes, expected_date)
def tencent_stock_quote(self, code: str, expected_date: str = "") -> dict[str, Any]:
symbol, _secid, ts_code = _a_share_identity(code)
raw, _cache_age = self._get_text(
f"{TENCENT_QUOTE_URL}{symbol}",
referer="https://gu.qq.com/",
encoding="gb18030",
)
quote = next(
(
item
for line in raw.splitlines()
if (item := _parse_tencent_stock_quote(line))
),
None,
)
if not quote:
raise RealtimeAggregateError(f"Tencent stock quote unavailable for {ts_code}")
return _require_quote_date(quote, expected_date)
def eastmoney_stock_quote(self, code: str, expected_date: str = "") -> dict[str, Any]:
_symbol, secid, ts_code = _a_share_identity(code)
payload = self._get_json(
EASTMONEY_STOCK_URL,
{
"secid": secid,
"invt": "2",
"fltt": "2",
"fields": EASTMONEY_STOCK_FIELDS,
},
referer="https://quote.eastmoney.com/",
)
quote = _normalize_eastmoney_stock_quote(payload.get("data") or {}, ts_code)
if not quote:
raise RealtimeAggregateError(f"Eastmoney stock quote unavailable for {ts_code}")
return _require_quote_date(quote, expected_date)
def eastmoney_stock_quotes(
self,
codes: list[str],
expected_date: str = "",
) -> list[dict[str, Any]]:
secids = []
for code in codes:
try:
_symbol, secid, _ts = _a_share_identity(code)
except RealtimeAggregateError:
continue
secids.append(secid)
quotes: list[dict[str, Any]] = []
for index in range(0, len(secids), 60):
payload = self._get_json(
EASTMONEY_INDEX_URL,
{
"secids": ",".join(secids[index:index + 60]),
"fltt": "2",
"invt": "2",
"fields": EASTMONEY_QUOTE_FIELDS,
},
referer="https://quote.eastmoney.com/",
)
for row in _diff_rows(payload.get("data") or {}):
quote = _normalize_eastmoney_quote(row)
if quote:
quotes.append(quote)
return self._filter_quotes_by_date(quotes, expected_date)
def eastmoney_shenwan_quote(
self,
ts_code: str,
expected_date: str = "",
) -> dict[str, Any]:
code = str(ts_code or "").split(".")[0]
if not code:
raise RealtimeAggregateError("Invalid Shenwan code")
payload = self._get_json(
EASTMONEY_INDEX_URL,
{
"secids": f"90.{code}",
"fltt": "2",
"invt": "2",
"fields": "f12,f14,f2,f3,f4,f15,f16,f17,f18,f6,f8,f104,f105,f128,f136,f140,f124",
},
referer="https://quote.eastmoney.com/",
)
row = next((item for item in _diff_rows(payload.get("data") or {}) if item), None)
if not row:
raise RealtimeAggregateError(f"Eastmoney Shenwan quote missing for {code}")
epoch = int(_number(row.get("f124")))
quote_time = (
datetime.fromtimestamp(epoch).astimezone().isoformat(timespec="seconds")
if epoch
else ""
)
close = _number(row.get("f2"))
previous = _number(row.get("f18"))
if close <= 0 or previous <= 0:
raise RealtimeAggregateError(f"Eastmoney Shenwan quote empty for {code}")
result = {
"ts_code": f"{code}.SI",
"code": f"{code}.SI",
"name": row.get("f14") or code,
"price": close,
"close": close,
"pre_close": previous,
"previous_close": previous,
"open": _number(row.get("f17")),
"high": _number(row.get("f15")),
"low": _number(row.get("f16")),
"change": _number(row.get("f3")),
"pct_change": _number(row.get("f3")),
"amount": _number(row.get("f6")),
"leader": row.get("f128") or "--",
"leader_code": row.get("f140") or "",
"leading_pct": _number(row.get("f136")),
"up_count": int(_number(row.get("f104"))),
"down_count": int(_number(row.get("f105"))),
"quote_time": quote_time,
"trade_time": quote_time,
"quote_date": datetime.fromtimestamp(epoch).astimezone().strftime("%Y%m%d") if epoch else "",
"quote_time_epoch": epoch,
"source": "eastmoney_sw",
}
return _require_quote_date(result, expected_date) if expected_date else result
def eastmoney_limit_pool(self, trade_date: str = "") -> list[dict[str, Any]]:
day = str(trade_date or "").replace("-", "")
rows: list[dict[str, Any]] = []
for url, limit_type in (
(EASTMONEY_ZT_POOL_URL, "U"),
(EASTMONEY_ZB_POOL_URL, "Z"),
):
try:
payload = self._get_json(
url,
{
"ut": "7eea3edcaed734bea9cbfc24409ed989",
"dpt": "wz.ztzt",
"PageIndex": "0",
"PageSize": "200",
"sort": "fbt:asc",
"date": day,
},
referer="https://quote.eastmoney.com/ztb/detail",
)
except RealtimeAggregateError:
continue
pool = (payload.get("data") or {}).get("pool") or []
if isinstance(pool, dict):
pool = list(pool.values())
for item in pool:
if not isinstance(item, dict):
continue
parsed = _normalize_eastmoney_limit_row(item, limit_type)
if parsed:
rows.append(parsed)
return rows
def tencent_indices(self) -> list[dict[str, Any]]:
raw, cache_age = self._get_text(
TENCENT_INDEX_URL,
@@ -185,11 +505,17 @@ class WebRealtimeAggregator:
if not matched:
raise RealtimeAggregateError(f"Eastmoney sector not found: {query}")
epoch = int(_number(matched.get("f124")))
quote_time = (
datetime.fromtimestamp(epoch).astimezone().isoformat(timespec="seconds")
if epoch else ""
)
return {
"code": matched.get("f12") or "",
"name": matched.get("f14") or query,
"price": _number(matched.get("f2")),
"close": _number(matched.get("f2")),
"change": _number(matched.get("f3")),
"pct_change": _number(matched.get("f3")),
"change_amount": _number(matched.get("f4")),
"turnover_rate": _number(matched.get("f8")),
"up_count": int(_number(matched.get("f104"))),
@@ -198,10 +524,9 @@ class WebRealtimeAggregator:
"leader_code": matched.get("f140") or "",
"leading_pct": _number(matched.get("f136")),
"quote_time_epoch": epoch,
"quote_time": (
datetime.fromtimestamp(epoch).astimezone().isoformat(timespec="seconds")
if epoch else ""
),
"quote_time": quote_time,
"trade_time": quote_time,
"quote_date": datetime.fromtimestamp(epoch).astimezone().strftime("%Y%m%d") if epoch else "",
"source": "eastmoney_push2",
"match_query": query,
}
@@ -397,6 +722,198 @@ class WebRealtimeAggregator:
) from last_error
def _diff_rows(data: dict[str, Any]) -> list[dict[str, Any]]:
diff = data.get("diff") or []
if isinstance(diff, dict):
return [row for row in diff.values() if isinstance(row, dict)]
return [row for row in diff if isinstance(row, dict)]
def _a_share_identity(code: str) -> tuple[str, str, str]:
raw = str(code or "").strip().upper()
symbol = raw.split(".")[0]
if not symbol.isdigit() or len(symbol) != 6:
raise RealtimeAggregateError("Invalid stock code")
if raw.endswith(".SH") or symbol.startswith(("5", "6", "9")):
return f"sh{symbol}", f"1.{symbol}", f"{symbol}.SH"
if raw.endswith(".BJ") or symbol.startswith(("4", "8")):
return f"bj{symbol}", f"0.{symbol}", f"{symbol}.BJ"
return f"sz{symbol}", f"0.{symbol}", f"{symbol}.SZ"
def _require_quote_date(quote: dict[str, Any], expected_date: str) -> dict[str, Any]:
want = str(expected_date or "").replace("-", "")
got = str(quote.get("quote_date") or "")
if want and got != want:
raise RealtimeAggregateError(f"quote date {got or 'empty'} is not {want}")
return quote
def _normalize_eastmoney_stock_quote(
row: dict[str, Any], ts_code: str
) -> dict[str, Any] | None:
close = _number(row.get("f43"))
previous_close = _number(row.get("f60"))
if close <= 0 or previous_close <= 0:
return None
epoch = int(_number(row.get("f86")))
quote_date = ""
if epoch > 0:
quote_date = datetime.fromtimestamp(epoch).astimezone().strftime("%Y%m%d")
return {
"ts_code": ts_code,
"name": row.get("f58") or ts_code.split(".")[0],
"pre_close": previous_close,
"open": _number(row.get("f46")),
"high": _number(row.get("f44")),
"low": _number(row.get("f45")),
"close": close,
"vol": _number(row.get("f47")) * 100,
"amount": _number(row.get("f48")),
"num": 0,
"quote_date": quote_date,
"quote_time_epoch": epoch,
"turnover_rate": _number(row.get("f168")),
"net_mf_amount": _eastmoney_flow_wan(row.get("f62")),
"large_amount": _eastmoney_flow_wan(row.get("f62")),
"medium_amount": _eastmoney_flow_wan(row.get("f78")),
"small_amount": _eastmoney_flow_wan(row.get("f84")),
"source": "eastmoney_stock",
}
def _parse_tencent_stock_quote(line: str) -> dict[str, Any] | None:
if '="' not in line:
return None
prefix, payload = line.split('="', 1)
fields = payload.rsplit('";', 1)[0].split("~")
if len(fields) < 38:
return None
symbol = fields[2]
if not symbol.isdigit() or len(symbol) != 6:
return None
close = _number(fields[3])
previous_close = _number(fields[4])
if close <= 0 or previous_close <= 0:
return None
marker = prefix.lower()
if "sh" in marker:
ts_code = f"{symbol}.SH"
elif "bj" in marker:
ts_code = f"{symbol}.BJ"
else:
ts_code = f"{symbol}.SZ"
try:
quote_time = datetime.strptime(fields[30], "%Y%m%d%H%M%S")
quote_date = quote_time.strftime("%Y%m%d")
epoch = int(quote_time.timestamp())
except ValueError:
quote_date = ""
epoch = 0
return {
"ts_code": ts_code,
"name": fields[1] or symbol,
"pre_close": previous_close,
"open": _number(fields[5]),
"high": _number(fields[33]),
"low": _number(fields[34]),
"close": close,
"vol": _number(fields[6]) * 100,
"amount": _number(fields[37]) * 10000,
"num": 0,
"quote_date": quote_date,
"quote_time_epoch": epoch,
"source": "tencent_qt",
}
def _normalize_eastmoney_quote(row: dict[str, Any]) -> dict[str, Any] | None:
symbol = str(row.get("f12") or "").strip()
if not symbol.isdigit() or len(symbol) != 6:
return None
close = _number(row.get("f2"))
previous_close = _number(row.get("f18"))
if close <= 0 or previous_close <= 0:
return None
market = int(_number(row.get("f13")))
if market == 1 or symbol.startswith(("5", "6", "9")):
ts_code = f"{symbol}.SH"
elif symbol.startswith(("4", "8")):
ts_code = f"{symbol}.BJ"
else:
ts_code = f"{symbol}.SZ"
epoch = int(_number(row.get("f124")))
quote_date = ""
if epoch > 0:
quote_date = datetime.fromtimestamp(epoch).astimezone().strftime("%Y%m%d")
return {
"ts_code": ts_code,
"name": row.get("f14") or symbol,
"pre_close": previous_close,
"open": _number(row.get("f17")),
"high": _number(row.get("f15")),
"low": _number(row.get("f16")),
"close": close,
"vol": _number(row.get("f5")) * 100,
"amount": _number(row.get("f6")),
"num": 0,
"quote_date": quote_date,
"quote_time_epoch": epoch,
"source": "eastmoney_clist",
}
def _eastmoney_flow_wan(value: Any) -> float | None:
if value in (None, "", "-"):
return None
amount = _number(value, default=float("nan"))
if amount != amount:
return None
return amount / 10000
def _board_clock(value: Any) -> str:
digits = "".join(character for character in str(value or "") if character.isdigit())
if len(digits) >= 6:
return f"{digits[:2]}:{digits[2:4]}:{digits[4:6]}"
if len(digits) == 5:
digits = digits.zfill(6)
return f"{digits[:2]}:{digits[2:4]}:{digits[4:6]}"
if len(digits) == 4:
return f"{digits[:2]}:{digits[2:]}:00"
return ""
def _normalize_eastmoney_limit_row(row: dict[str, Any], limit_type: str) -> dict[str, Any] | None:
symbol = str(row.get("c") or row.get("code") or "").strip()
if not symbol.isdigit() or len(symbol) != 6:
return None
market = int(_number(row.get("m") if row.get("m") not in (None, "") else row.get("market")))
if market == 1 or symbol.startswith(("5", "6", "9")):
ts_code = f"{symbol}.SH"
elif symbol.startswith(("4", "8")):
ts_code = f"{symbol}.BJ"
else:
ts_code = f"{symbol}.SZ"
first_time = _board_clock(row.get("fbt") if row.get("fbt") not in (None, "") else row.get("first_time"))
last_time = _board_clock(row.get("lbt") if row.get("lbt") not in (None, "") else row.get("last_time"))
fund = row.get("fund")
if fund in (None, ""):
fund = row.get("fd_amount")
return {
"ts_code": ts_code,
"name": row.get("n") or row.get("name") or symbol,
"limit_type": limit_type,
"first_time": first_time or None,
"last_time": last_time or None,
"open_times": int(_number(row.get("zbc") if row.get("zbc") not in (None, "") else row.get("open_times"))),
"limit_times": max(1, int(_number(row.get("lbc") if row.get("lbc") not in (None, "") else 1))),
"turnover_ratio": _number(row.get("hs") if row.get("hs") not in (None, "") else row.get("turnover_ratio")),
"fd_amount": _number(fund) if fund not in (None, "", "-") else None,
"source": "eastmoney_zt_pool",
}
def _normalize_sector(value: Any) -> str:
text = str(value or "").strip().replace(" ", "")
for suffix in ("板块", "概念", "行业", "", "", "(A股)", "A股)"):
@@ -0,0 +1,117 @@
{
"version": "2026.08.05-5",
"sources": {
"zhouyi": {
"title": "周易经文与十翼",
"scope": "卦辞、爻辞、彖传、象传",
"kind": "public_domain_primary",
"note": "观势与观心只引用本项目已校录的卦爻原文,不把现代网络释文当作原典。"
},
"jingfang": {
"title": "京氏易传",
"scope": "八宫与纳甲体系来源",
"kind": "public_domain_traditional",
"note": "确定性程序采用京房纳甲、八宫世应的通行排法。"
},
"huozhulin": {
"title": "火珠林",
"scope": "纳甲筮法、六亲与日月关系",
"kind": "public_domain_traditional",
"note": "用于观心规则脉络,不直接复制后世简化断语。"
},
"zengshan": {
"title": "增删卜易",
"scope": "用神、世应、动变、日月旺衰",
"kind": "public_domain_traditional",
"note": "只采用可明确编码且有一致输入条件的规则;争议规则单独标记。"
},
"neijing": {
"title": "黄帝内经·素问运气七篇",
"scope": "五运、司天在泉、主客气与运气关系",
"kind": "public_domain_primary",
"note": "观气将原典关系转成当日自我观察语言,不宣称对股价存在因果作用。"
}
},
"trend": {
"method": "本卦说明当下结构,实际动爻说明变化关节,之卦说明所趋结构;多动爻全部保留,不以固定口诀删去用户实际得到的爻。",
"rules": {
"stable": "无动爻时以本卦整体、上下卦关系和大象为主,说明结构的延续条件,不把静止等同于永远不变。",
"single": "一爻动时以该爻的时位、爻辞和象辞为变化核心,并用之卦检查变化后的结构。",
"multiple": "多爻动时逐一保留相关爻义,先找共同方向与冲突,再结合之卦给出有条件的倾向;不得用固定套话把不同动爻压成同一结论。"
}
},
"fortune": {
"principle": "先立中运与司天在泉的年纲,再察当前客气加临主气,最后以日辰说明当日触发;不使用产品权重推导传统结论。",
"movement": {
"太过": "太过表示该运之气偏于有余,解释时同时观察其本气表现与对所胜、所生关系的牵动,不直接等同于吉或凶。",
"不及": "不及表示该运之气偏于不足,解释时同时观察其所不胜来乘与所生受累的可能,不直接等同于弱势结论。"
},
"qi": {
"厥阴风木": "厥阴取风木之动,侧重疏泄、升发、变化与不定;偏盛时可表现为动摇、急变或升散不收。",
"少阴君火": "少阴取君火之明与热,侧重显化、温煦和内在驱动;偏盛时容易躁热,受制时则显而不畅。",
"太阴湿土": "太阴取湿土之濡与承载,侧重黏滞、蓄积和转化;偏盛时容易困重迟缓,得化时则能承接。",
"少阳相火": "少阳取相火之行与枢转,侧重外达、加速和往来;偏盛时容易浮越躁动,受阻时表现为枢机不利。",
"阳明燥金": "阳明取燥金之收与清肃,侧重收敛、裁决和边界;偏盛时容易干急严峻,得润时则清明有序。",
"太阳寒水": "太阳取寒水之藏与凝,侧重潜藏、收引和下行;偏盛时容易凝滞退缩,得温时则蓄势有根。"
},
"relations": {
"same": "客主同气表示同类气相并,重点看是否相得而彰,还是同气偏盛而亢;不能机械判为有利。",
"guest_generates_host": "客生主表示来气生助时令本气,气机较易衔接;仍需观察生助是否过度及年纲是否承接。",
"host_generates_guest": "主生客表示时令本气向来气流转,有相生也有外泄;不能只取相生而忽略主气受耗。",
"guest_controls_host": "客克主表示来气制约主气,传统称客胜为从;重点解释外来变化居上及原有节律受制。",
"host_controls_guest": "主克客表示主气制约来气,传统称主胜为逆;重点解释时令与来气相持而不把相克直接断凶。"
},
"day_trigger": "日辰只说明当日关系如何被触发,不与中运、司天在泉或主客气并列重复计权。",
"industry_boundary": "五行对应行业只作传统取象:可以说明本次已经出现的五行之气对相应行业形成的象征性关注、节奏或约束,但不得读取或猜测行业实时行情,不得预测涨跌,也不得把取象写成投资推荐。",
"personal_boundary": "personal.natal_day_master才是用户本命日主;today_relative_to_natal_day_master中的pillars是当日历法,stem_relations只是当日年、月、日三柱天干相对本命日主的确定性关系标签。只能使用本次检索到的关系释义,不得自行重算十神、扩展五行生克、使用藏干、库气或支的燥湿属性,也不得把当日日柱称为用户命局,或由这些字段推断命局中某一十神偏重、身强身弱或喜用神。",
"personal_relations": {
"比肩": "比肩作为当日天干关系标签,只提示用户可能更在意自主判断、同类比较或坚持原有立场;不能据此判断命局强弱或现实事件。",
"劫财": "劫财作为当日天干关系标签,只提示用户留意精力、注意力或可支配资源在同类事项间的分流与竞争感;不等同于破财或他人争夺。",
"食神": "食神作为当日天干关系标签,只提示用户留意表达、输出、舒缓与完成感;不等同于收益或确定的轻松结果。",
"伤官": "伤官作为当日天干关系标签,只提示用户留意质疑规则、急于表达或追求自主空间的倾向;不等同于冲突或违规。",
"偏财": "偏财作为当日天干关系标签,只提示用户留意机会分配、灵活取舍与非固定资源的吸引力;不等同于意外获利。",
"正财": "正财作为当日天干关系标签,只提示用户更关注可核对的结果、资源边界和务实落地;不等同于必得收益或现金变化。",
"七杀": "七杀作为当日天干关系标签,只提示用户留意紧迫感、外部压力和快速决断冲动;不等同于危险必然发生。",
"正官": "正官作为当日天干关系标签,只提示用户更在意规则、责任、秩序和可交付标准;不等同于结果必然受控。",
"偏印": "偏印作为当日天干关系标签,只提示用户留意内省、非惯常信息和反复推敲的倾向;不等同于退缩、失眠或方向错误。",
"正印": "正印作为当日天干关系标签,只提示用户更在意依据、支持、学习和安全边界;不等同于必然获得帮助。"
}
},
"heart": {
"presets": {
"trade": "关于我心中的这笔交易,此刻最需要看清的机会、阻碍与风险是什么?",
"mind": "此刻影响我交易判断的情绪、执念或盲点是什么?",
"unthemed": "不设具体问题,只观此刻一念。"
},
"focus": {
"trade": "以世爻、应爻、妻财爻及实际动变为主要检索对象,同时检查兄弟、官鬼和子孙的生克,不把任何单一六亲固定判吉凶。",
"mind": "以世爻和实际动爻为主,观察官鬼所示压力、子孙所示舒解及内外生克;不把心境问题强行翻译成价格方向。",
"unthemed": "不强选事项用神,以本卦、世爻、实际动爻和之卦作一般观照,不猜测用户没有提出的问题。",
"custom": "先依据用户明确写出的股票交易问题选择相关六亲;无法明确归类时退回世爻、动爻和卦变的一般解释,不擅自补全问题。"
},
"evidence_order": [
"用户问题与预设来源",
"本卦及卦宫",
"世应与所问相关六亲",
"月建日辰、旬空及冲合生克",
"实际动爻与变爻",
"之卦与整体卦义",
"六神辅助象义"
],
"limits": "六神只作辅助象义;空亡、月破、日冲、合冲刑害均需结合用神、世应和动变,不得单项宣布结果。",
"semantics": {
"self_response": "世爻表示求测者当前立场与承受状态,应爻表示所问事项的外部一端或对照面。应爻不是固定的合作方、庄家或资金方;只有用户问题明确给出该角色时,才可作对应解释。",
"calendar": "月建与日辰用于判断爻在起卦时刻的承受、生扶和制约。旬空表示该爻所象征的条件当下可能未落实、难发挥或有名无实,但不能单凭旬空判失败,也不能用填实日期预测何时涨跌或行动。月破、日冲、六合、六冲、六害和相刑同样必须与世应、相关六亲及动变合看。",
"movement": "动爻说明关系正在变化;变爻说明变化后的承接方向。回头生、回头克和原变爻生克只描述力量关系,不自动对应现实中的借贷、融资、合作或某个具体人物。进神退神只说明同类地支变化的进退趋势,不直接宣布价格方向。",
"six_spirits": "六神只补充表达色彩,不单独定成败。青龙不必然有利,白虎不必然紧急或凶险,朱雀不必然等同口舌,玄武不必然等同欺骗,勾陈与螣蛇也不得脱离爻位、六亲和动变独断。",
"timing_boundary": "观心不作应期预测。可以说明某项条件在起卦时刻尚未落实或受制,但不得给出未来若干日、某干支日、出空或填实后必然发生什么。",
"relatives": {
"兄弟": "兄弟是与卦宫五行同类的关系。在股票交易问题中可作为竞争、同类力量或资源分流的候选象义,但不直接等同合作方、亏损或他人拿走资金。",
"子孙": "子孙是卦宫所生的关系,可作为舒缓、产出、执行后的释放或对压力的制衡候选象义,但不直接等同收益、资金提供方或确定的利好。",
"妻财": "妻财是卦宫所克的关系,在股票交易问题中可作为价值、收益预期、持仓利益或可支配资源的候选象义,但不直接等同现金、融资、自有资金或必得之财。",
"官鬼": "官鬼是克制卦宫的关系,可作为压力、风险、规则约束或担忧的候选象义,但不直接等同借贷、坏消息、疾病或必然损失。",
"父母": "父母是生助卦宫的关系,可作为信息、依据、计划、规则、凭据或保护条件的候选象义,但不直接等同政策、合同或某一条消息。"
}
}
}
}
+13 -4
View File
@@ -3,15 +3,24 @@ from __future__ import annotations
import json
from http import HTTPStatus
from backend.features.heaven.knowledge import HeavenKnowledgeError
class HeavenHttpMixin:
def _send_heaven_client_error(self, exc: Exception) -> None:
payload: dict = {"error": str(exc)}
code = getattr(exc, "error_code", None)
if code:
payload["code"] = str(code)
self.send_json(payload, HTTPStatus.BAD_REQUEST)
def heaven_hexagram(self) -> None:
try:
body = self.read_json_body()
result = self.application_service.heaven_hexagram(body.get("lines"))
self.send_json({"ok": True, "hexagram": result})
except (ValueError, json.JSONDecodeError) as exc:
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
self._send_heaven_client_error(exc)
def heaven_personal(self) -> None:
try:
@@ -19,12 +28,12 @@ class HeavenHttpMixin:
result = self.application_service.heaven_personal(body)
self.send_json({"ok": True, "personal": result})
except (ValueError, json.JSONDecodeError) as exc:
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
self._send_heaven_client_error(exc)
def heaven_interpret(self) -> None:
try:
body = self.read_json_body()
result = self.application_service.heaven_interpret(body)
self.send_json({"ok": True, **result})
except (ValueError, json.JSONDecodeError) as exc:
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
except (HeavenKnowledgeError, ValueError, json.JSONDecodeError) as exc:
self._send_heaven_client_error(exc)
+53 -2
View File
@@ -2,12 +2,23 @@ from __future__ import annotations
import json
from functools import lru_cache
from pathlib import Path
from typing import Any
from backend.bootstrap.config import APP_DIR
KNOWLEDGE_FILE = APP_DIR / "data" / "heaven_knowledge.json"
# Baked into the image outside the ./data bind mount so volume overlay cannot hide it.
KNOWLEDGE_SEED_FILE = Path(__file__).resolve().parent / "assets" / "heaven_knowledge.json"
class HeavenKnowledgeError(ValueError):
"""Structured knowledge-file failure surfaced to HTTP as Chinese API errors."""
def __init__(self, message: str, *, code: str) -> None:
super().__init__(message)
self.error_code = code
def prepare_heaven_context(mode: str, calculation: dict[str, Any]) -> dict[str, Any]:
@@ -379,9 +390,49 @@ def _line_record(line: dict[str, Any]) -> dict[str, Any]:
}
def resolve_heaven_knowledge_path() -> Path:
"""Prefer the persisted data-dir file; fall back to the image-baked seed."""
if KNOWLEDGE_FILE.is_file():
return KNOWLEDGE_FILE
if KNOWLEDGE_SEED_FILE.is_file():
return KNOWLEDGE_SEED_FILE
raise HeavenKnowledgeError(
"问天知识文件缺失:未找到 heaven_knowledge.json。"
"请确认宿主机 data 目录或镜像内 seed 文件完整。",
code="heaven_knowledge_missing",
)
@lru_cache(maxsize=1)
def _knowledge_catalog() -> dict[str, Any]:
payload = json.loads(KNOWLEDGE_FILE.read_text(encoding="utf-8"))
path = resolve_heaven_knowledge_path()
try:
raw = path.read_text(encoding="utf-8")
except OSError as exc:
raise HeavenKnowledgeError(
f"问天知识文件无法读取({path.name}):{exc.strerror or exc}",
code="heaven_knowledge_missing",
) from exc
try:
payload = json.loads(raw)
except json.JSONDecodeError as exc:
raise HeavenKnowledgeError(
f"问天知识文件 JSON 损坏({path.name}),无法解析:"
f"{exc.lineno} 行附近。",
code="heaven_knowledge_invalid",
) from exc
if not isinstance(payload, dict):
raise HeavenKnowledgeError(
f"问天知识文件格式不正确({path.name}):根节点必须是对象。",
code="heaven_knowledge_invalid",
)
if not payload.get("version") or not isinstance(payload.get("sources"), dict):
raise ValueError("问天知识库格式不完整。")
raise HeavenKnowledgeError(
f"问天知识库格式不完整({path.name}):缺少 version 或 sources。",
code="heaven_knowledge_invalid",
)
return payload
def clear_heaven_knowledge_cache() -> None:
_knowledge_catalog.cache_clear()
+25 -9
View File
@@ -283,9 +283,9 @@ class HeavenMarketContextMixin:
) -> dict[str, Any] | None:
"""Return the Shenwan L2 sector context for heaven trend.
观势行业层只使用申万二级行业。外显盘中使用 rt_sw_k、历史使用
sw_daily;内核独立使用目标日期成分股行情聚合。收盘过渡期在
sw_daily 入库前接受同日15:00后的 rt_sw_k 收盘快照
观势行业层只使用申万二级行业。外显优先使用已发布的 sw_daily
盘中及收盘过渡期使用免费申万实时行情;内核使用数据中枢或免费
实时成分行情。不再调用无权限的 rt_sw_k / rt_k
"""
cache_key = f"{trade_date}:{identifier.strip().lower()}"
cached = self.database.get_data_snapshot("heaven_sector", cache_key)
@@ -299,6 +299,14 @@ class HeavenMarketContextMixin:
and not cached.get("realtime")
and int(cached.get("schema_version") or 0) >= 6
)
cached_quotes = int((cached or {}).get("quote_count") or 0)
cached_lkg = bool(
cached
and cached_date == trade_date
and cached.get("taxonomy") == "sw_l2"
and cached_quotes > 0
and int(cached.get("schema_version") or 0) >= 6
)
if market_mode != "intraday" and cached_valid:
return cached
if not self.configured:
@@ -311,8 +319,12 @@ class HeavenMarketContextMixin:
allow_realtime_close=market_mode == "closed",
)
except TushareError as exc:
if cached_valid:
return cached
if cached_lkg:
delayed = dict(cached)
delayed["delayed"] = True
delayed["delay_notice"] = "主备免费行情均暂不可用,显示最近一次真实快照"
delayed["realtime"] = market_mode == "intraday"
return delayed
return {
"name": "",
"code": "",
@@ -323,12 +335,16 @@ class HeavenMarketContextMixin:
"precise": False,
"inner_precise": False,
"outer_precise": False,
"coverage": 0,
"member_count": 0,
"quote_count": 0,
"error": f"申万二级行业数据获取失败:{exc}",
}
if not payload.get("realtime") and payload.get("precise"):
if int(payload.get("quote_count") or 0) > 0:
self.database.save_data_snapshot(
"heaven_sector",
cache_key,
str(payload.get("source") or "tushare"),
payload,
)
elif not payload.get("realtime") and payload.get("precise"):
self.database.save_data_snapshot(
"heaven_sector",
cache_key,
+4 -1
View File
@@ -243,6 +243,7 @@ class HeavenTrendMixin:
"detail": (
f"申万二级 {sector.get('name') or '--'} {sector.get('code') or '--'} "
f"成分覆盖 {int(sector.get('quote_count') or 0)}/{int(sector.get('member_count') or 0)}"
+ (";延迟快照" if sector.get("delayed") or sector.get("delay_notice") else "")
),
},
{
@@ -341,7 +342,9 @@ class HeavenTrendMixin:
issues.append("行业外显缺少申万官方行情")
if sector and sector_coverage_issue:
issues.append(sector_coverage_issue)
if sector.get("realtime") and not sector.get("relative_turnover"):
if sector.get("delay_notice"):
issues.append(str(sector.get("delay_notice")))
if sector.get("realtime") and not sector.get("relative_turnover") and not sector.get("delayed"):
issues.append("行业内核缺少相对全市场换手活跃度")
stock = stock or {}
+202
View File
@@ -0,0 +1,202 @@
"""Auditable recent-trading-day snapshot backfill helpers.
Planning and backup stay free of provider imports so feature boundary tests remain green.
The service layer supplies open trading dates from the live calendar and executes sync.
"""
from __future__ import annotations
import sqlite3
from datetime import date, datetime, timedelta
from pathlib import Path
from typing import Any, Iterable
MAX_RANGE_TRADING_DAYS = 15
MAX_RECENT_TRADING_DAYS = 60
DEFAULT_RECENT_TRADING_DAYS = 60
# Tables touched by a successful historical dashboard sync. User / token / model
# tables must never appear here.
SNAPSHOT_BACKFILL_WRITE_TABLES = frozenset(
{
"dashboard_snapshots",
"data_snapshots",
"sync_runs",
}
)
def clamp_recent_lookback(lookback: int) -> int:
value = int(lookback)
if value < 1:
raise ValueError("回补交易日数量至少为 1。")
if value > MAX_RECENT_TRADING_DAYS:
raise ValueError(f"单次最多回补最近 {MAX_RECENT_TRADING_DAYS} 个交易日。")
return value
def calendar_window_start(end_date: str, lookback: int) -> str:
"""Natural-day lower bound large enough to cover lookback open sessions."""
end = datetime.strptime(end_date, "%Y%m%d").date()
span = max(40, int(lookback * 2) + 20)
return (end - timedelta(days=span)).strftime("%Y%m%d")
def select_open_trade_dates(
calendar_rows: Iterable[dict[str, Any]],
end_date: str,
lookback: int,
) -> list[str]:
"""Pick the last ``lookback`` open SSE sessions on or before ``end_date``."""
lookback = clamp_recent_lookback(lookback)
end = normalize_compact_date(end_date)
open_dates = sorted(
{
normalize_compact_date(str(row.get("cal_date") or ""))
for row in calendar_rows
if int(row.get("is_open") or 0) == 1 and row.get("cal_date")
}
)
open_dates = [item for item in open_dates if item <= end]
if not open_dates:
raise ValueError("交易日历未返回可用交易日,请检查行情 Token。")
return open_dates[-lookback:]
def select_open_trade_dates_in_range(
calendar_rows: Iterable[dict[str, Any]],
start_date: str,
end_date: str,
*,
maximum: int = MAX_RANGE_TRADING_DAYS,
) -> tuple[list[str], list[str]]:
"""Return (open_dates, skipped_non_trading_days) inside an inclusive range."""
start = normalize_compact_date(start_date)
end = normalize_compact_date(end_date)
if start > end:
raise ValueError("开始日期不能晚于结束日期。")
open_set = {
normalize_compact_date(str(row.get("cal_date") or ""))
for row in calendar_rows
if int(row.get("is_open") or 0) == 1 and row.get("cal_date")
}
open_dates: list[str] = []
skipped: list[str] = []
cursor = datetime.strptime(start, "%Y%m%d").date()
last = datetime.strptime(end, "%Y%m%d").date()
while cursor <= last:
compact = cursor.strftime("%Y%m%d")
if compact in open_set:
open_dates.append(compact)
else:
skipped.append(compact)
cursor += timedelta(days=1)
if len(open_dates) > maximum:
raise ValueError(f"单次最多回补 {maximum} 个交易日。")
return open_dates, skipped
def classify_snapshot_coverage(
trade_dates: list[str],
existing_dates: Iterable[str],
) -> dict[str, Any]:
present_set = {
normalize_compact_date(item)
for item in existing_dates
if item
}
present = [item for item in trade_dates if item in present_set]
missing = [item for item in trade_dates if item not in present_set]
return {
"trade_dates": list(trade_dates),
"present": present,
"missing": missing,
"present_count": len(present),
"missing_count": len(missing),
}
def create_sqlite_backup(
source_path: Path,
backup_dir: Path,
*,
label: str = "pre-backfill",
stamped_at: datetime | None = None,
) -> Path:
"""Create a timestamped SQLite backup via the native backup API."""
source = Path(source_path)
if not source.exists():
raise FileNotFoundError(f"数据库不存在:{source}")
stamp = (stamped_at or datetime.now().astimezone()).strftime("%Y%m%d-%H%M%S")
safe_label = "".join(ch if ch.isalnum() or ch in "-_" else "-" for ch in label).strip("-") or "backup"
backup_dir = Path(backup_dir)
backup_dir.mkdir(parents=True, exist_ok=True)
target = backup_dir / f"review-{safe_label}-{stamp}.db"
source_conn = sqlite3.connect(f"file:{source}?mode=ro", uri=True)
try:
target_conn = sqlite3.connect(target)
try:
source_conn.backup(target_conn)
target_conn.commit()
finally:
target_conn.close()
finally:
source_conn.close()
return target
def display_date(compact: str) -> str:
value = normalize_compact_date(compact)
return f"{value[:4]}-{value[4:6]}-{value[6:8]}"
def normalize_compact_date(value: str) -> str:
compact = str(value or "").replace("-", "").strip()
if len(compact) != 8 or not compact.isdigit():
raise ValueError("日期格式应为 YYYY-MM-DD。")
datetime.strptime(compact, "%Y%m%d")
return compact
def build_backfill_audit(
*,
mode: str,
end_date: str,
lookback: int | None,
coverage: dict[str, Any],
skipped_non_trading_days: list[str] | None = None,
backup_path: str | None = None,
dry_run: bool = False,
results: list[dict[str, Any]] | None = None,
) -> dict[str, Any]:
results = list(results or [])
succeeded = [row for row in results if row.get("status") == "success"]
skipped = [row for row in results if row.get("status") == "skipped"]
failed = [row for row in results if row.get("status") == "failed"]
return {
"ok": not failed,
"mode": mode,
"dry_run": dry_run,
"end_date": display_date(end_date),
"lookback": lookback,
"backup_path": backup_path,
"write_tables": sorted(SNAPSHOT_BACKFILL_WRITE_TABLES),
"trade_dates": [display_date(item) for item in coverage.get("trade_dates") or []],
"present": [display_date(item) for item in coverage.get("present") or []],
"missing": [display_date(item) for item in coverage.get("missing") or []],
"skipped_non_trading_days": [
display_date(item) for item in (skipped_non_trading_days or [])
],
"present_count": int(coverage.get("present_count") or 0),
"missing_count": int(coverage.get("missing_count") or 0),
"results": results,
"succeeded_count": len(succeeded),
"skipped_count": len(skipped),
"failed_count": len(failed),
"created_dates": [
str(row.get("trade_date") or "")
for row in succeeded
if row.get("action") == "created"
],
}
+196 -39
View File
@@ -2,6 +2,7 @@ from __future__ import annotations
import http.client
import json
import logging
import re
import time
import urllib.error
@@ -13,14 +14,20 @@ from threading import Lock
from typing import Any, ClassVar
from backend.bootstrap.config import tushare_code as _stock_market_code
from backend.data.providers.ifind_client import IfindError, IfindHttpClient
from backend.data.providers.ifind_client import IfindError
LOGGER = logging.getLogger("xiaobai.charts")
class ChartDataError(RuntimeError):
pass
DAILY_CHART_LIMIT = 45
TRENDS_URL = "https://push2delay.eastmoney.com/api/qt/stock/trends2/get"
HIS_TRENDS_URL = "https://push2his.eastmoney.com/api/qt/stock/trends2/get"
BOARD_LIST_URL = "https://push2delay.eastmoney.com/api/qt/clist/get"
BROWSER_USER_AGENT = (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
@@ -35,55 +42,186 @@ INDEX_SECIDS = {
class MarketChartClient:
"""Prefer iFinD for display charts and retain Eastmoney as a last resort."""
"""Display charts are served by the data hub only."""
def __init__(self, ifind: IfindHttpClient, fallback: "EastmoneyChartClient") -> None:
self.ifind = ifind
self.fallback = fallback
def __init__(self, datahub: Any = None) -> None:
self.datahub = datahub
self.ifind = None
self.fallback = None
def stock_intraday(self, code: str) -> dict[str, Any]:
normalized = str(code or "").strip()
if not re.fullmatch(r"\d{6}", normalized):
raise ChartDataError("Invalid stock code")
ifind_code = _stock_market_code(normalized)
try:
return self._ifind_intraday(ifind_code, "stock", normalized)
except (IfindError, ChartDataError):
return self.fallback.stock_intraday(normalized)
hub_chart = self._datahub_intraday(normalized)
if hub_chart is not None:
return hub_chart
raise ChartDataError("分时图数据中枢暂不可用")
def stock_daily(self, code: str, end_date: str, limit: int = 90) -> list[dict[str, Any]]:
def stock_daily(self, code: str, end_date: str, limit: int = DAILY_CHART_LIMIT) -> list[dict[str, Any]]:
normalized = str(code or "").strip()
if not re.fullmatch(r"\d{6}", normalized):
raise ChartDataError("Invalid stock code")
return self._ifind_daily(_stock_market_code(normalized), end_date, limit)
hub_rows = self._datahub_daily(normalized, end_date, limit, "daily")
if hub_rows:
return hub_rows
raise ChartDataError("日K数据中枢暂不可用")
def index_daily(self, identifier: str, end_date: str, limit: int = 90) -> list[dict[str, Any]]:
def index_daily(self, identifier: str, end_date: str, limit: int = DAILY_CHART_LIMIT) -> list[dict[str, Any]]:
normalized = str(identifier or "").strip().upper()
if normalized not in INDEX_SECIDS:
raise ChartDataError("Unsupported index")
return self._ifind_daily(normalized, end_date, limit)
hub_rows = self._datahub_daily(normalized, end_date, limit, "index_daily")
if hub_rows:
return hub_rows
raise ChartDataError("指数日K数据中枢暂不可用")
def board_daily(self, identifier: str, end_date: str, limit: int = 90) -> list[dict[str, Any]]:
normalized = str(identifier or "").strip().upper()
if not normalized:
raise ChartDataError("Invalid board code")
return self._ifind_daily(normalized, end_date, limit)
hub_rows = self._datahub_daily(normalized, end_date, limit, "sector_daily")
if hub_rows:
return hub_rows
raise ChartDataError("板块日K数据中枢暂不可用")
def index_intraday(self, identifier: str) -> dict[str, Any]:
normalized = str(identifier or "").strip().upper()
if normalized not in INDEX_SECIDS:
raise ChartDataError("Unsupported index")
hub_chart = self._datahub_intraday(normalized)
if hub_chart is not None:
return hub_chart
raise ChartDataError("指数分时数据中枢暂不可用")
def _datahub_intraday(self, code: str) -> dict[str, Any] | None:
if self.datahub is None:
return None
try:
return self._ifind_intraday(normalized, "index", normalized)
except (IfindError, ChartDataError):
return self.fallback.index_intraday(normalized)
chart = self.datahub.try_intraday(code)
except Exception as exc:
LOGGER.warning("datahub intraday unexpected error: %s", exc)
return None
if not chart:
return None
points = list(chart.get("points") or [])
if not points:
return None
return chart
def _datahub_daily(
self,
code: str,
end_date: str,
limit: int,
dataset: str,
) -> list[dict[str, Any]] | None:
if self.datahub is None or not hasattr(self.datahub, "try_daily_chart"):
return None
try:
rows = self.datahub.try_daily_chart(code, end_date, limit, dataset)
except Exception as exc:
LOGGER.warning("datahub daily unexpected error: %s", exc)
rows = None
if not rows:
return None
compact_end = str(end_date or "").replace("-", "")
market_now = datetime.now().astimezone()
today = market_now.strftime("%Y%m%d")
market_open = (
market_now.weekday() < 5
and market_now.time().replace(tzinfo=None) >= dt_time(9, 30)
)
if compact_end == today and market_open:
overlay = self._datahub_today_bar(code, dataset, rows)
if overlay:
if rows and rows[-1]["trade_date"] == overlay["trade_date"]:
rows[-1] = overlay
else:
rows.append(overlay)
return rows
def _datahub_today_bar(
self,
code: str,
dataset: str,
history: list[dict[str, Any]],
) -> dict[str, Any] | None:
today_display = datetime.now().astimezone().date().isoformat()
previous = history[-1]["close"] if history and history[-1]["trade_date"] != today_display else (
history[-2]["close"] if len(history) >= 2 else 0.0
)
quote = None
if dataset == "index_daily" and hasattr(self.datahub, "try_index_quotes"):
quotes = self.datahub.try_index_quotes() or []
quote = next(
(
item for item in quotes
if str(item.get("ts_code") or "") == code or str(item.get("code") or "") == code.split(".")[0]
),
None,
)
elif hasattr(self.datahub, "try_quotes"):
quotes = self.datahub.try_quotes([code]) or []
quote = quotes[0] if quotes else None
if quote:
close = _number(quote.get("close") if quote.get("close") not in (None, "") else quote.get("price"))
open_price = _number(quote.get("open"))
high = _number(quote.get("high"))
low = _number(quote.get("low"))
previous_close = _number(
quote.get("pre_close") if quote.get("pre_close") not in (None, "") else quote.get("previous_close")
) or previous
volume = _number(quote.get("vol") if quote.get("vol") not in (None, "") else quote.get("volume"))
amount = _number(quote.get("amount"))
if close > 0 and open_price > 0:
return {
"trade_date": today_display,
"open": open_price,
"high": high or close,
"low": low or close,
"close": close,
"change": round((close / previous_close - 1) * 100, 4) if previous_close else 0.0,
"volume": volume,
"amount_billion": amount / 100_000_000,
"realtime": True,
}
chart = self._datahub_intraday(code)
points = list((chart or {}).get("points") or [])
if not points:
return None
closes = [_number(point.get("close")) for point in points if _number(point.get("close")) > 0]
if not closes:
return None
opens = [_number(point.get("open")) for point in points if _number(point.get("open")) > 0]
highs = [_number(point.get("high")) for point in points if _number(point.get("high")) > 0]
lows = [_number(point.get("low")) for point in points if _number(point.get("low")) > 0]
volume = sum(_number(point.get("volume")) for point in points)
amount = sum(_number(point.get("amount")) for point in points)
previous_close = _number((chart or {}).get("previous_close")) or previous
close = closes[-1]
open_price = opens[0] if opens else closes[0]
return {
"trade_date": today_display,
"open": open_price,
"high": max(highs or closes),
"low": min(lows or closes),
"close": close,
"change": round((close / previous_close - 1) * 100, 4) if previous_close else 0.0,
"volume": volume,
"amount_billion": amount / 100_000_000,
"realtime": True,
}
def board_intraday(self, identifier: str, name: str = "") -> dict[str, Any]:
normalized = str(identifier or "").strip().upper()
try:
return self._ifind_intraday(normalized, "board", normalized, name)
except (IfindError, ChartDataError):
return self.fallback.board_intraday(normalized, name)
hub_chart = self._datahub_intraday(normalized)
if hub_chart is not None:
if name:
hub_chart = dict(hub_chart)
hub_chart["name"] = name
return hub_chart
raise ChartDataError("板块分时数据中枢暂不可用")
def _ifind_intraday(
self,
@@ -92,7 +230,7 @@ class MarketChartClient:
identifier: str,
name: str = "",
) -> dict[str, Any]:
if not self.ifind.configured:
if not self.ifind or not self.ifind.configured:
raise ChartDataError("iFinD is not configured")
now = datetime.now().astimezone()
rows: list[dict[str, Any]] = []
@@ -129,7 +267,7 @@ class MarketChartClient:
def _ifind_daily(
self, ifind_code: str, end_date: str, limit: int
) -> list[dict[str, Any]]:
if not self.ifind.configured:
if not self.ifind or not self.ifind.configured:
raise ChartDataError("iFinD is not configured")
compact_end = str(end_date or "").replace("-", "")
if not re.fullmatch(r"\d{8}", compact_end):
@@ -231,9 +369,11 @@ class MarketChartClient:
pass
if not normalized:
raise ChartDataError("No iFinD daily chart data returned")
return normalized[-max(20, min(180, int(limit))):]
return normalized[-max(1, int(limit)):]
def _previous_close(self, code: str, trade_date: str, fallback: float) -> float:
if not self.ifind:
return fallback
today = datetime.now().astimezone().date().isoformat()
if trade_date == today:
try:
@@ -305,21 +445,29 @@ class EastmoneyChartClient:
if cached is not None:
return cached
payload = self._request_json(
TRENDS_URL,
{
"secid": secid,
"fields1": "f1,f2,f3,f4,f5,f6,f7,f8,f9,f10,f11,f12,f13",
"fields2": "f51,f52,f53,f54,f55,f56,f57,f58",
"iscr": "0",
"ndays": "1",
},
"https://quote.eastmoney.com/",
)
data = payload.get("data") or {}
points = [point for raw in data.get("trends") or [] if (point := _parse_trend(raw))]
params = {
"secid": secid,
"fields1": "f1,f2,f3,f4,f5,f6,f7,f8,f9,f10,f11,f12,f13",
"fields2": "f51,f52,f53,f54,f55,f56,f57,f58",
"iscr": "0",
}
last_error: Exception | None = None
data: dict[str, Any] = {}
points: list[dict[str, Any]] = []
for url, ndays in ((TRENDS_URL, "1"), (TRENDS_URL, "5"), (HIS_TRENDS_URL, "5")):
request_params = {**params, "ndays": ndays}
try:
payload = self._request_json(url, request_params, "https://quote.eastmoney.com/")
except ChartDataError as exc:
last_error = exc
continue
data = payload.get("data") or {}
parsed = [point for raw in data.get("trends") or [] if (point := _parse_trend(raw))]
points = _latest_session(parsed)
if points:
break
if not points:
raise ChartDataError("No intraday chart data returned")
raise ChartDataError("No intraday chart data returned") from last_error
result = {
"entity_type": entity_type,
@@ -433,6 +581,15 @@ class EastmoneyChartClient:
raise ChartDataError("Intraday chart request failed") from last_error
def _latest_session(points: list[dict[str, Any]]) -> list[dict[str, Any]]:
if not points:
return []
latest = max(str(point.get("date") or "") for point in points)
if not latest:
return points
return [point for point in points if str(point.get("date") or "") == latest]
def _parse_trend(raw: Any) -> dict[str, Any] | None:
fields = str(raw or "").split(",")
if len(fields) < 8 or " " not in fields[0]:
+25
View File
@@ -227,6 +227,31 @@ class MarketRepositoryMixin:
result.append(payload)
return result
def list_snapshot_trade_dates(
self,
start_date: str = "",
end_date: str = "",
) -> list[str]:
clauses: list[str] = []
parameters: list[Any] = []
if start_date:
clauses.append("trade_date >= ?")
parameters.append(start_date)
if end_date:
clauses.append("trade_date <= ?")
parameters.append(end_date)
where = f"WHERE {' AND '.join(clauses)}" if clauses else ""
with self.connect() as connection:
rows = connection.execute(
f"""
SELECT trade_date FROM dashboard_snapshots
{where}
ORDER BY trade_date
""",
parameters,
).fetchall()
return [str(row["trade_date"]) for row in rows]
def start_sync(self, trade_date: str, source: str) -> int:
started_at = datetime.now().astimezone().isoformat(timespec="seconds")
with self.connect() as connection:
+618 -67
View File
@@ -3,17 +3,32 @@ from __future__ import annotations
import copy
import re
from datetime import date, datetime, time as dt_time, timedelta
from pathlib import Path
from typing import Any
from backend.bootstrap.config import (
DATA_DIR,
normalize_date,
tushare_code,
validate_stock_code,
validate_text,
)
from backend.data.providers.ifind_client import IfindError
from backend.data.providers.tushare_client import TushareClient, TushareError
from backend.features.market.charts import ChartDataError
from backend.data.providers.tushare_client import TushareError
from backend.data.providers.tushare_helpers import _moneyflow_payload, _optional_number
from backend.data.realtime import RealtimeAggregateError
from backend.features.market.backfill_history import (
DEFAULT_RECENT_TRADING_DAYS,
MAX_RANGE_TRADING_DAYS,
build_backfill_audit,
calendar_window_start,
classify_snapshot_coverage,
create_sqlite_backup,
display_date,
select_open_trade_dates,
select_open_trade_dates_in_range,
)
from backend.features.market.charts import ChartDataError, DAILY_CHART_LIMIT
from backend.features.market.insights import MarketInsightsService
from backend.features.sentiment.engine import SENTIMENT_ENGINE_VERSION
@@ -29,6 +44,7 @@ SEARCH_TYPE_LABELS = {
"theme": "题材",
"index": "指数",
}
TODAY_DAILY_UNAVAILABLE_NOTICE = "今日日K暂不可用,仍显示最近收盘K线。"
THS_SEARCH_TYPES = {
"I": ("sector", "行业板块"),
"R": ("sector", "地域板块"),
@@ -45,16 +61,40 @@ class MarketServiceMixin:
self._tushare_client(),
ifind=self.ifind,
)
def _tushare_client(self) -> TushareClient:
def _tushare_client(self) -> Any:
override = getattr(self, "_market_client_override", None)
if override is not None:
return override
gateway = getattr(self, "data_gateway", None)
if gateway is not None:
return gateway.tushare()
# Compatibility for isolated legacy unit-test service stubs.
return TushareClient(self.token)
if gateway is None:
raise RuntimeError("数据中枢尚未装配。")
return gateway.tushare()
def _now(self) -> datetime:
clock = getattr(self, "clock", None)
if callable(clock):
return clock()
return datetime.now().astimezone()
def _is_requested_open_session(self, requested_date: str) -> bool:
now = self._now()
if requested_date != now.strftime("%Y%m%d"):
return False
if now.time().replace(tzinfo=None) < dt_time(9, 15):
return False
client = self._tushare_client() if self.configured else None
resolve = getattr(client, "resolve_trade_context", None) if client else None
if resolve is None:
return now.weekday() < 5
try:
trade_date, _ = resolve(requested_date)
except Exception:
return now.weekday() < 5
return str(trade_date or "") == requested_date
def get_dashboard(self, trade_date: str, force: bool = False) -> dict[str, Any]:
normalized_date = normalize_date(trade_date)
now = datetime.now().astimezone()
now = self._now()
if (
normalized_date == now.strftime("%Y%m%d")
and now.time().replace(tzinfo=None) < datetime.strptime("09:15", "%H:%M").time()
@@ -66,6 +106,8 @@ class MarketServiceMixin:
if not force:
snapshot = self.database.get_snapshot(normalized_date)
if snapshot and str((snapshot.get("meta") or {}).get("source") or "") != "demo":
if self._should_retry_incomplete_snapshot(snapshot, normalized_date):
return self.sync_dashboard(normalized_date)
snapshot = copy.deepcopy(snapshot)
if normalized_date != now.strftime("%Y%m%d"):
snapshot.setdefault("meta", {}).update(
@@ -84,6 +126,8 @@ class MarketServiceMixin:
"dashboard_request_v1", normalized_date
)
if resolved and str((resolved.get("meta") or {}).get("source") or "") != "demo":
if self._should_retry_incomplete_snapshot(resolved, normalized_date):
return self.sync_dashboard(normalized_date)
resolved = copy.deepcopy(resolved)
resolved.setdefault("meta", {})["requested_date"] = self._display_compact_date(
normalized_date
@@ -125,6 +169,70 @@ class MarketServiceMixin:
def _display_compact_date(compact: str) -> str:
return f"{compact[:4]}-{compact[4:6]}-{compact[6:8]}"
@staticmethod
def _chinese_month_day(value: str) -> str:
compact = str(value or "").replace("-", "").replace("/", "")
if len(compact) < 8 or not compact[:8].isdigit():
return "最近可用交易日"
return f"{int(compact[4:6])}{int(compact[6:8])}"
def _preparing_display_notice(self, actual_date: str, requested_date: str) -> str:
shown = self._chinese_month_day(actual_date)
requested = str(requested_date or "").replace("-", "")
if requested == self._now().strftime("%Y%m%d"):
return f"今日数据正在准备,当前展示 {shown}"
return f"所选日期数据尚未到齐,当前展示 {shown}"
@staticmethod
def _snapshot_age_seconds(meta: dict[str, Any]) -> float:
raw = str(meta.get("updated_at") or "")
if not raw:
return 10**9
try:
updated_at = datetime.fromisoformat(raw)
except ValueError:
return 10**9
now = datetime.now().astimezone()
if updated_at.tzinfo is None:
updated_at = updated_at.replace(tzinfo=now.tzinfo)
return (now - updated_at.astimezone(now.tzinfo)).total_seconds()
def _should_retry_incomplete_snapshot(
self, snapshot: dict[str, Any], requested_date: str
) -> bool:
if requested_date != self._now().strftime("%Y%m%d"):
return False
meta = snapshot.get("meta") or {}
actual = str(meta.get("trade_date") or "").replace("-", "")
stale_carry = bool(meta.get("carried_forward") or actual != requested_date)
if stale_carry and self._is_requested_open_session(requested_date):
return True
incomplete = meta.get("limit_data_source") == "derived" or stale_carry
return incomplete and self._snapshot_age_seconds(meta) >= 60
def _annotate_data_status(self, dashboard: dict[str, Any]) -> dict[str, Any]:
meta = dashboard.setdefault("meta", {})
notice = str(meta.get("notice") or "")
requested = str(meta.get("requested_date") or "").replace("-", "")
actual = str(meta.get("trade_date") or "").replace("-", "")
if meta.get("limit_data_source") == "derived" and not meta.get("carried_forward"):
meta["data_status"] = "partial"
meta["display_notice"] = notice or "部分正式数据尚未到齐,当前展示日线推算结果"
elif meta.get("carried_forward"):
if "非交易日" in notice or "盘前" in notice:
meta["data_status"] = "carried"
meta["display_notice"] = notice
else:
meta["data_status"] = "preparing"
meta["display_notice"] = self._preparing_display_notice(actual, requested)
elif meta.get("realtime"):
meta["data_status"] = "intraday"
meta.setdefault("display_notice", "")
else:
meta["data_status"] = "official"
meta.setdefault("display_notice", "")
return dashboard
def _carry_dashboard(
self, snapshot: dict[str, Any], requested_date: str, reason: str
) -> dict[str, Any]:
@@ -139,16 +247,16 @@ class MarketServiceMixin:
"notice": reason,
}
)
return carried
return self._annotate_data_status(carried)
def _realtime_snapshot_due(
self,
normalized_date: str,
snapshot: dict[str, Any],
) -> bool:
if not self.configured or normalized_date != date.today().strftime("%Y%m%d"):
if not self.configured or normalized_date != self._now().strftime("%Y%m%d"):
return False
now = datetime.now().astimezone()
now = self._now()
local_time = now.time().replace(tzinfo=None)
realtime_start = datetime.strptime("09:15", "%H:%M").time()
morning_end = datetime.strptime("11:35", "%H:%M").time()
@@ -184,14 +292,28 @@ class MarketServiceMixin:
if not self.configured:
raise TushareError("公共行情尚未配置")
dashboard = self._tushare_client().dashboard(normalized_date)
dashboard["meta"]["source"] = source
dashboard["meta"]["requested_date"] = self._display_compact_date(normalized_date)
meta = dashboard.setdefault("meta", {})
quote_source = str(meta.get("quote_source") or "")
meta["source"] = source
if quote_source:
meta["quote_source"] = quote_source
meta["requested_date"] = self._display_compact_date(normalized_date)
if meta.get("limit_data_source") == "derived":
meta.setdefault(
"notice",
"涨跌停高级接口当日数据尚未更新,已使用日线数据推算。",
)
dashboard = self._enrich_dashboard_sentiment(dashboard, normalized_date)
record_count = self._record_count(dashboard)
actual_date = normalize_date(
str(dashboard.get("meta", {}).get("trade_date") or normalized_date)
)
if actual_date != normalized_date and self._is_requested_open_session(
normalized_date
):
raise TushareError(
f"Intraday dashboard resolved {actual_date} instead of {normalized_date}"
)
self.database.save_snapshot(actual_date, source, dashboard)
if actual_date != normalized_date:
dashboard.setdefault("meta", {}).update(
@@ -213,10 +335,37 @@ class MarketServiceMixin:
)
return self._apply_reason_overrides(self._with_storage(dashboard, cached=False))
except TushareError as exc:
if self._is_requested_open_session(normalized_date):
existing = self.database.get_snapshot(normalized_date)
existing_date = str(
((existing or {}).get("meta") or {}).get("trade_date") or ""
).replace("-", "")
if existing and existing_date == normalized_date:
kept = copy.deepcopy(existing)
kept.setdefault("meta", {}).update(
{
"requested_date": self._display_compact_date(normalized_date),
}
)
self.database.finish_sync(
sync_id,
"fallback",
self._record_count(kept),
str(exc),
"tushare",
)
return self._apply_reason_overrides(
self._with_storage(kept, cached=True)
)
self.database.finish_sync(sync_id, "failed", message=str(exc))
raise ValueError("当天盘中行情暂时不可用,请稍后重试。") from exc
fallback = self.database.get_latest_real_snapshot(normalized_date)
if fallback:
actual = str((fallback.get("meta") or {}).get("trade_date") or "")
carried = self._carry_dashboard(
fallback, normalized_date, f"最新行情暂不可用,沿用最近收盘快照:{exc}"
fallback,
normalized_date,
self._preparing_display_notice(actual, normalized_date),
)
self.database.finish_sync(
sync_id, "fallback", self._record_count(carried), str(exc), "tushare"
@@ -526,7 +675,7 @@ class MarketServiceMixin:
"index_daily",
{
"ts_code": basic["id"],
"start_date": (end - timedelta(days=190)).strftime("%Y%m%d"),
"start_date": (end - timedelta(days=400)).strftime("%Y%m%d"),
"end_date": resolved_date,
},
"ts_code,trade_date,open,high,low,close,pct_chg,vol,amount",
@@ -542,10 +691,10 @@ class MarketServiceMixin:
"change": float(row.get("pct_chg") or 0),
"volume": float(row.get("vol") or 0),
}
for row in rows[-90:]
for row in rows[-DAILY_CHART_LIMIT:]
]
try:
chart_series = self.chart_data.index_daily(str(basic["id"]), resolved_date, 90)
chart_series = self.chart_data.index_daily(str(basic["id"]), resolved_date, DAILY_CHART_LIMIT)
if chart_series:
series = chart_series
except (AttributeError, ChartDataError):
@@ -653,7 +802,7 @@ class MarketServiceMixin:
result = copy.deepcopy(payload)
now = datetime.now().astimezone()
try:
result["prices"] = self.chart_data.stock_daily(code, requested_date, 90)
result["prices"] = self.chart_data.stock_daily(code, requested_date, DAILY_CHART_LIMIT)
result["meta"] = {**(result.get("meta") or {}), "chart_source": "market_chart"}
except (AttributeError, ChartDataError):
pass
@@ -665,27 +814,28 @@ class MarketServiceMixin:
"trade_date": f"{actual_date[:4]}-{actual_date[4:6]}-{actual_date[6:]}",
}
today = now.strftime("%Y%m%d")
latest_bar = (result.get("prices") or [{}])[-1] if result.get("prices") else {}
official_today = (
actual_date == today and not bool(latest_bar.get("realtime"))
)
after_close = now.time().replace(tzinfo=None) >= dt_time(15, 0)
should_merge = (
requested_date == today
and actual_date <= today
and now.weekday() < 5
and now.time().replace(tzinfo=None) >= dt_time(9, 30)
and not (official_today and after_close)
)
if should_merge:
quote = self._ifind_realtime_stock_quote(code)
quote = self._resolve_today_daily_quote(code, today, result)
if quote and self._valid_realtime_stock_quote(quote, today):
self._merge_realtime_stock_detail(result, quote, requested_date)
elif self.configured and actual_date < today:
client = self._tushare_client()
try:
resolved_date, _ = client.resolve_trade_context(requested_date)
if resolved_date == today:
quote = client.realtime_stock_quote(tushare_code(code), requested_date)
if self._valid_realtime_stock_quote(quote, today):
self._merge_realtime_stock_detail(result, quote, requested_date)
except TushareError:
pass
return self._enrich_stock_detail(result)
elif actual_date < today:
result["meta"] = {
**(result.get("meta") or {}),
"notice": TODAY_DAILY_UNAVAILABLE_NOTICE,
}
return self._enrich_stock_detail(result, requested_date)
@staticmethod
def _sanitize_stock_detail_prices(
@@ -799,6 +949,138 @@ class MarketServiceMixin:
"quote_time": str(row.get("time") or ""),
}
def _resolve_today_daily_quote(
self, code: str, today: str, payload: dict[str, Any]
) -> dict[str, Any] | None:
quote = self._ifind_realtime_stock_quote(code)
if quote and self._valid_realtime_stock_quote(quote, today):
return quote
if self.configured:
try:
client = self._tushare_client()
resolve = getattr(client, "resolve_trade_context", None)
resolved = today
if callable(resolve):
resolved, _ = resolve(today)
if str(resolved or "") == today:
quote = client.realtime_stock_quote(tushare_code(code), today)
if self._valid_realtime_stock_quote(quote, today):
return quote
except TushareError:
pass
quote = self._free_realtime_stock_quote(code, today)
if quote and self._valid_realtime_stock_quote(quote, today):
return quote
return self._intraday_realtime_stock_quote(code, today, payload)
def _free_realtime_stock_quote(self, code: str, today: str) -> dict[str, Any] | None:
aggregator = getattr(self, "realtime_aggregator", None)
if aggregator is None:
return None
ts_code = tushare_code(code)
for loader in (
getattr(aggregator, "tencent_stock_quote", None),
getattr(aggregator, "eastmoney_stock_quote", None),
):
if not callable(loader):
continue
try:
row = loader(ts_code, expected_date=today)
except (RealtimeAggregateError, Exception):
continue
quote = self._quote_from_free_row(code, today, row)
if quote:
return quote
return None
def _quote_from_free_row(
self, code: str, today: str, row: dict[str, Any]
) -> dict[str, Any] | None:
price = float(row.get("close") or 0)
previous_close = float(row.get("pre_close") or 0)
if price <= 0 or previous_close <= 0:
return None
try:
name, sector = self._stock_identity(code, today)
except Exception:
name, sector = "--", "其他"
epoch = int(row.get("quote_time_epoch") or 0)
if epoch > 0:
quote_time = datetime.fromtimestamp(epoch).astimezone().isoformat(timespec="seconds")
else:
quote_date = str(row.get("quote_date") or today)
quote_time = f"{quote_date[:4]}-{quote_date[4:6]}-{quote_date[6:]}"
quote = {
"name": str(row.get("name") or name or "--"),
"sector": sector,
"price": price,
"open": float(row.get("open") or 0),
"high": float(row.get("high") or 0),
"low": float(row.get("low") or 0),
"change": round((price / previous_close - 1) * 100, 4),
"volume": float(row.get("vol") or 0),
"amount_billion": float(row.get("amount") or 0) / 100_000_000,
"turnover_rate": float(row.get("turnover_rate") or 0),
"quote_time": quote_time,
}
flow = _moneyflow_payload(row)
if flow.get("available"):
quote["moneyflow"] = flow
return quote
def _intraday_realtime_stock_quote(
self, code: str, today: str, payload: dict[str, Any]
) -> dict[str, Any] | None:
chart_data = getattr(self, "chart_data", None)
if chart_data is None:
return None
try:
chart = chart_data.stock_intraday(code)
except (AttributeError, ChartDataError, Exception):
return None
points = [
point
for point in list(chart.get("points") or [])
if str(point.get("date") or "").replace("-", "") == today
]
if not points:
return None
opens = [float(point.get("open") or 0) for point in points if float(point.get("open") or 0) > 0]
highs = [float(point.get("high") or 0) for point in points if float(point.get("high") or 0) > 0]
lows = [float(point.get("low") or 0) for point in points if float(point.get("low") or 0) > 0]
closes = [float(point.get("close") or 0) for point in points if float(point.get("close") or 0) > 0]
if not opens or not highs or not lows or not closes:
return None
price = closes[-1]
previous_close = float(chart.get("previous_close") or 0)
if previous_close <= 0:
history = list(payload.get("prices") or [])
previous_close = float((history[-1] if history else {}).get("close") or 0)
if previous_close <= 0:
return None
volume = sum(float(point.get("volume") or 0) for point in points)
amount = sum(float(point.get("amount") or 0) for point in points)
if volume <= 0 and amount <= 0:
return None
try:
name, sector = self._stock_identity(code, today)
except Exception:
name, sector = "--", "其他"
return {
"name": name,
"sector": sector,
"price": price,
"open": opens[0],
"high": max(highs),
"low": min(lows),
"change": round((price / previous_close - 1) * 100, 4),
"volume": volume,
"volume_unit": "lots",
"amount_billion": amount / 100_000_000,
"turnover_rate": 0.0,
"quote_time": str(points[-1].get("date") or today),
}
@staticmethod
def _merge_realtime_stock_detail(
payload: dict[str, Any], quote: dict[str, Any], trade_date: str
@@ -820,23 +1102,29 @@ class MarketServiceMixin:
prices[-1] = realtime_bar
else:
prices.append(realtime_bar)
payload["prices"] = prices[-90:]
payload["prices"] = prices[-DAILY_CHART_LIMIT:]
stock = dict(payload.get("stock") or {})
stock.update(
{
"name": quote["name"],
"industry": quote["sector"],
"price": quote["price"],
"change": quote["change"],
"amount_billion": quote["amount_billion"],
"turnover_rate": quote["turnover_rate"],
}
)
updates = {
"name": quote["name"],
"industry": quote["sector"],
"price": quote["price"],
"change": quote["change"],
"amount_billion": quote["amount_billion"],
}
quote_turnover = _optional_number(quote.get("turnover_rate"))
if quote_turnover:
updates["turnover_rate"] = quote_turnover
stock.update(updates)
payload["stock"] = stock
quote_flow = quote.get("moneyflow")
current_flow = payload.get("moneyflow") or {}
if isinstance(quote_flow, dict) and quote_flow.get("available") and not current_flow.get("available"):
payload["moneyflow"] = quote_flow
payload["meta"] = {
**(payload.get("meta") or {}),
"trade_date": display_date,
"realtime": True,
"notice": "",
"updated_at": datetime.now().astimezone().isoformat(timespec="seconds"),
}
@@ -870,7 +1158,7 @@ class MarketServiceMixin:
intraday_status = "unavailable"
intraday_notice = "分时行情暂不可用,请稍后重试。"
prices = list(detail.get("prices") or [])[-60:]
prices = list(detail.get("prices") or [])[-DAILY_CHART_LIMIT:]
stock = dict(detail.get("stock") or {"code": code})
realtime = bool(detail_meta.get("realtime"))
return {
@@ -890,31 +1178,226 @@ class MarketServiceMixin:
"intraday": intraday_points,
}
def backfill(self, start_date: str, end_date: str) -> list[dict[str, Any]]:
start = datetime.strptime(normalize_date(start_date), "%Y%m%d").date()
end = datetime.strptime(normalize_date(end_date), "%Y%m%d").date()
if start > end:
raise ValueError("开始日期不能晚于结束日期。")
weekdays = []
current = start
while current <= end:
if current.weekday() < 5:
weekdays.append(current)
current += timedelta(days=1)
if len(weekdays) > 15:
raise ValueError("单次最多回补 15 个工作日。")
results = []
for day in weekdays:
dashboard = self.sync_dashboard(day.strftime("%Y%m%d"))
def backfill(
self,
start_date: str = "",
end_date: str = "",
*,
lookback: int | None = None,
dry_run: bool = False,
force: bool = False,
create_backup: bool = True,
) -> dict[str, Any]:
"""Backfill dashboard snapshots for real trading days only.
- Date-range mode keeps the admin UI contract (max 15 open sessions).
- Recent mode fills the last N open sessions (default/max 60).
Weekends and holidays are reported as skipped non-trading days, not errors.
"""
if not self.configured:
raise ValueError("公共行情尚未配置,无法回补历史快照。")
normalized_end = normalize_date(end_date or date.today().isoformat())
if lookback is not None or not (start_date and end_date):
target_lookback = (
DEFAULT_RECENT_TRADING_DAYS if lookback is None else int(lookback)
)
return self.backfill_recent_trading_days(
end_date=normalized_end,
lookback=target_lookback,
dry_run=dry_run,
force=force,
create_backup=create_backup,
)
return self._backfill_date_range(
start_date=normalize_date(start_date),
end_date=normalized_end,
dry_run=dry_run,
force=force,
create_backup=create_backup,
)
def backfill_recent_trading_days(
self,
end_date: str = "",
lookback: int = DEFAULT_RECENT_TRADING_DAYS,
*,
dry_run: bool = False,
force: bool = False,
create_backup: bool = True,
) -> dict[str, Any]:
normalized_end = normalize_date(end_date or date.today().isoformat())
trade_dates = self._load_recent_open_trade_dates(normalized_end, lookback)
existing = self.database.list_snapshot_trade_dates(
trade_dates[0], trade_dates[-1]
)
coverage = classify_snapshot_coverage(trade_dates, existing)
return self._execute_snapshot_backfill(
mode="recent",
end_date=normalized_end,
lookback=lookback,
coverage=coverage,
skipped_non_trading_days=[],
dry_run=dry_run,
force=force,
create_backup=create_backup,
)
def _backfill_date_range(
self,
start_date: str,
end_date: str,
*,
dry_run: bool = False,
force: bool = False,
create_backup: bool = True,
) -> dict[str, Any]:
window_start = calendar_window_start(end_date, MAX_RANGE_TRADING_DAYS)
calendar_rows = self._tushare_client().query(
"trade_cal",
{
"exchange": "SSE",
"start_date": min(window_start, start_date),
"end_date": end_date,
},
"cal_date,is_open,pretrade_date",
)
trade_dates, skipped = select_open_trade_dates_in_range(
calendar_rows,
start_date,
end_date,
maximum=MAX_RANGE_TRADING_DAYS,
)
if not trade_dates:
raise ValueError("选定区间内没有交易日,周末或节假日无需回补。")
existing = self.database.list_snapshot_trade_dates(trade_dates[0], trade_dates[-1])
coverage = classify_snapshot_coverage(trade_dates, existing)
return self._execute_snapshot_backfill(
mode="range",
end_date=end_date,
lookback=None,
coverage=coverage,
skipped_non_trading_days=skipped,
dry_run=dry_run,
force=force,
create_backup=create_backup,
)
def _load_recent_open_trade_dates(self, end_date: str, lookback: int) -> list[str]:
start_date = calendar_window_start(end_date, lookback)
calendar_rows = self._tushare_client().query(
"trade_cal",
{
"exchange": "SSE",
"start_date": start_date,
"end_date": end_date,
},
"cal_date,is_open,pretrade_date",
)
return select_open_trade_dates(calendar_rows, end_date, lookback)
def _execute_snapshot_backfill(
self,
*,
mode: str,
end_date: str,
lookback: int | None,
coverage: dict[str, Any],
skipped_non_trading_days: list[str],
dry_run: bool,
force: bool,
create_backup: bool,
) -> dict[str, Any]:
targets = list(coverage["trade_dates"] if force else coverage["missing"])
backup_path: str | None = None
if create_backup and not dry_run and targets:
backup = create_sqlite_backup(
Path(self.database.path),
DATA_DIR / "backups",
label=f"pre-{mode}-backfill",
)
backup_path = str(backup)
results: list[dict[str, Any]] = []
if dry_run:
for trade_date in coverage["trade_dates"]:
exists = trade_date in coverage["present"]
if exists and not force:
status = "skipped"
action = "exists"
else:
status = "planned"
action = "refresh" if exists else "create"
results.append(
{
"requested_date": display_date(trade_date),
"trade_date": display_date(trade_date),
"status": status,
"action": action,
}
)
return build_backfill_audit(
mode=mode,
end_date=end_date,
lookback=lookback,
coverage=coverage,
skipped_non_trading_days=skipped_non_trading_days,
backup_path=backup_path,
dry_run=True,
results=results,
)
present_before = set(coverage["present"])
for trade_date in targets:
existed = trade_date in present_before
try:
dashboard = self.sync_dashboard(trade_date)
actual = normalize_date(
str(dashboard.get("meta", {}).get("trade_date") or trade_date)
)
results.append(
{
"requested_date": display_date(trade_date),
"trade_date": display_date(actual),
"status": "success",
"action": "refreshed" if existed else "created",
"source": dashboard.get("meta", {}).get("source"),
"records": self._record_count(dashboard),
}
)
except Exception as exc:
results.append(
{
"requested_date": display_date(trade_date),
"trade_date": display_date(trade_date),
"status": "failed",
"action": "refresh" if existed else "create",
"error": str(exc),
}
)
for trade_date in coverage["present"]:
if force:
continue
results.append(
{
"requested_date": day.isoformat(),
"trade_date": dashboard["meta"]["trade_date"],
"source": dashboard["meta"]["source"],
"records": self._record_count(dashboard),
"requested_date": display_date(trade_date),
"trade_date": display_date(trade_date),
"status": "skipped",
"action": "exists",
}
)
return results
results.sort(key=lambda row: str(row.get("requested_date") or ""))
return build_backfill_audit(
mode=mode,
end_date=end_date,
lookback=lookback,
coverage=coverage,
skipped_non_trading_days=skipped_non_trading_days,
backup_path=backup_path,
dry_run=False,
results=results,
)
def _stock_identity(self, code: str, trade_date: str) -> tuple[str, str]:
snapshot = self.database.get_snapshot(trade_date) or {}
@@ -927,10 +1410,40 @@ class MarketServiceMixin:
return item["name"], item["sector"] or "其他"
return "--", "其他"
def _enrich_stock_detail(self, payload: dict[str, Any]) -> dict[str, Any]:
def _enrich_stock_detail(
self, payload: dict[str, Any], trade_date: str = ""
) -> dict[str, Any]:
result = dict(payload)
stock = dict(payload.get("stock") or {})
code = str(stock.get("code") or "")
compact_date = normalize_date(
str((payload.get("meta") or {}).get("trade_date") or trade_date)
)
board = self._limit_event_for_stock(code, compact_date)
if board:
if not stock.get("first_time") or stock.get("first_time") == "--":
stock["first_time"] = board.get("first_time") or "--"
if not stock.get("last_time") or stock.get("last_time") == "--":
stock["last_time"] = board.get("last_time") or "--"
if not stock.get("open_times"):
stock["open_times"] = board.get("open_times") or 0
if _optional_number(stock.get("seal_amount_million")) is None:
stock["seal_amount_million"] = board.get("seal_amount_million")
if not _optional_number(stock.get("turnover_rate")) and _optional_number(board.get("turnover_rate")):
stock["turnover_rate"] = board.get("turnover_rate")
flow = result.get("moneyflow") or {}
if not flow.get("available"):
live_flow = self._live_moneyflow_for_stock(code, compact_date)
if live_flow.get("available"):
result["moneyflow"] = live_flow
else:
result["moneyflow"] = {
"available": False,
"net_million": None,
"large_million": None,
"medium_million": None,
"small_million": None,
}
watched = {
item["code"]: item
for item in self.database.list_watchlist(self.current_user_id)
@@ -940,6 +1453,45 @@ class MarketServiceMixin:
result["notes"] = self.database.list_notes(self.current_user_id, code=code)
return result
def _limit_event_for_stock(self, code: str, trade_date: str) -> dict[str, Any]:
if not code or not trade_date:
return {}
ts_code = tushare_code(code)
client = self._tushare_client() if self.configured else None
rows: list[dict[str, Any]] = []
if client is not None:
try:
rows = client._load_limit_type(trade_date, "U") + client._load_limit_type(trade_date, "Z")
except Exception:
rows = []
if not rows:
try:
rows = list((client._free_board_map(trade_date) or {}).values())
except Exception:
rows = []
match = next((row for row in rows if str(row.get("ts_code") or "") == ts_code), None)
if not match:
return {}
fd = _optional_number(match.get("fd_amount"))
return {
"first_time": match.get("first_time") or "--",
"last_time": match.get("last_time") or "--",
"open_times": match.get("open_times") or 0,
"seal_amount_million": None if fd is None else round(fd / 10000, 0),
"turnover_rate": _optional_number(match.get("turnover_ratio")),
}
def _live_moneyflow_for_stock(self, code: str, trade_date: str) -> dict[str, Any]:
aggregator = getattr(self, "realtime_aggregator", None)
loader = getattr(aggregator, "eastmoney_stock_quote", None) if aggregator else None
if not callable(loader) or not code:
return _moneyflow_payload(None)
try:
quote = loader(tushare_code(code), expected_date=trade_date)
except Exception:
return _moneyflow_payload(None)
return _moneyflow_payload(quote)
def _with_storage(self, dashboard: dict[str, Any], cached: bool) -> dict[str, Any]:
result = dict(dashboard)
result["meta"] = {
@@ -947,7 +1499,7 @@ class MarketServiceMixin:
"storage": "sqlite",
"cached": cached,
}
return result
return self._annotate_data_status(result)
@staticmethod
def _record_count(dashboard: dict[str, Any]) -> int:
@@ -955,4 +1507,3 @@ class MarketServiceMixin:
len(dashboard.get(key) or [])
for key in ("limits", "broken", "down_limits", "yesterday_limits")
)
+3 -1
View File
@@ -26,13 +26,15 @@ class SystemHttpMixin:
def start_background_refresh(self) -> None:
try:
body = self.read_json_body(allow_empty=True)
started = self.application_service.request_background_sync(
refresh = self.application_service.request_background_sync(
str(body.get("trade_date") or date.today().isoformat())
)
started = bool(refresh.get("started"))
self.send_json(
{
"ok": True,
"started": started,
"job_key": str(refresh.get("job_key") or ""),
"message": "后台刷新已开始" if started else "已有后台刷新任务正在运行",
},
HTTPStatus.ACCEPTED,
+8 -2
View File
@@ -29,11 +29,17 @@ class SystemRoutesMixin:
def backfill_data(self) -> None:
try:
body = self.read_json_body()
results = self.application_service.backfill(
lookback_raw = body.get("lookback")
lookback = int(lookback_raw) if lookback_raw not in (None, "") else None
audit = self.application_service.backfill(
str(body.get("start_date") or ""),
str(body.get("end_date") or ""),
lookback=lookback,
dry_run=bool(body.get("dry_run")),
force=bool(body.get("force")),
create_backup=body.get("create_backup", True) is not False,
)
self.send_json({"ok": True, "results": results})
self.send_json({"ok": True, **audit, "results": audit.get("results") or []})
except ValueError as exc:
self.send_json({"error": str(exc)}, HTTPStatus.BAD_REQUEST)
except Exception as exc:
+42 -31
View File
@@ -4,7 +4,14 @@ import re
import secrets
from typing import Any
from backend.bootstrap.config import TOKEN_PATTERN, validate_text
from backend.bootstrap.config import validate_text
MARKET_SOURCE_SECRET_KEYS = {
"tushare_token",
"ifind_refresh_token",
"ifind_access_token",
}
class SystemServiceMixin:
@@ -18,9 +25,6 @@ class SystemServiceMixin:
first_encrypted = self.database.get_user_credentials(first_user_id)
first_personal = self.vault.decrypt_json(first_encrypted) if first_encrypted else {}
defaults = {
"tushare_token": environment.get("tushare_token") or first_personal.get("tushare_token") or "",
"ifind_refresh_token": environment.get("ifind_refresh_token") or "",
"ifind_access_token": environment.get("ifind_access_token") or "",
"platform_llm_primary_api_key": environment.get("platform_llm_primary_api_key") or first_personal.get("llm_primary_api_key") or "",
"platform_llm_primary_base_url": environment.get("platform_llm_primary_base_url") or first_personal.get("llm_primary_base_url") or "https://api.openai.com/v1",
"platform_llm_primary_model": environment.get("platform_llm_primary_model") or first_personal.get("llm_primary_model") or "",
@@ -34,6 +38,10 @@ class SystemServiceMixin:
if key not in current:
current[key] = value
changed = True
for key in MARKET_SOURCE_SECRET_KEYS:
if key in current:
current.pop(key, None)
changed = True
if not isinstance(current.get("llm_models"), list):
migrated_models: list[dict[str, str]] = []
for role, label in (("primary", "原主模型"), ("fallback", "原辅助模型")):
@@ -56,26 +64,27 @@ class SystemServiceMixin:
self.database.save_system_setting("credentials", self.vault.encrypt_json(current))
for row in self.database.list_user_credentials():
personal = self.vault.decrypt_json(str(row.get("encrypted_payload") or ""))
if "tushare_token" in personal:
personal.pop("tushare_token", None)
if any(key in personal for key in MARKET_SOURCE_SECRET_KEYS):
for key in MARKET_SOURCE_SECRET_KEYS:
personal.pop(key, None)
self.database.save_user_credentials(
int(row["user_id"]), self.vault.encrypt_json(personal)
)
return current
def _save_system_credentials(self, credentials: dict[str, Any]) -> None:
sanitized = {
key: value
for key, value in credentials.items()
if key not in MARKET_SOURCE_SECRET_KEYS
}
with self.system_lock:
self.database.save_system_setting("credentials", self.vault.encrypt_json(credentials))
self._system_credentials = dict(credentials)
if hasattr(self, "ifind"):
self.ifind.set_credentials(
str(credentials.get("ifind_refresh_token") or ""),
str(credentials.get("ifind_access_token") or ""),
)
self.database.save_system_setting("credentials", self.vault.encrypt_json(sanitized))
self._system_credentials = dict(sanitized)
@property
def configured(self) -> bool:
return bool(self.token)
return bool(self._datahub_status().get("configured"))
def _credentials(self) -> dict[str, str]:
credentials = getattr(self._request_context, "credentials", {})
@@ -99,7 +108,7 @@ class SystemServiceMixin:
@property
def token(self) -> str:
return str(self._system_credentials.get("tushare_token") or "")
return "datahub" if self.configured else ""
def system_status(self) -> dict[str, Any]:
platform = self._platform_llm_profile()
@@ -130,6 +139,7 @@ class SystemServiceMixin:
),
**self.database.status(),
"jobs": self.jobs.repository.recent(12),
"datahub": self._datahub_status(),
},
"llm": {
"primary_configured": self._profile_configured(platform["primary"]),
@@ -145,21 +155,24 @@ class SystemServiceMixin:
},
}
def _datahub_status(self) -> dict[str, Any]:
gateway = getattr(self, "data_gateway", None)
reporter = getattr(gateway, "datahub_status", None)
if callable(reporter):
return reporter()
return {
"configured": False,
"base_url": "",
"enabled_reads": 0,
"total_reads": 0,
"flags": [],
"routes": [],
"fallback_count": 0,
"fallback_labels": [],
}
def save_system_settings(self, payload: dict[str, Any]) -> dict[str, Any]:
current = dict(self._system_credentials)
token = str(payload.get("tushare_token") or current.get("tushare_token") or "").strip()
if token and not TOKEN_PATTERN.fullmatch(token):
raise ValueError("Tushare Token 格式不正确。")
ifind_refresh_token = str(
payload.get("ifind_refresh_token")
or current.get("ifind_refresh_token")
or ""
).strip()
if ifind_refresh_token and (
len(ifind_refresh_token) > 2048
or any(character.isspace() for character in ifind_refresh_token)
):
raise ValueError("iFinD Refresh Token 格式不正确。")
existing_models = {
str(item.get("id") or ""): item
for item in current.get("llm_models") or []
@@ -221,8 +234,6 @@ class SystemServiceMixin:
raise ValueError("会员每日额度应为 1 至 1000。") from exc
current.update(
{
"tushare_token": token,
"ifind_refresh_token": ifind_refresh_token,
"llm_models": models,
"primary_model_id": primary_model_id,
"fallback_model_id": fallback_model_id,
@@ -242,7 +253,7 @@ class SystemServiceMixin:
llm_access = self.llm_access_status()
return {
"configured": self.configured,
"mode": "tushare" if self.configured else "unavailable",
"mode": "datahub" if self.configured else "unavailable",
"llm_configured": self.llm_configured,
"llm_model": self.llm_primary_model if self.llm_configured else "",
"llm_fallback_configured": self.llm_fallback_configured,
+8 -1
View File
@@ -109,7 +109,14 @@ class HttpTransportMixin:
return {}
if length <= 0 or length > 65536:
raise ValueError("请求内容为空或过大。")
return json.loads(self.rfile.read(length).decode("utf-8"))
raw = self.rfile.read(length)
try:
payload = json.loads(raw.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError):
raise ValueError("请求不是合法 JSON。") from None
if not isinstance(payload, dict):
raise ValueError("请求不是合法 JSON。")
return payload
def serve_static(self, request_path: str) -> None:
relative = unquote(request_path).lstrip("/") or "index.html"
+48
View File
@@ -0,0 +1,48 @@
from __future__ import annotations
from datetime import datetime, time as dt_time
def dashboard_has_usable_data(dashboard: dict[str, object]) -> bool:
if not isinstance(dashboard, dict) or dashboard.get("status") == "failed":
return False
meta = dashboard.get("meta") or {}
overview = dashboard.get("overview") or {}
if isinstance(meta, dict) and (meta.get("trade_date") or meta.get("carried_forward")):
return True
return bool(isinstance(overview, dict) and overview)
def verified_dashboard_result(dashboard: dict[str, object]) -> dict[str, object]:
"""Manual refresh and automatic catch-up share this rule.
Derived limit lists or a previous usable snapshot are not whole-job failures.
Only a payload with no displayable market data is recorded as failed.
"""
if dashboard_has_usable_data(dashboard):
return dashboard
meta = dashboard.get("meta") if isinstance(dashboard, dict) else None
notice = ""
if isinstance(meta, dict):
notice = str(meta.get("notice") or meta.get("display_notice") or "")
return {
"status": "failed",
"error": notice or "未获取到可用行情",
}
def official_catchup_due(today: str, snapshot: dict[str, object]) -> bool:
now = datetime.now().astimezone().time().replace(tzinfo=None)
if not (dt_time(15, 5) <= now < dt_time(22, 0)):
return False
meta = snapshot.get("meta") if isinstance(snapshot.get("meta"), dict) else {}
actual = str(meta.get("trade_date") or "").replace("-", "")
if (
actual == today
and meta.get("limit_data_source") != "derived"
and not meta.get("carried_forward")
and not meta.get("realtime")
and meta.get("mode") != "realtime"
):
return False
return True
+14 -4
View File
@@ -5,6 +5,7 @@ import time
from datetime import date
from backend.bootstrap.config import normalize_date
from backend.jobs.refresh import official_catchup_due, verified_dashboard_result
class JobServiceMixin:
@@ -20,15 +21,16 @@ class JobServiceMixin:
workers_stopped = self.jobs.wait_for_idle(timeout_seconds)
return scheduler_stopped and workers_stopped
def request_background_sync(self, trade_date: str) -> bool:
def request_background_sync(self, trade_date: str) -> dict[str, object]:
normalized = normalize_date(trade_date)
key = f"manual:{normalized}:{time.time_ns()}"
return self.jobs.submit(
started = self.jobs.submit(
"market.refresh",
key,
lambda: self.sync_dashboard(normalized),
lambda: verified_dashboard_result(self.sync_dashboard(normalized)),
{"trade_date": normalized, "trigger": "administrator"},
)
return {"started": started, "job_key": key if started else ""}
def _background_refresh_tick(self) -> None:
if not (
@@ -43,7 +45,15 @@ class JobServiceMixin:
self.jobs.submit(
"market.refresh",
f"realtime:{today}:{bucket}",
lambda: self.sync_dashboard(today),
lambda: verified_dashboard_result(self.sync_dashboard(today)),
{"trade_date": today, "trigger": "realtime-poll"},
)
elif official_catchup_due(today, snapshot):
bucket = int(time.time() // 300)
self.jobs.submit(
"market.refresh",
f"catchup:{today}:{bucket}",
lambda: verified_dashboard_result(self.sync_dashboard(today)),
{"trade_date": today, "trigger": "official-catchup"},
)
self._schedule_automatic_screeners(today, snapshot)
+47
View File
@@ -0,0 +1,47 @@
# Optional overlay. Does not replace the existing xiaobai-review service.
# Start later (总工部署时) with:
# docker compose -f compose.yaml -f compose.datahub.yaml up -d
#
# Required .env keys: DATAHUB_ENCRYPTION_KEY, DATAHUB_TOKEN, DATAHUB_ADMIN_PASSWORD, TUSHARE_TOKEN
services:
xiaobai-datahub:
build:
context: ./xiaobai-datahub
dockerfile: Dockerfile
image: xiaobai-datahub:local
container_name: xiaobai-datahub
ports:
- "0.0.0.0:8766:8766/tcp"
env_file:
- ./xiaobai-datahub/.env
environment:
DATAHUB_ENCRYPTION_KEY: "${DATAHUB_ENCRYPTION_KEY:?DATAHUB_ENCRYPTION_KEY must be set}"
DATAHUB_TOKEN: "${DATAHUB_TOKEN:?DATAHUB_TOKEN must be set}"
DATAHUB_ADMIN_PASSWORD: "${DATAHUB_ADMIN_PASSWORD:?DATAHUB_ADMIN_PASSWORD must be set}"
TUSHARE_TOKEN: "${TUSHARE_TOKEN:-}"
IFIND_REFRESH_TOKEN: "${IFIND_REFRESH_TOKEN:-}"
IFIND_ACCESS_TOKEN: "${IFIND_ACCESS_TOKEN:-}"
DATAHUB_DB_PATH: /app/data/datahub.db
DATAHUB_BACKUP_DIR: /app/data/backups
TZ: Asia/Shanghai
PYTHONUTF8: "1"
volumes:
- type: bind
source: ./datahub-data
target: /app/data
restart: unless-stopped
init: true
read_only: true
tmpfs:
- /tmp:size=64m,mode=1777
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
stop_grace_period: 30s
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
+23 -1
View File
@@ -3,7 +3,9 @@ services:
build:
context: .
dockerfile: Dockerfile
image: xiaobai-review:latest
labels:
org.opencontainers.image.revision: "${XIAOBAI_GIT_REV:?必须先设置 XIAOBAI_GIT_REV=当前 main 完整提交号(或改用 tools/update_from_main.sh}"
image: xiaobai-review:main-${XIAOBAI_GIT_SHORT:?必须先设置 XIAOBAI_GIT_SHORT=7位提交短号(或改用 tools/update_from_main.sh}
container_name: xiaobai-review
ports:
- "0.0.0.0:8765:8765/tcp"
@@ -11,6 +13,26 @@ services:
- ./.env
environment:
APP_ENCRYPTION_KEY: "${APP_ENCRYPTION_KEY:?APP_ENCRYPTION_KEY must be set in .env}"
# Provider credentials are consumed only by xiaobai-datahub.
TUSHARE_TOKEN: ""
IFIND_REFRESH_TOKEN: ""
IFIND_ACCESS_TOKEN: ""
DATAHUB_BASE_URL: "${DATAHUB_BASE_URL:-http://192.168.200.11:8766}"
DATAHUB_READ_CALENDAR: "1"
DATAHUB_READ_STOCKS: "1"
DATAHUB_READ_DAILY: "1"
DATAHUB_READ_INDEX_DAILY: "1"
DATAHUB_READ_VALUATION: "1"
DATAHUB_READ_MONEYFLOW: "1"
DATAHUB_READ_AUCTION: "1"
DATAHUB_READ_LIMIT_EVENTS: "1"
DATAHUB_READ_POPULARITY: "1"
DATAHUB_READ_DRAGON_TIGER: "1"
DATAHUB_READ_SECTOR_DAILY: "1"
DATAHUB_READ_QUOTES: "1"
DATAHUB_READ_INDEX_QUOTES: "1"
DATAHUB_READ_INTRADAY: "1"
DATAHUB_READ_STATUS: "1"
TZ: Asia/Shanghai
PYTHONUTF8: "1"
volumes:
+6
View File
@@ -12,6 +12,12 @@ These registries describe the approved product surface of the standalone applica
providers, model entry points, CSS layers, and remaining code hotspots.
- `data-fields.config.json`: canonical data products, provider eligibility, intended use, and
known blocked datasets.
- `datahub.config.json`: official read-only client for `xiaobai-datahub`. Each dataset has its
own `read` / `shadow` flag; official reads default on. `compose.yaml` pins every
`DATAHUB_READ_*` to `"1"` so a leftover `.env` `=0` cannot silently keep official
pages on the old APIs. Environment variables can still override a single
`DATAHUB_SHADOW_*` without a master switch. The old website APIs stay as
emergency fallback only.
- `data-quality.config.json`: freshness, coverage, units, adjustment, point-in-time, and
fail-closed rules for every canonical data product.
- `jobs.config.json`: background schedules, dependencies, lock keys, retry policy, timeouts,
+129 -111
View File
@@ -204,25 +204,25 @@
"path": "backend/data/providers/tushare_client.py",
"runtime_role": "stable client facade for primary deterministic market data"
},
{
"provider": "datahub",
"path": "backend/data/datahub/client.py",
"runtime_role": "website-only read path; official EOD, live quotes, and licensed iFinD"
},
{
"provider": "ifind",
"path": "backend/data/providers/ifind_client.py",
"runtime_role": "realtime, charts, snapshots, enrichment"
"path": "xiaobai-datahub/datahub/adapters/ifind.py",
"runtime_role": "licensed iFinD source inside the data hub"
},
{
"provider": "eastmoney",
"path": "backend/features/market/charts.py",
"runtime_role": "display chart fallback"
},
{
"provider": "eastmoney",
"path": "backend/data/realtime.py",
"runtime_role": "isolated realtime observation"
"path": "xiaobai-datahub/datahub/adapters/eastmoney.py",
"runtime_role": "free realtime quotes and shenwan inside the data hub"
},
{
"provider": "tencent",
"path": "backend/data/realtime.py",
"runtime_role": "index observation fallback"
"path": "xiaobai-datahub/datahub/adapters/tencent.py",
"runtime_role": "free index and stock quotes inside the data hub"
}
],
"provider_domains": [
@@ -279,16 +279,28 @@
"compatibility_fallback": "backend/features/market/service.py"
},
{
"client": "IfindHttpClient",
"client": "DatahubClient",
"owner": "backend/data/gateway.py"
},
{
"client": "DatahubBridge",
"owner": "backend/data/gateway.py"
},
{
"client": "DatahubAwareTushareClient",
"owner": "backend/data/gateway.py"
},
{
"client": "HubIfindProxy",
"owner": "backend/data/gateway.py"
},
{
"client": "HubRealtimeProxy",
"owner": "backend/data/gateway.py"
},
{
"client": "MarketChartClient",
"owner": "backend/data/gateway.py"
},
{
"client": "WebRealtimeAggregator",
"owner": "backend/data/gateway.py"
}
],
"heaven_service_owners": {
@@ -313,6 +325,7 @@
"system_service": "backend/features/system/service.py",
"account_bridge": "backend/features/accounts/application.py",
"job_lifecycle": "backend/jobs/service.py",
"job_refresh_status": "backend/jobs/refresh.py",
"feature_routes": "backend/features/*/routes.py"
},
"numeric_normalization": [
@@ -374,11 +387,11 @@
}
],
"css_layers": [
"/shared/tokens.css?v=20260829-1",
"/shared/tokens.css?v=20260829-hel240",
"/shared/base.css?v=20260806-1",
"/shared/shell.css?v=20260820-8",
"/shared/auth.css?v=20260829-1",
"/shared/components/controls.css?v=20260820-2",
"/shared/shell.css?v=20260829-hel237",
"/shared/auth.css?v=20260829-hel240b",
"/shared/components/controls.css?v=20260829-hel237",
"/shared/components/navigation.css?v=20260820-1",
"/shared/components/cards.css?v=20260820-1",
"/shared/components/tables.css?v=20260820-1",
@@ -394,8 +407,8 @@
"/pages/popularity/foundation.css?v=20260820-1",
"/pages/dragon-tiger/foundation.css?v=20260820-1",
"/pages/screener/foundation.css?v=20260820-4",
"/pages/mentor/foundation.css?v=20260820-2",
"/pages/heaven/foundation.css?v=20260806-2",
"/pages/mentor/foundation.css?v=20260827-hel183",
"/pages/heaven/foundation.css?v=20260827-hel183",
"/pages/review/foundation.css?v=20260820-4"
],
"frontend_composition": {
@@ -440,8 +453,8 @@
"code_hotspots": [
{
"path": "frontend/pages/heaven/foundation.css",
"bytes": 185936,
"lines": 11734
"bytes": 182616,
"lines": 11494
},
{
"path": "frontend/pages/screener/foundation.css",
@@ -450,13 +463,13 @@
},
{
"path": "frontend/pages/heaven/page.js",
"bytes": 97189,
"lines": 2069
"bytes": 97770,
"lines": 2079
},
{
"path": "frontend/shared/shell.css",
"bytes": 63550,
"lines": 3757
"bytes": 63733,
"lines": 3767
},
{
"path": "backend/features/heaven/engine.py",
@@ -465,8 +478,13 @@
},
{
"path": "frontend/index.html",
"bytes": 48037,
"lines": 663
"bytes": 48403,
"lines": 665
},
{
"path": "backend/data/providers/tushare_industries.py",
"bytes": 37168,
"lines": 859
},
{
"path": "backend/features/screener/catalog.py",
@@ -478,6 +496,11 @@
"bytes": 35247,
"lines": 2416
},
{
"path": "backend/data/providers/tushare_dashboard.py",
"bytes": 33230,
"lines": 770
},
{
"path": "database.py",
"bytes": 32073,
@@ -488,16 +511,6 @@
"bytes": 31756,
"lines": 562
},
{
"path": "backend/data/providers/tushare_dashboard.py",
"bytes": 28051,
"lines": 644
},
{
"path": "backend/data/providers/tushare_industries.py",
"bytes": 26540,
"lines": 616
},
{
"path": "backend/features/heaven/manual.py",
"bytes": 24521,
@@ -505,8 +518,8 @@
},
{
"path": "frontend/pages/heaven/page.html",
"bytes": 19747,
"lines": 262
"bytes": 19885,
"lines": 269
},
{
"path": "frontend/pages/screener/page.html",
@@ -515,13 +528,13 @@
},
{
"path": "frontend/pages/market/preview.js",
"bytes": 18178,
"lines": 446
"bytes": 18230,
"lines": 447
},
{
"path": "backend/features/heaven/trend.py",
"bytes": 16772,
"lines": 370
"bytes": 17005,
"lines": 373
},
{
"path": "backend/features/market/insights_auction_scoring.py",
@@ -530,33 +543,38 @@
},
{
"path": "frontend/pages/market/charts.js",
"bytes": 15311,
"lines": 387
"bytes": 15743,
"lines": 401
},
{
"path": "frontend/shared/dashboard.js",
"bytes": 15063,
"lines": 321
},
{
"path": "frontend/pages/pools/page.html",
"bytes": 14942,
"lines": 235
},
{
"path": "frontend/shared/admin.js",
"bytes": 14836,
"lines": 283
},
{
"path": "backend/features/screener/data_sync.py",
"bytes": 14743,
"lines": 342
},
{
"path": "frontend/shared/admin.js",
"bytes": 14145,
"lines": 261
},
{
"path": "backend/features/heaven/market_context.py",
"bytes": 13681,
"lines": 338
"bytes": 14409,
"lines": 354
},
{
"path": "frontend/shared/session.js",
"bytes": 12848,
"lines": 283
"bytes": 13219,
"lines": 289
},
{
"path": "backend/features/market/insights_auction_data.py",
@@ -565,8 +583,8 @@
},
{
"path": "backend/features/system/service.py",
"bytes": 12392,
"lines": 254
"bytes": 12180,
"lines": 265
},
{
"path": "backend/features/market/insights_auction.py",
@@ -578,11 +596,6 @@
"bytes": 10539,
"lines": 244
},
{
"path": "frontend/shared/dashboard.js",
"bytes": 9993,
"lines": 220
},
{
"path": "backend/data/providers/tushare_sectors.py",
"bytes": 9876,
@@ -595,9 +608,14 @@
},
{
"path": "frontend/pages/market/entity-detail.js",
"bytes": 9119,
"bytes": 9139,
"lines": 199
},
{
"path": "backend/data/providers/tushare_daily.py",
"bytes": 9076,
"lines": 232
},
{
"path": "backend/data/providers/tushare_dragon_tiger.py",
"bytes": 9059,
@@ -608,6 +626,11 @@
"bytes": 8562,
"lines": 238
},
{
"path": "backend/data/providers/tushare_indices.py",
"bytes": 8447,
"lines": 189
},
{
"path": "frontend/pages/mentor/page.html",
"bytes": 8357,
@@ -618,16 +641,6 @@
"bytes": 6983,
"lines": 146
},
{
"path": "backend/data/providers/tushare_daily.py",
"bytes": 6837,
"lines": 160
},
{
"path": "backend/application.py",
"bytes": 6751,
"lines": 178
},
{
"path": "backend/features/market/insights_popularity.py",
"bytes": 6739,
@@ -644,8 +657,13 @@
"lines": 81
},
{
"path": "backend/data/providers/tushare_stocks.py",
"bytes": 6244,
"path": "backend/application.py",
"bytes": 6399,
"lines": 161
},
{
"path": "frontend/pages/market/stock-detail.js",
"bytes": 6325,
"lines": 137
},
{
@@ -664,25 +682,20 @@
"lines": 85
},
{
"path": "frontend/pages/market/stock-detail.js",
"bytes": 5690,
"lines": 124
"path": "backend/data/providers/tushare_stocks.py",
"bytes": 5592,
"lines": 123
},
{
"path": "backend/data/providers/tushare_indices.py",
"bytes": 5451,
"lines": 118
"path": "frontend/pages.config.js",
"bytes": 5385,
"lines": 130
},
{
"path": "frontend/pages/market/search.js",
"bytes": 5384,
"lines": 131
},
{
"path": "frontend/pages.config.js",
"bytes": 5380,
"lines": 130
},
{
"path": "frontend/pages/auction/page.html",
"bytes": 5350,
@@ -703,6 +716,11 @@
"bytes": 4712,
"lines": 106
},
{
"path": "backend/data/providers/tushare_helpers.py",
"bytes": 4406,
"lines": 124
},
{
"path": "backend/features/market/routes.py",
"bytes": 4276,
@@ -768,31 +786,31 @@
"bytes": 2514,
"lines": 63
},
{
"path": "backend/jobs/service.py",
"bytes": 2337,
"lines": 59
},
{
"path": "backend/features/mentor/routes.py",
"bytes": 2299,
"lines": 57
},
{
"path": "backend/data/providers/tushare_client.py",
"bytes": 2263,
"lines": 70
},
{
"path": "backend/features/screener/regime.py",
"bytes": 2202,
"lines": 53
},
{
"path": "backend/data/providers/tushare_client.py",
"bytes": 2166,
"lines": 68
},
{
"path": "frontend/pages/popularity/page.html",
"bytes": 2165,
"lines": 35
},
{
"path": "backend/data/providers/tushare_helpers.py",
"bytes": 2083,
"lines": 64
},
{
"path": "frontend/pages/market/breadth.js",
"bytes": 2071,
@@ -804,9 +822,14 @@
"lines": 45
},
{
"path": "backend/jobs/service.py",
"bytes": 1746,
"lines": 49
"path": "backend/jobs/refresh.py",
"bytes": 1808,
"lines": 48
},
{
"path": "backend/features/system/routes.py",
"bytes": 1791,
"lines": 46
},
{
"path": "backend/features/alerts/routes.py",
@@ -818,6 +841,11 @@
"bytes": 1642,
"lines": 53
},
{
"path": "backend/data/providers/tushare_transport.py",
"bytes": 1592,
"lines": 50
},
{
"path": "backend/features/market/insights.py",
"bytes": 1580,
@@ -828,16 +856,6 @@
"bytes": 1535,
"lines": 39
},
{
"path": "backend/data/providers/tushare_transport.py",
"bytes": 1455,
"lines": 48
},
{
"path": "backend/features/system/routes.py",
"bytes": 1423,
"lines": 40
},
{
"path": "backend/features/themes/routes.py",
"bytes": 1337,
+1
View File
@@ -2,6 +2,7 @@
"schema_version": 1,
"providers": {
"tushare": {"class": "licensed", "calculation_allowed": true},
"datahub": {"class": "licensed", "calculation_allowed": true},
"ifind": {"class": "licensed", "calculation_allowed": true},
"eastmoney": {"class": "public_web", "calculation_allowed": false},
"tencent": {"class": "public_web", "calculation_allowed": false},
+25
View File
@@ -0,0 +1,25 @@
{
"schema_version": 1,
"base_url": "http://127.0.0.1:8766",
"timeout_seconds": 8,
"retries": 1,
"page_limit": 5000,
"stale_seconds_max": 86400,
"datasets": {
"calendar": { "read": true, "shadow": false },
"stocks": { "read": true, "shadow": false },
"daily": { "read": true, "shadow": false },
"index_daily": { "read": true, "shadow": false },
"valuation": { "read": true, "shadow": false },
"moneyflow": { "read": true, "shadow": false },
"auction": { "read": true, "shadow": false },
"limit_events": { "read": true, "shadow": false },
"popularity": { "read": true, "shadow": false },
"dragon_tiger": { "read": true, "shadow": false },
"sector_daily": { "read": true, "shadow": false },
"quotes": { "read": true, "shadow": false },
"index_quotes": { "read": true, "shadow": false },
"intraday": { "read": true, "shadow": false },
"status": { "read": true, "shadow": false }
}
}
+1 -1
View File
@@ -32,4 +32,4 @@
- 旧文档不能删:被替代的旧文档开头要加一行「⚠️ 本文档已过时,仅留档备查,请勿删除」,再写新版。
- 用中文大白话写,专业词要带通俗解释,让不懂代码的人也能看懂。
- 「问天」板块是冻结区,任何改动都不许碰;写文档时别误导后来人去改它
- 「问天」不是永久冻结区:此前只冻结过界面视觉方案,现已解冻。问天可纳入后续数据与功能迁移,不要再写成“永远不碰”
+2 -2
View File
@@ -213,12 +213,12 @@
{
"provider": "eastmoney",
"path": "realtime_aggregator.py",
"runtime_role": "isolated realtime observation"
"runtime_role": "isolated realtime observation and intraday dashboard fallback"
},
{
"provider": "tencent",
"path": "realtime_aggregator.py",
"runtime_role": "index observation fallback"
"runtime_role": "index observation and intraday quote fallback"
}
],
"llm_entrypoints": [
+69
View File
@@ -0,0 +1,69 @@
# 行情历史补档(最近 60 个交易日)
用于修复 `dashboard_snapshots` 断档导致情绪周期 / 主题轮动 / 智能选股只剩当天的问题。
保留 `latest_contiguous_history` 连续性规则;通过真实交易日历回补缺失交易日快照。
## 适用场景
- 库中已有稀疏历史快照,但最近一个真实交易日缺失,接口 `available_days=1`
- 需要可重复执行、可审计、可回退的补档,而不是迁库或放宽算法。
## 前置
1. 使用与线上一致的代码分支。
2. 管理员账号已配置可用的公共 Tushare Token。
3. 只操作目标环境自己的 `data/review.db`;禁止 `.36``.11` 互拷。
## 上线步骤(总工执行)
在目标环境容器内执行(应用根目录;宿主机也可直接跑,脚本已自带仓库根 `sys.path` 引导):
```bash
# 1) 只读规划:区分已有、真正缺档;不会写入
docker compose exec xiaobai-review python tools/backfill_recent_snapshots.py --account <管理员账号> --lookback 60 --dry-run --json
# 2) 正式补档:先走 SQLite backup API 写 data/backups/review-pre-recent-backfill-*.db
# 再对缺失交易日调用现有 sync_dashboard
docker compose exec xiaobai-review python tools/backfill_recent_snapshots.py --account <管理员账号> --lookback 60 --json
# 3) 验证
# GET /api/sentiment/history?trade_date=YYYY-MM-DD&limit=60
# 期望 available_days >= 20,且不再只有 1 天
```
管理端日期区间回补(`/api/backfill`)已改为只处理交易日历中的开市日,周末/节假日会进入
`skipped_non_trading_days`,不再当成错误;单次仍限制 15 个交易日。最近 60 日请用本工具。
## 写入边界
只会通过现有同步路径写入:
- `dashboard_snapshots`
- 同步审计表 `sync_runs`
- 必要时的 `data_snapshots`(仅当请求日被解析到其他交易日)
不得改动用户、Token、模型绑定或系统配置表。
## 回滚
1. 优先按审计结果的 `created_dates` 精确删除新增行:
```sql
DELETE FROM dashboard_snapshots WHERE trade_date IN ('YYYYMMDD', ...);
```
2. 若需整库回退,停止写入后用补档前备份覆盖:
```bash
# 示例:把 data/backups/review-pre-recent-backfill-YYYYMMDD-HHMMSS.db
# 复制回 data/review.db 后重启容器
```
3. 代码回退:对该提交执行 Git revert 后重新部署镜像。
## 验收要点
- dry-run 与正式执行可重复跑;已有交易日默认跳过。
- 周末、节假日出现在 `skipped_non_trading_days`,不计入失败。
- 部分交易日同步失败时,其他日期仍会继续,并在审计结果中标 `failed`
- 情绪周期、主题轮动 9 列、智能选股置信度随连续交易日恢复。
@@ -320,6 +320,11 @@ PC端统一采用以下固定骨架:
### 6.1 数据源职责
运行边界:下表中的职责全部由独立的 `xiaobai-datahub` 数据中枢执行。主网站只按固定业务
协议请求“行情、日K、分时、申万、竞价”等数据,不接触任何提供方参数或凭据,也不决定优先级、
重试、降级和回填。数据中枢是主网站唯一的行情出口;中枢暂时取不到新数据时,网站只能读取
已经归档的真实快照,不能绕回旧提供方直连接口。
| 数据源 | 可用于正式计算 | 主要职责 |
|---|:---:|---|
| Tushare | 是 | 交易日历、股票主表、日线、估值、财务、资金流、申万行业、涨跌停、9:25竞价、热榜、龙虎榜 |
+1
View File
@@ -7,6 +7,7 @@
| 任务 | 说明 | 状态 |
|---|---|---|
| 全站视觉统一改造收尾 | 主线。17 个阶段已完成,正在最终验收、代码合并 | 收尾中 |
| 行情刷新误报与旧数据提示 | HEL-412:高级接口未到齐不再记整次失败;今日正式数据晚到时提示当前展示日期 | 施工中 |
| 手机端独立重新设计 | 先出视觉/交互规范和技术架构方案,等老板确认后再施工 | 方案送审中 |
## 已做完
+2 -2
View File
@@ -29,11 +29,11 @@
- **智能工具类(3 个)**:智能选股、问师、问天。
- **个人类(1 个)**:我的复盘。
其中「问天」是冻结区(见下面的硬规矩)
其中「问天」此前只在全站视觉改造阶段冻结过界面方案,现已解冻;问天可以纳入后续数据与功能迁移,但不等于本阶段要重做视觉
## 几条硬规矩(不能破坏的边界)
- 「问天」板块**冻结区**,任何改动都不许碰它
- 「问天」板块**不是永久冻结区**:此前冻结的是界面视觉方案,现已解冻。问天现有功能与界面不要破坏;后续数据与功能迁移可以纳入,不主动重做视觉
- **不用假数据冒充真行情**;数据缺失就明说“没有/不可用”,不能编。
- **每个用户自己的数据互相隔离**(自选、复盘、对话、问天历史等),看不到别人的。
- **计算由程序确定性完成**(情绪周期、智能选股、问天排盘等),AI 大模型(LLM,就是会聊天的那个 AI)只负责解释或编译自然语言条件,不能改计算结果。
+11 -9
View File
@@ -34,11 +34,11 @@
document.documentElement.style.colorScheme = theme;
})();
</script>
<link rel="stylesheet" href="/shared/tokens.css?v=20260829-1">
<link rel="stylesheet" href="/shared/tokens.css?v=20260829-hel240">
<link rel="stylesheet" href="/shared/base.css?v=20260806-1">
<link rel="stylesheet" href="/shared/shell.css?v=20260820-8">
<link rel="stylesheet" href="/shared/auth.css?v=20260829-1">
<link rel="stylesheet" href="/shared/components/controls.css?v=20260820-2">
<link rel="stylesheet" href="/shared/shell.css?v=20260829-hel237">
<link rel="stylesheet" href="/shared/auth.css?v=20260829-hel240b">
<link rel="stylesheet" href="/shared/components/controls.css?v=20260829-hel237">
<link rel="stylesheet" href="/shared/components/navigation.css?v=20260820-1">
<link rel="stylesheet" href="/shared/components/cards.css?v=20260820-1">
<link rel="stylesheet" href="/shared/components/tables.css?v=20260820-1">
@@ -54,8 +54,8 @@
<link rel="stylesheet" href="/pages/popularity/foundation.css?v=20260820-1">
<link rel="stylesheet" href="/pages/dragon-tiger/foundation.css?v=20260820-1">
<link rel="stylesheet" href="/pages/screener/foundation.css?v=20260820-4">
<link rel="stylesheet" href="/pages/mentor/foundation.css?v=20260820-2">
<link rel="stylesheet" href="/pages/heaven/foundation.css?v=20260806-2">
<link rel="stylesheet" href="/pages/mentor/foundation.css?v=20260827-hel183">
<link rel="stylesheet" href="/pages/heaven/foundation.css?v=20260827-hel183">
<link rel="stylesheet" href="/pages/review/foundation.css?v=20260820-4">
</head>
<body>
@@ -607,11 +607,13 @@
<div class="admin-panel" data-admin-panel="market">
<form id="systemMarketForm" class="settings-section">
<div class="settings-section-heading"><h3>公共行情</h3><span id="systemDataStatus">待检查</span></div>
<label class="form-field"><span>Tushare Token</span><input id="systemTokenInput" type="password" autocomplete="off" minlength="20" placeholder="留空保留现有 Token"></label>
<label class="form-field"><span>iFinD Refresh Token</span><input id="systemIfindTokenInput" type="password" autocomplete="off" maxlength="2048" placeholder="留空保留现有 Token"></label>
<label class="form-field"><span>行情来源凭据</span><input id="systemTokenInput" type="text" value="请在数据中枢后台统一管理" disabled></label>
<label class="form-field"><span>实时来源凭据</span><input id="systemIfindTokenInput" type="text" value="请在数据中枢后台统一管理" disabled></label>
<label class="switch-control"><input id="systemBackgroundRefresh" type="checkbox"><span>启用交易时段后台刷新</span></label>
<p class="form-hint">所有用户读取同一份后台快照,页面不会随后台任务自动重绘。</p>
<div class="dialog-actions admin-inline-actions"><button id="adminRefreshButton" class="button" type="button"><i data-lucide="refresh-cw"></i>立即后台刷新</button><button class="button primary" type="submit">保存行情配置</button></div>
<div id="datahubRouteStatus" class="admin-refresh-status" data-tone="idle" role="status" aria-live="polite"><i data-lucide="database"></i><span>数据中枢线路待检查</span></div>
<div id="adminRefreshStatus" class="admin-refresh-status" data-tone="idle" role="status" aria-live="polite"><i data-lucide="circle-dot"></i><span>尚未手动刷新</span></div>
<div class="dialog-actions admin-inline-actions"><button id="adminRefreshButton" class="button" type="button"><i data-lucide="refresh-cw"></i>立即后台刷新</button><button class="button primary" type="submit">保存刷新设置</button></div>
</form>
<section class="settings-section">
<div class="settings-section-heading"><h3>历史数据回补</h3><span>管理员任务</span></div>
+127 -24
View File
@@ -19,41 +19,144 @@
document.documentElement.style.colorScheme = theme;
})();
</script>
<link rel="stylesheet" href="/shared/tokens.css?v=20260829-1">
<link rel="stylesheet" href="/shared/tokens.css?v=20260829-hel251">
<link rel="stylesheet" href="/shared/base.css?v=20260806-1">
<link rel="stylesheet" href="/shared/auth.css?v=20260829-1">
<link rel="stylesheet" href="/shared/auth.css?v=20260829-hel251">
<link rel="stylesheet" href="/shared/components/controls.css?v=20260820-2">
</head>
<body class="login-portal">
<button id="loginThemeToggle" class="login-theme-toggle" type="button">🌙 夜间</button>
<aside class="login-brand" aria-hidden="true">
<div class="login-brand-mark"><span class="login-brand-glyph"></span></div>
<p class="login-brand-kicker">收盘之后 · 复盘开始</p>
<h1 class="login-brand-title">小白复盘</h1>
<p class="login-brand-lead">看懂情绪周期,把复盘变成下一次的先手。</p>
<dl class="login-brand-stats">
<div class="login-stat">
<dt>市场情绪</dt>
<dd>72 <span class="login-stat-tag">高热</span></dd>
<div class="login-brand-header">
<div class="login-brand-mark"><span class="login-brand-glyph"></span></div>
<div class="login-brand-identity">
<p class="login-brand-name">小白复盘</p>
<p class="login-brand-subtitle">A股个人复盘工作台</p>
</div>
<div class="login-stat">
<dt>涨停</dt>
<dd>63</dd>
</div>
<div class="login-mascots" id="loginMascots" aria-hidden="true" data-mood="idle">
<div class="login-mascots-stage">
<div class="login-mascot is-red">
<div class="login-mascot-breathe">
<div class="login-mascot-lean">
<svg class="login-mascot-figure" viewBox="0 0 96 210" focusable="false">
<ellipse class="login-mascot-shadow" cx="48" cy="200" rx="26" ry="5.5"></ellipse>
<line class="login-mascot-wick" x1="48" y1="10" x2="48" y2="50"></line>
<rect class="login-mascot-body" x="16" y="50" width="64" height="126" rx="14"></rect>
<line class="login-mascot-wick" x1="48" y1="176" x2="48" y2="190"></line>
<g class="login-mascot-face">
<circle class="login-mascot-eye" cx="36" cy="94" r="10"></circle>
<circle class="login-mascot-eye" cx="60" cy="94" r="10"></circle>
<circle class="login-mascot-pupil" cx="36" cy="94" r="4.2"></circle>
<circle class="login-mascot-pupil" cx="60" cy="94" r="4.2"></circle>
<rect class="login-mascot-lid" x="24" y="82" width="48" height="24" rx="10"></rect>
<path class="login-mascot-happy" d="M28 96 Q36 88 44 96"></path>
<path class="login-mascot-happy" d="M52 96 Q60 88 68 96"></path>
<circle class="login-mascot-sad" cx="36" cy="96" r="2.2"></circle>
<circle class="login-mascot-sad" cx="60" cy="96" r="2.2"></circle>
</g>
<ellipse class="login-mascot-arm is-left" cx="16" cy="128" rx="8" ry="5.5"></ellipse>
<ellipse class="login-mascot-arm is-right" cx="80" cy="128" rx="8" ry="5.5"></ellipse>
<ellipse class="login-mascot-hand is-left" cx="16" cy="128" rx="9" ry="7"></ellipse>
<ellipse class="login-mascot-hand is-right" cx="80" cy="128" rx="9" ry="7"></ellipse>
</svg>
</div>
</div>
</div>
<div class="login-mascot is-green">
<div class="login-mascot-breathe">
<div class="login-mascot-lean">
<div class="login-mascot-figure">
<svg class="login-mascot-front" viewBox="0 0 84 175" focusable="false">
<ellipse class="login-mascot-shadow" cx="42" cy="166" rx="22" ry="4.5"></ellipse>
<line class="login-mascot-wick" x1="42" y1="8" x2="42" y2="42"></line>
<rect class="login-mascot-body" x="15" y="42" width="54" height="104" rx="12"></rect>
<line class="login-mascot-wick" x1="42" y1="146" x2="42" y2="158"></line>
<g class="login-mascot-face">
<circle class="login-mascot-eye" cx="32" cy="76" r="8.5"></circle>
<circle class="login-mascot-eye" cx="52" cy="76" r="8.5"></circle>
<circle class="login-mascot-pupil" cx="32" cy="76" r="3.6"></circle>
<circle class="login-mascot-pupil" cx="52" cy="76" r="3.6"></circle>
<rect class="login-mascot-lid" x="22" y="66" width="40" height="20" rx="8"></rect>
<path class="login-mascot-happy" d="M25 78 Q32 71 39 78"></path>
<path class="login-mascot-happy" d="M45 78 Q52 71 59 78"></path>
<circle class="login-mascot-sad" cx="32" cy="78" r="1.9"></circle>
<circle class="login-mascot-sad" cx="52" cy="78" r="1.9"></circle>
</g>
<ellipse class="login-mascot-arm is-left" cx="15" cy="108" rx="7" ry="4.5"></ellipse>
<ellipse class="login-mascot-arm is-right" cx="69" cy="108" rx="7" ry="4.5"></ellipse>
</svg>
<svg class="login-mascot-back" viewBox="0 0 84 175" focusable="false">
<ellipse class="login-mascot-shadow" cx="42" cy="166" rx="22" ry="4.5"></ellipse>
<line class="login-mascot-wick" x1="42" y1="8" x2="42" y2="42"></line>
<rect class="login-mascot-body" x="15" y="42" width="54" height="104" rx="12"></rect>
<line class="login-mascot-wick" x1="42" y1="146" x2="42" y2="158"></line>
<rect class="login-mascot-slit" x="40.5" y="58" width="3" height="72" rx="1.5"></rect>
<ellipse class="login-mascot-arm is-left" cx="15" cy="108" rx="7" ry="4.5"></ellipse>
<ellipse class="login-mascot-arm is-right" cx="69" cy="108" rx="7" ry="4.5"></ellipse>
</svg>
</div>
</div>
</div>
</div>
</div>
<div class="login-stat">
<dt>跌停</dt>
<dd>4</dd>
</div>
<div class="login-stat">
<dt>两市成交</dt>
<dd>1.02万亿</dd>
</div>
</dl>
</div>
<div class="login-brand-copy">
<p class="login-brand-kicker">收盘之后 · 复盘开始</p>
<h1 class="login-brand-title">看懂情绪周期,把复盘变成下一次的先手。</h1>
<p class="login-brand-lead">情绪周期、涨停梯队、主题轮动、竞价、龙虎榜、人气榜、交易复盘,集中在一个安静的复盘空间。</p>
</div>
<div class="login-brand-market">
<svg class="login-brand-chart" viewBox="0 0 480 168" focusable="false">
<defs>
<linearGradient id="loginChartFade" x1="0" x2="0" y1="0" y2="1">
<stop offset="0%" stop-color="#d7e4ff" stop-opacity="0.18"></stop>
<stop offset="100%" stop-color="#d7e4ff" stop-opacity="0"></stop>
</linearGradient>
</defs>
<path class="login-chart-area" d="M8 118 C 52 108, 78 96, 112 102 S 168 128, 204 112 S 268 78, 312 86 S 372 118, 428 92 L 472 84 L 472 168 L 8 168 Z"></path>
<g class="login-candles">
<g class="is-up" transform="translate(36 0)"><line x1="8" y1="58" x2="8" y2="128"></line><rect x="2" y="72" width="12" height="40"></rect></g>
<g class="is-down" transform="translate(68 0)"><line x1="8" y1="64" x2="8" y2="132"></line><rect x="2" y="86" width="12" height="28"></rect></g>
<g class="is-up" transform="translate(100 0)"><line x1="8" y1="48" x2="8" y2="118"></line><rect x="2" y="60" width="12" height="44"></rect></g>
<g class="is-down" transform="translate(132 0)"><line x1="8" y1="70" x2="8" y2="136"></line><rect x="2" y="92" width="12" height="26"></rect></g>
<g class="is-up" transform="translate(164 0)"><line x1="8" y1="42" x2="8" y2="110"></line><rect x="2" y="54" width="12" height="38"></rect></g>
<g class="is-up" transform="translate(196 0)"><line x1="8" y1="36" x2="8" y2="98"></line><rect x="2" y="48" width="12" height="32"></rect></g>
<g class="is-down" transform="translate(228 0)"><line x1="8" y1="58" x2="8" y2="128"></line><rect x="2" y="78" width="12" height="36"></rect></g>
<g class="is-up" transform="translate(260 0)"><line x1="8" y1="40" x2="8" y2="104"></line><rect x="2" y="52" width="12" height="36"></rect></g>
<g class="is-down" transform="translate(292 0)"><line x1="8" y1="66" x2="8" y2="134"></line><rect x="2" y="88" width="12" height="30"></rect></g>
<g class="is-up" transform="translate(324 0)"><line x1="8" y1="44" x2="8" y2="112"></line><rect x="2" y="58" width="12" height="40"></rect></g>
<g class="is-down" transform="translate(356 0)"><line x1="8" y1="72" x2="8" y2="138"></line><rect x="2" y="96" width="12" height="24"></rect></g>
<g class="is-up" transform="translate(388 0)"><line x1="8" y1="38" x2="8" y2="108"></line><rect x="2" y="50" width="12" height="42"></rect></g>
<g class="is-up" transform="translate(420 0)"><line x1="8" y1="32" x2="8" y2="96"></line><rect x="2" y="44" width="12" height="34"></rect></g>
</g>
<path class="login-chart-line" d="M8 118 C 52 108, 78 96, 112 102 S 168 128, 204 112 S 268 78, 312 86 S 372 118, 428 92 L 472 84"></path>
</svg>
<dl class="login-brand-stats">
<div class="login-stat">
<dt>市场情绪</dt>
<dd>72 <span class="login-stat-tag">高热</span></dd>
</div>
<div class="login-stat">
<dt>涨停</dt>
<dd>63</dd>
</div>
<div class="login-stat">
<dt>跌停</dt>
<dd>4</dd>
</div>
<div class="login-stat login-stat-wide">
<dt>两市成交</dt>
<dd>1.02万亿</dd>
</div>
</dl>
</div>
<p class="login-brand-disclaimer">股市有风险,投资需谨慎 · 本工具仅供个人复盘学习使用</p>
</aside>
<main class="login-stage">
<button id="loginThemeToggle" class="login-theme-toggle" type="button">🌙 夜间</button>
<section class="login-card" id="loginCard" aria-live="polite"></section>
</main>
<script src="/shared/api.js?v=20260803-2"></script>
<script src="/login/page.js?v=20260829-1"></script>
<script src="/login/page.js?v=20260829-hel251"></script>
</body>
</html>
+360 -23
View File
@@ -13,6 +13,9 @@
loading: false,
confirmingId: null,
error: "",
username: "",
password: "",
passwordVisible: false,
};
function escapeHtml(value) {
@@ -52,48 +55,90 @@
state.error = message || "";
}
function membershipLabel(account) {
if (account.role === "admin") return account.membership?.subscribed ? "管理员 · 会员" : "管理员";
return account.membership?.subscribed ? "会员" : "普通用户";
function formatLastUsed(value) {
if (!value) return "";
const parsed = new Date(value);
if (Number.isNaN(parsed.getTime())) return "";
const now = new Date();
const hh = String(parsed.getHours()).padStart(2, "0");
const mm = String(parsed.getMinutes()).padStart(2, "0");
if (parsed.toDateString() === now.toDateString()) return `今天 ${hh}:${mm}`;
const yesterday = new Date(now);
yesterday.setDate(now.getDate() - 1);
if (parsed.toDateString() === yesterday.toDateString()) return `昨天 ${hh}:${mm}`;
return `${parsed.getMonth() + 1}${parsed.getDate()}`;
}
function chipsFor(account, current) {
const chips = [];
if (current) chips.push('<span class="login-chip login-chip-current">当前</span>');
if (account.role === "admin") chips.push('<span class="login-chip">管理员</span>');
if (account.membership?.subscribed) chips.push('<span class="login-chip login-chip-member">会员</span>');
else if (account.role !== "admin") chips.push('<span class="login-chip">普通用户</span>');
return chips.join("");
}
function returnPath() {
const raw = new URLSearchParams(global.location.search).get("next") || "";
if (!raw) return "/";
try {
const url = new URL(raw, global.location.origin);
if (url.origin !== global.location.origin) return "/";
const path = url.pathname || "/";
if (path === "/login" || path.startsWith("/login/")) return "/";
return `${path}${url.search}${url.hash}` || "/";
} catch (_error) {
return "/";
}
}
function enterApp() {
const next = new URLSearchParams(global.location.search).get("next");
global.location.replace(next && next.startsWith("/") ? next : "/");
global.location.replace(returnPath());
}
function formMarkup(options) {
const registering = state.mode === "register";
const submitLabel = options.submitLabel
|| (state.loading ? "正在登录..." : registering ? "注册并进入" : options.add ? "添加并进入" : "登录");
const lead = options.lead;
const hint = options.hint;
const invalid = state.error ? " is-invalid" : "";
const passwordType = state.passwordVisible ? "text" : "password";
const passwordToggle = state.passwordVisible ? "隐藏" : "显示";
return [
options.back
? '<button class="login-back" type="button" data-login-action="picker">返回账号列表</button>'
: "",
`<h2 class="login-card-title">${escapeHtml(options.title)}</h2>`,
`<p class="login-card-lead">${escapeHtml(options.lead)}</p>`,
`<p class="login-card-lead">${escapeHtml(lead)}</p>`,
'<div class="login-tabs" role="tablist">',
`<button class="login-tab${state.mode === "login" ? " is-active" : ""}" type="button" data-auth-mode="login">登录</button>`,
`<button class="login-tab${state.mode === "register" ? " is-active" : ""}" type="button" data-auth-mode="register">注册</button>`,
"</div>",
'<form class="login-form" id="loginForm">',
'<label class="form-field"><span>账号名</span><input id="loginUsername" type="text" minlength="3" maxlength="30" autocomplete="username" required></label>',
`<label class="form-field"><span>密码</span><input id="loginPassword" type="password" minlength="8" maxlength="128" autocomplete="${registering ? "new-password" : "current-password"}" required></label>`,
`<label class="form-field"><span>账号名</span><input id="loginUsername" type="text" minlength="3" maxlength="30" autocomplete="username" placeholder="请输入账号名" value="${escapeHtml(state.username)}" required></label>`,
`<label class="form-field"><span>密码</span><span class="login-password-wrap"><input id="loginPassword" class="${invalid.trim()}" type="${passwordType}" minlength="8" maxlength="128" autocomplete="${registering ? "new-password" : "current-password"}" placeholder="请输入密码" value="${escapeHtml(state.password)}" required><button class="login-password-toggle" type="button" data-login-action="toggle-password" aria-pressed="${state.passwordVisible ? "true" : "false"}" aria-label="${state.passwordVisible ? "隐藏密码" : "显示密码"}">${passwordToggle}</button></span></label>`,
`<label class="form-field" id="loginConfirmField"${registering ? "" : " hidden"}><span>确认密码</span><input id="loginPasswordConfirm" type="password" minlength="8" maxlength="128" autocomplete="new-password"${registering ? " required" : ""}></label>`,
state.error ? `<p class="login-error">${escapeHtml(state.error)}</p>` : '<p class="login-error" hidden></p>',
`<button class="button primary login-submit" type="submit"${state.loading ? " disabled" : ""}>`,
state.loading ? '<span class="login-spinner" aria-hidden="true"></span>' : "",
`<span>${escapeHtml(submitLabel)}</span></button>`,
"</form>",
'<p class="login-hint">密码连续输错 5 次将锁定 10 分钟。还没有账号?切换到「注册」创建。</p>',
`<p class="login-hint">${escapeHtml(hint)}</p>`,
].join("");
}
function accountRow(account) {
const current = Number(account.user_id) === Number(state.currentUserId);
const confirming = Number(state.confirmingId) === Number(account.user_id);
const classes = `login-account-row${current ? " is-current" : ""}${confirming ? " is-confirming" : ""}`;
if (state.view === "manage" && confirming) {
const managing = state.view === "manage";
const classes = [
"login-account-row",
current ? "is-current" : "",
confirming ? "is-confirming" : "",
!managing ? "is-switchable" : "",
].filter(Boolean).join(" ");
if (managing && confirming) {
return [
`<div class="${classes}" data-user-id="${account.user_id}">`,
`<p class="login-confirm-copy">移除「${escapeHtml(account.username)}」的本机记录?</p>`,
@@ -103,16 +148,25 @@
"</div></div>",
].join("");
}
const action = state.view === "manage"
? `<button class="login-account-remove" type="button" data-confirm-id="${account.user_id}">移除</button>`
const glyph = escapeHtml(String(account.username || "账").slice(0, 1));
const tone = Number(account.user_id || 0) % 4;
const used = formatLastUsed(account.last_used_at);
const action = managing
? `<button class="login-account-remove" type="button" data-confirm-id="${account.user_id}" aria-label="移除 ${escapeHtml(account.username)}"><svg width="16" height="16" viewBox="0 0 16 16" aria-hidden="true"><path fill="currentColor" d="M6 2h4l.5 1H14v1H2V3h3.5L6 2zm1 4v6H6V6h1zm3 0v6H9V6h1zM3.5 5H13l-.7 8.2A1.5 1.5 0 0 1 10.81 14H5.19a1.5 1.5 0 0 1-1.49-1.8L3.5 5z"></path></svg></button>`
: current
? '<span class="login-account-check" aria-hidden="true">✓</span>'
: `<button class="login-account-enter" type="button" data-switch-id="${account.user_id}">进入</button>`;
? '<span class="login-account-action"><span class="login-account-check" aria-hidden="true">✓</span>继续使用</span>'
: "";
const switchAttr = !managing && !current ? ` data-switch-id="${account.user_id}"` : "";
const resumeAttr = !managing && current ? ` data-resume-id="${account.user_id}"` : "";
return [
`<div class="${classes}" data-user-id="${account.user_id}">`,
`<div class="${classes}" data-user-id="${account.user_id}"${switchAttr}${resumeAttr}>`,
`<span class="login-avatar tone-${tone}" aria-hidden="true">${glyph}</span>`,
'<div class="login-account-meta">',
'<div class="login-account-name">',
`<strong>${escapeHtml(account.username)}</strong>`,
`<span>${escapeHtml(membershipLabel(account))}${current ? " · 当前" : ""}</span>`,
chipsFor(account, current),
"</div>",
used ? `<span class="login-account-used">上次登录 ${escapeHtml(used)}</span>` : "",
"</div>",
action,
"</div>",
@@ -123,10 +177,15 @@
const count = state.accounts.length;
const managing = state.view === "manage";
return [
`<h2 class="login-card-title">${managing ? "管理账号记录" : "选择账号"}</h2>`,
`<p class="login-card-lead">这台电脑已记录 ${count} 个账号,可直接进入,无需再次输入密码。</p>`,
managing
? '<button class="login-manage" type="button" data-login-action="picker">完成</button>'
? ""
: '<button class="login-back" type="button" data-login-action="resume">返回复盘</button>',
`<h2 class="login-card-title">${managing ? "管理账号记录" : "选择账号"}</h2>`,
`<p class="login-card-lead">${managing
? "移除只删除这台电脑上的登录记录,不会注销账号"
: `这台电脑已记录 ${count} 个账号,点选即可进入,无需再次输入密码。`}</p>`,
managing
? '<div class="login-manage-toolbar"><p class="login-manage-hint">点击右侧图标移除对应记录</p><button class="login-manage-done" type="button" data-login-action="picker">完成</button></div>'
: "",
`<div class="login-account-list">${state.accounts.map(accountRow).join("")}</div>`,
managing
@@ -136,7 +195,9 @@
? ""
: '<button class="login-manage" type="button" data-login-action="manage">管理已记录的账号</button>',
state.error ? `<p class="login-error">${escapeHtml(state.error)}</p>` : "",
'<p class="login-privacy"><span class="login-lock" aria-hidden="true">🔒</span>账号记录仅保存在这台电脑的浏览器中</p>',
managing
? '<p class="login-privacy"><span class="login-lock" aria-hidden="true">🔒</span>移除后再次登录该账号需重新输入密码</p>'
: '<p class="login-privacy"><span class="login-lock" aria-hidden="true">🔒</span>账号记录仅保存在这台电脑的浏览器中</p>',
].join("");
}
@@ -145,7 +206,10 @@
if (state.view === "first" || state.view === "add") {
card.innerHTML = formMarkup({
title: state.view === "add" ? "添加账号" : "欢迎回来",
lead: "登录后进入你的复盘空间",
lead: state.view === "add" ? "登录另一个账号,添加后可随时一键切换" : "登录后进入你的复盘空间",
hint: state.view === "add"
? "添加后账号会保存在这台电脑,方便随时切换。"
: "密码连续输错 5 次将锁定 10 分钟。还没有账号?切换到「注册」创建。",
add: state.view === "add",
back: state.view === "add",
});
@@ -153,6 +217,7 @@
card.innerHTML = pickerMarkup();
}
bindCard();
if (mascots) mascots.sync();
}
function bindCard() {
@@ -166,6 +231,14 @@
card.querySelectorAll("[data-login-action]").forEach((button) => {
button.addEventListener("click", () => {
const action = button.dataset.loginAction;
if (action === "toggle-password") {
togglePasswordVisible();
return;
}
if (action === "resume") {
resumeCurrentAccount();
return;
}
if (action === "picker") {
state.view = state.accounts.length ? "picker" : "first";
state.confirmingId = null;
@@ -184,8 +257,12 @@
card.querySelectorAll("[data-switch-id]").forEach((button) => {
button.addEventListener("click", () => switchAccount(Number(button.dataset.switchId)));
});
card.querySelectorAll("[data-resume-id]").forEach((button) => {
button.addEventListener("click", () => resumeCurrentAccount());
});
card.querySelectorAll("[data-confirm-id]").forEach((button) => {
button.addEventListener("click", () => {
button.addEventListener("click", (event) => {
event.stopPropagation();
state.confirmingId = Number(button.dataset.confirmId);
render();
});
@@ -212,6 +289,8 @@
event.preventDefault();
const username = document.querySelector("#loginUsername").value.trim();
const password = document.querySelector("#loginPassword").value;
state.username = username;
state.password = password;
if (state.mode === "register" && password !== document.querySelector("#loginPasswordConfirm").value) {
setError("两次输入的密码不一致。");
render();
@@ -222,11 +301,13 @@
render();
try {
await api.request(`/api/auth/${state.mode}`, "POST", { username, password });
await celebrateLogin();
enterApp();
} catch (error) {
state.loading = false;
setError(error.message || "账号操作失败");
render();
mascots.fail();
}
}
@@ -244,6 +325,27 @@
}
}
async function resumeCurrentAccount() {
state.loading = true;
setError("");
render();
try {
const session = await api.request("/api/auth/me");
const sessionUserId = session.user?.id;
const matches = Boolean(session.authenticated) && (
!state.currentUserId || Number(sessionUserId) === Number(state.currentUserId)
);
if (!matches) {
throw new Error("当前会话已失效,请重新登录");
}
enterApp();
} catch (error) {
state.loading = false;
setError(error.message || "当前会话已失效,请重新登录");
render();
}
}
async function forgetAccount(userId) {
try {
await api.request("/api/auth/forget", "POST", { user_id: userId });
@@ -258,6 +360,241 @@
}
}
function togglePasswordVisible() {
state.passwordVisible = !state.passwordVisible;
const input = document.querySelector("#loginPassword");
const button = document.querySelector(".login-password-toggle");
if (input) {
input.type = state.passwordVisible ? "text" : "password";
input.focus();
}
if (button) {
button.textContent = state.passwordVisible ? "隐藏" : "显示";
button.setAttribute("aria-pressed", state.passwordVisible ? "true" : "false");
button.setAttribute("aria-label", state.passwordVisible ? "隐藏密码" : "显示密码");
}
mascots.sync();
}
function reducedMotion() {
return Boolean(global.matchMedia && global.matchMedia("(prefers-reduced-motion: reduce)").matches);
}
function finePointer() {
return Boolean(global.matchMedia && global.matchMedia("(pointer: fine)").matches);
}
function wait(ms) {
return new Promise((resolve) => global.setTimeout(resolve, ms));
}
async function celebrateLogin() {
mascots.succeed();
if (!reducedMotion()) await wait(720);
}
const mascots = (() => {
const root = document.querySelector("#loginMascots");
const red = root && root.querySelector(".login-mascot.is-red");
const green = root && root.querySelector(".login-mascot.is-green");
const motion = {
pupilX: 0,
pupilY: 0,
targetX: 0,
targetY: 0,
leanRed: 0,
leanGreen: 0,
targetLeanRed: 0,
targetLeanGreen: 0,
};
let mood = "idle";
let locked = "";
let blinkTimer = 0;
let failTimer = 0;
let raf = 0;
function setVars() {
if (!red || !green) return;
const pupilX = `${motion.pupilX.toFixed(2)}px`;
const pupilY = `${motion.pupilY.toFixed(2)}px`;
red.style.setProperty("--pupil-x", pupilX);
red.style.setProperty("--pupil-y", pupilY);
green.style.setProperty("--pupil-x", pupilX);
green.style.setProperty("--pupil-y", pupilY);
red.style.setProperty("--lean", `${motion.leanRed.toFixed(2)}deg`);
green.style.setProperty("--lean", `${motion.leanGreen.toFixed(2)}deg`);
}
function poseFor(next) {
if (next === "account" || next === "busy") {
motion.targetX = 3.2;
motion.targetY = 0.8;
motion.targetLeanRed = 7;
motion.targetLeanGreen = 5.6;
return;
}
if (next === "password") {
motion.targetX = 0;
motion.targetY = 0;
motion.targetLeanRed = 0;
motion.targetLeanGreen = 0;
return;
}
if (next === "fail") {
motion.targetX = 0;
motion.targetY = 2.8;
motion.targetLeanRed = 8;
motion.targetLeanGreen = 8;
return;
}
if (next === "success") {
motion.targetX = 0;
motion.targetY = 0;
motion.targetLeanRed = 0;
motion.targetLeanGreen = 0;
return;
}
if (!finePointer()) {
motion.targetX = 2.4;
motion.targetY = 0.4;
motion.targetLeanRed = 4;
motion.targetLeanGreen = 3.2;
return;
}
motion.targetLeanRed = 0;
motion.targetLeanGreen = 0;
}
function applyMood(next) {
if (!root) return;
const changed = next !== mood;
if (changed) {
mood = next;
root.dataset.mood = next;
poseFor(next);
} else if (next !== "idle") {
poseFor(next);
}
if (reducedMotion()) {
motion.pupilX = motion.targetX;
motion.pupilY = motion.targetY;
motion.leanRed = motion.targetLeanRed;
motion.leanGreen = motion.targetLeanGreen;
setVars();
}
}
function focusedControl() {
const active = document.activeElement;
if (!active || !card.contains(active)) return "";
if (active.classList.contains("login-password-toggle")) return "loginPassword";
return active.id || "";
}
function desiredMood() {
if (locked === "success") return "success";
if (state.loading) return "busy";
const focused = focusedControl();
if (focused === "loginPassword" || focused === "loginPasswordConfirm") return "password";
if (focused === "loginUsername") return "account";
if (locked === "fail") return "fail";
return "idle";
}
function sync() {
applyMood(desiredMood());
}
function succeed() {
locked = "success";
applyMood("success");
}
function fail() {
locked = "fail";
applyMood("fail");
global.clearTimeout(failTimer);
failTimer = global.setTimeout(() => {
if (locked === "fail") locked = "";
sync();
}, 900);
}
function blink() {
if (!root || reducedMotion()) return;
if (mood !== "idle" && mood !== "account" && mood !== "busy") return;
root.classList.remove("is-blinking");
void root.offsetWidth;
root.classList.add("is-blinking");
global.setTimeout(() => root.classList.remove("is-blinking"), 160);
}
function scheduleBlink() {
global.clearTimeout(blinkTimer);
if (reducedMotion()) return;
const waitMs = 4000 + Math.random() * 2000;
blinkTimer = global.setTimeout(() => {
blink();
scheduleBlink();
}, waitMs);
}
function onMouseMove(event) {
if (reducedMotion() || !finePointer()) return;
if (desiredMood() !== "idle") return;
const rect = root.getBoundingClientRect();
const cx = rect.left + rect.width * 0.42;
const cy = rect.top + rect.height * 0.42;
const dx = event.clientX - cx;
const dy = event.clientY - cy;
const dist = Math.hypot(dx, dy) || 1;
const cap = 3.5;
motion.targetX = (dx / dist) * Math.min(cap, Math.abs(dx) / 90);
motion.targetY = (dy / dist) * Math.min(cap, Math.abs(dy) / 90);
const tilt = Math.max(-5, Math.min(5, (dx / Math.max(global.innerWidth, 1)) * 10));
motion.targetLeanRed = tilt;
motion.targetLeanGreen = tilt * 0.8;
}
function tick() {
if (!root) return;
if (!reducedMotion()) {
motion.pupilX += (motion.targetX - motion.pupilX) * 0.18;
motion.pupilY += (motion.targetY - motion.pupilY) * 0.18;
motion.leanRed += (motion.targetLeanRed - motion.leanRed) * 0.18;
motion.leanGreen += (motion.targetLeanGreen - motion.leanGreen) * 0.18;
setVars();
} else {
motion.pupilX = motion.targetX;
motion.pupilY = motion.targetY;
motion.leanRed = motion.targetLeanRed;
motion.leanGreen = motion.targetLeanGreen;
setVars();
}
raf = global.requestAnimationFrame(tick);
}
function onFocusChange() {
global.requestAnimationFrame(sync);
}
if (root) {
document.addEventListener("focusin", onFocusChange);
document.addEventListener("focusout", onFocusChange);
global.addEventListener("mousemove", onMouseMove, { passive: true });
if (!reducedMotion()) {
raf = global.requestAnimationFrame(tick);
scheduleBlink();
} else {
poseFor("idle");
setVars();
}
sync();
}
return { sync, succeed, fail };
})();
themeButton.addEventListener("click", () => {
applyTheme(document.documentElement.dataset.theme === "dark" ? "light" : "dark", true);
});
+415 -8
View File
@@ -3192,8 +3192,7 @@
.m-sys-grid div {
padding: 12px;
border-radius: 8px;
background: var(--surface);
box-shadow: var(--elevation-card);
background: var(--surface-muted);
}
.m-sys-grid span {
@@ -3209,16 +3208,424 @@
color: var(--text-primary);
}
.m-sys-account-actions {
display: grid;
.m-sys-home {
padding: 0 0 12px;
display: flex;
flex-direction: column;
gap: 8px;
margin-top: 16px;
}
.m-page[data-page^="system/"] .m-btn-primary {
margin-bottom: 8px;
.m-sys-body {
padding: 12px 12px 20px;
display: flex;
flex-direction: column;
gap: 12px;
}
.m-sys-home .m-card,
.m-sys-body .m-card {
margin-left: 0;
margin-right: 0;
}
.m-sys-profile-card {
display: flex;
gap: 12px;
align-items: flex-start;
padding: 14px;
}
.m-sys-avatar {
flex: 0 0 auto;
width: 48px;
height: 48px;
border-radius: 50%;
display: flex;
align-items: center;
justify-content: center;
background: var(--action-soft);
color: var(--action);
font-size: var(--font-size-page-title);
font-weight: 600;
}
.m-sys-profile-card strong {
display: block;
font-size: var(--font-size-card-title);
color: var(--text-primary);
}
.m-sys-profile-meta {
margin: 4px 0 8px;
font-size: var(--font-size-caption);
color: var(--text-secondary);
}
.m-sys-badges {
display: flex;
flex-wrap: wrap;
gap: 6px;
}
.m-sys-badge {
display: inline-flex;
align-items: center;
height: 22px;
padding: 0 8px;
border-radius: 999px;
font-size: var(--font-size-aux);
font-weight: 600;
background: var(--surface-muted);
color: var(--text-secondary);
}
.m-sys-badge--admin {
background: var(--action-soft);
color: var(--action);
}
.m-sys-badge--ok {
background: var(--market-down-soft);
color: var(--market-down);
}
.m-sys-group-title {
margin: 4px 0 0;
font-size: var(--font-size-caption);
font-weight: 600;
color: var(--text-tertiary);
}
.m-sys-list {
padding: 0;
overflow: hidden;
}
.m-sys-row {
width: 100%;
display: flex;
align-items: center;
gap: 12px;
padding: 10px 12px;
border: 0;
background: transparent;
color: inherit;
text-align: left;
cursor: pointer;
-webkit-tap-highlight-color: transparent;
}
.m-sys-row + .m-sys-row {
border-top: 1px solid var(--border);
}
.m-sys-row:active {
background: var(--surface-hover);
}
.m-sys-row-icon {
flex: 0 0 auto;
width: 36px;
height: 36px;
border-radius: 10px;
display: inline-flex;
align-items: center;
justify-content: center;
background: var(--action-soft);
color: var(--action);
}
.m-sys-row--danger .m-sys-row-icon {
background: var(--market-up-soft);
color: var(--market-up);
}
.m-sys-row-body {
flex: 1;
min-width: 0;
}
.m-sys-row-body strong {
display: block;
font-size: var(--font-size-body);
font-weight: 600;
color: var(--text-primary);
}
.m-sys-row-body small {
display: block;
margin-top: 2px;
font-size: var(--font-size-caption);
color: var(--text-secondary);
}
.m-sys-row-chevron {
flex: 0 0 auto;
color: var(--text-tertiary);
}
.m-sys-foot {
margin: 8px 0 0;
text-align: center;
font-size: var(--font-size-caption);
color: var(--text-tertiary);
}
.m-sys-notice {
padding: 12px;
border-radius: 12px;
background: var(--surface-muted);
}
.m-sys-notice p {
margin: 0;
font-size: var(--font-size-label);
color: var(--text-secondary);
line-height: 1.55;
}
.m-sys-notice .m-sys-badges {
margin-top: 8px;
}
.m-sys-section {
padding: 12px;
}
.m-sys-section > strong,
.m-sys-section-title {
display: block;
margin-bottom: 10px;
font-size: var(--font-size-card-title);
color: var(--text-primary);
}
.m-sys-hint {
margin: 8px 0 0;
font-size: var(--font-size-caption);
color: var(--text-tertiary);
line-height: 1.45;
}
.m-sys-status-list {
display: grid;
gap: 8px;
}
.m-sys-status-item {
display: flex;
align-items: center;
justify-content: space-between;
gap: 8px;
font-size: var(--font-size-body);
color: var(--text-primary);
}
.m-sys-dot {
width: 8px;
height: 8px;
border-radius: 50%;
background: var(--text-tertiary);
}
.m-sys-dot--ok {
background: var(--market-down);
}
.m-sys-switch-row {
display: flex;
align-items: center;
justify-content: space-between;
gap: 12px;
}
.m-sys-switch-row strong {
display: block;
font-size: var(--font-size-body);
margin-bottom: 0;
}
.m-btn-outline,
.m-btn-outline-danger {
width: 100%;
height: 44px;
display: flex;
align-items: center;
justify-content: center;
gap: 8px;
border-radius: 8px;
background: var(--surface);
font-size: var(--font-size-body);
font-weight: 600;
cursor: pointer;
-webkit-tap-highlight-color: transparent;
}
.m-btn-outline {
border: 1px solid var(--action);
color: var(--action);
}
.m-btn-outline:active {
background: var(--action-soft);
}
.m-btn-outline-danger {
border: 1px solid var(--market-up);
color: var(--market-up);
}
.m-btn-outline-danger:active {
background: var(--market-up-soft);
}
.m-btn-outline:disabled,
.m-btn-outline-danger:disabled {
opacity: 0.45;
cursor: default;
pointer-events: none;
}
.m-sys-model-card .m-sys-badges,
.m-sys-user-row .m-sys-badges {
margin-top: 6px;
}
.m-sys-model-card {
display: flex;
align-items: flex-start;
justify-content: space-between;
gap: 8px;
width: 100%;
padding: 12px;
border: 0;
background: transparent;
text-align: left;
cursor: pointer;
}
.m-sys-model-card + .m-sys-model-card {
border-top: 1px solid var(--border);
}
.m-sys-model-card:active {
background: var(--surface-hover);
}
.m-sys-user-row {
display: flex;
align-items: center;
gap: 10px;
padding: 12px;
}
.m-sys-user-row + .m-sys-user-row {
border-top: 1px solid var(--border);
}
.m-sys-user-row .m-btn-outline {
width: auto;
height: 32px;
padding: 0 12px;
flex: 0 0 auto;
}
.m-sys-pair {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 8px;
}
.m-sheet-root.is-dialog .m-sheet {
display: none;
}
.m-dialog {
position: absolute;
left: 50%;
top: 42%;
width: calc(100% - 48px);
max-width: 320px;
transform: translate(-50%, -46%) scale(0.96);
border-radius: 12px;
background: var(--surface);
box-shadow: var(--elevation-float);
padding: 18px 16px 14px;
z-index: 3;
opacity: 0;
transition: opacity var(--motion-enter) var(--ease-enter),
transform var(--motion-enter) var(--ease-enter);
}
.m-sheet-root.is-open .m-dialog {
opacity: 1;
transform: translate(-50%, -50%) scale(1);
}
.m-dialog h2 {
margin: 0 0 8px;
font-size: var(--font-size-card-title);
color: var(--text-primary);
text-align: center;
}
.m-dialog p {
margin: 0 0 16px;
font-size: var(--font-size-label);
color: var(--text-secondary);
line-height: 1.5;
text-align: center;
}
.m-dialog-actions {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 8px;
}
.m-dialog-actions .m-btn-outline,
.m-dialog-actions .m-btn-outline-danger,
.m-dialog-actions .m-btn-primary {
height: 40px;
}
.m-sys-sheet-actions {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 8px;
margin-top: 12px;
}
.m-sys-test-row {
display: flex;
align-items: center;
gap: 10px;
margin: 8px 0 12px;
}
.m-sys-test-row .m-btn-outline {
width: auto;
flex: 0 0 auto;
height: 36px;
padding: 0 12px;
}
.m-sys-test-row [data-model-test-status] {
flex: 1;
font-size: var(--font-size-caption);
color: var(--text-secondary);
}
.m-sys-hero {
padding: 14px;
}
.m-sys-hero strong {
display: block;
font-size: var(--font-size-page-title);
margin-bottom: 6px;
}
.m-page[data-page^="system/"] .m-card {
margin-bottom: 12px;
margin-bottom: 0;
}
+541 -168
View File
File diff suppressed because it is too large Load Diff
+6 -2
View File
@@ -187,9 +187,9 @@
const dark = document.getElementById("m-app").dataset.theme === "dark";
const toggle = document.querySelector("[data-theme-toggle]");
if (toggle) toggle.setAttribute("aria-checked", dark ? "true" : "false");
const rowIcon = document.querySelector(".m-theme-row-icon");
const rowIcon = document.querySelector(".m-theme-row-icon, [data-theme-row-icon]");
if (rowIcon) rowIcon.innerHTML = icon(dark ? "moon" : "sun");
const rowBodySmall = document.querySelector(".m-theme-row-body small");
const rowBodySmall = document.querySelector(".m-theme-row-body small, [data-theme-row-label]");
if (rowBodySmall) rowBodySmall.textContent = dark ? "当前:夜间模式" : "当前:日间模式";
}
@@ -206,6 +206,10 @@
replace(DEFAULT_HASH);
return;
}
if (key === "system" && global.MobilePages && typeof global.MobilePages.renderSystemHome === "function") {
global.MobilePages.renderSystemHome();
return;
}
const items = visibleHubItems(hub);
updateHeader({ title: hub.title, back: false });
const section = key === "system" ? themeToggleSection() : "";
+5 -5
View File
@@ -68,10 +68,10 @@
"/pages/sentiment/page.js?v=20260729-1",
"/pages/pools/page.js?v=20260820-1",
"/pages/market/breadth.js?v=20260803-1",
"/pages/market/charts.js?v=20260803-1",
"/pages/market/entity-detail.js?v=20260803-1",
"/pages/market/stock-detail.js?v=20260803-1",
"/pages/market/preview.js?v=20260806-1",
"/pages/market/charts.js?v=20260908-1",
"/pages/market/entity-detail.js?v=20260908-1",
"/pages/market/stock-detail.js?v=20260908-1",
"/pages/market/preview.js?v=20260908-1",
"/pages/market/search.js?v=20260803-1",
"/pages/market/bindings.js?v=20260803-1",
"/pages/ladder/page.js?v=20260820-1",
@@ -95,7 +95,7 @@
"/shared/table.js?v=20260803-1",
"/shared/theme.js?v=20260803-1",
"/shared/dashboard.js?v=20260820-1",
"/shared/session.js?v=20260803-1",
"/shared/session.js?v=20260829-hel243",
"/shared/admin.js?v=20260803-1",
"/app.js?v=20260803-2",
];
+3 -243
View File
@@ -93,28 +93,6 @@ body[data-active-view="heavenView"] .workspace-view {
display: none;
}
:where(#heavenView) .heaven-tabs {
display: flex;
border-bottom: 1px solid var(--line);
}
:where(#heavenView) .heaven-tab {
border-bottom: 3px solid transparent;
background: transparent;
cursor: pointer;
}
:where(#heavenView) .heaven-tab.active {
border-bottom-color: var(--coral);
}
:where(#heavenView) .heaven-tab:hover {
border-bottom-color: var(--coral);
}
:where(#heavenView) .heaven-panel {
display: none;
}
@@ -1875,60 +1853,6 @@ body[data-active-view="heavenView"] .workspace-view {
color: var(--heaven-ink-faint);
}
#heavenView .heaven-tab {
font-family: var(--heaven-serif);
height: 52px;
position: relative;
padding: 0px 2px;
border: 0px;
color: var(--heaven-ink-soft);
font-size: 14px;
font-weight: 600;
}
#heavenView .heaven-tab::after {
content: "";
position: absolute;
right: 0px;
bottom: 0px;
left: 0px;
height: 2px;
background: var(--heaven-cinnabar);
opacity: 0;
transform: scaleX(0.3);
transition: opacity 220ms ease, transform 260ms var(--ease-out);
}
#heavenView .heaven-tab.active {
color: var(--heaven-ink);
}
#heavenView .heaven-tab:hover {
color: var(--heaven-ink);
}
#heavenView .heaven-tab.active::after {
opacity: 1;
transform: scaleX(1);
}
:where(#heavenView) .heaven-proverb {
margin: 0px;
@@ -1960,7 +1884,7 @@ body[data-active-view="heavenView"] .workspace-view {
}
#heavenView .button:focus-visible,
#heavenView .heaven-tab:focus-visible,
#heavenView .segment:focus-visible,
#heavenView summary:focus-visible {
outline: 2px solid var(--heaven-cinnabar);
@@ -2623,12 +2547,6 @@ body[data-active-view="heavenView"] .workspace-view {
padding: 0px 14px;
}
#heavenView .heaven-tabs {
gap: 22px;
padding: 0px 14px;
}
.heaven-proverb {
padding: 8px 14px;
@@ -3093,7 +3011,7 @@ body[data-active-view="heavenView"] .workspace-view {
#heavenView.heaven-data-loading .heaven-panel,
#heavenView.heaven-data-loading .heaven-proverb,
#heavenView.heaven-data-loading .heaven-tabs {
#heavenView.heaven-data-loading .heaven-page-head {
opacity: 0.42;
pointer-events: none;
@@ -3695,7 +3613,7 @@ body[data-active-view="heavenView"] .workspace-view {
@media (max-width: 900px) {
#heavenView .heaven-panel > ,
#heavenView .heaven-proverb,
#heavenView .heaven-tabs,
#heavenView .heaven-page-head,
#heavenView .heaven-toolbar {
width: min(100% - 28px, 1280px);
}
@@ -3746,22 +3664,6 @@ body[data-active-view="heavenView"] .workspace-view {
display: none;
}
#heavenView .heaven-tabs {
display: grid;
grid-template-columns: repeat(3, minmax(0px, 1fr));
gap: 0px;
padding: 0px;
}
#heavenView .heaven-tab {
width: 100%;
min-width: 0px;
}
#heavenFortunePanel .fortune-heading {
display: flex;
@@ -3867,18 +3769,6 @@ body[data-active-view="heavenView"] .workspace-view {
}
@media (max-width: 520px) {
.heaven-tabs {
gap: 0px;
padding: 0px 8px;
}
.heaven-tab {
min-width: 0px;
flex: 1 1 0%;
}
.heaven-controls {
grid-template-columns: 1fr;
@@ -6232,26 +6122,6 @@ body[data-active-view="heavenView"] .workspace-view {
padding: 12px 20px;
}
#heavenView .heaven-tabs {
align-items: stretch;
gap: 30px;
border-color: var(--heaven-rule);
background: rgba(253, 252, 248, 0.96);
width: min(100% - 40px, 1280px);
margin-right: auto;
margin-left: auto;
min-height: 54px;
padding: 0px 20px;
}
#heavenView .heaven-proverb {
margin-right: auto;
@@ -6297,12 +6167,6 @@ body[data-active-view="heavenView"] .workspace-view {
padding: 10px 14px;
}
#heavenView .heaven-tabs {
min-height: 52px;
padding: 0px 14px;
}
#heavenView .heaven-proverb {
padding: 9px 14px;
}
@@ -6668,18 +6532,6 @@ body[data-active-view="heavenView"] .workspace-view {
color: var(--wt-faint);
}
:root[data-theme="light"] #heavenView .wt-tabs .wt-tab {
color: var(--wt-muted);
}
:root[data-theme="light"] #heavenView .wt-tabs .wt-tab small {
color: var(--wt-faint);
}
:root[data-theme="light"] #heavenView .wt-tabs .wt-tab.on {
color: var(--wt-gold-bright);
}
:root[data-theme="light"] #heavenView .wt-empty {
color: var(--wt-muted);
}
@@ -7128,88 +6980,6 @@ body[data-active-view="heavenView"] .workspace-view {
letter-spacing: 4px;
}
.wt-tabs {
display: flex;
justify-content: center;
gap: 34px;
margin-top: 20px;
}
.wt-tabs .wt-tab {
position: relative;
padding: 8px 4px;
border: 0px;
background: transparent;
color: rgba(216, 210, 189, 0.5);
font-size: 15px;
letter-spacing: 3px;
transition: color 0.2s;
}
.wt-tabs .wt-tab small {
display: block;
margin-top: 3px;
color: rgba(216, 210, 189, 0.3);
font-family: inherit;
font-size: 10px;
letter-spacing: 1px;
}
.wt-tabs .wt-tab::after {
content: "";
position: absolute;
bottom: -2px;
left: 50%;
width: 0px;
height: 1.5px;
background: var(--wt-gold);
transform: translateX(-50%);
transition: 0.25s;
}
.wt-tabs .wt-tab.on {
color: var(--wt-gold-bright);
}
.wt-tabs .wt-tab.on::after {
width: 100%;
}
.wt-tabs .wt-tab:disabled {
cursor: default;
}
.wt-tabs .wt-tab:focus {
outline: none;
}
.wt-tabs .wt-tab:focus-visible {
box-shadow: rgba(201, 165, 92, 0.45) 0px 2px 0px;
}
.heaven-proverb {
margin: 8px 0px 0px;
@@ -9072,16 +8842,6 @@ body[data-active-view="heavenView"] .workspace-view {
gap: 5px;
}
.wt-tabs {
gap: 16px;
}
.wt-tabs .wt-tab {
font-size: 13px;
letter-spacing: 2px;
}
.heaven-proverb {
text-align: center;
}
+13 -6
View File
@@ -1,16 +1,23 @@
<section id="heavenView" class="workspace-view member-feature-view heaven-shell wt">
<div class="member-gate" hidden><div class="member-gate-icon"><i data-lucide="lock-keyhole"></i></div><div><strong>问天仅对会员开放</strong><span>开通会员后可使用观势、观气、观心及平台解读。会员状态可从顶部账号标识进入。</span></div></div>
<div class="section-toolbar redesigned-page-head heaven-page-head">
<div class="section-title-group">
<h2>问天</h2>
<span class="section-subtitle"><b id="heavenDataDate">--</b></span>
</div>
<div class="toolbar-controls">
<div class="segmented" role="group" aria-label="问天模块">
<button class="segment active on" type="button" data-heaven-panel="trend" aria-current="page">观势</button>
<button class="segment" type="button" data-heaven-panel="fortune">观气</button>
<button class="segment" type="button" data-heaven-panel="heart">观心</button>
</div>
</div>
</div>
<header class="wt-head">
<div class="wt-title-line">
<h1 class="wt-serif">问 天</h1>
<span id="heavenDataDate">--</span>
</div>
<div class="verse wt-serif">观天之道 · 执天之行</div>
<nav class="wt-tabs" aria-label="问天模块">
<button class="wt-tab wt-serif on" type="button" data-heaven-panel="trend" aria-current="page">观势<small>三才六爻 · 量化成卦</small></button>
<button class="wt-tab wt-serif" type="button" data-heaven-panel="fortune">观气<small>五运六气 · 日辰生克</small></button>
<button class="wt-tab wt-serif" type="button" data-heaven-panel="heart">观心<small>静心占卜 · 第一念</small></button>
</nav>
</header>
<p class="heaven-proverb wt-serif">遇事不决可问春风,春风不语即随本心</p>
<div id="heavenNotice" class="inline-notice" role="status" hidden></div>
+15 -5
View File
@@ -113,9 +113,12 @@ async function loadHeavenSetup(force = false, sector = "", stockCode = "") {
document.querySelector("#resetHeavenCalibrationButton"),
].filter(Boolean);
cancelHeavenPerformance();
heavenView?.classList.add("heaven-data-loading");
const blocking = !state.heavenSetup;
if (blocking) heavenView?.classList.add("heaven-data-loading");
if (loadButton) loadButton.disabled = true;
calibrationButtons.forEach((button) => { button.disabled = true; });
const controller = new AbortController();
const timeoutId = window.setTimeout(() => controller.abort(), 25_000);
try {
if (state.heavenSetup?.requestedKey && state.heavenSetup.requestedKey !== requestedKey) {
state.personalField = null;
@@ -126,7 +129,7 @@ async function loadHeavenSetup(force = false, sector = "", stockCode = "") {
if (sector) query.set("sector", sector);
if (stockCode) query.set("stock_code", stockCode);
if (manualData) query.set("manual_data", JSON.stringify(manualData));
const payload = await apiRequest(`/api/heaven/setup?${query}`);
const payload = await apiRequest(`/api/heaven/setup?${query}`, "GET", null, { signal: controller.signal });
if (
requestSequence !== state.heavenRequestSequence
|| calendarDate !== document.querySelector("#qiObservationDate")?.value
@@ -152,9 +155,15 @@ async function loadHeavenSetup(force = false, sector = "", stockCode = "") {
if (payload.chart.selection_notice) showHeavenNotice(payload.chart.selection_notice);
} catch (error) {
if (requestSequence !== state.heavenRequestSequence) return;
showHeavenNotice(error.message || "问天数据加载失败");
showToast(error.message || "问天数据加载失败");
const aborted = error?.payload?.aborted || /abort|超时|cancel/i.test(String(error?.message || ""));
const message = aborted
? "问天数据仍在准备,页面可继续输入和操作"
: (error.message || "问天数据加载失败");
showHeavenNotice(message);
if (!aborted) showToast(message);
if (!state.heavenSetup) renderHeavenWorkspace();
} finally {
window.clearTimeout(timeoutId);
if (requestSequence === state.heavenRequestSequence) {
heavenView?.classList.remove("heaven-data-loading");
if (loadButton) loadButton.disabled = false;
@@ -1363,7 +1372,8 @@ async function interpretHeaven(mode) {
} catch (error) {
stopHeavenReadingAnimation();
state.heavenReadingLoading = false;
state.heavenReadingError = error.message || "问天解读失败";
const detail = error?.payload?.message || error?.payload?.error || error.message;
state.heavenReadingError = detail || "问天解读失败";
renderHeavenReadingDialog();
showHeavenNotice(state.heavenReadingError);
showToast(state.heavenReadingError);
+23 -9
View File
@@ -1,3 +1,16 @@
const DAILY_CHART_BARS = 45;
function visibleDailyPrices(prices) {
return (prices || []).slice(-DAILY_CHART_BARS);
}
function dailyChartSourceLabel(prices, notice) {
const count = visibleDailyPrices(prices).length;
const base = `日 K 行情 · ${count} 个交易日`;
const text = String(notice || "").trim();
return text ? `${base} · ${text}` : base;
}
function currentChartPalette() {
const style = getComputedStyle(document.documentElement);
const color = (token, fallback) => style.getPropertyValue(token).trim() || fallback;
@@ -56,7 +69,8 @@ function drawCandlestick(context, x, item, priceY, candleWidth, palette = curren
function drawPriceChart(prices) {
const canvas = elements.priceChart;
if (!prices?.length) {
const visible = visibleDailyPrices(prices);
if (!visible.length) {
clearPriceChart("暂无日 K 数据");
return;
}
@@ -81,15 +95,15 @@ function drawPriceChart(prices) {
const gap = 12;
const priceBottom = height - bottom - volumeHeight - gap;
const plotWidth = width - left - right;
const highs = prices.map((item) => number(item.high));
const lows = prices.map((item) => number(item.low));
const highs = visible.map((item) => number(item.high));
const lows = visible.map((item) => number(item.low));
const maximum = Math.max(...highs);
const minimum = Math.min(...lows);
const range = Math.max(maximum - minimum, maximum * 0.01, 0.01);
const volumes = prices.map((item) => number(item.volume));
const volumes = visible.map((item) => number(item.volume));
const maxVolume = Math.max(...volumes, 1);
const priceY = (value) => top + (maximum - value) / range * (priceBottom - top);
const step = plotWidth / prices.length;
const step = plotWidth / visible.length;
const candleWidth = clamp(step * 0.62, 2, 8);
context.strokeStyle = palette.grid;
@@ -105,7 +119,7 @@ function drawPriceChart(prices) {
context.fillText((maximum - range * line / 4).toFixed(2), left - 5, y + 4);
}
prices.forEach((item, index) => {
visible.forEach((item, index) => {
const x = left + step * index + step / 2;
const color = drawCandlestick(context, x, item, priceY, candleWidth, palette);
const volumeBarHeight = number(item.volume) / maxVolume * volumeHeight;
@@ -117,10 +131,10 @@ function drawPriceChart(prices) {
context.textAlign = "center";
context.fillStyle = palette.axis;
const labelIndexes = [0, Math.floor((prices.length - 1) / 2), prices.length - 1];
const labelIndexes = [0, Math.floor((visible.length - 1) / 2), visible.length - 1];
labelIndexes.forEach((index) => {
const x = left + step * index + step / 2;
context.fillText(String(prices[index].trade_date).slice(5), x, height - 5);
context.fillText(String(visible[index].trade_date).slice(5), x, height - 5);
});
}
@@ -301,7 +315,7 @@ function drawIntradayPreviewChart(points, dailyPrices, referenceClose = 0) {
function drawDailyPreviewChart(prices) {
const { context, width, height, palette } = prepareStockPreviewCanvas();
const visible = prices.slice(-45);
const visible = visibleDailyPrices(prices);
const visibleStart = prices.length - visible.length;
const left = 45;
const right = 10;
+2 -2
View File
@@ -113,13 +113,13 @@ function renderEntityDetailMetrics(metrics) {
}
function drawEntityDetailChart(series, canvas = elements.entityDetailChart) {
const candles = (series || []).filter((item) => number(item.close) > 0).map((item) => {
const candles = visibleDailyPrices((series || []).filter((item) => number(item.close) > 0).map((item) => {
const close = number(item.close);
const open = number(item.open) || close;
const high = Math.max(number(item.high) || close, open, close);
const low = Math.min(number(item.low) || close, open, close);
return { ...item, open, high, low, close };
});
}));
if (!candles.length) {
clearEntityDetailChart("暂无日 K 数据", canvas);
return;
+2 -1
View File
@@ -367,7 +367,8 @@ function selectStockPreviewChart(chart) {
}
} else if ((payload.prices || []).length) {
setText("stockPreviewDate", payload.meta?.trade_date || "最新行情");
setText("stockPreviewSource", `日 K 行情 · ${payload.prices.length} 个交易日`);
const notice = String(payload.meta?.notice || "").trim();
setText("stockPreviewSource", dailyChartSourceLabel(payload.prices, notice));
drawDailyPreviewChart(payload.prices);
} else {
setText("stockPreviewDate", payload.meta?.trade_date || "最新行情");
+25 -12
View File
@@ -20,17 +20,9 @@ async function openStock(code, fallback = null) {
setText("detailStreak", row.status === "涨停" ? streakLabel(row.streak) : row.status || "--");
setText("detailReason", row.reason || "--");
setText("detailSector", row.sector || "其他");
setText("detailFirst", row.first_time || "--");
setText("detailLast", row.last_time || "--");
setText("detailOpen", `${number(row.open_times)}`);
setText("detailTurnover", `${formatNumber(row.turnover_rate, 2)}%`);
setText("detailAmount", `${formatNumber(row.amount_billion, 2)} 亿`);
setText("detailSeal", `${formatNumber(row.seal_amount_million, 0)}`);
setStockBoardFields(row);
setText("chartSource", "正在加载行情");
setText("flowNet", "--");
setText("flowLarge", "--");
setText("flowMedium", "--");
setText("flowSmall", "--");
renderMoneyflow({});
document.querySelector("#reasonInput").value = row.reason || "";
document.querySelector("#stockNoteContent").value = "";
document.querySelector("#stockNotePlan").value = "";
@@ -48,11 +40,13 @@ async function openStock(code, fallback = null) {
setText("detailName", stock.name || row.name);
setText("detailPrice", formatNumber(stock.price || row.price, 2));
setText("detailChange", `${signed(stock.change ?? row.change)}%`);
setStockBoardFields({ ...row, ...stock });
renderMoneyflow(payload.moneyflow || {});
renderStockNotes(payload.notes || []);
updateWatchButton();
if (state.stockDetailChartMode === "daily") {
setText("chartSource", `日 K 行情 · ${payload.prices.length} 个交易日`);
const notice = String(payload.meta?.notice || "").trim();
setText("chartSource", dailyChartSourceLabel(payload.prices, notice));
requestAnimationFrame(() => drawPriceChart(payload.prices || []));
}
} catch (error) {
@@ -69,7 +63,11 @@ async function selectStockDetailChart(mode) {
syncDetailChartButtons("stock", selected);
if (selected === "daily") {
const prices = state.stockDetail?.prices || [];
setText("chartSource", prices.length ? `日 K 行情 · ${prices.length} 个交易日` : "正在加载行情");
const notice = String(state.stockDetail?.meta?.notice || "").trim();
setText(
"chartSource",
prices.length ? dailyChartSourceLabel(prices, notice) : "正在加载行情",
);
if (prices.length) requestAnimationFrame(() => drawPriceChart(prices));
else clearPriceChart("正在加载日 K 数据");
return;
@@ -111,6 +109,21 @@ function renderStockDetailIntraday(payload) {
});
}
function setStockBoardFields(row) {
const firstTime = String(row.first_time || "").trim();
const lastTime = String(row.last_time || "").trim();
setText("detailFirst", firstTime && firstTime !== "--" ? firstTime : "--");
setText("detailLast", lastTime && lastTime !== "--" ? lastTime : "--");
setText("detailOpen", row.open_times === null || row.open_times === undefined || row.open_times === "" ? "--" : `${number(row.open_times)}`);
setText("detailTurnover", presentMetric(row.turnover_rate) ? `${formatNumber(row.turnover_rate, 2)}%` : "--");
setText("detailAmount", presentMetric(row.amount_billion) ? `${formatNumber(row.amount_billion, 2)} 亿` : "--");
setText("detailSeal", presentMetric(row.seal_amount_million) ? `${formatNumber(row.seal_amount_million, 0)}` : "--");
}
function presentMetric(value) {
return meaningfulNumber(value) && Number(value) !== 0;
}
function openActiveStockInHeaven() {
const code = state.activeStock?.code;
if (!/^\d{6}$/.test(String(code || ""))) return;
+1 -1
View File
@@ -49,7 +49,7 @@ body[data-active-view="mentorView"] .app-page-context span {
height: 100%;
min-height: 0;
flex-direction: column;
padding: 0;
padding: var(--page-pad-y) 0 0;
color: var(--qp-text-1);
font-family: "PingFang SC", "Microsoft YaHei", system-ui, sans-serif;
}
+11 -1
View File
@@ -408,8 +408,18 @@ async function saveReasonOverride(event) {
}
function renderMoneyflow(flow) {
for (const [id, value] of [["flowNet", flow.net_million], ["flowLarge", flow.large_million], ["flowMedium", flow.medium_million], ["flowSmall", flow.small_million]]) {
const payload = flow || {};
const available = payload.available !== false && [
payload.net_million, payload.large_million, payload.medium_million, payload.small_million,
].some((value) => value !== null && value !== undefined && value !== "");
for (const [id, value] of [["flowNet", payload.net_million], ["flowLarge", payload.large_million], ["flowMedium", payload.medium_million], ["flowSmall", payload.small_million]]) {
const element = document.getElementById(id);
if (!element) continue;
if (!available || value === null || value === undefined || value === "") {
element.textContent = "--";
element.className = "";
continue;
}
element.textContent = formatMoneyMillion(value);
element.className = changeClass(value);
}
+31 -9
View File
@@ -7,7 +7,14 @@ async function backfillData() {
start_date: document.querySelector("#backfillStart").value,
end_date: document.querySelector("#backfillEnd").value,
});
showToast(`历史回补完成,共处理 ${payload.results.length} 个工作日`);
const failed = (payload.failed_count || 0);
const skipped = (payload.skipped_non_trading_days || []).length;
const suffix = failed
? `,失败 ${failed}`
: skipped
? `,跳过 ${skipped} 个非交易日`
: "";
showToast(`历史回补完成,共处理 ${payload.results.length} 个交易日${suffix}`);
state.sentimentHistory = null;
state.sentimentHistoryKey = "";
if (state.activeView === "sentimentCycleView") {
@@ -34,11 +41,10 @@ async function openAdminSettings(refreshOnly = false) {
const ifind = data.ifind || {};
const llm = payload.llm || {};
const membership = payload.membership || {};
status.textContent = `Tushare ${data.configured ? "已配置" : "未配置"} · iFinD ${ifind.configured ? "已配置" : "未配置"} · ${number(data.snapshot_dates)} 个交易日`;
status.textContent = `数据中枢 ${data.configured ? "已连接" : "未连接"} · iFinD ${ifind.configured ? "已配置" : "未配置"} · ${number(data.snapshot_dates)} 个交易日`;
status.classList.toggle("connected", Boolean(data.configured));
setText("systemDataStatus", data.background_refresh_enabled ? "后台刷新已启用" : "后台刷新已暂停");
document.querySelector("#systemTokenInput").value = "";
document.querySelector("#systemIfindTokenInput").value = "";
renderDatahubRouteStatus(data.datahub || {});
document.querySelector("#systemBackgroundRefresh").checked = Boolean(data.background_refresh_enabled);
document.querySelector("#memberDailyLimit").value = number(membership.member_daily_limit) || 50;
renderModelPool(llm.models || [], llm.primary_model_id || "", llm.fallback_model_id || "");
@@ -48,6 +54,26 @@ async function openAdminSettings(refreshOnly = false) {
}
}
function renderDatahubRouteStatus(hub) {
const box = document.querySelector("#datahubRouteStatus");
if (!box) return;
const label = box.querySelector("span");
const enabled = number(hub.enabled_reads);
const total = number(hub.total_reads) || enabled;
const fallbacks = hub.fallback_labels || [];
if (fallbacks.length) {
box.dataset.tone = "warning";
if (label) label.textContent = `数据中枢主线路 ${enabled}/${total} · 备用 ${fallbacks.length} 类:${fallbacks.join("、")}`;
return;
}
box.dataset.tone = hub.configured ? "success" : "idle";
if (label) {
label.textContent = hub.configured
? `数据中枢主线路 ${enabled}/${total},当前无备用`
: "数据中枢未配置,网站只保留已有真实快照";
}
}
function selectAdminPanel(panel) {
const selected = ["market", "models", "members"].includes(panel) ? panel : "market";
document.querySelector("#adminSectionSelect").value = selected;
@@ -177,13 +203,9 @@ async function saveMarketSettings(event) {
button.disabled = true;
try {
await apiRequest("/api/admin/settings", "POST", {
tushare_token: document.querySelector("#systemTokenInput").value.trim(),
ifind_refresh_token: document.querySelector("#systemIfindTokenInput").value.trim(),
background_refresh_enabled: document.querySelector("#systemBackgroundRefresh").checked,
});
document.querySelector("#systemTokenInput").value = "";
document.querySelector("#systemIfindTokenInput").value = "";
showToast("行情配置已保存");
showToast("行情刷新设置已保存");
await openAdminSettings(true);
} catch (error) {
showToast(error.message || "系统配置保存失败");
+22 -2
View File
@@ -46,12 +46,32 @@
}
}
function readableRequestError(error) {
const message = String(error?.message || "");
if (
error instanceof TypeError
|| /failed to fetch|networkerror|load failed|network request failed/i.test(message)
) {
return "网络请求失败,服务暂时不可用,请稍后重试。";
}
return message || "请求失败";
}
async function request(url, method = "GET", body = null, options = {}) {
const response = await fetch(url, requestOptions(method, body, options.signal));
let response;
try {
response = await fetch(url, requestOptions(method, body, options.signal));
} catch (error) {
if (error?.name === "AbortError") {
throw new ApiError("请求已取消或超时", 0, { aborted: true });
}
throw new ApiError(readableRequestError(error), 0, null);
}
const payload = await parseJson(response);
handleUnauthorized(response, url);
if (!response.ok || payload.error) {
throw new ApiError(payload.error || "请求失败", response.status, payload);
const message = payload.message || payload.error || "请求失败";
throw new ApiError(message, response.status, payload);
}
return payload;
}
+812 -67
View File
File diff suppressed because it is too large Load Diff
+10 -2
View File
@@ -660,7 +660,9 @@ body.sidebar-collapsed .sidebar-collapse-button .lucide {
text-align: left;
min-height: 38px;
height: var(--size-statusbar);
min-height: var(--size-statusbar);
display: flex;
@@ -670,7 +672,7 @@ body.sidebar-collapsed .sidebar-collapse-button .lucide {
margin: auto -8px -8px;
padding: 10px 16px;
padding: 0 16px;
border-right: 0px;
@@ -691,6 +693,12 @@ body.sidebar-collapsed .sidebar-collapse-button .lucide {
font-size: 12px;
}
.sidebar-collapse-button .lucide {
width: 14px;
height: 14px;
}
body.sidebar-collapsed .sidebar-collapse-button span {
display: none;
}
+107 -6
View File
@@ -40,17 +40,109 @@ async function loadDashboard(force = false, background = false, showOverlay = tr
async function startAdminRefresh() {
const buttons = [document.querySelector("#syncButton"), document.querySelector("#adminRefreshButton")].filter(Boolean);
buttons.forEach((button) => { button.disabled = true; });
const requestedDate = elements.tradeDate.value;
setAdminRefreshStatus("running", `正在刷新 ${requestedDate} 的行情,请稍候…`, "loader-circle");
try {
const payload = await apiRequest("/api/admin/refresh", "POST", { trade_date: elements.tradeDate.value });
showToast(payload.message || "后台刷新已提交");
setStatus("后台刷新运行中,当前页面保持不变");
const payload = await apiRequest("/api/admin/refresh", "POST", { trade_date: requestedDate });
if (!payload.started || !payload.job_key) {
setAdminRefreshStatus("warning", "已有刷新任务正在运行,请稍后再试。", "clock-3");
showToast(payload.message || "已有后台刷新任务正在运行");
return;
}
setStatus(`正在刷新 ${requestedDate} 的行情`);
const job = await waitForAdminRefresh(payload.job_key);
if (job.status === "failed") {
const reason = job.message || job.error_code || "数据源未返回结果";
setAdminRefreshStatus("failure", `刷新失败:${reason}`, "circle-x");
setStatus("后台刷新失败");
showToast("后台刷新失败");
return;
}
const query = new URLSearchParams({ trade_date: requestedDate });
const dashboard = await apiRequest(`/api/dashboard?${query}`);
applyDashboard(dashboard);
const meta = dashboard.meta || {};
const actualDate = String(meta.trade_date || "").slice(0, 10);
const requestedCompact = requestedDate.replaceAll("-", "");
const actualCompact = actualDate.replaceAll("-", "");
const updated = formatTimestamp(meta.updated_at);
const freshness = dashboardFreshnessMessage(meta);
if (meta.realtime && actualCompact === requestedCompact && !meta.carried_forward) {
setAdminRefreshStatus("success", `刷新成功:已获取 ${actualDate} 的盘中行情,更新时间 ${updated}`, "circle-check");
showToast(`刷新成功:已获取 ${actualDate} 的盘中行情`);
return;
}
if (freshness || actualCompact !== requestedCompact || meta.carried_forward || meta.limit_data_source === "derived") {
setAdminRefreshStatus("warning", freshness || `部分正式数据尚未到齐,当前展示 ${actualDate || "最近可用数据"}`, "triangle-alert");
setStatus(freshness || "部分正式数据尚未到齐,当前展示最近可用数据");
return;
}
if (meta.notice) {
setAdminRefreshStatus("warning", `已刷新到 ${actualDate}${updated}),但数据源提示:${meta.notice}`, "triangle-alert");
showToast(`已刷新到 ${actualDate},请留意数据源提示`);
} else {
setAdminRefreshStatus("success", `刷新成功:已获取 ${actualDate} 的最新行情,更新时间 ${updated}`, "circle-check");
showToast(`刷新成功:已获取 ${actualDate} 的最新行情`);
}
} catch (error) {
showToast(error.message || "后台刷新启动失败");
const message = error.message || "后台刷新失败";
setAdminRefreshStatus("failure", `刷新失败:${message}`, "circle-x");
setStatus("后台刷新失败");
showToast(message);
} finally {
buttons.forEach((button) => { button.disabled = false; });
}
}
function setAdminRefreshStatus(tone, message, icon = "circle-dot") {
const status = document.querySelector("#adminRefreshStatus");
if (!status) return;
status.dataset.tone = tone;
status.innerHTML = `<i data-lucide="${icon}"></i><span>${escapeHtml(message)}</span>`;
refreshIcons();
}
async function waitForAdminRefresh(jobKey) {
for (let attempt = 0; attempt < 120; attempt += 1) {
const payload = await apiRequest("/api/admin/settings");
const job = (payload.data?.jobs || []).find((item) => item.idempotency_key === jobKey);
if (job && ["success", "failed"].includes(job.status)) return job;
await new Promise((resolve) => setTimeout(resolve, 1000));
}
throw new Error("刷新等待超时,请稍后重试");
}
let dashboardCatchupTimer = 0;
function chineseMonthDay(value) {
const compact = String(value || "").replaceAll("-", "").replaceAll("/", "");
if (!/^\d{8}/.test(compact)) return "";
return `${Number(compact.slice(4, 6))}${Number(compact.slice(6, 8))}`;
}
function dashboardFreshnessMessage(meta = {}) {
if (meta.display_notice) return String(meta.display_notice);
const requested = String(meta.requested_date || "").replaceAll("-", "");
const actual = String(meta.trade_date || "").replaceAll("-", "");
const shown = chineseMonthDay(actual);
if (meta.data_status === "preparing" || (meta.carried_forward && actual && requested && actual !== requested)) {
return shown ? `今日数据正在准备,当前展示 ${shown}` : "今日数据正在准备,当前展示最近可用数据";
}
if (meta.data_status === "partial" || meta.limit_data_source === "derived") {
return meta.notice || "部分正式数据尚未到齐,当前展示日线推算结果";
}
return "";
}
function scheduleDashboardCatchup(meta = {}) {
window.clearTimeout(dashboardCatchupTimer);
const status = String(meta.data_status || "");
if (status !== "preparing" && status !== "partial") return;
dashboardCatchupTimer = window.setTimeout(() => {
loadDashboard(false, true, false);
}, 60000);
}
function applyDashboard(payload, background = false) {
state.dashboard = payload;
const selectedDate = payload.meta.requested_date || payload.meta.trade_date;
@@ -58,7 +150,11 @@ function applyDashboard(payload, background = false) {
document.querySelector("#qiObservationDate").value = selectedDate;
document.querySelector("#journalDate").value = selectedDate;
renderDashboard();
setStatus(`${dashboardSourceLabel(payload.meta)} · 数据已更新`);
const freshness = dashboardFreshnessMessage(payload.meta || {});
setStatus(freshness || `${dashboardSourceLabel(payload.meta)} · 数据已更新`);
const updatedAt = document.querySelector("#updatedAt");
if (updatedAt) updatedAt.dataset.tone = freshness ? "warning" : "ok";
scheduleDashboardCatchup(payload.meta || {});
if (!background) {
if (state.activeView === "dragonView") loadDragonTiger();
if (state.activeView === "screenerView") loadScreenerSetup();
@@ -126,7 +222,12 @@ function renderDashboard() {
}
}
updateSentimentGauge(overview.sentiment_score);
setText("updatedAt", `${dashboardSourceLabel(meta)} · 更新 ${formatTimestamp(meta.updated_at)}`);
const freshness = dashboardFreshnessMessage(meta);
setText("updatedAt", freshness
? freshness
: `${dashboardSourceLabel(meta)} · 更新 ${formatTimestamp(meta.updated_at)}`);
const updatedAt = document.querySelector("#updatedAt");
if (updatedAt) updatedAt.dataset.tone = freshness ? "warning" : "ok";
renderLimitTable();
renderLadderMini(ladders || []);
+7 -1
View File
@@ -196,7 +196,13 @@ async function changeAccountPassword(event) {
async function switchAccount() {
toggleAccountDropdown(false);
window.location.assign("/login/");
const params = new URLSearchParams();
const next = `${window.location.pathname}${window.location.search}${window.location.hash}`;
if (next.startsWith("/") && !next.startsWith("//") && next !== "/login" && !next.startsWith("/login/") && !next.startsWith("/login?")) {
params.set("next", next);
}
const query = params.toString();
window.location.assign("/login/" + (query ? `?${query}` : ""));
}
+14 -4
View File
@@ -921,6 +921,10 @@ body.sidebar-collapsed .app-main {
text-align: right;
}
.status-bar #updatedAt[data-tone="warning"] {
color: var(--warning);
}
.status-bar .risk-note {
display: block;
@@ -1294,7 +1298,9 @@ body.sidebar-collapsed {
}
.sidebar-brand {
min-height: 55px;
height: var(--size-topbar);
min-height: var(--size-topbar);
display: flex;
@@ -1304,7 +1310,7 @@ body.sidebar-collapsed {
margin: 0px -8px 7px;
padding: 0px 16px;
padding: 0 16px;
border-bottom: 1px solid var(--r2-line-soft);
@@ -2146,13 +2152,17 @@ body.sidebar-collapsed .status-bar {
}
.module-nav .sidebar-brand {
height: var(--size-topbar);
min-height: var(--size-topbar);
display: flex;
align-items: center;
gap: 8px;
padding: 14px 16px;
padding: 0 16px;
border-bottom: 1px solid var(--line-soft);
}
@@ -3494,7 +3504,7 @@ body.sidebar-collapsed .status-bar {
flex: 0 0 auto;
align-items: center;
gap: var(--header-action-gap);
margin-left: 0;
margin-left: auto;
overflow: visible;
}
+29
View File
@@ -147,6 +147,31 @@
--duration-fast: 150ms;
--duration-normal: 220ms;
--login-brand-gradient: linear-gradient(165deg, #0c1e4a, #16307c, #2153cc);
--login-brand-share: 34%;
--login-brand-min: 420px;
--login-brand-cap: 560px;
--login-brand-wide-share: 29.2%;
--login-brand-pad: 36px 40px 24px;
--login-brand-mark-size: 44px;
--login-brand-name-size: 16px;
--login-hero-size: 28px;
--login-card-width: 408px;
--login-card-pad: 32px;
--login-card-title-size: 22px;
--login-account-row-min: 72px;
--login-account-avatar: 40px;
--login-submit-height: 40px;
--login-stat-chip-bg: rgba(8, 12, 24, 0.48);
--login-candle-up: #e07078;
--login-candle-down: #3db88a;
--login-trend-line: rgba(244, 247, 255, 0.88);
--login-mascot-red: #e8605a;
--login-mascot-red-shade: #c74a45;
--login-mascot-green: #46be93;
--login-mascot-green-shade: #37997a;
--login-mascot-eye: #f7f9fc;
--login-mascot-pupil: #1a2440;
--login-mascot-height: clamp(150px, 15vw, 260px);
--font-size-aux: 11.5px;
--font-size-caption: 12.5px;
@@ -516,6 +541,10 @@
--warning-line-strong: #66502d;
--control-shadow: 0 1px 3px rgba(0, 0, 0, .3);
--login-brand-gradient: linear-gradient(165deg, #080c18, #0e1730, #14224a);
--login-stat-chip-bg: rgba(6, 8, 16, 0.58);
--login-candle-up: #f06d73;
--login-candle-down: #43bc8a;
--login-trend-line: rgba(232, 236, 244, 0.9);
--dialog-backdrop: var(--backdrop);
--ladder-level-1: #2d2426;
--ladder-level-2: #2b2822;
+96
View File
@@ -3868,3 +3868,99 @@ test("desktop header keeps commands in view and tape text unclipped across works
await page.screenshot({ path: path.join(shotDir, "admin-390-night.png") });
fs.writeFileSync(path.join(shotDir, "measurements.json"), `${JSON.stringify(measurements, null, 2)}\n`);
});
test("HEL-183 heaven tools right-align and shell heights unify", async ({ page }) => {
const fs = require("node:fs");
const path = require("node:path");
const shotDir = path.join(__dirname, "../../runtime/hel183-shots");
fs.mkdirSync(shotDir, { recursive: true });
await page.setViewportSize({ width: 1440, height: 900 });
await mockApplication(page, session("admin", true));
await page.goto("/index.html");
const measure = () => page.evaluate(() => {
const box = (node) => {
if (!node) return null;
const r = node.getBoundingClientRect();
return { x: r.x, y: r.y, right: r.right, width: r.width, height: r.height };
};
const brand = document.querySelector(".module-nav .sidebar-brand") || document.querySelector(".sidebar-brand");
const header = document.querySelector(".app-header");
const actions = document.querySelector(".header-actions");
const collapse = document.querySelector(".sidebar-collapse-button");
const status = document.querySelector(".status-bar");
const overview = document.querySelector(".overview-strip");
const brandBox = box(brand);
const headerBox = box(header);
const actionsBox = box(actions);
const collapseBox = box(collapse);
const statusBox = box(status);
return {
brandHeight: brandBox ? Math.round(brandBox.height) : null,
headerHeight: headerBox ? Math.round(headerBox.height) : null,
brandBottom: brandBox ? Math.round(brandBox.y + brandBox.height) : null,
headerBottom: headerBox ? Math.round(headerBox.y + headerBox.height) : null,
collapseHeight: collapseBox ? Math.round(collapseBox.height) : null,
statusHeight: statusBox ? Math.round(statusBox.height) : null,
actionsNearRight: actionsBox && headerBox ? (headerBox.right - actionsBox.right) < 24 : false,
actionsMarginLeft: actions ? getComputedStyle(actions).marginLeft : null,
overviewDisplay: overview ? getComputedStyle(overview).display : null,
mentorPadTop: (() => {
const mentor = document.querySelector("#mentorView");
return mentor ? getComputedStyle(mentor).paddingTop : null;
})(),
};
});
await page.locator('[data-view="sentimentCycleView"]').first().click();
await expect(page.locator("#sentimentCycleView")).toHaveClass(/active-view/);
let geo = await measure();
expect(geo.brandHeight, "logo height").toBe(64);
expect(geo.headerHeight, "header height").toBe(64);
expect(geo.brandBottom, "logo/header bottom align").toBe(geo.headerBottom);
expect(geo.collapseHeight, "collapse height").toBe(28);
expect(geo.statusHeight, "status height").toBe(28);
expect(geo.actionsNearRight, "sentiment tools right").toBe(true);
await page.locator(".app-header").screenshot({ path: path.join(shotDir, "sentiment-header-day.png") });
await page.screenshot({ path: path.join(shotDir, "sentiment-page-day.png") });
await page.locator('[data-view="heavenView"]').first().click();
await expect(page.locator("#heavenView")).toHaveClass(/active-view/);
await expect(page.locator("#heavenView .heaven-page-head")).toBeVisible();
await expect(page.locator("#heavenView .heaven-page-head .segment")).toHaveCount(3);
geo = await measure();
expect(geo.overviewDisplay, "heaven hides overview").toBe("none");
expect(geo.actionsMarginLeft, "tools margin-left resolved").not.toBe("0px");
expect(Number.parseFloat(geo.actionsMarginLeft), "tools left auto gap").toBeGreaterThan(40);
expect(geo.actionsNearRight, "heaven tools right").toBe(true);
expect(geo.brandHeight).toBe(64);
expect(geo.collapseHeight).toBe(28);
await page.locator(".app-header").screenshot({ path: path.join(shotDir, "heaven-header-day.png") });
await page.screenshot({ path: path.join(shotDir, "heaven-page-day.png") });
await page.locator('[data-heaven-panel="fortune"]').click();
await expect(page.locator('[data-heaven-panel="fortune"]')).toHaveClass(/active/);
await expect(page.locator("#heavenFortunePanel")).toHaveClass(/active-heaven-panel/);
await page.locator('[data-view="mentorView"]').first().click();
await expect(page.locator("#mentorView")).toHaveClass(/active-view/);
geo = await measure();
expect(geo.mentorPadTop, "mentor top padding").toBe("14px");
await page.screenshot({ path: path.join(shotDir, "mentor-page-day.png") });
await page.locator("#themeToggle").click();
await page.locator('[data-view="heavenView"]').first().click();
await expect(page.locator("#heavenView")).toHaveClass(/active-view/);
geo = await measure();
expect(geo.actionsNearRight, "heaven night tools right").toBe(true);
expect(geo.brandHeight).toBe(64);
await page.locator(".app-header").screenshot({ path: path.join(shotDir, "heaven-header-night.png") });
await page.screenshot({ path: path.join(shotDir, "heaven-page-night.png") });
await page.locator('[data-view="sentimentCycleView"]').first().click();
await page.locator(".app-header").screenshot({ path: path.join(shotDir, "sentiment-header-night.png") });
await page.screenshot({ path: path.join(shotDir, "sentiment-page-night.png") });
await page.locator('[data-view="mentorView"]').first().click();
await page.screenshot({ path: path.join(shotDir, "mentor-page-night.png") });
});
+407 -2
View File
@@ -1,4 +1,9 @@
const { test, expect } = require("@playwright/test");
const fs = require("fs");
const path = require("path");
const SHOT_DIR = path.resolve(__dirname, "../../../verify-shots");
fs.mkdirSync(SHOT_DIR, { recursive: true });
function loginPayload(user) {
return {
@@ -54,6 +59,15 @@ async function mockLoginPortal(page, options = {}) {
return;
}
if ((url.pathname === "/api/auth/login" || url.pathname === "/api/auth/register") && method === "POST") {
if (options.loginDelay) await new Promise((resolve) => setTimeout(resolve, options.loginDelay));
if (options.loginFails) {
await route.fulfill({
status: 401,
contentType: "application/json",
body: JSON.stringify({ error: "账号名或密码不正确,请重新输入。" }),
});
return;
}
await route.fulfill({
status: 200,
contentType: "application/json",
@@ -67,14 +81,58 @@ async function mockLoginPortal(page, options = {}) {
return;
}
if (url.pathname === "/api/auth/me") {
const current = accounts.find((item) => Number(item.user_id) === Number(currentUserId)) || null;
const authenticated = Boolean(current) && !options.sessionExpired;
await route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({
ok: true,
authenticated: Boolean(currentUserId),
authenticated,
csrf_token: "portal-csrf",
user: accounts.find((item) => Number(item.user_id) === Number(currentUserId)) || null,
user: authenticated ? {
id: current.user_id,
username: current.username,
role: current.role,
membership: current.membership,
} : null,
}),
});
return;
}
if (url.pathname === "/api/dashboard") {
await route.fulfill({
status: 200,
contentType: "application/json",
body: JSON.stringify({
ok: true,
meta: {
trade_date: "2026-07-22",
requested_date: "2026-07-22",
source: "tushare",
realtime: false,
cached: true,
market_status: "closed",
updated_at: "2026-07-22T15:00:00+08:00",
},
overview: {
up_count: 2100,
down_count: 2800,
limit_up_count: 42,
limit_down_count: 8,
broken_count: 17,
seal_rate: 71.2,
amount_billion: 12600,
sentiment_score: 48,
},
limits: [],
broken: [],
down_limits: [],
yesterday_limits: [],
limit_performance: [],
ladders: [],
sectors: [],
sector_rotation: [],
}),
});
return;
@@ -149,3 +207,350 @@ test("managing accounts removes a local record after inline confirmation", async
await expect(page.locator(".login-account-row")).toHaveCount(1);
await expect(page.locator(".login-account-row")).toContainText("alpha_user");
});
async function assertConfirmedSkeleton(page, { width, height }) {
await expect(page.locator(".login-brand-title")).toHaveText("看懂情绪周期,把复盘变成下一次的先手。");
await expect(page.locator(".login-brand-header")).toBeVisible();
await expect(page.locator(".login-brand-chart")).toBeVisible();
await expect(page.locator(".login-brand-stats")).toBeVisible();
await expect(page.locator(".login-brand-kicker")).toHaveText("收盘之后 · 复盘开始");
const brand = await page.locator(".login-brand").boundingBox();
const header = await page.locator(".login-brand-header").boundingBox();
const mark = await page.locator(".login-brand-mark").boundingBox();
const name = await page.locator(".login-brand-name").boundingBox();
const title = await page.locator(".login-brand-title").boundingBox();
const stats = await page.locator(".login-brand-stats").boundingBox();
const chart = await page.locator(".login-brand-chart").boundingBox();
const card = await page.locator(".login-card").boundingBox();
expect(brand).toBeTruthy();
expect(header.y - brand.y).toBeLessThan(48);
expect(Math.abs(mark.y - name.y)).toBeLessThan(16);
expect(title.y).toBeGreaterThan(height * 0.28);
expect(title.y).toBeLessThan(height * 0.72);
expect(stats.y).toBeGreaterThan(height * 0.55);
expect(chart.height).toBeGreaterThan(80);
await expect(page.locator("#loginMascots")).toBeVisible();
expect(card.width).toBeGreaterThan(380);
expect(card.width).toBeLessThan(450);
if (width === 1440) {
expect(brand.width).toBeGreaterThan(470);
expect(brand.width).toBeLessThan(520);
expect(brand.height).toBe(height);
const mascots = await page.locator("#loginMascots").boundingBox();
const kicker = await page.locator(".login-brand-kicker").boundingBox();
expect(mascots).toBeTruthy();
expect(kicker).toBeTruthy();
expect(mascots.y).toBeGreaterThan(header.y + header.height - 4);
expect(mascots.y + mascots.height).toBeLessThan(kicker.y + 8);
const gapTop = header.y + header.height;
const gapBottom = kicker.y;
const mid = (gapTop + gapBottom) / 2;
const mascotMid = mascots.y + mascots.height / 2;
expect(Math.abs(mascotMid - mid)).toBeLessThan(48);
expect(title.y).toBeGreaterThan(470);
expect(title.y).toBeLessThan(580);
} else if (width === 1920) {
expect(brand.width).toBeGreaterThan(540);
expect(brand.width).toBeLessThan(580);
} else {
expect(brand.width).toBeGreaterThan(560);
}
}
async function openPortal(page, { theme, width, height, accounts, currentUserId, loginFails, loginDelay }) {
await page.addInitScript((nextTheme) => {
localStorage.setItem("xiaobaiTheme", nextTheme);
}, theme);
await page.setViewportSize({ width, height });
await mockLoginPortal(page, { accounts, currentUserId, loginFails, loginDelay });
await page.goto("/login/");
}
for (const theme of ["light", "dark"]) {
for (const [width, height] of [[1440, 900], [1920, 1080]]) {
test(`confirmed skeleton ${theme} ${width}x${height}`, async ({ page }) => {
await openPortal(page, { theme, width, height, accounts: [] });
await assertConfirmedSkeleton(page, { width, height });
await expect(page.locator("#loginThemeToggle")).toHaveText(theme === "dark" ? "☀ 日间" : "🌙 夜间");
await page.screenshot({ path: path.join(SHOT_DIR, `first-${theme}-${width}.png`), fullPage: true });
});
}
}
test("ultrawide keeps the left brand from collapsing into a strip", async ({ page }) => {
await openPortal(page, { theme: "dark", width: 2560, height: 1080, accounts: [] });
await assertConfirmedSkeleton(page, { width: 2560, height: 1080 });
});
test("picker add remove error and loading share the same desktop skeleton", async ({ page }) => {
const accounts = SAVED_ACCOUNTS.map((item) => ({ ...item }));
await openPortal(page, {
theme: "dark",
width: 1440,
height: 900,
accounts,
currentUserId: 1,
});
await assertConfirmedSkeleton(page, { width: 1440, height: 900 });
await expect(page.locator(".login-card-title")).toHaveText("选择账号");
await expect(page.locator(".login-avatar")).toHaveCount(2);
await expect(page.locator(".login-add")).toBeVisible();
await page.screenshot({ path: path.join(SHOT_DIR, "picker-dark-1440.png"), fullPage: true });
await page.locator('[data-login-action="add"]').click();
await expect(page.locator(".login-card-title")).toHaveText("添加账号");
await assertConfirmedSkeleton(page, { width: 1440, height: 900 });
await page.screenshot({ path: path.join(SHOT_DIR, "add-dark-1440.png"), fullPage: true });
await page.locator('[data-login-action="picker"]').click();
await page.locator('[data-login-action="manage"]').click();
await expect(page.locator(".login-card-title")).toHaveText("管理账号记录");
await page.locator('[data-confirm-id="2"]').click();
await expect(page.locator(".login-confirm-copy")).toContainText("beta_user");
await page.screenshot({ path: path.join(SHOT_DIR, "remove-dark-1440.png"), fullPage: true });
});
test("login failure and loading keep the confirmed first-login skeleton", async ({ page }) => {
await openPortal(page, {
theme: "light",
width: 1440,
height: 900,
accounts: [],
loginFails: true,
});
await page.locator("#loginUsername").fill("baiqizhi");
await page.locator("#loginPassword").fill("wrong-password");
await page.locator(".login-submit").click();
await expect(page.locator(".login-error")).toContainText("账号名或密码不正确");
await expect(page.locator("#loginPassword")).toHaveClass(/is-invalid/);
await assertConfirmedSkeleton(page, { width: 1440, height: 900 });
await page.screenshot({ path: path.join(SHOT_DIR, "error-light-1440.png"), fullPage: true });
});
test("loading button appears on the confirmed first-login skeleton", async ({ page }) => {
await openPortal(page, {
theme: "dark",
width: 1440,
height: 900,
accounts: [],
loginDelay: 2500,
});
await page.evaluate(() => {
window.location.replace = () => {};
});
await page.locator("#loginUsername").fill("baiqizhi");
await page.locator("#loginPassword").fill("password12");
const submit = page.locator(".login-submit").click();
await expect(page.locator(".login-submit")).toContainText("正在登录...");
await expect(page.locator(".login-spinner")).toBeVisible();
await assertConfirmedSkeleton(page, { width: 1440, height: 900 });
await page.screenshot({ path: path.join(SHOT_DIR, "loading-dark-1440.png"), fullPage: true });
await submit;
});
async function openPicker(page, options = {}) {
const accounts = options.accounts || SAVED_ACCOUNTS.map((item) => ({ ...item }));
const currentUserId = options.currentUserId ?? 1;
const next = options.next || "/index.html?view=sentimentCycleView";
await page.unroute("**/api/**").catch(() => {});
await mockLoginPortal(page, {
accounts,
currentUserId,
sessionExpired: options.sessionExpired,
switchFails: options.switchFails,
});
await page.goto(`/login/?next=${encodeURIComponent(next)}`);
await expect(page.locator(".login-card-title")).toHaveText("选择账号");
}
test("clicking the current account from two workspace pages returns without switching", async ({ page }) => {
const views = ["sentimentCycleView", "ladderView"];
for (const viewId of views) {
const next = `/index.html?view=${viewId}`;
const switchCalls = [];
const onRequest = (request) => {
if (request.url().includes("/api/auth/switch") && request.method() === "POST") {
switchCalls.push(request);
}
};
page.on("request", onRequest);
await openPicker(page, { next });
await expect(page.locator('[data-resume-id="1"]')).toContainText("继续使用");
await expect(page.locator('[data-resume-id="1"]')).toContainText("当前");
await page.locator('[data-resume-id="1"]').click();
await expect(page).toHaveURL(new RegExp(`[?&]view=${viewId}\\b`));
expect(switchCalls).toEqual([]);
page.off("request", onRequest);
}
});
test("a lone current account can return from the picker instead of dead-ending", async ({ page }) => {
await openPicker(page, {
accounts: [SAVED_ACCOUNTS[0]],
currentUserId: 1,
next: "/index.html?view=reviewWorkspaceView",
});
await expect(page.locator(".login-account-row")).toHaveCount(1);
await expect(page.locator('[data-switch-id]')).toHaveCount(0);
await page.locator('[data-resume-id="1"]').click();
await expect(page).toHaveURL(/view=reviewWorkspaceView/);
});
test("the return control also restores the originating workspace page", async ({ page }) => {
await openPicker(page, { next: "/index.html?view=ladderView" });
await page.locator('[data-login-action="resume"]').click();
await expect(page).toHaveURL(/view=ladderView/);
});
test("refreshing the picker still returns to the originating page", async ({ page }) => {
await openPicker(page, { next: "/index.html?view=sentimentCycleView" });
await page.reload();
await expect(page.locator(".login-card-title")).toHaveText("选择账号");
await page.locator('[data-resume-id="1"]').click();
await expect(page).toHaveURL(/view=sentimentCycleView/);
});
test("an expired current session asks for login instead of pretending to return", async ({ page }) => {
await openPicker(page, {
next: "/index.html?view=sentimentCycleView",
sessionExpired: true,
});
await page.locator('[data-resume-id="1"]').click();
await expect(page.locator(".login-error")).toHaveText("当前会话已失效,请重新登录");
await expect(page).toHaveURL(/\/login\/?/);
});
test("other saved accounts still switch while the current row only resumes", async ({ page }) => {
await openPicker(page, { next: "/index.html?view=auctionView" });
const switched = page.waitForRequest((request) => (
request.url().includes("/api/auth/switch") && request.method() === "POST"
));
await page.locator('[data-switch-id="2"]').click();
const request = await switched;
expect(JSON.parse(request.postData() || "{}")).toEqual({ user_id: 2 });
});
test("workspace switch-account menu carries the current page back to the picker", async ({ page }) => {
await mockLoginPortal(page, {
accounts: SAVED_ACCOUNTS.map((item) => ({ ...item })),
currentUserId: 1,
});
await page.goto("/index.html?view=sentimentCycleView");
await expect(page.locator("#accountButton")).toBeVisible();
await page.locator("#accountButton").click();
await page.locator("#switchAccountMenuButton").click();
await expect(page).toHaveURL(/\/login\/\?next=/);
await expect(page.locator(".login-card-title")).toHaveText("选择账号");
await page.locator('[data-resume-id="1"]').click();
await expect(page).toHaveURL(/view=sentimentCycleView/);
});
test("workspace switch-account from a second page also returns to that page", async ({ page }) => {
await mockLoginPortal(page, {
accounts: SAVED_ACCOUNTS.map((item) => ({ ...item })),
currentUserId: 1,
});
await page.goto("/index.html?view=ladderView");
await expect(page.locator("#accountButton")).toBeVisible();
await page.locator("#accountButton").click();
await page.locator("#switchAccountMenuButton").click();
await expect(page.locator(".login-card-title")).toHaveText("选择账号");
await page.locator('[data-resume-id="1"]').click();
await expect(page).toHaveURL(/view=ladderView/);
});
test("desktop mascots react to account focus, password, toggle, loading, success and failure", async ({ page }) => {
await openPortal(page, { theme: "light", width: 1440, height: 900, accounts: [] });
const mascots = page.locator("#loginMascots");
await expect(mascots).toHaveAttribute("data-mood", "idle");
await page.locator("#loginUsername").focus();
await expect(mascots).toHaveAttribute("data-mood", "account");
await page.locator("#loginPassword").focus();
await expect(mascots).toHaveAttribute("data-mood", "password");
await expect.poll(async () => (
page.locator(".login-mascot.is-red .login-mascot-hand.is-left").evaluate((node) => getComputedStyle(node).opacity)
)).toBe("1");
await page.locator(".login-password-toggle").click();
await expect(page.locator("#loginPassword")).toHaveAttribute("type", "text");
await expect(mascots).toHaveAttribute("data-mood", "password");
await page.locator(".login-password-toggle").click();
await expect(page.locator("#loginPassword")).toHaveAttribute("type", "password");
await expect(mascots).toHaveAttribute("data-mood", "password");
await page.screenshot({ path: path.join(SHOT_DIR, "mascots-password-light-1440.png"), fullPage: true });
});
test("login loading and failure drive short mascot feedback", async ({ page }) => {
await openPortal(page, {
theme: "dark",
width: 1440,
height: 900,
accounts: [],
loginFails: true,
loginDelay: 800,
});
await page.locator("#loginUsername").fill("baiqizhi");
await page.locator("#loginPassword").fill("wrong-password");
const submit = page.locator(".login-submit").click();
await expect(page.locator("#loginMascots")).toHaveAttribute("data-mood", "busy");
await expect(page.locator(".login-error")).toContainText("账号名或密码不正确");
await expect(page.locator("#loginMascots")).toHaveAttribute("data-mood", "fail");
await page.screenshot({ path: path.join(SHOT_DIR, "mascots-fail-dark-1440.png"), fullPage: true });
await submit;
});
test("login success plays a hop before leaving the portal", async ({ page }) => {
await openPortal(page, { theme: "light", width: 1440, height: 900, accounts: [] });
await page.evaluate(() => {
window.location.replace = () => {};
});
await page.locator("#loginUsername").fill("baiqizhi");
await page.locator("#loginPassword").fill("password12");
const submit = page.locator(".login-submit").click();
await expect(page.locator("#loginMascots")).toHaveAttribute("data-mood", /busy|success/);
await expect(page.locator("#loginMascots")).toHaveAttribute("data-mood", "success", { timeout: 4000 });
await page.screenshot({ path: path.join(SHOT_DIR, "mascots-success-light-1440.png"), fullPage: true });
await submit;
});
test("reduced motion keeps static mascots without mouse tracking", async ({ page }) => {
await page.emulateMedia({ reducedMotion: "reduce" });
await openPortal(page, { theme: "dark", width: 1440, height: 900, accounts: [] });
const mascots = page.locator("#loginMascots");
await expect(mascots).toBeVisible();
await expect(mascots).toHaveAttribute("data-mood", "idle");
const before = await mascots.evaluate((node) => getComputedStyle(node.querySelector(".login-mascot.is-red")).getPropertyValue("--pupil-x"));
await page.mouse.move(1200, 120);
await page.waitForTimeout(120);
const after = await mascots.evaluate((node) => getComputedStyle(node.querySelector(".login-mascot.is-red")).getPropertyValue("--pupil-x"));
expect(after).toBe(before);
await page.locator("#loginPassword").focus();
await expect(mascots).toHaveAttribute("data-mood", "password");
await page.screenshot({ path: path.join(SHOT_DIR, "mascots-reduced-dark-1440.png"), fullPage: true });
});
test("mouse follow updates mascot pupils on a fine pointer", async ({ page }) => {
await openPortal(page, { theme: "light", width: 1440, height: 900, accounts: [] });
const mascots = page.locator("#loginMascots");
await page.mouse.move(80, 160);
await page.waitForTimeout(180);
const left = await mascots.evaluate((node) => getComputedStyle(node.querySelector(".login-mascot.is-red")).getPropertyValue("--pupil-x"));
await page.mouse.move(1280, 200);
await page.waitForTimeout(180);
const right = await mascots.evaluate((node) => getComputedStyle(node.querySelector(".login-mascot.is-red")).getPropertyValue("--pupil-x"));
expect(Number.parseFloat(right)).toBeGreaterThan(Number.parseFloat(left));
});
test("narrow screens hide mascots without moving the login card", async ({ page }) => {
await openPortal(page, { theme: "light", width: 800, height: 900, accounts: [] });
await expect(page.locator("#loginMascots")).toBeHidden();
await expect(page.locator(".login-card-title")).toHaveText("欢迎回来");
await expect(page.locator(".login-brand-title")).toBeHidden();
});
test("theme toggle keeps mascots in the brand gap", async ({ page }) => {
await openPortal(page, { theme: "light", width: 1440, height: 900, accounts: [] });
await page.locator("#loginThemeToggle").click();
await expect(page.locator("html")).toHaveAttribute("data-theme", "dark");
await assertConfirmedSkeleton(page, { width: 1440, height: 900 });
await page.screenshot({ path: path.join(SHOT_DIR, "mascots-idle-dark-1440.png"), fullPage: true });
});
+73 -6
View File
@@ -138,6 +138,11 @@ async function mockMobileApi(page, options = {}) {
payload = { items: [] };
} else if (path === "/api/search") {
payload = { groups: { stocks: [{ id: "002141", code: "002141", name: "贤丰控股", type: "stock", industry: "电子元件" }], sectors: [], themes: [], indices: [] } };
} else if (path === "/api/auth/accounts") {
payload = {
accounts: [{ user_id: auth.user.id, username: auth.user.username, role: auth.user.role, last_used_at: "2026-07-22T09:12:00+08:00" }],
current_user_id: auth.user.id,
};
} else if (path === "/api/account/status") {
payload = {
birth_profile_configured: true,
@@ -284,12 +289,16 @@ test("mobile login renders before authentication", async ({ page }) => {
test("four hub pages render their icon grids", async ({ page }) => {
await mockMobileApi(page);
await openMobile(page);
for (const hub of ["market", "tools", "review", "system"]) {
for (const hub of ["market", "tools", "review"]) {
await page.evaluate((h) => { window.MobileRouter.navigate("#/hub/" + h); }, hub);
await expect(page.locator(".m-hub-grid")).toBeVisible();
await expect(page.locator(".m-hub-grid .m-grid-item").first()).toBeVisible();
expect(await measureOverflow(page)).toBeLessThanOrEqual(1);
}
await page.evaluate(() => { window.MobileRouter.navigate("#/hub/system"); });
await expect(page.locator("[data-system-page='home']")).toBeVisible();
await expect(page.locator(".m-sys-row").first()).toBeVisible();
expect(await measureOverflow(page)).toBeLessThanOrEqual(1);
});
for (const theme of ["day", "night"]) {
@@ -344,7 +353,7 @@ test("system management pages render real content instead of placeholders", asyn
}
await navigateToFeature(page, "system/profile");
await expect(page.locator("#m-sys-birth-date")).toBeVisible();
await expect(page.locator("[data-system-switch]")).toBeVisible();
await expect(page.locator("[data-system-save-birth]")).toBeVisible();
await navigateToFeature(page, "system/password");
await expect(page.locator("#m-sys-password-current")).toBeVisible();
await navigateToFeature(page, "system/membership");
@@ -355,7 +364,64 @@ test("system management pages render real content instead of placeholders", asyn
await expect(page.locator("#m-sys-member-limit")).toBeVisible();
});
test("empty profile save click shows a toast instead of a dead button", async ({ page }) => {
test("system home groups entries and keeps admin-only items gated", async ({ page }) => {
await mockMobileApi(page);
await openMobile(page);
await page.evaluate(() => { window.MobileRouter.navigate("#/hub/system"); });
await expect(page.locator("[data-system-page='home']")).toBeVisible();
await expect(page.locator("#m-view")).toContainText("账号");
await expect(page.locator("#m-view")).toContainText("偏好");
await expect(page.locator("#m-view")).toContainText("管理员专区");
await expect(page.locator("[data-theme-toggle]")).toBeVisible();
await expect(page.locator("[data-system-switch]")).toBeVisible();
expect(await measureOverflow(page)).toBeLessThanOrEqual(1);
});
test("system settings tabs, model editor, delete confirm and theme toggle work", async ({ page }) => {
await mockMobileApi(page);
await openMobile(page);
await navigateToFeature(page, "system/admin");
await expect(page.locator("[data-system-admin-panel='market']")).toBeVisible();
await page.locator("[data-system-admin-tab='models']").click();
await expect(page.locator("[data-system-admin-panel='models']")).toBeVisible();
await page.locator("[data-system-edit-model]").first().click();
await expect(page.locator(".m-sheet-root.is-open")).toBeVisible();
await expect(page.locator(".m-sheet-head h2")).toHaveText("编辑模型");
await page.locator("[data-sheet-close]").click();
await page.locator("[data-system-admin-tab='market']").click();
await expect(page.locator("#m-sys-token")).toBeVisible();
await navigateToFeature(page, "system/profile");
await page.locator("[data-system-delete-birth]").click();
await expect(page.locator(".m-dialog")).toBeVisible();
await expect(page.locator(".m-dialog")).toContainText("删除命理资料");
await page.locator("[data-sheet-close]").click();
await page.evaluate(() => { window.MobileRouter.navigate("#/hub/system"); });
await expect(page.locator("[data-theme-toggle]")).toBeVisible();
const before = await page.locator("#m-app").getAttribute("data-theme");
await page.locator("[data-theme-toggle]").click();
await expect.poll(async () => page.locator("#m-app").getAttribute("data-theme")).not.toBe(before);
await navigateToFeature(page, "system/members");
await page.locator("[data-system-open-member]").click();
await expect(page.locator(".m-sheet-root.is-open")).toBeVisible();
await expect(page.locator(".m-sheet-head h2")).toContainText("管理会员");
});
test("password mismatch shows inline error instead of a silent submit", async ({ page }) => {
await mockMobileApi(page);
await openMobile(page);
await navigateToFeature(page, "system/password");
await page.locator("#m-sys-password-current").fill("OldPass12");
await page.locator("#m-sys-password-new").fill("NewPass123");
await page.locator("#m-sys-password-confirm").fill("OtherPass123");
await page.locator("[data-system-save-password]").click();
await expect(page.locator("[data-field-error='confirm']")).toBeVisible();
await expect(page.locator("[data-field-error='confirm']")).toContainText("两次输入的密码不一致");
});
test("empty birth profile save shows a validation toast", async ({ page }) => {
await mockMobileApi(page);
await openMobile(page);
await navigateToFeature(page, "system/profile");
@@ -371,9 +437,10 @@ test("non-admin cannot open system admin pages as placeholders", async ({ page }
await mockMobileApi(page, { auth: authSession("user", true) });
await openMobile(page);
await page.evaluate(() => { window.MobileRouter.navigate("#/hub/system"); });
await expect(page.locator(".m-hub-grid")).toBeVisible();
await expect(page.locator('.m-grid-item[data-route="#/feature/system/admin"]')).toHaveCount(0);
await expect(page.locator('.m-grid-item[data-route="#/feature/system/members"]')).toHaveCount(0);
await expect(page.locator("[data-system-page='home']")).toBeVisible();
await expect(page.locator('[data-route="#/feature/system/admin"]')).toHaveCount(0);
await expect(page.locator('[data-route="#/feature/system/members"]')).toHaveCount(0);
await expect(page.locator("[data-system-switch]")).toBeVisible();
await navigateToFeature(page, "system/admin");
await expect(page.locator("#m-view")).not.toContainText(PLACEHOLDER_COPY);
await expect(page.locator("[data-system-page='forbidden']")).toBeVisible();
+453
View File
@@ -0,0 +1,453 @@
from __future__ import annotations
import copy
import threading
import unittest
from datetime import date, datetime, timedelta, timezone, time as dt_time
from unittest.mock import patch
from pathlib import Path
from backend.features.market.service import MarketServiceMixin
from backend.jobs.refresh import (
dashboard_has_usable_data,
official_catchup_due,
verified_dashboard_result,
)
from backend.data.providers.tushare_transport import TushareError
class AdminRefreshStatusTests(unittest.TestCase):
def test_carried_snapshot_is_usable_not_failed_job(self):
result = verified_dashboard_result(
{
"meta": {
"trade_date": "2026-09-01",
"requested_date": "2026-09-02",
"carried_forward": True,
"notice": "今日数据正在准备,当前展示 9 月 1 日",
"data_status": "preparing",
},
"overview": {"limit_up_count": 12},
}
)
self.assertNotEqual(result.get("status"), "failed")
self.assertEqual(result["meta"]["data_status"], "preparing")
self.assertTrue(dashboard_has_usable_data(result))
def test_derived_limit_snapshot_is_usable_not_failed_job(self):
dashboard = {
"meta": {
"trade_date": "2026-09-02",
"limit_data_source": "derived",
"notice": "涨跌停高级接口当日数据尚未更新,已使用日线数据推算。",
"data_status": "partial",
},
"overview": {"limit_up_count": 8},
}
self.assertIs(verified_dashboard_result(dashboard), dashboard)
def test_current_snapshot_is_reported_as_successful_job(self):
dashboard = {"meta": {"trade_date": "2026-08-28", "carried_forward": False}}
self.assertIs(verified_dashboard_result(dashboard), dashboard)
def test_empty_payload_is_still_failed(self):
result = verified_dashboard_result({"meta": {}, "overview": {}})
self.assertEqual(result["status"], "failed")
class FakeSyncDatabase:
def __init__(self, latest=None):
self.latest = latest
self.saved = []
self.finished = []
def start_sync(self, *_args, **_kwargs):
return 1
def save_snapshot(self, trade_date, source, payload):
self.saved.append((trade_date, source, copy.deepcopy(payload)))
def save_data_snapshot(self, *_args, **_kwargs):
return None
def finish_sync(self, *args, **kwargs):
self.finished.append((args, kwargs))
def get_latest_real_snapshot(self, *_args, **_kwargs):
return copy.deepcopy(self.latest)
def get_snapshot(self, *_args, **_kwargs):
return None
def get_data_snapshot(self, *_args, **_kwargs):
return None
def reason_overrides(self, *_args, **_kwargs):
return {}
class FakeDerivedClient:
def dashboard(self, trade_date: str):
return {
"meta": {
"trade_date": f"{trade_date[:4]}-{trade_date[4:6]}-{trade_date[6:8]}",
"limit_data_source": "derived",
"notice": "涨跌停高级接口当日数据尚未更新,已使用日线数据推算。",
"updated_at": datetime.now().astimezone().isoformat(timespec="seconds"),
},
"overview": {"limit_up_count": 3},
"limits": [{"code": "000001"}],
"broken": [],
"down_limits": [],
"yesterday_limits": [],
}
SHANGHAI = timezone(timedelta(hours=8))
TRADE_DAY = date(2026, 9, 8)
def at_clock(hour: int, minute: int, day: date = TRADE_DAY) -> datetime:
return datetime(day.year, day.month, day.day, hour, minute, tzinfo=SHANGHAI)
class FakeMissingDailyClient:
def __init__(self, open_today: bool = True):
self.open_today = open_today
def dashboard(self, trade_date: str):
raise TushareError(f"No daily data returned for {trade_date}")
def resolve_trade_context(self, requested: str):
if self.open_today:
return requested, "20260907"
return "20260907", "20260904"
class FakeRealtimeTodayClient:
def dashboard(self, trade_date: str):
return {
"meta": {
"trade_date": f"{trade_date[:4]}-{trade_date[4:6]}-{trade_date[6:8]}",
"requested_date": f"{trade_date[:4]}-{trade_date[4:6]}-{trade_date[6:8]}",
"realtime": True,
"mode": "realtime",
"market_status": "trading",
"notice": "盘中行情由 Tushare rt_k 实时计算;涨停原因、封板时间和开板次数以盘后榜单校正为准。",
"updated_at": datetime.now().astimezone().isoformat(timespec="seconds"),
},
"overview": {"limit_up_count": 15},
"limits": [{"code": "000001"}],
"broken": [],
"down_limits": [],
"yesterday_limits": [],
}
def resolve_trade_context(self, requested: str):
return requested, "20260907"
class FakeFreeRealtimeTodayClient:
def dashboard(self, trade_date: str):
return {
"meta": {
"trade_date": f"{trade_date[:4]}-{trade_date[4:6]}-{trade_date[6:8]}",
"requested_date": f"{trade_date[:4]}-{trade_date[4:6]}-{trade_date[6:8]}",
"realtime": True,
"mode": "realtime",
"quote_source": "eastmoney_clist",
"source": "eastmoney",
"market_status": "trading",
"notice": "盘中行情由东财免费实时快照计算;涨停原因、封板时间和开板次数以盘后榜单校正为准。",
"updated_at": datetime.now().astimezone().isoformat(timespec="seconds"),
"indices": [{"code": "000001", "price": 3800.1, "change": 0.5}],
},
"overview": {"limit_up_count": 18, "up_count": 2100, "amount_billion": 12345.6},
"limits": [{"code": "000001"}],
"broken": [],
"down_limits": [],
"yesterday_limits": [],
}
def resolve_trade_context(self, requested: str):
return requested, "20260907"
class SyncHarness(MarketServiceMixin):
def __init__(self, client, latest=None, clock=None):
self.configured = True
self.sync_lock = threading.Lock()
self.database = FakeSyncDatabase(latest)
self._client = client
self.current_user_id = 1
self.clock = clock
def _tushare_client(self):
return self._client
def _enrich_dashboard_sentiment(self, dashboard, _trade_date):
return dashboard
def _apply_reason_overrides(self, dashboard):
return dashboard
class DashboardFreshnessTests(unittest.TestCase):
def test_derived_limits_are_kept_as_partial_success(self):
today = date.today().strftime("%Y%m%d")
harness = SyncHarness(FakeDerivedClient())
payload = harness.sync_dashboard(today)
meta = payload["meta"]
self.assertEqual(meta["limit_data_source"], "derived")
self.assertEqual(meta["data_status"], "partial")
self.assertFalse(meta.get("carried_forward"))
self.assertIn("日线数据推算", meta["display_notice"])
self.assertEqual(harness.database.finished[0][0][1], "success")
self.assertEqual(verified_dashboard_result(payload), payload)
def test_intraday_refresh_keeps_today_and_does_not_fall_back_to_yesterday(self):
today = TRADE_DAY.strftime("%Y%m%d")
latest = {
"meta": {"trade_date": "2026-09-07", "source": "tushare"},
"overview": {"limit_up_count": 20},
}
harness = SyncHarness(
FakeRealtimeTodayClient(),
latest,
clock=lambda: at_clock(10, 5),
)
payload = harness.sync_dashboard(today)
meta = payload["meta"]
self.assertFalse(meta.get("carried_forward"))
self.assertTrue(meta["realtime"])
self.assertEqual(meta["data_status"], "intraday")
self.assertEqual(str(meta["trade_date"]).replace("-", ""), today)
self.assertNotIn("今日数据正在准备", meta.get("display_notice") or "")
self.assertEqual(harness.database.saved[0][0], today)
def test_intraday_free_source_keeps_today_and_indices(self):
today = TRADE_DAY.strftime("%Y%m%d")
latest = {
"meta": {"trade_date": "2026-09-07", "source": "tushare"},
"overview": {"limit_up_count": 20},
}
harness = SyncHarness(
FakeFreeRealtimeTodayClient(),
latest,
clock=lambda: at_clock(10, 5),
)
payload = harness.sync_dashboard(today)
meta = payload["meta"]
self.assertFalse(meta.get("carried_forward"))
self.assertTrue(meta["realtime"])
self.assertEqual(meta["data_status"], "intraday")
self.assertEqual(str(meta["trade_date"]).replace("-", ""), today)
self.assertEqual(meta["quote_source"], "eastmoney_clist")
self.assertEqual(payload["overview"]["amount_billion"], 12345.6)
self.assertEqual(meta["indices"][0]["price"], 3800.1)
self.assertEqual(harness.database.saved[0][0], today)
def test_intraday_missing_quotes_do_not_carry_yesterday(self):
today = TRADE_DAY.strftime("%Y%m%d")
latest = {
"meta": {"trade_date": "2026-09-07", "source": "tushare"},
"overview": {"limit_up_count": 20},
}
harness = SyncHarness(
FakeMissingDailyClient(),
latest,
clock=lambda: at_clock(10, 5),
)
with self.assertRaises(ValueError) as ctx:
harness.sync_dashboard(today)
self.assertIn("当天盘中行情", str(ctx.exception))
self.assertFalse(harness.database.saved)
def test_intraday_keeps_existing_today_snapshot_when_refresh_fails(self):
today = TRADE_DAY.strftime("%Y%m%d")
existing = {
"meta": {
"trade_date": "2026-09-08",
"realtime": True,
"mode": "realtime",
"source": "tushare",
},
"overview": {"limit_up_count": 11},
"limits": [{"code": "600000"}],
"broken": [],
"down_limits": [],
"yesterday_limits": [],
}
harness = SyncHarness(
FakeMissingDailyClient(),
clock=lambda: at_clock(10, 5),
)
harness.database.get_snapshot = lambda *_args, **_kwargs: copy.deepcopy(existing)
payload = harness.sync_dashboard(today)
meta = payload["meta"]
self.assertEqual(str(meta["trade_date"]).replace("-", ""), today)
self.assertTrue(meta["realtime"])
self.assertEqual(meta["data_status"], "intraday")
self.assertFalse(meta.get("carried_forward"))
def test_lunch_and_after_hours_keep_today_until_official_arrives(self):
today = TRADE_DAY.strftime("%Y%m%d")
for clock in (lambda: at_clock(12, 0), lambda: at_clock(16, 10)):
harness = SyncHarness(
FakeRealtimeTodayClient(),
clock=clock,
)
payload = harness.sync_dashboard(today)
self.assertEqual(str(payload["meta"]["trade_date"]).replace("-", ""), today)
self.assertFalse(payload["meta"].get("carried_forward"))
def test_preopen_and_weekend_still_carry_last_session(self):
latest = {
"meta": {"trade_date": "2026-09-07", "source": "tushare"},
"overview": {"limit_up_count": 20},
}
preopen = SyncHarness(
FakeMissingDailyClient(),
latest,
clock=lambda: at_clock(8, 30),
)
preopen_payload = preopen.sync_dashboard(TRADE_DAY.strftime("%Y%m%d"))
self.assertTrue(preopen_payload["meta"]["carried_forward"])
self.assertEqual(preopen_payload["meta"]["data_status"], "preparing")
self.assertIn("今日数据正在准备,当前展示", preopen_payload["meta"]["display_notice"])
weekend = SyncHarness(
FakeMissingDailyClient(open_today=False),
latest,
clock=lambda: at_clock(10, 5, date(2026, 9, 5)),
)
weekend_payload = weekend.sync_dashboard("20260905")
self.assertTrue(weekend_payload["meta"]["carried_forward"])
def test_history_date_still_uses_official_or_preparing_notice(self):
latest = {
"meta": {"trade_date": "2026-09-01", "source": "tushare"},
"overview": {"limit_up_count": 8},
}
harness = SyncHarness(
FakeMissingDailyClient(),
latest,
clock=lambda: at_clock(10, 5),
)
payload = harness.sync_dashboard("20260902")
self.assertTrue(payload["meta"]["carried_forward"])
self.assertIn("所选日期数据尚未到齐", payload["meta"]["display_notice"])
def test_carried_today_snapshot_is_retried_immediately_in_session(self):
today = TRADE_DAY.strftime("%Y%m%d")
snapshot = {
"meta": {
"source": "tushare",
"trade_date": "2026-09-07",
"carried_forward": True,
"requested_date": "2026-09-08",
"updated_at": at_clock(10, 0).isoformat(),
},
"overview": {"limit_up_count": 1},
}
harness = SyncHarness(
FakeRealtimeTodayClient(),
clock=lambda: at_clock(10, 5),
)
harness.database.get_snapshot = lambda *_args, **_kwargs: copy.deepcopy(snapshot)
payload = harness.get_dashboard(today)
self.assertEqual(str(payload["meta"]["trade_date"]).replace("-", ""), today)
self.assertEqual(payload["meta"]["data_status"], "intraday")
self.assertTrue(harness.database.saved)
def test_weekend_carry_is_not_labeled_as_preparing(self):
snapshot = {
"meta": {"trade_date": "2026-07-24", "source": "tushare", "updated_at": "2026-07-24T15:00:00+08:00"},
"overview": {"limit_up_count": 1},
}
harness = SyncHarness(FakeMissingDailyClient())
carried = harness._carry_dashboard(snapshot, "20260725", "非交易日沿用最近交易日收盘行情")
self.assertEqual(carried["meta"]["data_status"], "carried")
self.assertIn("非交易日", carried["meta"]["display_notice"])
def test_stale_derived_snapshot_is_retried(self):
today = date.today().strftime("%Y%m%d")
old = datetime.now(timezone.utc) - timedelta(minutes=5)
snapshot = {
"meta": {
"source": "tushare",
"trade_date": f"{today[:4]}-{today[4:6]}-{today[6:8]}",
"limit_data_source": "derived",
"updated_at": old.isoformat(),
},
"overview": {"limit_up_count": 1},
}
harness = SyncHarness(FakeDerivedClient())
harness.database.get_snapshot = lambda *_args, **_kwargs: copy.deepcopy(snapshot)
payload = harness.get_dashboard(today)
self.assertEqual(payload["meta"]["data_status"], "partial")
self.assertTrue(harness.database.saved)
def test_official_catchup_skips_complete_today_snapshot(self):
today = date.today().strftime("%Y%m%d")
iso = f"{today[:4]}-{today[4:6]}-{today[6:8]}"
due = official_catchup_due(
today,
{"meta": {"trade_date": iso, "limit_data_source": "official"}},
)
derived_due = official_catchup_due(
today,
{"meta": {"trade_date": iso, "limit_data_source": "derived"}},
)
now = datetime.now().astimezone().time().replace(tzinfo=None)
if dt_time(15, 5) <= now < dt_time(22, 0):
self.assertFalse(due)
self.assertTrue(derived_due)
else:
self.assertFalse(due)
self.assertFalse(derived_due)
def test_official_catchup_is_due_for_intraday_snapshot_after_close(self):
today = TRADE_DAY.strftime("%Y%m%d")
snapshot = {
"meta": {
"trade_date": "2026-09-08",
"realtime": True,
"mode": "realtime",
}
}
with patch("backend.jobs.refresh.datetime") as mocked:
mocked.now.return_value = at_clock(16, 10)
mocked.strptime = datetime.strptime
self.assertTrue(official_catchup_due(today, snapshot))
official = {
"meta": {
"trade_date": "2026-09-08",
"limit_data_source": "official",
"realtime": False,
}
}
self.assertFalse(official_catchup_due(today, official))
class FrontendRefreshCopyTests(unittest.TestCase):
def test_dashboard_script_distinguishes_partial_from_failure(self):
script = (Path(__file__).resolve().parents[1] / "frontend" / "shared" / "dashboard.js").read_text(encoding="utf-8")
self.assertIn("今日数据正在准备,当前展示", script)
self.assertIn("部分正式数据尚未到齐", script)
self.assertIn("盘中行情", script)
self.assertIn("meta.realtime && actualCompact === requestedCompact", script)
self.assertIn('job.status === "failed"', script)
failed_block = script.split("if (job.status === \"failed\")", 1)[1].split("const query", 1)[0]
self.assertIn("后台刷新失败", failed_block)
success_block = script.split("const freshness = dashboardFreshnessMessage(meta);", 1)[1]
self.assertNotIn("后台刷新失败", success_block.split("} else {", 1)[0])
if __name__ == "__main__":
unittest.main()
+6 -5
View File
@@ -10,7 +10,7 @@ from database import ReviewDatabase
class BootstrapContainerTests(unittest.TestCase):
def test_environment_credentials_preserve_legacy_model_fallbacks(self) -> None:
def test_environment_credentials_exclude_provider_secrets_and_preserve_llm_fallbacks(self) -> None:
result = environment_credentials(
{
"TUSHARE_TOKEN": " tushare ",
@@ -20,8 +20,8 @@ class BootstrapContainerTests(unittest.TestCase):
"LLM_MODEL": "legacy-model",
}
)
self.assertEqual(result["tushare_token"], "tushare")
self.assertEqual(result["ifind_refresh_token"], "refresh")
self.assertNotIn("tushare_token", result)
self.assertNotIn("ifind_refresh_token", result)
self.assertEqual(result["platform_llm_primary_api_key"], "legacy-key")
self.assertEqual(result["platform_llm_primary_base_url"], "https://legacy.example/v1")
self.assertEqual(result["platform_llm_primary_model"], "legacy-model")
@@ -45,8 +45,9 @@ class BootstrapContainerTests(unittest.TestCase):
self.assertIs(container.strategy_tracking.repository.database, database)
self.assertIs(container.alert_service.repository.database, database)
self.assertIs(container.trade_journal.repository.database, database)
self.assertIs(container.chart_data.ifind, container.ifind)
self.assertTrue(container.ifind.configured)
self.assertIs(container.ifind, container.data_gateway.ifind)
self.assertIs(container.chart_data.datahub, container.data_gateway.datahub)
self.assertIsNone(container.chart_data.ifind)
if __name__ == "__main__":
+34
View File
@@ -0,0 +1,34 @@
import logging
import unittest
from backend.bootstrap.runtime import configure_logging
class ConfigureLoggingTest(unittest.TestCase):
def setUp(self) -> None:
self._saved_handlers = logging.getLogger().handlers[:]
self._saved_level = logging.getLogger().level
logging.getLogger().handlers.clear()
def tearDown(self) -> None:
logging.getLogger().handlers[:] = self._saved_handlers
logging.getLogger().setLevel(self._saved_level)
def test_configures_root_logger_at_info(self) -> None:
configure_logging()
root = logging.getLogger()
self.assertTrue(root.handlers)
self.assertEqual(root.level, logging.INFO)
with self.assertLogs("xiaobai.datahub", level="INFO") as captured:
logging.getLogger("xiaobai.datahub").info("datahub shadow %s", {"dataset": "daily"})
self.assertIn("datahub shadow", captured.output[0])
def test_keeps_existing_configuration(self) -> None:
handler = logging.NullHandler()
logging.getLogger().addHandler(handler)
configure_logging()
self.assertEqual(logging.getLogger().handlers, [handler])
if __name__ == "__main__":
unittest.main()
+165 -1
View File
@@ -2,7 +2,7 @@ from __future__ import annotations
import unittest
from backend.features.market.charts import ChartDataError, EastmoneyChartClient
from backend.features.market.charts import ChartDataError, EastmoneyChartClient, HIS_TRENDS_URL, MarketChartClient, TRENDS_URL
from server import DashboardService
@@ -72,6 +72,170 @@ class ChartDataProviderTests(unittest.TestCase):
self.client.stock_intraday("abc")
class LookbackChartClient(EastmoneyChartClient):
def __init__(self) -> None:
super().__init__(cache_ttl_seconds=20)
self.requests: list[tuple[str, dict[str, str]]] = []
def _request_json(self, url, params, referer):
self.requests.append((url, params))
if url == TRENDS_URL and params.get("ndays") == "1":
return {"data": {"code": "601318", "name": "中国平安", "preClose": 56.0, "trends": []}}
if url == TRENDS_URL and params.get("ndays") == "5":
return {"data": {"code": "601318", "name": "中国平安", "preClose": 56.0, "trends": []}}
if url == HIS_TRENDS_URL:
return {
"data": {
"code": "601318",
"name": "中国平安",
"preClose": 55.8,
"trends": [
"2026-09-07 09:30,55.80,55.90,56.00,55.70,100,5580.00,55.900",
"2026-09-07 15:00,56.10,56.20,56.30,56.00,200,11240.00,56.150",
"2026-09-08 09:30,0,0,0,0,0,0.00,0",
],
}
}
raise ChartDataError("unexpected url")
class ChartLookbackTests(unittest.TestCase):
def setUp(self) -> None:
EastmoneyChartClient._cache.clear()
self.client = LookbackChartClient()
def test_empty_today_falls_back_to_latest_available_session(self):
payload = self.client.stock_intraday("601318")
urls = [url for url, _ in self.client.requests]
self.assertEqual(urls[0], TRENDS_URL)
self.assertEqual(self.client.requests[0][1]["ndays"], "1")
self.assertEqual(urls[1], TRENDS_URL)
self.assertEqual(self.client.requests[1][1]["ndays"], "5")
self.assertEqual(urls[2], HIS_TRENDS_URL)
self.assertEqual(payload["trade_date"], "2026-09-07")
self.assertEqual([point["time"] for point in payload["points"]], ["09:30", "15:00"])
self.assertEqual(payload["points"][0]["close"], 55.9)
def test_delay_multiday_can_recover_without_his(self):
class DelayFive(EastmoneyChartClient):
def __init__(self):
super().__init__(cache_ttl_seconds=20)
self.requests = []
def _request_json(self, url, params, referer):
self.requests.append((url, params))
if params.get("ndays") == "1":
return {"data": {"code": "000001", "name": "平安银行", "preClose": 11.7, "trends": []}}
return {
"data": {
"code": "000001",
"name": "平安银行",
"preClose": 11.5,
"trends": [
"2026-09-07 09:30,11.50,11.60,11.70,11.40,100,1160.00,11.600",
"2026-09-07 15:00,11.70,11.80,11.90,11.60,200,2360.00,11.750",
],
}
}
EastmoneyChartClient._cache.clear()
client = DelayFive()
payload = client.stock_intraday("000001")
self.assertEqual(payload["trade_date"], "2026-09-07")
self.assertEqual(len(payload["points"]), 2)
self.assertEqual([url for url, _ in client.requests], [TRENDS_URL, TRENDS_URL])
def test_sh_sz_cyb_codes_use_correct_secid(self):
for code, secid in (("601318", "1.601318"), ("000001", "0.000001"), ("300750", "0.300750")):
EastmoneyChartClient._cache.clear()
client = LookbackChartClient()
client.stock_intraday(code)
self.assertEqual(client.requests[0][1]["secid"], secid)
class FakeHub:
def __init__(self, chart=None, error=None, daily=None):
self.chart = chart
self.error = error
self.daily = daily
self.calls: list[str] = []
self.legacy: list[str] = []
def try_intraday(self, code):
self.calls.append(code)
if self.error:
raise self.error
return self.chart
def try_daily_chart(self, code, end_date, limit=90, dataset="daily"):
self.calls.append(f"{dataset}:{code}")
if self.error:
raise self.error
return self.daily
def record_legacy(self, dataset, source="", error=""):
self.legacy.append(dataset)
class DatahubChartFallbackTests(unittest.TestCase):
def setUp(self) -> None:
EastmoneyChartClient._cache.clear()
def test_datahub_success_skips_old_channel(self):
hub = FakeHub(
{
"entity_type": "stock",
"identifier": "601318",
"name": "中国平安",
"code": "601318",
"trade_date": "2026-09-08",
"previous_close": 56.36,
"points": [{"date": "2026-09-08", "time": "09:30", "close": 56.5, "average": 56.4}],
"source": "datahub",
}
)
fallback = LookbackChartClient()
client = MarketChartClient(hub)
payload = client.stock_intraday("601318")
self.assertEqual(payload["source"], "datahub")
self.assertEqual(hub.calls, ["601318"])
self.assertEqual(fallback.requests, [])
def test_datahub_timeout_or_empty_does_not_use_old_channel(self):
fallback = LookbackChartClient()
for hub in (
FakeHub(chart=None),
FakeHub(error=RuntimeError("timeout")),
FakeHub(error=RuntimeError("datahub exploded")),
FakeHub(chart={"points": []}),
):
EastmoneyChartClient._cache.clear()
fallback.requests.clear()
client = MarketChartClient(hub)
with self.assertRaises(ChartDataError):
client.stock_intraday("000001")
self.assertEqual(fallback.requests, [])
def test_datahub_daily_skips_ifind(self):
hub = FakeHub(
daily=[
{
"trade_date": "2026-09-07",
"open": 10.0,
"high": 10.4,
"low": 9.9,
"close": 10.2,
"volume": 1000,
"amount_billion": 0.02,
}
]
)
client = MarketChartClient(hub)
rows = client.stock_daily("600000", "20260907")
self.assertEqual(rows[-1]["trade_date"], "2026-09-07")
self.assertIn("daily:600000", hub.calls)
class ChartServiceStub:
@staticmethod
def _payload(code: str, name: str):
+41 -15
View File
@@ -12,6 +12,7 @@ from backend.data import (
QualityEvidence,
build_data_gateway,
)
from backend.data.datahub.settings import DATASETS, DatahubSettings, DatasetFlags
from backend.data.quality import market_timezone
@@ -35,16 +36,30 @@ class DataGatewayTests(unittest.TestCase):
with self.assertRaises(DataPolicyError):
policy.assert_allowed("market.level2", "unresolved", "display")
def test_gateway_uses_live_token_supplier_and_shared_ifind(self) -> None:
token = {"value": "first"}
gateway = build_data_gateway(
{"ifind_refresh_token": "refresh", "ifind_access_token": "access"},
lambda: token["value"],
def test_gateway_uses_hub_facade_and_proxies(self) -> None:
settings = DatahubSettings(
base_url="http://127.0.0.1:8766",
token="hub-token",
datasets={name: DatasetFlags(name, read=True) for name in DATASETS},
)
self.assertEqual(gateway.tushare().token, "first")
token["value"] = "second"
self.assertEqual(gateway.tushare().token, "second")
self.assertIs(gateway.chart_data.ifind, gateway.ifind)
gateway = build_data_gateway(
{},
datahub_settings=settings,
)
client = gateway.tushare()
self.assertEqual(client.token, "datahub")
self.assertIsNone(client.realtime_aggregator)
self.assertFalse(hasattr(client, "_legacy"))
self.assertIs(gateway.ifind, gateway.ifind_provider.client)
self.assertIs(gateway.chart_data.datahub, gateway.datahub)
self.assertIsNone(gateway.chart_data.ifind)
from backend.data.datahub.bridge import DatahubAwareTushareClient
from backend.data.datahub.ifind_proxy import HubIfindProxy
from backend.data.datahub.realtime_proxy import HubRealtimeProxy
self.assertIsInstance(client, DatahubAwareTushareClient)
self.assertIsInstance(gateway.ifind, HubIfindProxy)
self.assertIsInstance(gateway.realtime_observer, HubRealtimeProxy)
def test_server_has_no_direct_runtime_tushare_construction(self) -> None:
source = (
@@ -54,21 +69,29 @@ class DataGatewayTests(unittest.TestCase):
/ "market"
/ "service.py"
).read_text(encoding="utf-8")
self.assertEqual(source.count("TushareClient(self.token)"), 1)
self.assertNotIn("TushareClient(self.token)", source)
self.assertIn("return gateway.tushare()", source)
def test_provider_construction_has_unique_declared_owners(self) -> None:
root = Path(__file__).resolve().parents[1]
owners = {
"EastmoneyChartClient": {"backend/data/gateway.py"},
"IfindHttpClient": {"backend/data/gateway.py"},
"IfindProvider": {"backend/data/gateway.py"},
"MarketChartClient": {"backend/data/gateway.py"},
"TushareClient": {"backend/features/market/service.py"},
"TushareProvider": {"backend/data/gateway.py"},
"WebRealtimeAggregator": {"backend/data/gateway.py"},
"TushareClient": set(),
"DatahubClient": {"backend/data/gateway.py"},
"DatahubAwareTushareClient": {"backend/data/gateway.py"},
"DatahubBridge": {"backend/data/gateway.py"},
"HubIfindProxy": {"backend/data/gateway.py"},
"HubRealtimeProxy": {"backend/data/gateway.py"},
}
found = {name: set() for name in owners}
forbidden = {
"IfindHttpClient": set(),
"EastmoneyChartClient": set(),
"WebRealtimeAggregator": set(),
"TushareProvider": set(),
}
found_forbidden = {name: set() for name in forbidden}
for path in (root / "backend").rglob("*.py"):
relative = path.relative_to(root).as_posix()
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
@@ -78,7 +101,10 @@ class DataGatewayTests(unittest.TestCase):
name = getattr(node.func, "id", None) or getattr(node.func, "attr", None)
if name in found:
found[name].add(relative)
if name in found_forbidden:
found_forbidden[name].add(relative)
self.assertEqual(found, owners)
self.assertEqual(found_forbidden, forbidden)
provider_source = (root / "backend/data/providers/tushare.py").read_text(
encoding="utf-8"
)
+611
View File
@@ -0,0 +1,611 @@
from __future__ import annotations
import ast
import json
import unittest
from pathlib import Path
from typing import Any
from backend.data import build_data_gateway
from backend.data.datahub.bridge import DatahubAwareTushareClient, DatahubBridge, looks_like_heaven
from backend.data.datahub.client import DatahubClient, DatahubResponse
from backend.data.datahub.compare import compare_rows
from backend.data.datahub.errors import DatahubError
from backend.data.datahub.native import to_canonical_row, to_native_row
from backend.data.datahub.route_state import LEDGER
from backend.data.providers.tushare_transport import TushareError
from backend.data.datahub.settings import DATASETS, DatahubSettings, DatasetFlags
ROOT = Path(__file__).resolve().parents[1]
TOKEN = "super-secret-datahub-token"
LEGACY_DAILY = {
"ts_code": "600000.SH",
"trade_date": "20240902",
"open": 10.11,
"high": 10.25,
"low": 10.01,
"close": 10.20,
"pct_chg": 1.2345,
"vol": 1000.0,
"amount": 2000.0,
}
HUB_DAILY = {
"ts_code": "600000.SH",
"trade_date": "20240902",
"open": 10.11,
"high": 10.25,
"low": 10.01,
"close": 10.20,
"pct_chg": 1.2345,
"volume": 100000.0,
"amount": 2000000.0,
}
class FakeLegacy:
def __init__(self, rows: list[dict[str, Any]] | Exception | None = None) -> None:
self.token = "legacy-token"
self.timeout = 30
self.rows = [] if rows is None else rows
self.calls: list[tuple[str, dict[str, Any] | None, str]] = []
def query(self, api_name: str, params: dict[str, Any] | None = None, fields: str = "") -> list[dict[str, Any]]:
self.calls.append((api_name, params, fields))
if isinstance(self.rows, Exception):
raise self.rows
return [dict(row) for row in self.rows]
class FakeClient(DatahubClient):
def __init__(self, error: DatahubError | None = None, response: DatahubResponse | None = None) -> None:
super().__init__(DatahubSettings(base_url="http://127.0.0.1:9", token=TOKEN))
self.error = error
self.response = response or DatahubResponse(
data=[dict(HUB_DAILY)],
meta={"tier": "official", "trade_date": "20240902", "stale": False, "staleness_seconds": 0},
)
self.paths: list[str] = []
self.calls: list[tuple[str, dict[str, Any]]] = []
def get(self, path: str, params: dict[str, Any] | None = None) -> DatahubResponse:
return self._record(path, params)
def post(self, path: str, body: dict[str, Any] | None = None) -> DatahubResponse:
return self._record(path, body)
def _record(self, path: str, payload: dict[str, Any] | None) -> DatahubResponse:
self.paths.append(path)
self.calls.append((path, {key: value for key, value in (payload or {}).items()}))
packed = json.dumps(payload or {})
if TOKEN in packed or TOKEN in path:
raise AssertionError("token leaked into url")
if self.error:
raise self.error
return self.response
def flags(**enabled: tuple[bool, bool]) -> DatahubSettings:
datasets = {name: DatasetFlags(name) for name in DATASETS}
for name, pair in enabled.items():
datasets[name] = DatasetFlags(name, read=pair[0], shadow=pair[1])
return DatahubSettings(base_url="http://127.0.0.1:9", token=TOKEN, datasets=datasets)
class DatahubBridgeTests(unittest.TestCase):
def setUp(self) -> None:
LEDGER.clear()
def test_default_config_enables_official_reads(self) -> None:
settings = DatahubSettings.load(environ={}, credentials={})
self.assertTrue(settings.any_enabled())
self.assertTrue(all(settings.flags(name).read and not settings.flags(name).shadow for name in DATASETS))
client = FakeClient()
legacy = FakeLegacy([LEGACY_DAILY])
wrapped = DatahubAwareTushareClient(legacy, DatahubBridge(settings, client))
rows = wrapped.query("daily", {"trade_date": "20240902"}, "ts_code,close,vol,amount")
self.assertEqual(rows[0]["amount"], 2000.0)
self.assertEqual(client.paths, ["/v1/bars/daily"])
self.assertEqual(legacy.calls, [])
self.assertEqual(LEDGER.snapshot()[0]["route"], "datahub")
def test_each_dataset_has_independent_read_flag(self) -> None:
settings = flags(daily=(True, False), auction=(False, False))
self.assertTrue(settings.flags("daily").read)
self.assertFalse(settings.flags("auction").read)
self.assertFalse(any(settings.flags(name).read for name in DATASETS if name != "daily"))
source = (ROOT / "config" / "datahub.config.json").read_text(encoding="utf-8")
self.assertNotIn("master", source)
self.assertNotIn("DATAHUB_READ_ALL", source)
compose = (ROOT / "compose.yaml").read_text(encoding="utf-8")
for env_key in (
"CALENDAR", "STOCKS", "DAILY", "INDEX_DAILY", "VALUATION", "MONEYFLOW",
"AUCTION", "LIMIT_EVENTS", "POPULARITY", "DRAGON_TIGER", "SECTOR_DAILY",
"QUOTES", "INDEX_QUOTES", "INTRADAY", "STATUS",
):
self.assertIn(f'DATAHUB_READ_{env_key}: "1"', compose)
def test_read_flag_replaces_only_that_dataset_and_converts_units(self) -> None:
shadows: list[dict[str, Any]] = []
client = FakeClient()
legacy = FakeLegacy([LEGACY_DAILY])
wrapped = DatahubAwareTushareClient(
legacy,
DatahubBridge(flags(daily=(True, False)), client, shadow_sink=shadows.append),
)
rows = wrapped.query("daily", {"trade_date": "20240902"}, "ts_code,vol,amount")
self.assertEqual(rows[0]["vol"], 1000.0)
self.assertEqual(rows[0]["amount"], 2000.0)
self.assertEqual(legacy.calls, [])
self.assertEqual(client.paths, ["/v1/bars/daily"])
calendar_legacy = FakeLegacy([{"cal_date": "20240902", "is_open": 1}])
calendar_client = FakeClient(
response=DatahubResponse(
data=[{"cal_date": "20240902", "is_open": 1, "pretrade_date": "20240830"}],
meta={"source": "datahub", "stale": False, "staleness_seconds": 0},
)
)
calendar_wrapped = DatahubAwareTushareClient(
calendar_legacy,
DatahubBridge(flags(daily=(True, False)), calendar_client),
)
calendar = calendar_wrapped.query("trade_cal", {"start_date": "20240902", "end_date": "20240902"}, "")
self.assertEqual(calendar[0]["is_open"], 1)
self.assertEqual(calendar_legacy.calls, [])
self.assertEqual(calendar_client.paths, ["/v1/query"])
def test_hub_failure_does_not_call_website_legacy(self) -> None:
cases = [
DatahubError("UNAVAILABLE", "down"),
DatahubError("UNAUTHORIZED", "401"),
DatahubError("TIMEOUT", "late"),
DatahubError("EMPTY", "no rows"),
DatahubError("DATASET_NOT_PUBLISHED", "not ready"),
DatahubError("STALE", "old"),
DatahubError("INCOMPLETE", "truncated"),
]
for error in cases:
with self.subTest(error=error.code):
client = FakeClient(error=error)
legacy = FakeLegacy([LEGACY_DAILY])
wrapped = DatahubAwareTushareClient(legacy, DatahubBridge(flags(daily=(True, False)), client))
with self.assertRaises(TushareError):
wrapped.query("daily", {"trade_date": "20240902"}, "ts_code,amount")
self.assertEqual(legacy.calls, [])
def test_shadow_mode_no_longer_calls_website_tushare(self) -> None:
reports: list[dict[str, Any]] = []
client = FakeClient(
response=DatahubResponse(
data=[dict(LEGACY_DAILY)],
meta={"source": "tushare", "stale": False, "staleness_seconds": 0, "row_shape": "tushare"},
)
)
legacy = FakeLegacy([LEGACY_DAILY])
wrapped = DatahubAwareTushareClient(
legacy,
DatahubBridge(flags(daily=(False, True)), client, shadow_sink=reports.append),
)
rows = wrapped.query("daily", {"trade_date": "20240902"}, "ts_code,amount,vol")
self.assertEqual(rows[0]["amount"], 2000.0)
self.assertEqual(legacy.calls, [])
self.assertEqual(client.paths, ["/v1/query"])
failed = FakeClient(error=DatahubError("UNAVAILABLE", TOKEN))
fail_legacy = FakeLegacy([LEGACY_DAILY])
fail_wrapped = DatahubAwareTushareClient(
fail_legacy,
DatahubBridge(flags(daily=(False, True)), failed, shadow_sink=reports.append),
)
with self.assertRaises(TushareError):
fail_wrapped.query("daily", {"trade_date": "20240902"}, "amount")
self.assertEqual(fail_legacy.calls, [])
self.assertNotIn(TOKEN, str(failed.calls))
def test_compare_classifies_unit_conversion_missing_row_and_value_diff(self) -> None:
equal = compare_rows("daily", [LEGACY_DAILY], [HUB_DAILY], {"stale": False, "staleness_seconds": 0})
self.assertTrue(equal["equal"])
unit = compare_rows("daily", [LEGACY_DAILY], [{**HUB_DAILY, "amount": 2000.0, "volume": 1000.0}])
self.assertGreater(unit["unit_conversion_count"], 0)
missing = compare_rows("daily", [LEGACY_DAILY], [])
self.assertEqual(missing["missing_hub_count"], 1)
value = compare_rows("daily", [LEGACY_DAILY], [{**HUB_DAILY, "close": 99.0}])
self.assertEqual(value["value_diff_count"], 1)
skew = compare_rows("daily", [LEGACY_DAILY], [HUB_DAILY], {"stale": False, "staleness_seconds": 12})
self.assertTrue(skew["time_skew"])
def test_shadow_extra_hub_columns_are_not_false_diffs_when_projected(self) -> None:
hub_full = {**HUB_DAILY, "adj_factor": 1.1}
legacy_close_only = {k: LEGACY_DAILY[k] for k in ("ts_code", "trade_date", "close")}
report = compare_rows(
"daily", [legacy_close_only], [hub_full],
{"stale": False, "staleness_seconds": 0},
fields="ts_code,trade_date,close",
)
self.assertTrue(report["equal"])
self.assertEqual(report["value_diff_count"], 0)
self.assertEqual(report["fields_compared"], ["close", "trade_date", "ts_code"])
# without projection the same pair shows the historic false diff
unprojected = compare_rows("daily", [legacy_close_only], [hub_full])
self.assertFalse(unprojected["equal"])
legacy_stocks = {"ts_code": "600000.SH", "name": "浦发银行"}
hub_stocks = {
"ts_code": "600000.SH", "symbol": "600000", "name": "浦发银行", "area": "上海",
"industry": "银行", "market": "主板", "list_status": "L", "list_date": "19991110",
}
stocks = compare_rows("stocks", [legacy_stocks], [hub_stocks], {}, fields="ts_code,name")
self.assertTrue(stocks["equal"])
legacy_cal = {"cal_date": "20240902", "is_open": 1}
hub_cal = {
"cal_date": "20240902", "is_open": True,
"pretrade_date": "20240830", "prev_open": "20240830",
}
calendar = compare_rows(
"calendar", [legacy_cal], [hub_cal], {}, fields="cal_date,is_open"
)
self.assertTrue(calendar["equal"])
def test_shadow_projection_still_alarms_on_requested_field_problems(self) -> None:
hub_missing_field = {k: v for k, v in HUB_DAILY.items() if k != "close"}
legacy_close_only = {k: LEGACY_DAILY[k] for k in ("ts_code", "trade_date", "close")}
lost = compare_rows(
"daily", [legacy_close_only], [hub_missing_field], fields="ts_code,trade_date,close"
)
self.assertFalse(lost["equal"])
self.assertEqual(lost["value_diff_count"], 1)
changed = compare_rows(
"daily", [legacy_close_only], [{**HUB_DAILY, "close": 99.0}],
fields="ts_code,trade_date,close",
)
self.assertFalse(changed["equal"])
self.assertEqual(changed["value_diff_count"], 1)
self.assertEqual(changed["value_diffs"][0]["fields"][0]["field"], "close")
gone = compare_rows("daily", [LEGACY_DAILY], [], fields="ts_code,trade_date,close")
self.assertEqual(gone["missing_hub_count"], 1)
self.assertFalse(gone["equal"])
unit = compare_rows(
"daily", [LEGACY_DAILY], [{**HUB_DAILY, "amount": 2000.0, "volume": 1000.0}],
fields="ts_code,trade_date,vol,amount",
)
self.assertGreater(unit["unit_conversion_count"], 0)
self.assertFalse(unit["equal"])
def test_bridge_shadow_report_uses_website_request_fields(self) -> None:
hub_full = {**HUB_DAILY, "adj_factor": 1.1}
legacy_close_only = {k: LEGACY_DAILY[k] for k in ("ts_code", "trade_date", "close", "vol", "amount")}
reports: list[dict[str, Any]] = []
client = FakeClient(
response=DatahubResponse(
data=[hub_full],
meta={"tier": "official", "trade_date": "20240902", "stale": False, "staleness_seconds": 0},
)
)
wrapped = DatahubAwareTushareClient(
FakeLegacy([legacy_close_only]),
DatahubBridge(flags(daily=(True, False)), client, shadow_sink=reports.append),
)
rows = wrapped.query("daily", {"trade_date": "20240902"}, "ts_code,trade_date,close,vol,amount")
self.assertEqual(rows[0]["close"], 10.20)
self.assertEqual(rows[0]["vol"], 1000.0)
self.assertEqual(client.paths, ["/v1/bars/daily"])
def test_native_roundtrip_matches_known_scales(self) -> None:
native = to_native_row("daily", HUB_DAILY)
self.assertEqual(native["vol"], 1000.0)
self.assertEqual(native["amount"], 2000.0)
canonical = to_canonical_row("daily", native)
self.assertEqual(canonical["vol"], 100000.0)
self.assertEqual(canonical["amount"], 2000000.0)
def test_heaven_can_use_hub_when_dataset_flag_is_on(self) -> None:
"""问天按数据依赖接入:已映射 API 跟随开关,不再整栈强制旧链路。"""
self.assertTrue(looks_like_heaven("backend.features.heaven.market_context", "backend/features/heaven/market_context.py"))
self.assertFalse(looks_like_heaven("backend.features.market.service", "backend/features/market/service.py"))
client = FakeClient()
legacy = FakeLegacy([LEGACY_DAILY])
wrapped = DatahubAwareTushareClient(
legacy,
DatahubBridge(flags(daily=(True, False)), client, heaven_guard=lambda: True),
)
rows = wrapped.query("daily", {"trade_date": "20240902"}, "amount")
self.assertEqual(rows[0]["amount"], 2000.0)
self.assertEqual(client.paths, ["/v1/bars/daily"])
self.assertEqual(legacy.calls, [])
def test_status_flag_does_not_run_when_off_and_falls_back_when_on(self) -> None:
off = DatahubBridge(flags(), FakeClient(error=DatahubError("UNAVAILABLE", "down")))
self.assertIsNone(off.dataset_status("20240902"))
reports: list[dict[str, Any]] = []
failed = DatahubBridge(
flags(status=(True, True)),
FakeClient(error=DatahubError("UNAUTHORIZED", "nope")),
shadow_sink=reports.append,
)
self.assertIsNone(failed.dataset_status("20240902"))
self.assertTrue(reports[0]["hub_error"])
ok = DatahubBridge(
flags(status=(True, False)),
FakeClient(response=DatahubResponse(data=[{"dataset": "daily", "state": "published"}], meta={"stale": False, "staleness_seconds": 0})),
)
self.assertEqual(ok.dataset_status("20240902")[0]["state"], "published")
def test_default_gateway_wraps_tushare_without_calling_datahub(self) -> None:
gateway = build_data_gateway({}, datahub_settings=flags())
client = gateway.tushare()
self.assertIsInstance(client, DatahubAwareTushareClient)
self.assertFalse(gateway.datahub.settings.any_enabled())
def test_stock_detail_range_query_is_not_silently_accepted_when_incomplete(self) -> None:
source = (ROOT / "backend" / "data" / "providers" / "tushare_stocks.py").read_text(encoding="utf-8")
self.assertIn('"daily"', source)
self.assertIn("start_date", source)
self.assertIn("end_date", source)
client = FakeClient(error=DatahubError("INCOMPLETE", "truncated"))
legacy = FakeLegacy([LEGACY_DAILY])
wrapped = DatahubAwareTushareClient(legacy, DatahubBridge(flags(daily=(True, False)), client))
with self.assertRaises(TushareError):
wrapped.query(
"daily",
{"ts_code": "600000.SH", "start_date": "20240301", "end_date": "20240902"},
"ts_code,amount",
)
self.assertEqual(legacy.calls, [])
self.assertIn("/v1/query", client.paths)
def test_try_intraday_respects_switch_and_falls_back_on_bad_payload(self) -> None:
closed = DatahubBridge(flags(), FakeClient(error=DatahubError("INTERNAL", "should not run")))
self.assertIsNone(closed.try_intraday("601318"))
empty = DatahubBridge(
flags(intraday=(True, False)),
FakeClient(response=DatahubResponse(data={"points": []}, meta={"stale": False})),
)
self.assertIsNone(empty.try_intraday("601318"))
stale = DatahubBridge(
flags(intraday=(True, False)),
FakeClient(response=DatahubResponse(
data={
"entity_type": "stock",
"code": "601318",
"trade_date": "2026-09-07",
"previous_close": 55.8,
"points": [{"date": "2026-09-07", "time": "09:30", "close": 55.9, "avg_price": 55.85}],
},
meta={"stale": True},
)),
)
self.assertIsNone(stale.try_intraday("601318"))
ok = DatahubBridge(
flags(intraday=(True, False)),
FakeClient(response=DatahubResponse(
data={
"entity_type": "stock",
"identifier": "601318",
"name": "中国平安",
"code": "601318",
"trade_date": "2026-09-08",
"previous_close": 56.36,
"points": [
{"date": "2026-09-08", "time": "09:30", "close": 0},
{"date": "2026-09-08", "time": "09:31", "close": 56.5, "avg_price": 56.4},
],
},
meta={"stale": False},
)),
)
chart = ok.try_intraday("601318")
self.assertEqual(chart["source"], "datahub")
self.assertEqual(len(chart["points"]), 1)
self.assertEqual(chart["points"][0]["average"], 56.4)
self.assertEqual(ok.client.paths, ["/v1/intraday/points"])
self.assertEqual(ok.client.calls, [("/v1/intraday/points", {"code": "601318"})])
self.assertNotIn("date", ok.client.calls[0][1])
timeout = DatahubBridge(
flags(intraday=(True, False)),
FakeClient(error=DatahubError("TIMEOUT", "datahub request timed out")),
)
self.assertIsNone(timeout.try_intraday("601318"))
broken = DatahubBridge(
flags(intraday=(True, False)),
FakeClient(error=DatahubError("INTERNAL", "datahub exploded")),
)
self.assertIsNone(broken.try_intraday("601318"))
self.assertTrue(DatahubSettings.load(environ={}, credentials={}).flags("intraday").read)
def test_try_market_quotes_and_visible_fallback(self) -> None:
quotes = [
{
"ts_code": f"{600000 + index:06d}.SH",
"name": f"股票{index}",
"close": 10.2,
"pre_close": 10.0,
"open": 10.1,
"high": 10.3,
"low": 9.9,
"vol": 1000,
"amount": 2000000,
"quote_date": "20240902",
}
for index in range(220)
]
ok = DatahubBridge(
flags(quotes=(True, False)),
FakeClient(
response=DatahubResponse(
data=quotes,
meta={"stale": False, "staleness_seconds": 0, "source": "eastmoney:clist"},
)
),
)
rows = ok.try_market_quotes("20240902")
self.assertEqual(len(rows), 220)
self.assertEqual(rows[0]["pre_close"], 10.0)
self.assertEqual(ok.client.paths, ["/v1/quotes/latest"])
self.assertEqual(LEDGER.snapshot()[0]["route"], "datahub")
failed = DatahubBridge(
flags(quotes=(True, False)),
FakeClient(error=DatahubError("UNAVAILABLE", "down")),
)
self.assertIsNone(failed.try_market_quotes("20240902"))
snap = next(item for item in LEDGER.snapshot() if item["dataset"] == "quotes")
self.assertEqual(snap["route"], "datahub")
self.assertEqual(snap["source"], "unavailable")
gateway = build_data_gateway({}, datahub_settings=flags(quotes=(True, False)))
status = gateway.datahub_status()
self.assertEqual(status["enabled_reads"], 1)
self.assertEqual(status["total_reads"], len(DATASETS))
self.assertEqual(status["fallback_count"], 0)
def test_try_daily_chart_converts_hub_bars(self) -> None:
rows = [
{
"ts_code": "600000.SH",
"trade_date": "20240901",
"open": 10.0,
"high": 10.4,
"low": 9.9,
"close": 10.2,
"volume": 100000,
"amount": 2000000,
},
{
"ts_code": "600000.SH",
"trade_date": "20240902",
"open": 10.2,
"high": 10.5,
"low": 10.1,
"close": 10.4,
"volume": 120000,
"amount": 2400000,
},
]
hub = DatahubBridge(
flags(daily=(True, False)),
FakeClient(
response=DatahubResponse(
data=rows,
meta={"stale": False, "staleness_seconds": 0, "source": "tushare:daily"},
)
),
)
chart = hub.try_daily_chart("600000.SH", "20240902", 90, "daily")
self.assertEqual(chart[-1]["trade_date"], "2024-09-02")
self.assertEqual(chart[-1]["close"], 10.4)
self.assertAlmostEqual(chart[-1]["amount_billion"], 0.024)
def test_try_daily_chart_keeps_usable_bars_when_coverage_incomplete(self) -> None:
rows = [
{
"ts_code": "000001.SZ",
"trade_date": "20240901",
"open": 10.0,
"high": 10.4,
"low": 9.9,
"close": 10.2,
"volume": 100000,
"amount": 2000000,
},
{
"ts_code": "000001.SZ",
"trade_date": "20240902",
"open": 10.2,
"high": 10.5,
"low": 10.1,
"close": 10.4,
"volume": 120000,
"amount": 2400000,
},
]
hub = DatahubBridge(
flags(daily=(True, False)),
FakeClient(
response=DatahubResponse(
data=rows,
meta={
"stale": False,
"staleness_seconds": 0,
"incomplete": True,
"coverage": {"complete": False, "missing_count": 127},
"source": "tushare:daily",
},
)
),
)
chart = hub.try_daily_chart("000001.SZ", "20240902", 90, "daily")
self.assertIsNotNone(chart)
self.assertEqual(chart[-1]["trade_date"], "2024-09-02")
self.assertEqual(chart[-1]["close"], 10.4)
def test_gateway_tushare_facade_has_no_legacy_client(self) -> None:
quotes = [
{
"ts_code": f"{index:06d}.SZ",
"name": f"S{index}",
"pre_close": 10.0,
"open": 10.0,
"high": 10.5,
"low": 9.8,
"close": 10.2,
"vol": 100.0,
"amount": 1000.0,
"quote_date": "20240902",
}
for index in range(1, 221)
]
hub_client = FakeClient(
response=DatahubResponse(
data=quotes,
meta={"stale": False, "staleness_seconds": 0, "source": "eastmoney_clist"},
)
)
gateway = build_data_gateway(
{"tushare_token": "tok"},
datahub_settings=flags(quotes=(True, False), daily=(True, False)),
)
gateway.datahub.client = hub_client
wrapped = gateway.tushare()
self.assertFalse(hasattr(wrapped, "_legacy"))
self.assertIsNone(getattr(type(wrapped), "__getattr__", None))
self.assertTrue(callable(getattr(type(wrapped), "try_market_quotes", None)))
self.assertTrue(callable(getattr(type(wrapped), "try_index_quotes", None)))
self.assertTrue(callable(getattr(type(wrapped), "record_datahub_legacy", None)))
self.assertTrue(callable(getattr(type(wrapped), "dashboard", None)))
rows = wrapped.try_market_quotes("20240902")
self.assertGreaterEqual(len(rows or []), 200)
self.assertIn("/v1/quotes/latest", hub_client.paths)
hub_client.response = DatahubResponse(
data=[dict(HUB_DAILY)],
meta={"stale": False, "staleness_seconds": 0, "source": "tushare:daily"},
)
daily = wrapped.query("daily", {"trade_date": "20240902"}, "ts_code,amount")
self.assertEqual(daily[0]["amount"], 2000.0)
self.assertIn("/v1/bars/daily", hub_client.paths)
def test_features_do_not_import_datahub_client(self) -> None:
violations = []
for path in (ROOT / "backend" / "features").rglob("*.py"):
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
for node in ast.walk(tree):
names = []
if isinstance(node, ast.Import):
names = [alias.name for alias in node.names]
elif isinstance(node, ast.ImportFrom) and node.module:
names = [node.module]
for name in names:
if "datahub" in name.split("."):
violations.append(f"{path.relative_to(ROOT)} -> {name}")
self.assertEqual(violations, [])
if __name__ == "__main__":
unittest.main()
+185
View File
@@ -0,0 +1,185 @@
from __future__ import annotations
import io
import json
import logging
import threading
import unittest
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from urllib.parse import parse_qs, urlparse
from backend.data.datahub.client import DatahubClient
from backend.data.datahub.errors import DatahubError
from backend.data.datahub.redact import redact_text
from backend.data.datahub.settings import DatahubSettings
TOKEN = "super-secret-datahub-token"
class FakeHubState:
def __init__(self) -> None:
self.mode = "ok"
self.hits = 0
self.paths: list[str] = []
STATE = FakeHubState()
class FakeHubHandler(BaseHTTPRequestHandler):
def log_message(self, format: str, *args: object) -> None:
return
def do_GET(self) -> None: # noqa: N802
STATE.hits += 1
parsed = urlparse(self.path)
STATE.paths.append(parsed.path)
token = self.headers.get("X-Datahub-Token", "")
if STATE.mode == "timeout":
raise TimeoutError("simulated timeout")
if token != TOKEN:
self._json(401, {"error": {"code": "UNAUTHORIZED", "message": "missing or invalid X-Datahub-Token"}})
return
if STATE.mode == "unpublished":
self._json(404, {"error": {"code": "DATASET_NOT_PUBLISHED", "message": "daily 19990101 尚未发布", "expected_at": "15:05+08:00"}})
return
if STATE.mode == "empty":
self._json(200, {"schema_version": 1, "data": [], "meta": {"tier": "official", "stale": False, "staleness_seconds": 0}})
return
if STATE.mode == "stale":
self._json(200, {"schema_version": 1, "data": [{"ts_code": "600000.SH", "trade_date": "20240902", "close": 10.2, "volume": 100000, "amount": 2000000}], "meta": {"tier": "official", "stale": True, "staleness_seconds": 999999}})
return
if STATE.mode == "invalid":
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.end_headers()
self.wfile.write(b"not-json")
return
if parsed.path == "/v1/health":
self._json(200, {"schema_version": 1, "data": {"status": "ok"}, "meta": {"tier": "official", "source": "datahub", "stale": False, "staleness_seconds": 0}})
return
if parsed.path == "/v1/calendar":
self._json(200, {"schema_version": 1, "data": [{"cal_date": "20240902", "is_open": True, "pretrade_date": "20240830"}], "meta": {"tier": "official", "trade_date": "20240902", "stale": False, "staleness_seconds": 0}})
return
if parsed.path == "/v1/bars/daily":
query = {key: values[-1] for key, values in parse_qs(parsed.query).items()}
self._json(200, {
"schema_version": 1,
"data": [{
"ts_code": "600000.SH",
"trade_date": query.get("date") or "20240902",
"open": 10.11, "high": 10.25, "low": 10.01, "close": 10.20,
"pct_chg": 1.2345, "volume": 100000.0, "amount": 2000000.0, "adj_factor": 1.1,
}],
"meta": {"tier": "official", "trade_date": "20240902", "stale": False, "staleness_seconds": 0, "source": "tushare:daily"},
})
return
if parsed.path == "/v1/datasets/status":
self._json(200, {"schema_version": 1, "data": [{"dataset": "daily", "state": "published", "trade_date": "20240902"}], "meta": {"tier": "official", "stale": False, "staleness_seconds": 0}})
return
self._json(400, {"error": {"code": "INVALID_ARGUMENT", "message": f"unknown endpoint: {parsed.path}"}})
def _json(self, status: int, payload: dict) -> None:
body = json.dumps(payload).encode("utf-8")
self.send_response(status)
self.send_header("Content-Type", "application/json; charset=utf-8")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
class DatahubClientTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.server = ThreadingHTTPServer(("127.0.0.1", 0), FakeHubHandler)
cls.thread = threading.Thread(target=cls.server.serve_forever, daemon=True)
cls.thread.start()
cls.base = f"http://127.0.0.1:{cls.server.server_address[1]}"
@classmethod
def tearDownClass(cls) -> None:
cls.server.shutdown()
cls.server.server_close()
def setUp(self) -> None:
STATE.mode = "ok"
STATE.hits = 0
STATE.paths = []
self.client = DatahubClient(DatahubSettings(base_url=self.base, token=TOKEN, retries=1, timeout_seconds=2))
def test_health_envelope(self) -> None:
response = self.client.health()
self.assertEqual(response.schema_version, 1)
self.assertEqual(response.data["status"], "ok")
self.assertIn("stale", response.meta)
def test_missing_and_bad_token_401(self) -> None:
missing = DatahubClient(DatahubSettings(base_url=self.base, token=""))
with self.assertRaises(DatahubError) as raised:
missing.health()
self.assertEqual(raised.exception.code, "NOT_CONFIGURED")
bad = DatahubClient(DatahubSettings(base_url=self.base, token="wrong"))
with self.assertRaises(DatahubError) as raised:
bad.health()
self.assertEqual(raised.exception.code, "UNAUTHORIZED")
self.assertNotIn(TOKEN, str(raised.exception))
def test_unpublished_and_empty_and_stale_codes(self) -> None:
STATE.mode = "unpublished"
with self.assertRaises(DatahubError) as raised:
self.client.daily_bars(date="19990101")
self.assertEqual(raised.exception.code, "DATASET_NOT_PUBLISHED")
STATE.mode = "empty"
response = self.client.daily_bars(date="20240902")
self.assertEqual(response.data, [])
STATE.mode = "stale"
stale = self.client.daily_bars(date="20240902")
self.assertTrue(stale.meta["stale"])
def test_invalid_json_maps_to_internal(self) -> None:
STATE.mode = "invalid"
with self.assertRaises(DatahubError) as raised:
self.client.health()
self.assertEqual(raised.exception.code, "INTERNAL")
def test_timeout_maps_and_retries(self) -> None:
hits = {"n": 0}
def boom(_request, timeout=None):
hits["n"] += 1
raise TimeoutError("late")
client = DatahubClient(
DatahubSettings(base_url=self.base, token=TOKEN, retries=1, timeout_seconds=1),
urlopen=boom,
)
with self.assertRaises(DatahubError) as raised:
client.health()
self.assertEqual(raised.exception.code, "TIMEOUT")
self.assertEqual(hits["n"], 2)
def test_token_never_appears_in_error_text_or_logs(self) -> None:
stream = io.StringIO()
logger = logging.getLogger("xiaobai.datahub")
handler = logging.StreamHandler(stream)
logger.addHandler(handler)
logger.setLevel(logging.DEBUG)
try:
with self.assertRaises(DatahubError):
DatahubClient(DatahubSettings(base_url=self.base, token="wrong")).health()
blob = stream.getvalue() + redact_text("header " + TOKEN, (TOKEN,))
self.assertNotIn(TOKEN, blob)
self.assertIn("***", redact_text(TOKEN, (TOKEN,)))
finally:
logger.removeHandler(handler)
def test_calendar_and_status_contract(self) -> None:
calendar = self.client.calendar("20240901", "20240902")
self.assertEqual(calendar.data[0]["cal_date"], "20240902")
status = self.client.dataset_status("20240902")
self.assertEqual(status.data[0]["dataset"], "daily")
if __name__ == "__main__":
unittest.main()
+124
View File
@@ -0,0 +1,124 @@
from __future__ import annotations
import os
import stat
import subprocess
import tempfile
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
CHECK = ROOT / "tools" / "check_deploy_baseline.sh"
BUILD = ROOT / "tools" / "build_image.sh"
def run_check(repo: Path, candidate: str, live: str) -> subprocess.CompletedProcess[str]:
env = os.environ.copy()
env["GIT_DIR"] = str(repo / ".git")
env["GIT_WORK_TREE"] = str(repo)
return subprocess.run(
["bash", str(CHECK), candidate, "--live-revision", live],
cwd=repo,
capture_output=True,
text=True,
env=env,
check=False,
)
def git(repo: Path, *args: str) -> str:
result = subprocess.run(
["git", *args],
cwd=repo,
capture_output=True,
text=True,
check=True,
)
return result.stdout.strip()
class DeployBaselineGateTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.tmpdir = tempfile.TemporaryDirectory()
cls.repo = Path(cls.tmpdir.name) / "repo"
cls.repo.mkdir()
git(cls.repo, "init")
git(cls.repo, "config", "user.email", "gate@example.com")
git(cls.repo, "config", "user.name", "Gate")
(cls.repo / "README").write_text("base\n", encoding="utf-8")
git(cls.repo, "add", "README")
git(cls.repo, "commit", "-m", "base")
cls.base = git(cls.repo, "rev-parse", "HEAD")
(cls.repo / "online.txt").write_text("live\n", encoding="utf-8")
git(cls.repo, "add", "online.txt")
git(cls.repo, "commit", "-m", "online")
cls.live = git(cls.repo, "rev-parse", "HEAD")
git(cls.repo, "checkout", "-b", "successor")
(cls.repo / "next.txt").write_text("next\n", encoding="utf-8")
git(cls.repo, "add", "next.txt")
git(cls.repo, "commit", "-m", "successor of live")
cls.successor = git(cls.repo, "rev-parse", "HEAD")
git(cls.repo, "checkout", "-B", "lagging-main", cls.base)
(cls.repo / "stale.txt").write_text("stale main\n", encoding="utf-8")
git(cls.repo, "add", "stale.txt")
git(cls.repo, "commit", "-m", "lagging main")
cls.lagging = git(cls.repo, "rev-parse", "HEAD")
git(cls.repo, "checkout", "-B", "side", cls.base)
(cls.repo / "side.txt").write_text("side branch\n", encoding="utf-8")
git(cls.repo, "add", "side.txt")
git(cls.repo, "commit", "-m", "unrelated side branch")
cls.side = git(cls.repo, "rev-parse", "HEAD")
git(cls.repo, "checkout", "-B", "successor", cls.successor)
@classmethod
def tearDownClass(cls) -> None:
cls.tmpdir.cleanup()
def test_check_script_is_executable(self) -> None:
self.assertTrue(CHECK.exists())
self.assertTrue(stat.S_IXUSR & CHECK.stat().st_mode)
def test_successor_of_live_passes(self) -> None:
result = run_check(self.repo, self.successor, self.live)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn(self.live, result.stdout)
self.assertIn(self.successor, result.stdout)
self.assertIn("next.txt", result.stdout)
self.assertIn("祖先关系通过", result.stdout)
def test_lagging_main_is_blocked(self) -> None:
result = run_check(self.repo, self.lagging, self.live)
self.assertNotEqual(result.returncode, 0)
self.assertIn("拒绝", result.stderr)
def test_side_branch_is_blocked(self) -> None:
result = run_check(self.repo, self.side, self.live)
self.assertNotEqual(result.returncode, 0)
self.assertIn("拒绝", result.stderr)
def test_unknown_commit_is_blocked(self) -> None:
result = run_check(self.repo, "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef", self.live)
self.assertNotEqual(result.returncode, 0)
self.assertIn("无法解析", result.stderr)
def test_build_image_calls_the_gate_and_rejects_latest(self) -> None:
source = BUILD.read_text(encoding="utf-8")
self.assertIn("check_deploy_baseline.sh", source)
self.assertIn("禁止构建 latest", source)
self.assertIn("org.opencontainers.image.revision", source)
gate = CHECK.read_text(encoding="utf-8")
self.assertIn("org.opencontainers.image.revision", gate)
self.assertIn("merge-base --is-ancestor", gate)
self.assertIn("候选将丢失的提交", gate)
self.assertIn("禁止人工填写", gate)
if __name__ == "__main__":
unittest.main()
+13
View File
@@ -459,6 +459,19 @@ class FrontendContractTests(unittest.TestCase):
self.assertIn('payload.question_preset = state.heartQuestionPreset;', self.script)
self.assertIn('payload.cast_at = state.heartCastAt;', self.script)
def test_heaven_loading_timeout_clears_dimmed_state(self):
self.assertIn("controller.abort()", self.script)
self.assertIn('heavenView?.classList.remove("heaven-data-loading")', self.script)
self.assertIn("问天数据仍在准备,页面可继续输入和操作", self.script)
self.assertIn("const blocking = !state.heavenSetup;", self.script)
self.assertIn("payload?.aborted", self.script)
def test_stock_detail_does_not_display_missing_metrics_as_zero(self):
self.assertIn("function setStockBoardFields(row)", self.script)
self.assertIn("function presentMetric(value)", self.script)
self.assertIn("payload.available !== false", self.script)
self.assertIn('element.textContent = "--"', self.script)
if __name__ == "__main__":
unittest.main()
+1 -1
View File
@@ -94,7 +94,7 @@ class GlobalSearchTests(unittest.TestCase):
self.assertIn('event.key.toLowerCase() !== "k"', script)
self.assertIn('openStock(item.id, { code: item.code', script)
self.assertNotIn('include_notes', script)
self.assertIn('const candles = (series || [])', script)
self.assertIn('const candles = visibleDailyPrices((series || [])', script)
self.assertIn('renderStockNotes(payload.notes || [])', script)
+150 -1
View File
@@ -1,13 +1,162 @@
from __future__ import annotations
import json
import tempfile
import unittest
from http import HTTPStatus
from pathlib import Path
from unittest.mock import patch
from backend.features.heaven.engine import build_five_phase_field, hexagram_from_lines
from backend.features.heaven.knowledge import prepare_heaven_context
from backend.features.heaven.http import HeavenHttpMixin
from backend.features.heaven.knowledge import (
HeavenKnowledgeError,
clear_heaven_knowledge_cache,
prepare_heaven_context,
resolve_heaven_knowledge_path,
_knowledge_catalog,
)
from backend.features.heaven.six_yao import build_six_yao_chart
class HeavenKnowledgeTests(unittest.TestCase):
def tearDown(self) -> None:
clear_heaven_knowledge_cache()
def test_catalog_loads_from_trusted_repo_file(self):
clear_heaven_knowledge_cache()
path = resolve_heaven_knowledge_path()
catalog = _knowledge_catalog()
self.assertTrue(path.is_file())
self.assertEqual(path.name, "heaven_knowledge.json")
self.assertTrue(str(catalog.get("version") or "").startswith("2026."))
self.assertIn("zhouyi", catalog["sources"])
self.assertIn("neijing", catalog["sources"])
self.assertEqual(len(catalog["fortune"]["qi"]), 6)
self.assertEqual(len(catalog["fortune"]["personal_relations"]), 10)
def test_missing_knowledge_file_raises_chinese_structured_error(self):
clear_heaven_knowledge_cache()
missing = Path(tempfile.mkdtemp()) / "missing-heaven_knowledge.json"
with patch(
"backend.features.heaven.knowledge.KNOWLEDGE_FILE", missing
), patch(
"backend.features.heaven.knowledge.KNOWLEDGE_SEED_FILE",
missing.with_name("missing-seed.json"),
):
with self.assertRaises(HeavenKnowledgeError) as raised:
_knowledge_catalog()
self.assertEqual(raised.exception.error_code, "heaven_knowledge_missing")
self.assertIn("缺失", str(raised.exception))
def test_corrupt_knowledge_json_raises_chinese_structured_error(self):
clear_heaven_knowledge_cache()
with tempfile.TemporaryDirectory() as temp_dir:
broken = Path(temp_dir) / "heaven_knowledge.json"
broken.write_text("{not-json", encoding="utf-8")
with patch(
"backend.features.heaven.knowledge.KNOWLEDGE_FILE", broken
), patch(
"backend.features.heaven.knowledge.KNOWLEDGE_SEED_FILE",
Path(temp_dir) / "unused-seed.json",
):
with self.assertRaises(HeavenKnowledgeError) as raised:
_knowledge_catalog()
self.assertEqual(raised.exception.error_code, "heaven_knowledge_invalid")
self.assertIn("损坏", str(raised.exception))
def test_interpret_http_returns_structured_chinese_error_for_missing_file(self):
class FakeHandler(HeavenHttpMixin):
def __init__(self) -> None:
self.payload = None
self.status = None
self.application_service = type(
"Svc",
(),
{
"heaven_interpret": staticmethod(
lambda _body: (_ for _ in ()).throw(
HeavenKnowledgeError(
"问天知识文件缺失:未找到 heaven_knowledge.json。",
code="heaven_knowledge_missing",
)
)
)
},
)()
def read_json_body(self):
return {"mode": "trend", "trade_date": "2026-08-04"}
def send_json(self, payload, status=HTTPStatus.OK, headers=None):
self.payload = payload
self.status = status
handler = FakeHandler()
handler.heaven_interpret()
self.assertEqual(handler.status, HTTPStatus.BAD_REQUEST)
self.assertIn("缺失", handler.payload["error"])
self.assertEqual(handler.payload["code"], "heaven_knowledge_missing")
def test_interpret_http_returns_structured_chinese_error_for_corrupt_json(self):
class FakeHandler(HeavenHttpMixin):
def __init__(self) -> None:
self.payload = None
self.status = None
self.application_service = type(
"Svc",
(),
{
"heaven_interpret": staticmethod(
lambda _body: (_ for _ in ()).throw(
HeavenKnowledgeError(
"问天知识文件 JSON 损坏(heaven_knowledge.json),无法解析:第 1 行附近。",
code="heaven_knowledge_invalid",
)
)
)
},
)()
def read_json_body(self):
return {"mode": "trend", "trade_date": "2026-08-04"}
def send_json(self, payload, status=HTTPStatus.OK, headers=None):
self.payload = payload
self.status = status
handler = FakeHandler()
handler.heaven_interpret()
self.assertEqual(handler.status, HTTPStatus.BAD_REQUEST)
self.assertIn("损坏", handler.payload["error"])
self.assertEqual(handler.payload["code"], "heaven_knowledge_invalid")
def test_seed_fallback_when_data_file_missing(self):
clear_heaven_knowledge_cache()
with tempfile.TemporaryDirectory() as temp_dir:
seed = Path(temp_dir) / "seed.json"
seed.write_text(
json.dumps(
{
"version": "test-seed",
"sources": {"zhouyi": {"title": "周易"}},
"trend": {"method": "m", "rules": {"stable": "s", "single": "a", "multiple": "b"}},
"fortune": {},
"heart": {},
},
ensure_ascii=False,
),
encoding="utf-8",
)
missing_data = Path(temp_dir) / "data-heaven_knowledge.json"
with patch(
"backend.features.heaven.knowledge.KNOWLEDGE_FILE", missing_data
), patch(
"backend.features.heaven.knowledge.KNOWLEDGE_SEED_FILE", seed
):
catalog = _knowledge_catalog()
self.assertEqual(catalog["version"], "test-seed")
def test_fortune_context_excludes_weighted_summary_and_adds_bounded_industry_symbols(self):
field = build_five_phase_field("2026-08-04")
prepared = prepare_heaven_context(
+83 -1
View File
@@ -3,9 +3,10 @@ from __future__ import annotations
import http.client
import json
import unittest
from datetime import datetime
from unittest.mock import MagicMock, patch
from backend.data.realtime import WebRealtimeAggregator
from backend.data.realtime import RealtimeAggregateError, WebRealtimeAggregator
from backend.features.heaven.engine import _market_line_scores, build_manual_market_hexagram
from server import DashboardService
from backend.data.providers.tushare_client import (
@@ -377,6 +378,87 @@ class RealtimeAggregatorTests(unittest.TestCase):
self.assertEqual(rows[0]["quote_time"][:10], "2026-07-20")
self.assertAlmostEqual(rows[0]["amount_billion"], 12946.52)
@patch.object(WebRealtimeAggregator, "_get_json")
def test_eastmoney_market_quotes_normalize_and_keep_expected_date(self, get_json: MagicMock):
epoch = datetime(2026, 7, 20, 10, 5).timestamp()
rows = []
for index in range(200):
sz = index < 100
rows.append(
{
"f12": f"{index:06d}" if sz else f"{600000 + index - 100:06d}",
"f13": 0 if sz else 1,
"f14": f"股票{index}",
"f2": 11.2,
"f3": 2.0,
"f5": 10,
"f6": 50000000,
"f15": 11.3,
"f16": 11.0,
"f17": 11.1,
"f18": 11.0,
"f124": epoch,
}
)
def fake_get_json(_url, params, referer=""):
page = int(params.get("pn") or 1)
start = (page - 1) * 100
return {"rc": 0, "data": {"total": 200, "diff": rows[start:start + 100]}}
get_json.side_effect = fake_get_json
aggregator = WebRealtimeAggregator()
aggregator._response_cache.clear()
quotes = aggregator.eastmoney_market_quotes("20260720")
self.assertEqual(len(quotes), 200)
self.assertEqual(quotes[0]["ts_code"], "000000.SZ")
self.assertTrue(quotes[100]["ts_code"].endswith(".SH"))
self.assertEqual(quotes[0]["vol"], 1000)
self.assertEqual(quotes[0]["quote_date"], "20260720")
@patch.object(WebRealtimeAggregator, "_get_text")
def test_tencent_stock_quote_keeps_expected_date(self, get_text: MagicMock):
fields = [""] * 38
fields[1] = "浦发银行"
fields[2] = "600000"
fields[3] = "11.20"
fields[4] = "11.00"
fields[5] = "11.10"
fields[6] = "1234"
fields[30] = "20260720103000"
fields[33] = "11.30"
fields[34] = "11.00"
fields[37] = "1380"
get_text.return_value = (f'v_sh600000="{"~".join(fields)}";', 0)
quote = WebRealtimeAggregator().tencent_stock_quote("600000", "20260720")
self.assertEqual(quote["ts_code"], "600000.SH")
self.assertEqual(quote["quote_date"], "20260720")
self.assertEqual(quote["vol"], 123400)
self.assertAlmostEqual(quote["amount"], 13_800_000)
@patch.object(WebRealtimeAggregator, "_get_json")
def test_eastmoney_stock_quote_rejects_stale_date(self, get_json: MagicMock):
epoch = datetime(2026, 7, 19, 15, 0).timestamp()
get_json.return_value = {
"rc": 0,
"data": {
"f43": 11.2,
"f44": 11.3,
"f45": 11.0,
"f46": 11.1,
"f47": 10,
"f48": 50000000,
"f57": "300750",
"f58": "宁德时代",
"f60": 11.0,
"f86": epoch,
},
}
with self.assertRaises(RealtimeAggregateError):
WebRealtimeAggregator().eastmoney_stock_quote("300750.SZ", "20260720")
if __name__ == "__main__":
unittest.main()
+342
View File
@@ -0,0 +1,342 @@
from __future__ import annotations
import unittest
from unittest.mock import MagicMock, patch
from backend.data.providers.tushare_client import TushareClient, TushareError
from backend.data.providers.tushare_helpers import _moneyflow_payload
from backend.data.realtime import (
WebRealtimeAggregator,
_normalize_eastmoney_limit_row,
_normalize_eastmoney_stock_quote,
)
from backend.data.providers.tushare_daily import DailyMarketMixin
class MoneyflowPayloadTests(unittest.TestCase):
def test_missing_row_is_not_zero(self) -> None:
payload = _moneyflow_payload(None)
self.assertFalse(payload["available"])
self.assertIsNone(payload["net_million"])
self.assertIsNone(payload["large_million"])
def test_empty_row_is_not_zero(self) -> None:
payload = _moneyflow_payload({})
self.assertFalse(payload["available"])
self.assertIsNone(payload["net_million"])
def test_real_zero_net_is_kept_when_source_exists(self) -> None:
payload = _moneyflow_payload(
{
"net_mf_amount": 0,
"buy_lg_amount": 1,
"sell_lg_amount": 1,
"buy_elg_amount": 0,
"sell_elg_amount": 0,
"buy_md_amount": 0,
"sell_md_amount": 0,
"buy_sm_amount": 0,
"sell_sm_amount": 0,
}
)
self.assertTrue(payload["available"])
self.assertEqual(payload["net_million"], 0)
class LimitOverlayTests(unittest.TestCase):
def test_normalize_limit_keeps_missing_seal_as_none(self) -> None:
row = DailyMarketMixin._normalize_limit(
{
"ts_code": "000737.SZ",
"name": "北方铜业",
"close": 12.3,
"pct_chg": 10,
"amount": 1e8,
"amount_unit": "yuan",
},
"涨停",
)
self.assertIsNone(row["seal_amount_million"])
self.assertEqual(row["first_time"], "--")
def test_overlay_fills_board_times_from_official_list(self) -> None:
mixin = DailyMarketMixin()
mixin._load_limit_lists = lambda trade_date: [
{
"ts_code": "000737.SZ",
"first_time": "09:31:02",
"last_time": "10:18:11",
"fd_amount": 82000000,
"open_times": 1,
"turnover_ratio": 18.4,
}
]
mixin.realtime_aggregator = None
rows = mixin._overlay_board_fields(
[{"ts_code": "000737.SZ", "close": 12.3, "limit_type": "U"}],
"20260908",
)
self.assertEqual(rows[0]["first_time"], "09:31:02")
self.assertEqual(rows[0]["fd_amount"], 82000000)
self.assertEqual(rows[0]["turnover_ratio"], 18.4)
class ShenwanRealtimeSourceTests(unittest.TestCase):
def test_transport_refuses_rt_sw_k(self) -> None:
client = TushareClient(token="demo")
with self.assertRaisesRegex(TushareError, "rt_sw_k is disabled"):
client.query("rt_sw_k", {"ts_code": "801074.SI"})
def test_outer_realtime_uses_hub_sector_quote_not_rt_sw_k(self) -> None:
client = TushareClient(token="demo")
client.query = MagicMock(side_effect=AssertionError("should not call tushare"))
client.try_sector_quote = MagicMock(return_value={
"code": "801074.SI",
"name": "工业金属",
"close": 1234.5,
"pre_close": 1200,
"change": 2.88,
"pct_change": 2.88,
"quote_date": "20260908",
"quote_time": "2026-09-08T14:50:00+08:00",
"source": "eastmoney_sw",
})
row, source, error = client._sw_outer_realtime("801074.SI", "工业金属", "20260908")
self.assertEqual(source, "eastmoney_sw")
self.assertEqual(error, "")
self.assertEqual(row["change"], 2.88)
client.query.assert_not_called()
def test_outer_waiting_state_has_no_permission_error(self) -> None:
client = TushareClient(token="demo")
client.realtime_aggregator = None
row, source, error = client._sw_outer_realtime(
"801074.SI", "工业金属", "20260908", finalized=True
)
self.assertEqual(row, {})
self.assertIn("尚未入库", error)
self.assertNotIn("权限", error)
self.assertNotIn("rt_sw_k", error)
class EastmoneyHelperTests(unittest.TestCase):
def test_limit_pool_row_keeps_board_clock(self) -> None:
parsed = _normalize_eastmoney_limit_row(
{
"c": "000737",
"m": 0,
"n": "北方铜业",
"fbt": 93102,
"lbt": 101811,
"zbc": 1,
"lbc": 2,
"hs": 18.4,
"fund": 82000000,
},
"U",
)
self.assertEqual(parsed["ts_code"], "000737.SZ")
self.assertEqual(parsed["first_time"], "09:31:02")
self.assertEqual(parsed["last_time"], "10:18:11")
self.assertEqual(parsed["fd_amount"], 82000000)
def test_stock_quote_keeps_moneyflow_when_present(self) -> None:
quote = _normalize_eastmoney_stock_quote(
{
"f43": 12.3,
"f60": 11.18,
"f46": 11.2,
"f44": 12.3,
"f45": 11.1,
"f47": 1000,
"f48": 150000000,
"f58": "北方铜业",
"f86": 0,
"f168": 8.5,
"f62": 25000000,
"f78": 3000000,
"f84": -1000000,
},
"000737.SZ",
)
self.assertEqual(quote["net_mf_amount"], 2500)
payload = _moneyflow_payload(quote)
self.assertTrue(payload["available"])
self.assertEqual(payload["net_million"], 25)
@patch.object(WebRealtimeAggregator, "_get_json")
def test_shenwan_quote_uses_eastmoney_90_prefix(self, get_json: MagicMock) -> None:
get_json.return_value = {
"rc": 0,
"data": {
"diff": [
{
"f12": "801074",
"f14": "工业金属",
"f2": 1234.5,
"f3": 2.88,
"f18": 1200,
"f17": 1205,
"f15": 1240,
"f16": 1198,
"f6": 1,
"f124": 1757319000,
}
]
},
}
quote = WebRealtimeAggregator().eastmoney_shenwan_quote("801074.SI")
self.assertEqual(quote["source"], "eastmoney_sw")
self.assertAlmostEqual(quote["change"], 2.88)
params = get_json.call_args.args[1]
self.assertEqual(params["secids"], "90.801074")
class ChartWindowTests(unittest.TestCase):
def test_display_window_is_45_not_250(self) -> None:
from backend.features.market.charts import DAILY_CHART_LIMIT
self.assertEqual(DAILY_CHART_LIMIT, 45)
class MemberQuoteCoverageTests(unittest.TestCase):
def test_prefers_full_hub_market_over_truncated_named_quotes(self) -> None:
client = TushareClient(token="demo")
wanted = [f"{index:06d}.SZ" for index in range(205)]
market = [
{"ts_code": code, "close": 10.0, "pre_close": 9.0}
for code in wanted
]
client.try_market_quotes = MagicMock(return_value=market)
client.try_quotes = MagicMock(return_value=market[:60])
client.realtime_aggregator = MagicMock()
rows, source = client._load_member_realtime_quotes(wanted, "20260908")
self.assertEqual(len(rows), 205)
self.assertEqual(source, "datahub")
client.try_quotes.assert_not_called()
def test_hub_named_quotes_cover_members_when_market_missing(self) -> None:
client = TushareClient(token="demo")
wanted = ["000737.SZ", "000630.SZ"]
client.try_market_quotes = MagicMock(return_value=None)
client.try_quotes = MagicMock(return_value=[
{"ts_code": "000737.SZ", "close": 12.3, "pre_close": 11.2},
{"ts_code": "000630.SZ", "close": 4.5, "pre_close": 4.4},
])
client.realtime_aggregator = MagicMock()
rows, source = client._load_member_realtime_quotes(wanted, "20260908")
self.assertEqual(len(rows), 2)
self.assertEqual(source, "datahub")
client.try_quotes.assert_called()
client.realtime_aggregator.eastmoney_stock_quotes.assert_not_called()
client.realtime_aggregator.tencent_stock_quotes.assert_not_called()
def test_delayed_hub_quotes_are_kept_not_zeroed(self) -> None:
client = TushareClient(token="demo")
delayed = [
{
"ts_code": "000737.SZ",
"close": 12.3,
"pre_close": 11.2,
"delayed": True,
"delay_seconds": 90,
"delay_notice": "主备免费行情均暂不可用,显示 90 秒前的真实快照",
}
]
client.try_market_quotes = MagicMock(return_value=delayed)
client.try_quotes = MagicMock()
client.realtime_aggregator = MagicMock()
rows, source = client._load_member_realtime_quotes(["000737.SZ"], "20260908")
self.assertEqual(source, "datahub_delayed")
self.assertEqual(rows[0]["close"], 12.3)
client.try_quotes.assert_not_called()
def test_ignores_non_member_quotes_from_market_snapshot(self) -> None:
client = TushareClient(token="demo")
client.try_market_quotes = MagicMock(
return_value=[
{"ts_code": "000737.SZ", "close": 12.3, "pre_close": 11.2},
{"ts_code": "600000.SH", "close": 10.0, "pre_close": 9.9},
]
)
client.try_quotes = MagicMock(return_value=[])
client._free_realtime_quotes = MagicMock(return_value=([], "empty"))
rows, _source = client._load_member_realtime_quotes(
["000737.SZ", "000630.SZ"], "20260908"
)
self.assertEqual([row["ts_code"] for row in rows], ["000737.SZ"])
def test_local_sw_members_survive_tushare_outage(self) -> None:
import tempfile
from pathlib import Path
from backend.data.providers import tushare_industries as module
client = TushareClient(token="demo")
stored = [
{
"ts_code": "000737.SZ",
"name": "北方铜业",
"l2_code": "801074.SI",
"in_date": "20200101",
"out_date": "",
}
]
with tempfile.TemporaryDirectory() as tmp:
original = module._SW_MEMBER_DIR
module._SW_MEMBER_DIR = Path(tmp)
try:
client._sw_member_cache.clear()
client._write_local_sw_members("801074.SI", stored)
client.query = MagicMock(side_effect=TushareError("index_member_all down"))
members = client._sw_sector_members("801074.SI", "20260908")
finally:
module._SW_MEMBER_DIR = original
client._sw_member_cache.clear()
self.assertEqual([item["ts_code"] for item in members], ["000737.SZ"])
client.query.assert_not_called()
def test_closed_keeps_daily_inner_when_sw_daily_missing(self) -> None:
client = TushareClient(token="demo")
client.resolve_trade_context = lambda _date: ("20260908", "20260907")
client.sw_stock_industry = MagicMock(
return_value={"l2_code": "801074.SI", "l2_name": "工业金属"}
)
client._sw_sector_members = MagicMock(
return_value=[
{"ts_code": "000737.SZ", "name": "北方铜业"},
{"ts_code": "000630.SZ", "name": "铜陵有色"},
]
)
client._stock_listing_reference = MagicMock(return_value={})
client._load_daily = MagicMock(
return_value=[
{"ts_code": "000737.SZ", "name": "北方铜业", "pct_chg": 2, "amount": 1e8},
{"ts_code": "000630.SZ", "name": "铜陵有色", "pct_chg": 1, "amount": 1e8},
]
)
client._confirmed_suspended_members = MagicMock(return_value=[])
client.query = MagicMock(return_value=[])
client._sw_realtime_sector_snapshot = MagicMock(
side_effect=AssertionError("daily inner should be kept")
)
client.try_sector_quote = MagicMock(return_value={
"code": "801074.SI",
"name": "工业金属",
"change": 1.5,
"pct_change": 1.5,
"quote_date": "20260908",
"quote_time": "2026-09-08T15:00:00+08:00",
"source": "eastmoney_sw",
})
snapshot = client.sw_sector_snapshot(
"000737.SZ", "20260908", allow_realtime_close=True
)
self.assertEqual(snapshot["quote_count"], 2)
self.assertEqual(snapshot["member_count"], 2)
self.assertTrue(snapshot["inner_precise"])
self.assertTrue(snapshot["outer_precise"])
self.assertEqual(snapshot["inner_source"], "tushare_member_daily")
self.assertEqual(snapshot["change"], 1.5)
self.assertNotIn("权限", snapshot.get("outer_error") or "")
self.assertNotIn("rt_sw_k", snapshot.get("outer_error") or "")

Some files were not shown because too many files have changed in this diff Show More