leefer/xiaobaifupan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(migration): normalize legacy screener archives

Browse Source
This commit is contained in:
leefer
2026-07-30 19:49:24 +08:00
parent 198806c1bd
commit e5f7d836ac
8 changed files with 151 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
next/docs/evidence/local-preflight-2026-07-30.md
+5 -3
View File
@@ -5,7 +5,7 @@
## 范围
- 新版地址：`http://127.0.0.1:8877`
- 新版数据库：`next/data/local-preflight/xiaobai-normalized-v5.db`
- 新版数据库：`next/data/local-preflight/xiaobai-normalized-v7.db`
- 旧版`8765`未启动、未占用、未修改。
- 本轮未连接NAS，未执行Docker构建或容器切换。
@@ -14,6 +14,8 @@
- SQLite integrity：`ok`
- schema版本：12
- 用户：3；自选：6；复盘笔记：3；问师消息：28；问天记录：31；策略跟踪：16
- 旧库250条选股运行记录归一为107条有效归档：内置策略映射到当前36套目录的稳定ID，同日同策略同版本重复执行只保留最新一条；被去重运行关联的跟踪记录重映射到保留记录。
- 内置阶段/精选结果改为系统共享归档；未知或用户自建策略继续保留原账号归属。旧“多因子综合打分(IC动态加权)”按当前产品口径显示为“动态多因子（基础版）”。
- 未映射旧表：0；未映射快照类型：0；外键异常：0
- 情绪快照按真实交易日去重；旧金额单位、涨跌停字段、情绪样本和个人出生信息已转换为新版契约。
@@ -31,7 +33,7 @@
- 1920×1080：情绪、轮动、智能选股、问天、复盘、系统管理均整页纵向滚动且无横向溢出。
- 3840×2160：内容区最大宽度2200px，居中且无横向溢出，基础字号不随视口缩放。
- 390×844：侧栏隐藏、底部导航和移动上下文栏显示；表格转卡片；交互控件触控高度不低于44px。
- 390×844：侧栏隐藏、底部导航和移动上下文栏显示；表格转卡片；交互控件触控高度不低于44px；长页切换工作区会回到新页顶部，前进/后退仍恢复历史位置。
- 停止并重启`8877`后：健康检查通过，管理员会话、默认日期、20行情绪明细、6只自选和系统配置保持。
## 自动质量门
@@ -41,7 +43,7 @@
- Vue类型检查：通过
- Vitest：7项通过
- Vite生产构建：通过
- Playwright：26项通过；在全新空白测试数据库中重跑，包含注册、权限、16工作区、日夜、390px和4K结构断言。
- Playwright：26项通过；在隔离测试数据库中重跑，包含注册、权限、16工作区、日夜、390px、长页路由复位和4K结构断言。
## 未关闭项
next/docs/final/completion-audit.md
+3
View File
@@ -35,6 +35,7 @@
- 管理员会员、会员非管理员、普通用户三个真实账户已验证双维权限；普通用户智能工具保持完整结构并锁定。
- 情绪、四类股池、天梯、轮动、竞价、题材、热榜、龙虎榜归档、智能选股、问师、问天、复盘和系统管理均读取到迁移后的真实数据。
- 1920×1080、3840×2160和390×844均无页面级横向溢出；移动端侧栏、底部导航、表格卡片化和44px触控目标已直接验证。
- 移动端长页切换工作区统一回到新页顶部，浏览器前进/后退恢复历史滚动位置；该行为已加入端到端回归。
- 服务进程停止并重新启动后，会话、默认数据日期、20行情绪明细、6只自选和系统配置均保持一致。
## 已在本轮纠正
@@ -45,6 +46,8 @@
- 上涨K线空心实体和上下影线不穿体已有几何断言。
- 情绪退潮警示无选股建议、60日末行可达、低行数股池状态栏固定已有直接 E2E。
- 阶段、精选、自定义三类结果切换和策略来源隔离已有直接 E2E。
- 旧库选股归档迁移会将内置策略映射到当前36套目录的稳定ID，统一共享归属并按日期、策略和版本去重；关联跟踪记录重映射到保留运行，未知自建策略仍保持账号私有。
- 旧“多因子综合打分(IC动态加权)”迁移后统一显示为“动态多因子（基础版）”，不继承过度宣称。
- 股票、板块、题材和指数共用唯一实体详情接口；详情涨跌幅按所选真实交易日及对应前收计算。
- 个股详情已包含交易指标、资金流、事件逻辑、自选操作、观势入口和私有个股笔记；题材详情展示成分股。
- 股池、天梯、轮动、题材、人气榜、龙虎榜、智能选股、自选、交易日志、策略跟踪和提醒中的股票代码
next/docs/final/page-state-matrix.md
+1 -1
View File
@@ -10,7 +10,7 @@
- `B04`：`stage4.spec.js`，账户权限、Shell、账户弹窗、主题即时切换和首次空系统。
- `B05-B12`：`stage5.spec.js`至`stage12.spec.js`，各领域正常交互、夜间和390px流程。
- `B13`：`stage13.spec.js`，16工作区移动端可达、320px至4K结构边界和触控目标。
- `B13`：`stage13.spec.js`，16工作区移动端可达、320px至4K结构边界、触控目标及长页切换后的滚动复位。
- `B14`：`stage14-matrix.spec.js`，逐页加载、空、失败和错误正文隐藏。
- `A账户`：`test_accounts.py`；`A行情`：`test_market_data.py`、`test_market_insights.py`。
- `A智能`：`test_screener.py`、`test_mentor.py`、`test_heaven.py`；`A私有`：`test_review.py`。
next/docs/final/spec-coverage.md
+1 -1
View File
@@ -130,7 +130,7 @@
| U02 | 阶段6的1920×1080整页滚动与明细可达 | 浏览器通过 |
| U03 | 阶段11观气/观心滚动与背景断言 | 浏览器通过 |
| U04 | `stage13.spec.js`自动验证4K最大宽度、字号、Shell和无溢出；信息密度 | 人工待验 |
| U05 | `stage13.spec.js`390×844、五入口与44px触控目标 | 浏览器通过 |
| U05 | `stage13.spec.js`390×844、五入口、44px触控目标及长页切换滚动复位 | 浏览器通过 |
| U06 | `stage13.spec.js`320px 16工作区、纵向记录/局部横滚 | 浏览器通过 |
## 汇总
next/frontend/src/app/router.ts
+3
View File
@@ -8,6 +8,9 @@ import { findWorkspace } from "./workspaceRegistry";
export default createRouter({
history: createWebHistory(),
scrollBehavior(_to, _from, savedPosition) {
return savedPosition ?? { left: 0, top: 0 };
},
routes: [
{ path: "/", redirect: "/workspace/emotion" },
{
next/tests/e2e/stage13.spec.js
+8
View File
@@ -76,6 +76,14 @@ test("mobile information architecture, touch targets and responsive tables", asy
await expect(page.locator(".data-table tbody td").filter({ hasText: "移动样本1" })).toHaveAttribute("data-label", "股票");
expect(await page.locator(".data-table tbody tr").first().evaluate((row) => row.scrollWidth <= row.clientWidth)).toBe(true);
await page.evaluate(() => window.scrollTo(0, document.documentElement.scrollHeight));
expect(await page.evaluate(() => window.scrollY)).toBeGreaterThan(0);
await page.getByRole("navigation", { name: "移动端主导航" }).getByRole("link", { name: "选股" }).click();
await expect(page).toHaveURL(/\/workspace\/screener$/);
await expect.poll(() => page.evaluate(() => window.scrollY)).toBe(0);
await page.getByRole("navigation", { name: "移动端主导航" }).getByRole("link", { name: "行情" }).click();
await page.getByLabel("切换行情页面").selectOption("pool");
for (const viewport of [[320, 568], [375, 667], [390, 844], [430, 932], [768, 1024], [844, 390]]) {
await assertViewport(page, viewport[0], viewport[1]);
}
next/tests/test_legacy_migration.py
+42 -2
View File
@@ -10,6 +10,7 @@ import sqlite3
from cryptography.fernet import Fernet
from backend.features.market.events import apply_event_revisions
from backend.features.screener.catalog import strategy_by_id
from backend.security.passwords import PasswordHasher
from tools.legacy_migration import LegacyMigrator
@@ -181,6 +182,20 @@ def _legacy_database(path, key: str) -> None:
"INSERT INTO screener_runs VALUES (1,'20260729','','策略','{}',?, ?,8,'curated')",
(result, now),
)
connection.execute(
"INSERT INTO screener_runs VALUES (2,'20260729','','退潮防守观察','{}',?, ?,8,'smart')",
(result, now),
)
connection.execute(
"INSERT INTO screener_runs VALUES (3,'20260729','','退潮防守观察','{}',?, ?,NULL,'smart')",
(result, now),
)
multi_factor = strategy_by_id("curated-25")
assert multi_factor is not None
connection.execute(
"INSERT INTO screener_runs VALUES (4,'20260729','','多因子综合打分(IC动态加权)',?, ?, ?,NULL,'curated')",
(json.dumps(multi_factor["formula"]), result, now),
)
connection.execute(
"INSERT INTO screener_strategies VALUES (1,'自定义','','[]','{}',0,?,?,8)",
(now, now),
@@ -193,6 +208,10 @@ def _legacy_database(path, key: str) -> None:
"INSERT INTO strategy_tracks VALUES (1,8,1,'20260729','策略','000001.SZ','000001','平安银行','银行',10.5,?,?)",
(now, now),
)
connection.execute(
"INSERT INTO strategy_tracks VALUES (2,8,2,'20260729','退潮防守观察','000001.SZ','000001','平安银行','银行',10.5,?,?)",
(now, now),
)
connection.execute(
"INSERT INTO data_snapshots VALUES ('popularity_v1','20260729','legacy','{}',?)", (now,)
)
@@ -386,8 +405,29 @@ def test_legacy_migration_is_idempotent_and_preserves_login(tmp_path) -> None:
"birth_time": "08:30",
"gender": "male",
}
assert connection.execute("SELECT count(*) FROM screener_runs").fetchone()[0] == 1
assert connection.execute("SELECT count(*) FROM strategy_tracks").fetchone()[0] == 1
assert connection.execute("SELECT count(*) FROM screener_runs").fetchone()[0] == 3
stage_run = connection.execute(
"""SELECT id,owner_user_id,strategy_id,strategy_version FROM screener_runs
WHERE mode='stage'"""
).fetchone()
assert dict(stage_run) == {
"id": 3,
"owner_user_id": None,
"strategy_id": "stage-06",
"strategy_version": 1,
}
renamed = connection.execute(
"""SELECT strategy_id,strategy_name FROM screener_runs
WHERE strategy_id='curated-25'"""
).fetchone()
assert dict(renamed) == {
"strategy_id": "curated-25",
"strategy_name": "动态多因子（基础版）",
}
assert connection.execute("SELECT count(*) FROM strategy_tracks").fetchone()[0] == 2
assert connection.execute(
"SELECT run_id FROM strategy_tracks WHERE id=2"
).fetchone()[0] == 3
assert connection.execute("SELECT daily_llm_limit FROM memberships").fetchone()[0] == 61
assert connection.execute("SELECT count(*) FROM seat_aliases").fetchone()[0] == 1
assert connection.execute(
next/tools/legacy_migration.py
+88 -14
View File
@@ -15,6 +15,7 @@ from cryptography.fernet import Fernet
from backend.data.sentiment import calculate_sentiment
from backend.database import MIGRATIONS, Database, MigrationRunner
from backend.features.screener.catalog import strategy_catalog
ARCHIVE_VERSION = "legacy-archive-v1"
@@ -109,6 +110,17 @@ def _dump(value: Any) -> str:
return json.dumps(value, ensure_ascii=False, separators=(",", ":"))
def _formula_fingerprint(value: Any) -> str:
formula = _json(value, {})
if not isinstance(formula, dict):
return ""
normalized = json.loads(json.dumps(formula, ensure_ascii=False))
metadata = normalized.get("meta")
if isinstance(metadata, dict):
metadata.pop("library_version", None)
return json.dumps(normalized, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
def _hash_file(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
@@ -286,6 +298,7 @@ class LegacyMigrator:
self.row_skips: dict[str, dict[str, Any]] = {}
self.user_ids: set[int] = set()
self.run_ids: set[int] = set()
self.run_id_map: dict[int, int] = {}
self.stock_ids: dict[str, str] = {}
self.admin_id: int | None = None
self.dashboard_events: dict[str, dict[str, str]] = {}
@@ -847,9 +860,73 @@ class LegacyMigrator:
self.counts["review_assistant_messages"] += 1
def _screener(self, source: sqlite3.Connection, target: sqlite3.Connection) -> None:
snapshots: dict[str, int] = {}
builtin_by_name = {
str(item["name"]): item
for item in strategy_catalog()
if item.get("kind") in {"stage", "curated"}
}
builtin_by_formula = {
(str(item["kind"]), _formula_fingerprint(item["formula"])): item
for item in builtin_by_name.values()
}
prepared: list[dict[str, Any]] = []
selected_by_key: dict[tuple[Any, ...], dict[str, Any]] = {}
for row in source.execute("SELECT * FROM screener_runs ORDER BY id"):
trade_date = _iso_date(row["trade_date"])
mode = {"smart": "stage", "curated": "curated", "quant": "custom"}.get(
str(row["mode"]), "custom"
)
formula = _json(row["formula"], {})
builtin = builtin_by_name.get(str(row["strategy_name"] or "")) or (
builtin_by_formula.get((mode, _formula_fingerprint(formula)))
if formula
else None
)
if builtin is not None and builtin["kind"] == mode:
strategy_id = str(builtin["id"])
strategy_name = str(builtin.get("display_name") or builtin["name"])
strategy_version = int(builtin["version"])
owner_user_id = None
else:
strategy_id = str(formula.get("id") or f"legacy-{row['id']}")
strategy_name = str(row["strategy_name"] or strategy_id)
raw_version = formula.get("version", 1)
strategy_version = (
int(raw_version)
if isinstance(raw_version, int) and not isinstance(raw_version, bool)
and raw_version > 0
else 1
)
owner_user_id = row["user_id"]
item = {
"row": row,
"mode": mode,
"formula": formula,
"strategy_id": strategy_id,
"strategy_name": strategy_name,
"strategy_version": strategy_version,
"owner_user_id": owner_user_id,
"trade_date": _iso_date(row["trade_date"]),
}
key = (
owner_user_id,
mode,
strategy_id,
item["trade_date"],
strategy_version,
)
item["key"] = key
prepared.append(item)
selected_by_key[key] = item
for item in prepared:
source_id = int(item["row"]["id"])
self.run_id_map[source_id] = int(selected_by_key[item["key"]]["row"]["id"])
snapshots: dict[str, int] = {}
selected = sorted(selected_by_key.values(), key=lambda item: int(item["row"]["id"]))
for item in selected:
row = item["row"]
trade_date = item["trade_date"]
if trade_date not in snapshots:
target.execute(
"""INSERT OR IGNORE INTO screener_factor_snapshots
@@ -871,23 +948,19 @@ class LegacyMigrator:
candidate["identifier"] = (
candidate.get("ts_code") or candidate.get("code") or ""
)
mode = {"smart": "stage", "curated": "curated", "quant": "custom"}.get(
str(row["mode"]), "custom"
)
formula = _json(row["formula"], {})
strategy_id = str(formula.get("id") or f"legacy-{row['id']}")
target.execute(
"""INSERT OR REPLACE INTO screener_runs
(id,owner_user_id,mode,strategy_id,strategy_name,strategy_version,
selection_date,factor_snapshot_id,status,started_at,completed_at,coverage,
missing_fields_json,result_json,error_message)
VALUES (?,?,?,?,?,1,?,?,?, ?,?,0,'[]',?,'')""",
VALUES (?,?,?,?,?,?,?,?,?, ?,?,0,'[]',?,'')""",
(
row["id"],
row["user_id"],
mode,
strategy_id,
row["strategy_name"],
item["owner_user_id"],
item["mode"],
item["strategy_id"],
item["strategy_name"],
item["strategy_version"],
trade_date,
snapshots[trade_date],
"completed" if candidates else "no_signal",
@@ -924,7 +997,8 @@ class LegacyMigrator:
)
self.counts["custom_screener_strategies"] += 1
for row in source.execute("SELECT * FROM strategy_tracks"):
if int(row["run_id"]) not in self.run_ids:
mapped_run_id = self.run_id_map.get(int(row["run_id"]))
if mapped_run_id not in self.run_ids:
continue
target.execute(
"""INSERT OR REPLACE INTO strategy_tracks
@@ -933,7 +1007,7 @@ class LegacyMigrator:
(
row["id"],
row["user_id"],
row["run_id"],
mapped_run_id,
row["ts_code"],
row["code"],
row["name"],