B-43: 双边流水归并与规范事件层——迁移5、匹配引擎、个人过账映射与事件API

This commit is contained in:
腾讯WorkBuddy
2026-08-18 21:06:08 +08:00
parent 486842963e
commit df517d4a68
10 changed files with 4565 additions and 49 deletions
@@ -0,0 +1,62 @@
# 005 规范转账事件与双边归并技术决策
对应 IssueB-43/B-83/B-114`docs/issues/006-p1-canonical-transfer-matching.md`)。
## 分层模型
- 银行源行是不可变观察;匹配层从不修改 `source_rows`
- 新增“不可变决定日志 + 可重建当前投影”的规范事件层:
- `transfer_match_decisions` 等证据表只增不改(UPDATE/DELETE 被触发器拒绝)。
- `current_transfer_decisions``transfer_observation_claims` 是可更新、可重建的
当前投影,`source_row_id` 主键保证一条观察不可能同时属于两个当前事件。
- B-44 余额计算只读 `eligible_intercompany_events` 视图,只含
`active + current + classification='intercompany'`;单边内部、待审、同公司调拨、
外部事件全部排除。
## 已确认业务口径(B-114
1. 单边流水即使双方公司都能由批准账号唯一确认,也先进入未决金额,不进入 B-44
已确认往来余额;只有完成双边归并(`intercompany` + `paired`)或管理员按证据确认
`intercompany` + locked 单边)后才可计算。
2. 跨日事件经济日期取付款方 outgoing 银行入账时间,与导入顺序无关。
3. 自动窗口 v1:非空参考号相同且账号镜像 <= 3 自然日(M1);无参考号精确账号镜像
<= 1 自然日(M2);别名/个人映射镜像 <= 1 自然日且参考号或摘要相等(M3)。
超窗或同层多候选全部进入人工审核,禁止按行号、导入顺序或名字破平局。
4. 个人过账映射绑定具体账号、生效区间、允许方向和代表公司,必须管理员批准;
姓名只作辅助证据,不能单独定案。
## 参与方解析
- 本方:源行 `own_account` 精确命中生效期内已批准账户(`own_exact`);缺失时才用
上传时持久化的批准上传账户(`upload_account`),且必须与批次公司一致。冲突进入
审核,不以 `company_id` 或户名反推。
- 对方:严格按账号层解析——精确账号 -> 已审核账号别名 -> 已批准且在窗口内、方向
允许的个人过账映射;每层唯一最佳结果才继续。户名/户名别名只用于一致性/冲突证据,
冲突降级到审核。
- 方向:`expense > 0 且 income = 0` 为 outgoing`income > 0 且 expense = 0`
incoming;双正/双零/负数/缺币种进入 `unresolved`。金额用 `Decimal` 精确相等,
不设手续费容差。
## 匹配与状态机
- 候选硬门槛:双方工作表 confirmed、方向相反、金额与币种相等、公司端点互为反向。
- M1/M2/M3 唯一候选自动 paired;R1(端点可确认但镜像/参考号证据不足)与 R2(同层
多候选)进入 `needs_review`;待审行不能被其他自动匹配抢占。
- 状态:`unresolved -> internal_single -> matched`、`needs_review -> matched |
same_company_transfer | external`;单边确认两端同公司立即为同公司调拨。
- 幂等:rule_version + 当前观察集合 + 证据未变时零写入;locked 人工决定自动重跑
跳过;纠错只能通过 `reverse``mode='reversal'` 新记录)+ 原因,之后允许重跑。
## 事务、并发与回滚
- “确认工作表 + 为新增确认行建立/更新事件 + audit”在同一个服务事务,任一失败整体
回滚,不存在 confirmed 但未匹配的半成品。
- 独立重跑使用 `BEGIN IMMEDIATE`,按 `source_row_id` 升序写,SQLite 30 秒 busy
timeout 保留;投影可从只增日志重建(`rebuild_current_projection`)。
- migration down 只用于测试/上线前回退;生产产生决定后默认保留 migration 5 与审计
数据,不执行破坏性 down。
## Decimal
金额全程以规范十进制字符串存储和比较,任何路径都不转 `float``100.0` 与 `100.00`
视为相等,`100.00` 与 `100.01` 不匹配。
+735 -22
View File
@@ -10,7 +10,7 @@ from http.cookies import SimpleCookie
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
from urllib.parse import parse_qs, urlparse from urllib.parse import parse_qs, urlparse
from bank_importer import auth, importing, master_data, multipart from bank_importer import auth, importing, master_data, matching, multipart, personal_transit
from bank_importer.db import connect, migrate, utc_now from bank_importer.db import connect, migrate, utc_now
@@ -77,6 +77,29 @@ class AppHandler(SimpleHTTPRequestHandler):
if path == "/api/admin/audit-log": if path == "/api/admin/audit-log":
self._handle_admin_audit_log(query) self._handle_admin_audit_log(query)
return return
if path == "/api/admin/transfer-events":
self._handle_admin_transfer_events(query)
return
if path == "/api/admin/match-exceptions":
self._handle_admin_match_exceptions(query)
return
if path == "/api/admin/personal-transit-mappings":
self._handle_admin_personal_mappings(query)
return
event_match = re.fullmatch(r"/api/admin/transfer-events/(\d+)", path)
if event_match:
self._handle_admin_transfer_event_detail(int(event_match.group(1)))
return
if path == "/api/company/transfer-events":
self._handle_company_transfer_events(query)
return
if path == "/api/company/match-exceptions":
self._handle_company_match_exceptions(query)
return
company_event_match = re.fullmatch(r"/api/company/transfer-events/(\d+)", path)
if company_event_match:
self._handle_company_transfer_event_detail(int(company_event_match.group(1)))
return
if path == "/admin.html" and not self._guard_page("admin"): if path == "/admin.html" and not self._guard_page("admin"):
return return
@@ -130,6 +153,20 @@ class AppHandler(SimpleHTTPRequestHandler):
if ignore_match: if ignore_match:
self._handle_batch_review(int(ignore_match.group(1)), "ignore") self._handle_batch_review(int(ignore_match.group(1)), "ignore")
return return
if path == "/api/admin/transfer-events/reconcile":
self._handle_admin_reconcile()
return
if path == "/api/admin/personal-transit-mappings":
self._handle_admin_create_personal_mapping()
return
decision_match = re.fullmatch(r"/api/admin/transfer-events/(\d+)/decisions", path)
if decision_match:
self._handle_admin_transfer_decision(int(decision_match.group(1)))
return
mapping_review = re.fullmatch(r"/api/admin/personal-transit-mappings/(\d+)/review", path)
if mapping_review:
self._handle_admin_review_personal_mapping(int(mapping_review.group(1)))
return
self._send_json(404, {"status": "error", "message": "接口不存在。"}) self._send_json(404, {"status": "error", "message": "接口不存在。"})
# ------------------------------------------------------------------ # ------------------------------------------------------------------
@@ -339,7 +376,10 @@ class AppHandler(SimpleHTTPRequestHandler):
# Tenant binding comes from the session only; any # Tenant binding comes from the session only; any
# company_id in the multipart body is ignored. # company_id in the multipart body is ignored.
company_id = user["company_id"] company_id = user["company_id"]
if not self._validate_upload_account(connection, user, fields): upload_account_id, invalid = self._resolve_upload_account(
connection, user, fields, company_id
)
if invalid:
return return
else: else:
company_id = self._parse_company_field( company_id = self._parse_company_field(
@@ -347,10 +387,16 @@ class AppHandler(SimpleHTTPRequestHandler):
) )
if company_id is None: if company_id is None:
return return
upload_account_id, invalid = self._resolve_upload_account(
connection, user, fields, company_id
)
if invalid:
return
result = importing.import_statement_path( result = importing.import_statement_path(
connection, STORAGE_DIR, filename, upload.path, connection, STORAGE_DIR, filename, upload.path,
company_id=company_id, company_id=company_id,
upload_bank_account_id=upload_account_id,
) )
auth.audit( auth.audit(
connection, connection,
@@ -410,32 +456,33 @@ class AppHandler(SimpleHTTPRequestHandler):
raise ValueError(str(exc)) from exc raise ValueError(str(exc)) from exc
return fields, upload return fields, upload
def _validate_upload_account(self, connection, user, fields) -> bool: def _resolve_upload_account(self, connection, user, fields, company_id) -> tuple[int | None, bool]:
"""Validate the optional bank_account_id on a company upload. """Validate and return the approved upload account for a batch.
When supplied, the account must belong to the session company and be Returns ``(account_id, False)`` when the account is accepted or when
enabled inside its effective interval — pending, returned or disabled none was supplied, and ``(None, True)`` after sending the rejection.
accounts never accept uploads. The approved account is persisted on the batch so ownership can be
resolved later even when source rows lack an own account.
""" """
raw = fields.get("bank_account_id") raw = fields.get("bank_account_id")
if not raw: if not raw:
return True return None, False
try: try:
account_id = int(raw) account_id = int(raw)
except ValueError: except ValueError:
self._send_json(400, {"status": "error", "message": "bank_account_id 参数无效。"}) self._send_json(400, {"status": "error", "message": "bank_account_id 参数无效。"})
return False return None, True
account = master_data.get_account(connection, account_id) account = master_data.get_account(connection, account_id)
if account is None or account["company_id"] != user["company_id"]: if account is None or account["company_id"] != company_id:
self._send_json(404, {"status": "error", "message": "账户不存在。"}) self._send_json(404, {"status": "error", "message": "账户不存在。"})
return False return None, True
if not master_data.is_usable(account, master_data.utc_today()): if not master_data.is_usable(account, master_data.utc_today()):
self._send_json( self._send_json(
409, 409,
{"status": "error", "message": "该账户未启用或已超出有效期,不能上传流水。"}, {"status": "error", "message": "该账户未启用或已超出有效期,不能上传流水。"},
) )
return False return None, True
return True return account_id, False
def _parse_company_field(self, connection, raw) -> int | None: def _parse_company_field(self, connection, raw) -> int | None:
try: try:
@@ -600,16 +647,24 @@ class AppHandler(SimpleHTTPRequestHandler):
except ValueError as exc: except ValueError as exc:
self._send_json(400, {"status": "error", "message": str(exc)}) self._send_json(400, {"status": "error", "message": str(exc)})
return return
except Exception:
# Confirmation and matching share one transaction; any failure
# rolls both back (the sheet stays pending).
self._send_json(
500,
{"status": "error", "message": "确认失败,本次确认与匹配已整体回滚。"},
)
return
rows = importing.sheet_review_rows(connection, batch_id) rows = importing.sheet_review_rows(connection, batch_id)
self._send_json( payload: dict[str, object] = {
200, "status": "ok",
{ "updated": outcome["updated"],
"status": "ok", "already": outcome["already"],
"updated": outcome["updated"], "sheets": [self._sheet_payload(row) for row in rows],
"already": outcome["already"], }
"sheets": [self._sheet_payload(row) for row in rows], if outcome.get("matching") is not None:
}, payload["matching"] = outcome["matching"]
) self._send_json(200, payload)
finally: finally:
connection.close() connection.close()
@@ -1261,6 +1316,664 @@ class AppHandler(SimpleHTTPRequestHandler):
finally: finally:
connection.close() connection.close()
# ------------------------------------------------------------------
# Canonical transfer events (admin)
# ------------------------------------------------------------------
@staticmethod
def _event_base_sql(company_scope: bool) -> str:
"""Shared list SELECT; ``company_scope`` filters to one company."""
where = (
"WHERE payer.company_id = ? OR payee.company_id = ?"
if company_scope
else ""
)
return f"""
SELECT c.event_id, d.id AS decision_id, d.revision, d.classification,
d.pairing, d.amount, d.currency, d.effective_at, d.mode,
d.locked, d.rule_version, d.created_at,
payer.company_id AS payer_company_id,
payee.company_id AS payee_company_id,
payer.bank_account_id AS payer_account_id,
payee.bank_account_id AS payee_account_id,
cpayer.name AS payer_company_name,
cpayee.name AS payee_company_name,
(SELECT COUNT(*) FROM transfer_decision_observations o
WHERE o.decision_id = d.id) AS evidence_count
FROM current_transfer_decisions c
JOIN transfer_match_decisions d ON d.id = c.decision_id
LEFT JOIN transfer_decision_participants payer
ON payer.decision_id = d.id AND payer.role = 'payer'
LEFT JOIN transfer_decision_participants payee
ON payee.decision_id = d.id AND payee.role = 'payee'
LEFT JOIN companies cpayer ON cpayer.id = payer.company_id
LEFT JOIN companies cpayee ON cpayee.id = payee.company_id
{where}
"""
def _handle_admin_transfer_events(self, query: dict[str, list[str]]) -> None:
connection = connect(DB_PATH)
try:
user = self._require_admin(connection)
if user is None:
return
conditions: list[str] = []
params: list[object] = []
raw_company = (query.get("company_id") or [None])[0]
if raw_company:
try:
params.append(int(raw_company))
params.append(int(raw_company))
except ValueError:
self._send_json(400, {"status": "error", "message": "company_id 参数无效。"})
return
conditions.append("(payer.company_id = ? OR payee.company_id = ?)")
raw_from = (query.get("from") or [None])[0]
if raw_from:
conditions.append("d.effective_at >= ?")
params.append(raw_from)
raw_to = (query.get("to") or [None])[0]
if raw_to:
conditions.append("d.effective_at <= ?")
params.append(raw_to)
raw_limit = (query.get("limit") or ["200"])[0]
try:
limit = max(1, min(int(raw_limit), 500))
except ValueError:
limit = 200
where = f"WHERE {' AND '.join(conditions)}" if conditions else ""
rows = connection.execute(
self._event_base_sql(False) + where + " ORDER BY d.id DESC LIMIT ?",
(*params, limit),
).fetchall()
items = [self._admin_event_list_item(row) for row in rows]
status = (query.get("status") or [None])[0]
if status:
items = [item for item in items if item["status"] == status]
self._send_json(200, {"status": "ok", "events": items})
finally:
connection.close()
@staticmethod
def _admin_event_list_item(row) -> dict[str, object]:
return {
"event_id": row["event_id"],
"decision_id": row["decision_id"],
"revision": row["revision"],
"classification": row["classification"],
"pairing": row["pairing"],
"status": matching.exposed_status(row),
"amount": row["amount"],
"currency": row["currency"],
"effective_at": row["effective_at"],
"mode": row["mode"],
"locked": bool(row["locked"]),
"rule_version": row["rule_version"],
"payer_company_id": row["payer_company_id"],
"payer_company_name": row["payer_company_name"],
"payee_company_id": row["payee_company_id"],
"payee_company_name": row["payee_company_name"],
"evidence_count": row["evidence_count"],
"created_at": row["created_at"],
}
def _handle_admin_transfer_event_detail(self, event_id: int) -> None:
connection = connect(DB_PATH)
try:
user = self._require_admin(connection)
if user is None:
return
payload = self._admin_event_detail(connection, event_id)
if payload is None:
self._send_json(404, {"status": "error", "message": "事件不存在。"})
return
self._send_json(200, {"status": "ok", "event": payload})
finally:
connection.close()
def _admin_event_detail(self, connection, event_id: int) -> dict[str, object] | None:
current = matching._current_decision_for_event(connection, event_id)
if current is None:
return None
decisions = connection.execute(
"SELECT * FROM transfer_match_decisions WHERE event_id = ? ORDER BY revision",
(event_id,),
).fetchall()
participants = matching._decision_participants(connection, current["id"])
observations = matching._decision_observations(connection, current["id"])
candidates = connection.execute(
"SELECT * FROM transfer_match_candidates WHERE decision_id = ? ORDER BY id",
(current["id"],),
).fetchall()
observation_rows = []
for observation in observations:
row = connection.execute(
"""
SELECT r.id, r.source_row, r.transaction_at, r.income, r.expense,
r.own_account, r.own_name, r.counterparty_account,
r.counterparty_name, r.summary, r.reference, r.currency,
s.sheet_name, f.original_filename,
b.company_id AS batch_company_id
FROM source_rows r
JOIN sheet_batches s ON s.id = r.sheet_batch_id
JOIN import_batches b ON b.id = s.import_batch_id
JOIN source_files f ON f.id = b.source_file_id
WHERE r.id = ?
""",
(observation["source_row_id"],),
).fetchone()
observation_rows.append(
{
"source_row_id": observation["source_row_id"],
"role": observation["role"],
"source_row": row["source_row"] if row else None,
"sheet_name": row["sheet_name"] if row else None,
"original_filename": row["original_filename"] if row else None,
"company_id": row["batch_company_id"] if row else None,
"transaction_at": row["transaction_at"] if row else None,
"income": row["income"] if row else None,
"expense": row["expense"] if row else None,
"counterparty_name": row["counterparty_name"] if row else None,
}
)
history = [
{
"decision_id": decision["id"],
"revision": decision["revision"],
"classification": decision["classification"],
"pairing": decision["pairing"],
"status": matching.exposed_status(decision),
"amount": decision["amount"],
"currency": decision["currency"],
"effective_at": decision["effective_at"],
"mode": decision["mode"],
"locked": bool(decision["locked"]),
"rule_version": decision["rule_version"],
"reason": decision["reason"],
"actor_username": decision["actor_username"],
"supersedes_decision_id": decision["supersedes_decision_id"],
"created_at": decision["created_at"],
}
for decision in decisions
]
return {
"event_id": event_id,
"decision_id": current["id"],
"revision": current["revision"],
"classification": current["classification"],
"pairing": current["pairing"],
"status": matching.exposed_status(current),
"amount": current["amount"],
"currency": current["currency"],
"effective_at": current["effective_at"],
"mode": current["mode"],
"locked": bool(current["locked"]),
"rule_version": current["rule_version"],
"participants": [
{
"role": participant["role"],
"company_id": participant["company_id"],
"bank_account_id": participant["bank_account_id"],
"resolve_method": participant["resolve_method"],
"evidence": json.loads(participant["evidence"])
if participant["evidence"]
else None,
}
for participant in participants
],
"observations": observation_rows,
"candidates": [
{
"source_row_id": candidate["source_row_id"],
"rule_tier": candidate["rule_tier"],
"date_diff_days": candidate["date_diff_days"],
"account_mirror": bool(candidate["account_mirror"]),
"reference_match": candidate["reference_match"],
"summary_match": bool(candidate["summary_match"]),
"accepted": candidate["accepted"],
"reason": candidate["reason"],
}
for candidate in candidates
],
"history": history,
}
def _handle_admin_match_exceptions(self, query: dict[str, list[str]]) -> None:
connection = connect(DB_PATH)
try:
user = self._require_admin(connection)
if user is None:
return
conditions = ["d.classification IN ('unresolved', 'needs_review')"]
params: list[object] = []
raw_company = (query.get("company_id") or [None])[0]
if raw_company:
try:
company_id = int(raw_company)
except ValueError:
self._send_json(400, {"status": "error", "message": "company_id 参数无效。"})
return
conditions.append("(payer.company_id = ? OR payee.company_id = ?)")
params.extend([company_id, company_id])
rows = connection.execute(
self._event_base_sql(False)
+ f"WHERE {' AND '.join(conditions)} ORDER BY d.id DESC LIMIT 500",
params,
).fetchall()
items = [self._admin_event_list_item(row) for row in rows]
self._send_json(200, {"status": "ok", "exceptions": items})
finally:
connection.close()
def _handle_admin_reconcile(self) -> None:
connection = connect(DB_PATH)
try:
user = self._require_admin(connection)
if user is None:
return
data = self._read_json_body()
if data is None:
return
raw_rows = data.get("source_row_ids")
row_ids: list[int] = []
if raw_rows is not None:
if not isinstance(raw_rows, list):
self._send_json(400, {"status": "error", "message": "source_row_ids 必须是数组。"})
return
try:
row_ids = [int(item) for item in raw_rows]
except (TypeError, ValueError):
self._send_json(400, {"status": "error", "message": "source_row_ids 必须是整数数组。"})
return
raw_batch = data.get("batch_id")
if raw_batch is not None and not row_ids:
try:
batch_id = int(raw_batch)
except (TypeError, ValueError):
self._send_json(400, {"status": "error", "message": "batch_id 参数无效。"})
return
rows = connection.execute(
"""
SELECT r.id FROM source_rows r
JOIN sheet_batches s ON s.id = r.sheet_batch_id
JOIN sheet_reviews rv ON rv.sheet_batch_id = s.id
WHERE s.import_batch_id = ? AND rv.review_status = 'confirmed'
ORDER BY r.id
""",
(batch_id,),
).fetchall()
row_ids = [item["id"] for item in rows]
if not row_ids:
self._send_json(400, {"status": "error", "message": "必须指定 source_row_ids 或 batch_id。"})
return
try:
result = matching.reconcile_rows(connection, row_ids, actor=user)
except Exception as exc:
self._send_json(500, {"status": "error", "message": f"重跑匹配失败:{exc}"})
return
auth.audit(
connection, "transfer_reconcile", actor=user,
target=f"rows:{len(row_ids)}",
detail=f"created:{result['created_events']};updated:{result['updated_events']}",
ip=self._client_ip,
)
self._send_json(200, {"status": "ok", "matching": result})
finally:
connection.close()
def _handle_admin_transfer_decision(self, event_id: int) -> None:
connection = connect(DB_PATH)
try:
user = self._require_admin(connection)
if user is None:
return
data = self._read_json_body()
if data is None:
return
action = str(data.get("action") or "")
try:
expected_revision = (
int(data["expected_revision"])
if data.get("expected_revision") is not None
else None
)
except (TypeError, ValueError):
self._send_json(400, {"status": "error", "message": "expected_revision 参数无效。"})
return
source_row_ids = data.get("source_row_ids")
if source_row_ids is not None and not isinstance(source_row_ids, list):
self._send_json(400, {"status": "error", "message": "source_row_ids 必须是数组。"})
return
try:
payload = matching.apply_manual_decision(
connection,
event_id,
action,
reason=str(data.get("reason") or ""),
expected_revision=expected_revision,
request_key=str(data.get("request_key") or "") or None,
actor=user,
source_row_ids=[int(item) for item in source_row_ids]
if source_row_ids
else None,
participant=data.get("participant"),
)
except matching.MatchConflictError as exc:
self._send_json(409, {"status": "error", "message": str(exc)})
return
except matching.MatchInputError as exc:
self._send_json(400, {"status": "error", "message": str(exc)})
return
self._send_json(200, {"status": "ok", "decision": payload})
finally:
connection.close()
# ------------------------------------------------------------------
# Personal transit mappings (admin)
# ------------------------------------------------------------------
def _handle_admin_personal_mappings(self, query: dict[str, list[str]]) -> None:
connection = connect(DB_PATH)
try:
user = self._require_admin(connection)
if user is None:
return
raw_company = (query.get("company_id") or [None])[0]
raw_status = (query.get("status") or [None])[0]
try:
company_id = int(raw_company) if raw_company else None
except ValueError:
self._send_json(400, {"status": "error", "message": "company_id 参数无效。"})
return
try:
rows = personal_transit.list_mappings(
connection, company_id=company_id, status=raw_status or None
)
except ValueError as exc:
self._send_json(400, {"status": "error", "message": str(exc)})
return
self._send_json(
200,
{"status": "ok",
"mappings": [personal_transit.mapping_payload(row, full=True) for row in rows]},
)
finally:
connection.close()
def _handle_admin_create_personal_mapping(self) -> None:
connection = connect(DB_PATH)
try:
user = self._require_admin(connection)
if user is None:
return
data = self._read_json_body()
if data is None:
return
try:
company_id = int(str(data.get("represented_company_id")))
except (TypeError, ValueError):
self._send_json(400, {"status": "error", "message": "represented_company_id 参数无效。"})
return
try:
mapping = personal_transit.submit_mapping(
connection,
account_number=data.get("account_number"),
account_name=data.get("account_name"),
represented_company_id=company_id,
allowed_direction=str(data.get("allowed_direction") or ""),
effective_from=data.get("effective_from"),
actor=user,
)
except personal_transit.ConflictError as exc:
self._send_json(409, {"status": "error", "message": str(exc)})
return
except ValueError as exc:
self._send_json(400, {"status": "error", "message": str(exc)})
return
auth.audit(
connection, "personal_transit_submit", actor=user,
target=f"personal_transit:{mapping['id']}",
detail=f"company:{company_id}",
ip=self._client_ip,
)
self._send_json(
200,
{"status": "ok",
"mapping": personal_transit.mapping_payload(mapping, full=True)},
)
finally:
connection.close()
def _handle_admin_review_personal_mapping(self, mapping_id: int) -> None:
connection = connect(DB_PATH)
try:
user = self._require_admin(connection)
if user is None:
return
data = self._read_json_body()
if data is None:
return
try:
mapping = personal_transit.review_mapping(
connection,
mapping_id,
str(data.get("decision") or ""),
str(data.get("reason") or ""),
user,
effective_from=data.get("effective_from"),
effective_to=data.get("effective_to"),
)
except LookupError as exc:
self._send_json(404, {"status": "error", "message": str(exc)})
return
except personal_transit.ConflictError as exc:
self._send_json(409, {"status": "error", "message": str(exc)})
return
except ValueError as exc:
self._send_json(400, {"status": "error", "message": str(exc)})
return
auth.audit(
connection, "personal_transit_review", actor=user,
target=f"personal_transit:{mapping_id}",
detail=str(data.get("decision")),
ip=self._client_ip,
)
self._send_json(
200,
{"status": "ok",
"mapping": personal_transit.mapping_payload(mapping, full=True)},
)
finally:
connection.close()
# ------------------------------------------------------------------
# Canonical transfer events (company, own-company scope only)
# ------------------------------------------------------------------
def _handle_company_transfer_events(self, query: dict[str, list[str]]) -> None:
connection = connect(DB_PATH)
try:
user = self._require_user(connection)
if user is None:
return
if user["role"] != "company":
self._send_json(403, {"status": "error", "message": "该操作仅限公司用户。"})
return
conditions = []
params: list[object] = [user["company_id"], user["company_id"]]
status = (query.get("status") or [None])[0]
raw_limit = (query.get("limit") or ["200"])[0]
try:
limit = max(1, min(int(raw_limit), 500))
except ValueError:
limit = 200
rows = connection.execute(
self._event_base_sql(True) + " ORDER BY d.id DESC LIMIT ?",
(*params, limit),
).fetchall()
items = [
self._company_event_list_item(row, user["company_id"]) for row in rows
]
if status:
items = [item for item in items if item["status"] == status]
self._send_json(200, {"status": "ok", "events": items})
finally:
connection.close()
@staticmethod
def _company_event_list_item(row, own_company_id: int) -> dict[str, object]:
counterparty_company_id = (
row["payee_company_id"]
if row["payer_company_id"] == own_company_id
else row["payer_company_id"]
)
counterparty_company_name = (
row["payee_company_name"]
if row["payer_company_id"] == own_company_id
else row["payer_company_name"]
)
return {
"event_id": row["event_id"],
"classification": row["classification"],
"pairing": row["pairing"],
"status": matching.exposed_status(row),
"amount": row["amount"],
"currency": row["currency"],
"effective_at": row["effective_at"],
"mode": row["mode"],
"rule_version": row["rule_version"],
"own_company_id": own_company_id,
"counterparty_company_id": counterparty_company_id,
"counterparty_company_name": counterparty_company_name,
"evidence_count": row["evidence_count"],
}
def _handle_company_match_exceptions(self, query: dict[str, list[str]]) -> None:
connection = connect(DB_PATH)
try:
user = self._require_user(connection)
if user is None:
return
if user["role"] != "company":
self._send_json(403, {"status": "error", "message": "该操作仅限公司用户。"})
return
params: list[object] = [user["company_id"], user["company_id"]]
rows = connection.execute(
self._event_base_sql(True)
+ "AND d.classification IN ('unresolved', 'needs_review') "
"ORDER BY d.id DESC LIMIT 500",
params,
).fetchall()
items = [
self._company_event_list_item(row, user["company_id"]) for row in rows
]
self._send_json(200, {"status": "ok", "exceptions": items})
finally:
connection.close()
def _handle_company_transfer_event_detail(self, event_id: int) -> None:
connection = connect(DB_PATH)
try:
user = self._require_user(connection)
if user is None:
return
if user["role"] != "company":
self._send_json(403, {"status": "error", "message": "该操作仅限公司用户。"})
return
current = matching._current_decision_for_event(connection, event_id)
if current is None:
self._send_json(404, {"status": "error", "message": "事件不存在。"})
return
participants = matching._decision_participants(connection, current["id"])
participant_company_ids = {p["company_id"] for p in participants}
if user["company_id"] not in participant_company_ids:
self._send_json(404, {"status": "error", "message": "事件不存在。"})
return
observations = matching._decision_observations(connection, current["id"])
own_rows = self._company_observation_rows(connection, observations, user["company_id"])
other_side = next(
(
p for p in participants
if p["company_id"] != user["company_id"]
),
None,
)
counterparty = None
if other_side is not None:
company = connection.execute(
"SELECT name FROM companies WHERE id = ?", (other_side["company_id"],)
).fetchone()
counterparty = {
"company_id": other_side["company_id"],
"company_name": company["name"] if company else None,
}
if other_side["bank_account_id"] is not None:
account = master_data.get_account(connection, other_side["bank_account_id"])
if account is not None:
counterparty["account_number_masked"] = master_data.mask_account_number(
account["account_number"]
)
self._send_json(
200,
{
"status": "ok",
"event": {
"event_id": event_id,
"classification": current["classification"],
"pairing": current["pairing"],
"status": matching.exposed_status(current),
"amount": current["amount"],
"currency": current["currency"],
"effective_at": current["effective_at"],
"mode": current["mode"],
"reason": current["reason"],
"counterparty": counterparty,
"observations": own_rows,
},
},
)
finally:
connection.close()
def _company_observation_rows(self, connection, observations, own_company_id: int) -> list[dict]:
rows = []
for observation in observations:
row = connection.execute(
"""
SELECT r.id, r.source_row, r.transaction_at, r.income, r.expense,
r.own_account, r.own_name, r.counterparty_account,
r.counterparty_name, r.summary, r.reference,
s.sheet_name, b.company_id AS batch_company_id
FROM source_rows r
JOIN sheet_batches s ON s.id = r.sheet_batch_id
JOIN import_batches b ON b.id = s.import_batch_id
WHERE r.id = ?
""",
(observation["source_row_id"],),
).fetchone()
if row is None or row["batch_company_id"] != own_company_id:
continue
rows.append(
{
"source_row_id": row["id"],
"role": observation["role"],
"source_row": row["source_row"],
"sheet_name": row["sheet_name"],
"transaction_at": row["transaction_at"],
"income": row["income"],
"expense": row["expense"],
"batch_company_id": row["batch_company_id"],
"own_account_masked": master_data.mask_account_number(row["own_account"])
if row["own_account"]
else None,
"counterparty_name": row["counterparty_name"],
"summary": row["summary"],
"reference": row["reference"],
}
)
return rows
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# Request/response plumbing # Request/response plumbing
# ------------------------------------------------------------------ # ------------------------------------------------------------------
+214
View File
@@ -354,6 +354,220 @@ MIGRATIONS: tuple[Migration, ...] = (
DROP TABLE IF EXISTS sheet_reviews; DROP TABLE IF EXISTS sheet_reviews;
""", """,
), ),
Migration(
version=5,
name="0005_canonical_transfer_matching",
# Canonical transfer event layer (B-43). Immutable source rows stay
# untouched; every matching decision, its participant/observation
# bindings and its candidate evidence are append-only history, and the
# current decision + row claims are a rebuildable projection that the
# B-44 balance layer reads through the eligible view.
up="""
ALTER TABLE import_batches ADD COLUMN upload_bank_account_id INTEGER REFERENCES bank_accounts (id);
CREATE TABLE master_data_changes_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
entity_type TEXT NOT NULL
CHECK (entity_type IN ('company', 'user', 'bank_account', 'account_alias', 'personal_transit_mapping')),
entity_id INTEGER NOT NULL,
action TEXT NOT NULL,
before_json TEXT,
after_json TEXT,
reason TEXT,
actor_user_id INTEGER REFERENCES users (id),
actor_username TEXT,
created_at TEXT NOT NULL
);
INSERT INTO master_data_changes_new SELECT * FROM master_data_changes;
DROP TABLE master_data_changes;
ALTER TABLE master_data_changes_new RENAME TO master_data_changes;
CREATE TABLE personal_transit_mappings (
id INTEGER PRIMARY KEY AUTOINCREMENT,
account_number TEXT NOT NULL UNIQUE,
account_name TEXT,
represented_company_id INTEGER NOT NULL REFERENCES companies (id),
allowed_direction TEXT NOT NULL
CHECK (allowed_direction IN ('outgoing', 'incoming', 'both')),
status TEXT NOT NULL DEFAULT 'pending'
CHECK (status IN ('pending', 'active', 'returned', 'disabled')),
effective_from TEXT,
effective_to TEXT,
submitted_by INTEGER REFERENCES users (id),
reviewed_by INTEGER REFERENCES users (id),
reviewed_at TEXT,
review_reason TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE TABLE canonical_transfer_events (
id INTEGER PRIMARY KEY AUTOINCREMENT,
lifecycle TEXT NOT NULL DEFAULT 'active'
CHECK (lifecycle IN ('active', 'superseded')),
created_at TEXT NOT NULL
);
CREATE TABLE transfer_match_decisions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
event_id INTEGER NOT NULL REFERENCES canonical_transfer_events (id),
revision INTEGER NOT NULL,
classification TEXT NOT NULL CHECK (classification IN (
'unresolved', 'needs_review', 'internal_single',
'intercompany', 'same_company', 'external'
)),
pairing TEXT NOT NULL
CHECK (pairing IN ('single', 'paired', 'not_applicable')),
amount TEXT,
currency TEXT,
effective_at TEXT,
mode TEXT NOT NULL CHECK (mode IN ('auto', 'manual', 'reversal')),
rule_version TEXT,
locked INTEGER NOT NULL DEFAULT 0 CHECK (locked IN (0, 1)),
reason TEXT,
idempotency_key TEXT,
actor_user_id INTEGER REFERENCES users (id),
actor_username TEXT,
supersedes_decision_id INTEGER REFERENCES transfer_match_decisions (id),
created_at TEXT NOT NULL,
UNIQUE (event_id, revision)
);
CREATE TABLE transfer_decision_observations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
decision_id INTEGER NOT NULL REFERENCES transfer_match_decisions (id),
source_row_id INTEGER NOT NULL REFERENCES source_rows (id),
role TEXT NOT NULL CHECK (role IN ('outgoing', 'incoming')),
created_at TEXT NOT NULL,
UNIQUE (decision_id, source_row_id)
);
CREATE TABLE transfer_decision_participants (
id INTEGER PRIMARY KEY AUTOINCREMENT,
decision_id INTEGER NOT NULL REFERENCES transfer_match_decisions (id),
role TEXT NOT NULL CHECK (role IN ('payer', 'payee')),
company_id INTEGER REFERENCES companies (id),
bank_account_id INTEGER REFERENCES bank_accounts (id),
resolve_method TEXT NOT NULL CHECK (resolve_method IN (
'own_exact', 'upload_account', 'counterparty_exact',
'account_alias', 'personal_mapping', 'manual'
)),
alias_id INTEGER REFERENCES account_aliases (id),
mapping_id INTEGER REFERENCES personal_transit_mappings (id),
evidence TEXT,
created_at TEXT NOT NULL,
UNIQUE (decision_id, role)
);
CREATE TABLE transfer_match_candidates (
id INTEGER PRIMARY KEY AUTOINCREMENT,
decision_id INTEGER NOT NULL REFERENCES transfer_match_decisions (id),
source_row_id INTEGER NOT NULL REFERENCES source_rows (id),
rule_tier TEXT NOT NULL,
date_diff_days INTEGER,
account_mirror INTEGER NOT NULL DEFAULT 0,
reference_match TEXT,
summary_match INTEGER NOT NULL DEFAULT 0,
accepted INTEGER,
reason TEXT,
rule_version TEXT,
created_at TEXT NOT NULL
);
CREATE TABLE current_transfer_decisions (
event_id INTEGER PRIMARY KEY REFERENCES canonical_transfer_events (id),
decision_id INTEGER NOT NULL UNIQUE REFERENCES transfer_match_decisions (id)
);
CREATE TABLE transfer_observation_claims (
source_row_id INTEGER PRIMARY KEY REFERENCES source_rows (id),
event_id INTEGER NOT NULL REFERENCES canonical_transfer_events (id),
decision_id INTEGER NOT NULL REFERENCES transfer_match_decisions (id)
);
CREATE INDEX idx_obs_decision ON transfer_decision_observations (decision_id);
CREATE INDEX idx_participants_decision ON transfer_decision_participants (decision_id);
CREATE INDEX idx_candidates_decision ON transfer_match_candidates (decision_id);
CREATE INDEX idx_claims_event ON transfer_observation_claims (event_id);
CREATE VIEW eligible_intercompany_events AS
SELECT e.id AS event_id, d.id AS decision_id, d.revision AS revision,
d.effective_at AS effective_at, d.amount AS amount, d.currency AS currency,
payer.company_id AS payer_company_id,
payer.bank_account_id AS payer_account_id,
payee.company_id AS payee_company_id,
payee.bank_account_id AS payee_account_id,
d.pairing AS pairing, d.rule_version AS rule_version,
(SELECT COUNT(*) FROM transfer_decision_observations o
WHERE o.decision_id = d.id) AS evidence_count
FROM current_transfer_decisions c
JOIN canonical_transfer_events e ON e.id = c.event_id
JOIN transfer_match_decisions d ON d.id = c.decision_id
JOIN transfer_decision_participants payer
ON payer.decision_id = d.id AND payer.role = 'payer'
JOIN transfer_decision_participants payee
ON payee.decision_id = d.id AND payee.role = 'payee'
WHERE e.lifecycle = 'active' AND d.classification = 'intercompany';
CREATE TRIGGER canonical_transfer_events_no_delete BEFORE DELETE ON canonical_transfer_events
BEGIN SELECT RAISE (ABORT, 'canonical_transfer_events rows are immutable'); END;
CREATE TRIGGER canonical_transfer_events_no_update BEFORE UPDATE ON canonical_transfer_events
BEGIN
SELECT RAISE (ABORT, 'canonical_transfer_events only allow lifecycle changes')
WHERE OLD.lifecycle = NEW.lifecycle
OR OLD.id IS NOT NEW.id
OR OLD.created_at IS NOT NEW.created_at;
END;
CREATE TRIGGER transfer_match_decisions_no_update BEFORE UPDATE ON transfer_match_decisions
BEGIN SELECT RAISE (ABORT, 'transfer_match_decisions rows are immutable'); END;
CREATE TRIGGER transfer_match_decisions_no_delete BEFORE DELETE ON transfer_match_decisions
BEGIN SELECT RAISE (ABORT, 'transfer_match_decisions rows are immutable'); END;
CREATE TRIGGER transfer_decision_observations_no_update BEFORE UPDATE ON transfer_decision_observations
BEGIN SELECT RAISE (ABORT, 'transfer_decision_observations rows are immutable'); END;
CREATE TRIGGER transfer_decision_observations_no_delete BEFORE DELETE ON transfer_decision_observations
BEGIN SELECT RAISE (ABORT, 'transfer_decision_observations rows are immutable'); END;
CREATE TRIGGER transfer_decision_participants_no_update BEFORE UPDATE ON transfer_decision_participants
BEGIN SELECT RAISE (ABORT, 'transfer_decision_participants rows are immutable'); END;
CREATE TRIGGER transfer_decision_participants_no_delete BEFORE DELETE ON transfer_decision_participants
BEGIN SELECT RAISE (ABORT, 'transfer_decision_participants rows are immutable'); END;
CREATE TRIGGER transfer_match_candidates_no_update BEFORE UPDATE ON transfer_match_candidates
BEGIN SELECT RAISE (ABORT, 'transfer_match_candidates rows are immutable'); END;
CREATE TRIGGER transfer_match_candidates_no_delete BEFORE DELETE ON transfer_match_candidates
BEGIN SELECT RAISE (ABORT, 'transfer_match_candidates rows are immutable'); END;
""",
down="""
DROP VIEW IF EXISTS eligible_intercompany_events;
DROP TABLE IF EXISTS transfer_observation_claims;
DROP TABLE IF EXISTS current_transfer_decisions;
DROP TABLE IF EXISTS transfer_match_candidates;
DROP TABLE IF EXISTS transfer_decision_participants;
DROP TABLE IF EXISTS transfer_decision_observations;
DROP TABLE IF EXISTS transfer_match_decisions;
DROP TABLE IF EXISTS canonical_transfer_events;
DROP TABLE IF EXISTS personal_transit_mappings;
CREATE TABLE master_data_changes_new (
id INTEGER PRIMARY KEY AUTOINCREMENT,
entity_type TEXT NOT NULL
CHECK (entity_type IN ('company', 'user', 'bank_account', 'account_alias')),
entity_id INTEGER NOT NULL,
action TEXT NOT NULL,
before_json TEXT,
after_json TEXT,
reason TEXT,
actor_user_id INTEGER REFERENCES users (id),
actor_username TEXT,
created_at TEXT NOT NULL
);
INSERT INTO master_data_changes_new SELECT * FROM master_data_changes;
DROP TABLE master_data_changes;
ALTER TABLE master_data_changes_new RENAME TO master_data_changes;
ALTER TABLE import_batches DROP COLUMN upload_bank_account_id;
""",
),
) )
+85 -23
View File
@@ -22,8 +22,10 @@ import os
from pathlib import Path from pathlib import Path
import shutil import shutil
import sqlite3 import sqlite3
import tempfile
from . import auth from . import auth
from . import matching
from .db import utc_now from .db import utc_now
from .models import SheetResult, StatementBatch from .models import SheetResult, StatementBatch
from .parser import StatementParseError, analyze_workbook from .parser import StatementParseError, analyze_workbook
@@ -52,6 +54,7 @@ def import_statement(
original_filename: str, original_filename: str,
content: bytes, content: bytes,
company_id: int | None = None, company_id: int | None = None,
upload_bank_account_id: int | None = None,
) -> ImportResult: ) -> ImportResult:
sha256 = hashlib.sha256(content).hexdigest() sha256 = hashlib.sha256(content).hexdigest()
existing_file = connection.execute( existing_file = connection.execute(
@@ -64,7 +67,8 @@ def import_statement(
stored_path = _store_immutable(Path(storage_dir), original_filename, content, sha256) stored_path = _store_immutable(Path(storage_dir), original_filename, content, sha256)
try: try:
source_file_id, batch_id = _create_batch_records( source_file_id, batch_id = _create_batch_records(
connection, sha256, original_filename, len(content), stored_path, company_id connection, sha256, original_filename, len(content), stored_path,
company_id, upload_bank_account_id,
) )
except sqlite3.IntegrityError: except sqlite3.IntegrityError:
# Lost a concurrent-insert race on the sha256 UNIQUE constraint: the # Lost a concurrent-insert race on the sha256 UNIQUE constraint: the
@@ -87,13 +91,16 @@ def import_statement_path(
original_filename: str, original_filename: str,
upload_path: str | Path, upload_path: str | Path,
company_id: int | None = None, company_id: int | None = None,
upload_bank_account_id: int | None = None,
) -> ImportResult: ) -> ImportResult:
"""Import from an already-downloaded upload file (streaming-friendly). """Import from an already-downloaded upload file (streaming-friendly).
``upload_path`` names the temp file the multipart handler streamed to ``upload_path`` names the temp file the multipart handler streamed to
disk; it is hashed incrementally and either discarded (duplicate) or disk; it is hashed incrementally and either discarded (duplicate) or
published into the content-addressed store. The upload file itself is published into the content-addressed store. The upload file itself is
never modified. never modified. The approved ``upload_bank_account_id`` used at upload time
is persisted on the batch so ownership can be resolved later even when the
source rows carry no own account.
""" """
source = Path(upload_path) source = Path(upload_path)
sha256 = _file_sha256(source) sha256 = _file_sha256(source)
@@ -109,7 +116,8 @@ def import_statement_path(
size = stored_path.stat().st_size size = stored_path.stat().st_size
try: try:
source_file_id, batch_id = _create_batch_records( source_file_id, batch_id = _create_batch_records(
connection, sha256, original_filename, size, stored_path, company_id connection, sha256, original_filename, size, stored_path,
company_id, upload_bank_account_id,
) )
except sqlite3.IntegrityError: except sqlite3.IntegrityError:
# Lost a concurrent-insert race on the sha256 UNIQUE constraint: the # Lost a concurrent-insert race on the sha256 UNIQUE constraint: the
@@ -133,6 +141,7 @@ def _create_batch_records(
size_bytes: int, size_bytes: int,
stored_path: Path, stored_path: Path,
company_id: int | None, company_id: int | None,
upload_bank_account_id: int | None,
) -> tuple[int, int]: ) -> tuple[int, int]:
now = utc_now() now = utc_now()
with connection: with connection:
@@ -146,10 +155,10 @@ def _create_batch_records(
source_file_id = cursor.lastrowid source_file_id = cursor.lastrowid
cursor = connection.execute( cursor = connection.execute(
""" """
INSERT INTO import_batches (source_file_id, status, company_id, created_at, updated_at) INSERT INTO import_batches (source_file_id, status, company_id, upload_bank_account_id, created_at, updated_at)
VALUES (?, 'parsing', ?, ?, ?) VALUES (?, 'parsing', ?, ?, ?, ?)
""", """,
(source_file_id, company_id, now, now), (source_file_id, company_id, upload_bank_account_id, now, now),
) )
batch_id = cursor.lastrowid batch_id = cursor.lastrowid
return source_file_id, batch_id return source_file_id, batch_id
@@ -278,17 +287,24 @@ def _store_immutable(
target_dir = storage_dir / sha256[:2] target_dir = storage_dir / sha256[:2]
target_dir.mkdir(parents=True, exist_ok=True) target_dir.mkdir(parents=True, exist_ok=True)
target = target_dir / f"{sha256}{suffix}" target = target_dir / f"{sha256}{suffix}"
# Publish via a temporary file + hard link: the content-addressed target # Publish via a unique temporary file + hard link: the content-addressed
# either appears complete or not at all, and is never overwritten. # target either appears complete or not at all, and is never overwritten.
temp = target_dir / f".{sha256}.tmp" # The temp name is unique per writer, so concurrent uploads of the same
temp.write_bytes(content) # bytes cannot corrupt each other's staging file (B-43 baseline fix).
fd, temp_path = tempfile.mkstemp(prefix=f".{sha256}.", suffix=".tmp", dir=target_dir)
try: try:
os.link(temp, target) with os.fdopen(fd, "wb") as handle:
except FileExistsError: handle.write(content)
# Content-addressed name means identical bytes; never overwrite. try:
pass os.link(temp_path, target)
except FileExistsError:
# Content-addressed name means identical bytes; never overwrite.
pass
finally: finally:
temp.unlink(missing_ok=True) try:
os.unlink(temp_path)
except FileNotFoundError:
pass
return target return target
@@ -299,17 +315,24 @@ def _publish_immutable(
target_dir = storage_dir / sha256[:2] target_dir = storage_dir / sha256[:2]
target_dir.mkdir(parents=True, exist_ok=True) target_dir.mkdir(parents=True, exist_ok=True)
target = target_dir / f"{sha256}{suffix}" target = target_dir / f"{sha256}{suffix}"
temp = target_dir / f".{sha256}.tmp" fd, temp_path = tempfile.mkstemp(prefix=f".{sha256}.", suffix=".tmp", dir=target_dir)
try: try:
# Copy keeps the same-filesystem guarantee even if the upload temp with os.fdopen(fd, "wb") as handle:
# lives elsewhere; the hard link then publishes atomically. # Copy keeps the same-filesystem guarantee even if the upload temp
shutil.copyfile(upload_path, temp) # lives elsewhere; the hard link then publishes atomically. The
# staging file is unique per writer, so two concurrent uploads of
# the same bytes never corrupt each other's copy.
with upload_path.open("rb") as source:
shutil.copyfileobj(source, handle, length=1024 * 1024)
try: try:
os.link(temp, target) os.link(temp_path, target)
except FileExistsError: except FileExistsError:
pass pass
finally: finally:
temp.unlink(missing_ok=True) try:
os.unlink(temp_path)
except FileNotFoundError:
pass
upload_path.unlink(missing_ok=True) upload_path.unlink(missing_ok=True)
return target return target
@@ -572,7 +595,15 @@ def review_sheets(
now = utc_now() now = utc_now()
updated: list[str] = [] updated: list[str] = []
already: list[str] = [] already: list[str] = []
with connection: matching_result: dict[str, object] | None = None
# BEGIN IMMEDIATE serializes concurrent confirmations: the write lock is
# taken before the read, so two cashiers confirming matching worksheets can
# never derive their match decisions from stale snapshots.
began = False
if not connection.in_transaction:
connection.execute("BEGIN IMMEDIATE")
began = True
try:
for row in rows: for row in rows:
if row["review_status"] == target: if row["review_status"] == target:
already.append(row["sheet_name"]) already.append(row["sheet_name"])
@@ -596,6 +627,34 @@ def review_sheets(
if cursor.rowcount: if cursor.rowcount:
updated.append(row["sheet_name"]) updated.append(row["sheet_name"])
# Confirming a worksheet and reconciling its rows must succeed or fail
# together: any matching failure rolls the confirmation back, so there
# is never a "confirmed but unmatched" half state.
if updated and decision == "confirm":
placeholders = ",".join("?" for _ in updated)
confirmed_rows = connection.execute(
f"""
SELECT r.id FROM source_rows r
JOIN sheet_batches s ON s.id = r.sheet_batch_id
JOIN sheet_reviews rv ON rv.sheet_batch_id = s.id
WHERE rv.sheet_name IN ({placeholders})
AND rv.import_batch_id = ?
ORDER BY r.id
""",
(*updated, batch_id),
).fetchall()
matching_result = matching.reconcile_rows(
connection,
[item["id"] for item in confirmed_rows],
actor=actor,
)
if began:
connection.commit()
except Exception:
if began:
connection.rollback()
raise
if updated: if updated:
auth.audit( auth.audit(
connection, connection,
@@ -604,4 +663,7 @@ def review_sheets(
target=f"batch:{batch_id}", target=f"batch:{batch_id}",
detail=f"sheets:{','.join(updated)}" + (f";reason:{reason}" if reason else ""), detail=f"sheets:{','.join(updated)}" + (f";reason:{reason}" if reason else ""),
) )
return {"updated": updated, "already": already} payload: dict[str, object] = {"updated": updated, "already": already}
if matching_result is not None:
payload["matching"] = matching_result
return payload
File diff suppressed because it is too large Load Diff
+281
View File
@@ -0,0 +1,281 @@
"""Personal transit account mappings and their administrator approval.
A personal transit mapping states that a personal bank account represents a
company for a specific flow direction inside an effective interval. The key is
the normalized account number; the account holder name is display-only and
never resolves a counterparty on its own (``AGENTS.md``: never infer a
financial fact from a name alone). Only administrator-approved (``active``)
mappings inside their effective window participate in counterparty resolution.
"""
from __future__ import annotations
import sqlite3
from .db import utc_now
from .master_data import (
ConflictError,
mask_account_number,
normalize_account_number,
record_change,
validate_date,
)
MAPPING_STATUSES = ("pending", "active", "returned", "disabled")
ALLOWED_DIRECTIONS = ("outgoing", "incoming", "both")
def get_mapping(connection: sqlite3.Connection, mapping_id: int) -> sqlite3.Row | None:
return connection.execute(
"SELECT * FROM personal_transit_mappings WHERE id = ?", (mapping_id,)
).fetchone()
def submit_mapping(
connection: sqlite3.Connection,
*,
account_number: object,
account_name: object,
represented_company_id: int,
allowed_direction: str,
effective_from: object = None,
actor: sqlite3.Row | None,
) -> sqlite3.Row:
"""Create a pending personal transit mapping; returns the new row.
The account number is normalized (digits only) and unique: resubmitting a
returned request from the same company reopens the same row as pending;
any other existing row is a conflict guarded by the UNIQUE constraint.
"""
number = normalize_account_number(account_number)
holder = str(account_name or "").strip() or None
direction = str(allowed_direction or "").strip()
if direction not in ALLOWED_DIRECTIONS:
raise ValueError(f"允许方向必须是:{''.join(ALLOWED_DIRECTIONS)}")
if not holder:
raise ValueError("账户户名不能为空(仅作展示,不作匹配键)。")
company = connection.execute(
"SELECT id FROM companies WHERE id = ?", (represented_company_id,)
).fetchone()
if company is None:
raise ValueError("代表的公司不存在。")
start = validate_date(effective_from, "生效日期")
now = utc_now()
existing = connection.execute(
"SELECT * FROM personal_transit_mappings WHERE account_number = ?", (number,)
).fetchone()
if existing is None:
try:
with connection:
cursor = connection.execute(
"""
INSERT INTO personal_transit_mappings (
account_number, account_name, represented_company_id,
allowed_direction, status, effective_from, submitted_by,
created_at, updated_at
) VALUES (?, ?, ?, ?, 'pending', ?, ?, ?, ?)
""",
(
number, holder, represented_company_id, direction,
start, actor["id"] if actor else None, now, now,
),
)
except sqlite3.IntegrityError as exc:
raise ConflictError("该个人过账账号已登记,请等待现有申请处理。") from exc
mapping_id = int(cursor.lastrowid)
with connection:
record_change(
connection, "personal_transit_mapping", mapping_id, "submit", None,
{"account_number": number, "account_name": holder,
"represented_company_id": represented_company_id,
"allowed_direction": direction, "status": "pending",
"effective_from": start},
None, actor,
)
return get_mapping(connection, mapping_id)
if existing["status"] == "returned":
with connection:
connection.execute(
"""
UPDATE personal_transit_mappings
SET account_name = ?, represented_company_id = ?,
allowed_direction = ?, status = 'pending', effective_from = ?,
submitted_by = ?, reviewed_by = NULL, reviewed_at = NULL,
review_reason = NULL, updated_at = ?
WHERE id = ? AND status = 'returned'
""",
(
holder, represented_company_id, direction, start,
actor["id"] if actor else None, now, existing["id"],
),
)
record_change(
connection, "personal_transit_mapping", existing["id"], "resubmit",
None, {"account_number": number, "status": "pending",
"allowed_direction": direction, "effective_from": start},
"退回后重新提交", actor,
)
return get_mapping(connection, existing["id"])
raise ConflictError("该个人过账账号已登记,请等待现有申请处理。")
def review_mapping(
connection: sqlite3.Connection,
mapping_id: int,
decision: str,
reason: str | None,
actor: sqlite3.Row,
*,
effective_from: object = None,
effective_to: object = None,
) -> sqlite3.Row:
"""Approve, return or disable a mapping; returns the updated row."""
mapping = get_mapping(connection, mapping_id)
if mapping is None:
raise LookupError("个人过账映射不存在。")
reason = (reason or "").strip() or None
today = utc_now()[:10]
if decision == "approve":
if mapping["status"] != "pending":
raise ConflictError("只有待复核的映射可以审核通过。")
start = validate_date(effective_from, "生效日期") or mapping["effective_from"] or today
with connection:
connection.execute(
"""
UPDATE personal_transit_mappings
SET status = 'active', effective_from = ?, effective_to = NULL,
reviewed_by = ?, reviewed_at = ?, review_reason = ?, updated_at = ?
WHERE id = ?
""",
(start, actor["id"], utc_now(), reason, utc_now(), mapping_id),
)
record_change(
connection, "personal_transit_mapping", mapping_id, "approve", None,
{"status": "active", "effective_from": start}, reason, actor,
)
elif decision == "return":
if mapping["status"] != "pending":
raise ConflictError("只有待复核的映射可以退回。")
if reason is None:
raise ValueError("退回必须填写原因。")
with connection:
connection.execute(
"""
UPDATE personal_transit_mappings
SET status = 'returned', reviewed_by = ?, reviewed_at = ?,
review_reason = ?, updated_at = ?
WHERE id = ?
""",
(actor["id"], utc_now(), reason, utc_now(), mapping_id),
)
record_change(
connection, "personal_transit_mapping", mapping_id, "return", None,
{"status": "returned"}, reason, actor,
)
elif decision == "disable":
if mapping["status"] != "active":
raise ConflictError("只有已启用的映射可以停用。")
if reason is None:
raise ValueError("停用必须填写原因。")
end = validate_date(effective_to, "停用日期") or today
with connection:
connection.execute(
"""
UPDATE personal_transit_mappings
SET status = 'disabled', effective_to = ?,
reviewed_by = ?, reviewed_at = ?, review_reason = ?, updated_at = ?
WHERE id = ?
""",
(end, actor["id"], utc_now(), reason, utc_now(), mapping_id),
)
record_change(
connection, "personal_transit_mapping", mapping_id, "disable", None,
{"status": "disabled", "effective_to": end}, reason, actor,
)
else:
raise ValueError("审核决定必须是 approve、return 或 disable。")
return get_mapping(connection, mapping_id)
def list_mappings(
connection: sqlite3.Connection,
company_id: int | None = None,
status: str | None = None,
) -> list[sqlite3.Row]:
conditions: list[str] = []
params: list[object] = []
if company_id is not None:
conditions.append("m.represented_company_id = ?")
params.append(company_id)
if status is not None:
if status not in MAPPING_STATUSES:
raise ValueError("无效的映射状态。")
conditions.append("m.status = ?")
params.append(status)
where = f"WHERE {' AND '.join(conditions)}" if conditions else ""
return connection.execute(
f"""
SELECT m.*, c.name AS company_name
FROM personal_transit_mappings m
JOIN companies c ON c.id = m.represented_company_id
{where}
ORDER BY m.id
""",
params,
).fetchall()
def is_mapping_active(mapping: sqlite3.Row, on_date: str) -> bool:
"""True when the mapping may resolve a counterparty on ``on_date``.
``effective_to`` is the last participating day (inclusive). Disabled
mappings stop resolving new rows but keep identifying historical rows
inside their effective window, matching ``bank_accounts`` semantics.
"""
if mapping["status"] not in ("active", "disabled"):
return False
if mapping["effective_from"] and on_date < mapping["effective_from"]:
return False
if mapping["effective_to"] and on_date > mapping["effective_to"]:
return False
return True
def direction_allowed(mapping: sqlite3.Row, row_direction: str) -> bool:
"""True when the personal account may represent the company for this flow.
``allowed_direction`` describes the transfer from the represented company's
perspective: when the personal account appears as the counterparty of an
outgoing row, the represented company is receiving (incoming); for an
incoming row the represented company is paying out (outgoing).
"""
counterparty_flow = "incoming" if row_direction == "outgoing" else "outgoing"
allowed = mapping["allowed_direction"]
return allowed == "both" or allowed == counterparty_flow
def mapping_payload(mapping: sqlite3.Row, *, full: bool) -> dict[str, object]:
"""Serialize a mapping; company-facing views only ever see masked numbers."""
payload: dict[str, object] = {
"id": mapping["id"],
"account_name": mapping["account_name"],
"represented_company_id": mapping["represented_company_id"],
"allowed_direction": mapping["allowed_direction"],
"status": mapping["status"],
"effective_from": mapping["effective_from"],
"effective_to": mapping["effective_to"],
"reviewed_at": mapping["reviewed_at"],
"review_reason": mapping["review_reason"],
"created_at": mapping["created_at"],
}
if "company_name" in mapping.keys():
payload["company_name"] = mapping["company_name"]
if full:
payload["account_number"] = mapping["account_number"]
else:
payload["account_number_masked"] = mask_account_number(mapping["account_number"])
return payload
+148
View File
@@ -17,6 +17,7 @@ from pathlib import Path
import tempfile import tempfile
import threading import threading
import unittest import unittest
from unittest import mock
from openpyxl import Workbook from openpyxl import Workbook
@@ -49,6 +50,14 @@ CCB_DATA = [
] ]
def ccb_row(own, cp, *, expense="", income="", at="2026-01-05 10:00:00"):
return [own, "测试公司", at, expense or "", income or "", "99900.00", "RMB", "对方", cp, "某银行", "货款", ""]
def ccb_bytes(rows):
return workbook_bytes([("正常流水", [CCB_HEADER, *rows])])
def workbook_bytes(sheets): def workbook_bytes(sheets):
"""Build an xlsx in memory. """Build an xlsx in memory.
@@ -582,6 +591,145 @@ class ImportApiServerTests(unittest.TestCase):
finally: finally:
connection.close() connection.close()
# ------------------------------------------------------------------
# Worksheet confirmation drives canonical matching in the same transaction
# ------------------------------------------------------------------
def _approve_test_account(self, company_client, number: str) -> int:
status, _, data = company_client.post_json(
"/api/company/accounts",
{"bank_name": "中信银行", "account_type": "基本户",
"account_number": number, "start_date": "2026-01-01"},
)
self.assertEqual(200, status, data)
account_id = as_json(data)["account"]["id"]
status, _, data = self.admin.post_json(
f"/api/admin/accounts/{account_id}/review",
{"decision": "approve", "reason": "测试启用", "effective_from": "2026-01-01"},
)
self.assertEqual(200, status, data)
return account_id
def _company_cashier(self, username: str, company_id: int) -> Client:
"""A company cashier bound to the given company (the class fixture binds
both cashiers to company A, so tests needing a real tenant B create one)."""
status, _, data = self.admin.post_json(
"/api/admin/users", {"username": username, "company_id": company_id}
)
self.assertEqual(200, status, data)
initial = as_json(data)["initial_password"]
client = self.fresh_client()
status, _, data = client.post_json(
"/api/login",
{"username": username, "password": initial, "portal": "company"},
)
self.assertEqual(200, status, data)
status, _, data = client.post_json(
"/api/password/change",
{"old_password": initial, "new_password": "Changed456"},
)
self.assertEqual(200, status, data)
return client
def test_confirm_drives_bilateral_matching_atomically(self) -> None:
account_a = self._approve_test_account(self.cashier_a, "6222000000000001")
cashier_b = self._company_cashier("cashier-b2", self.company_b)
account_b = self._approve_test_account(cashier_b, "6222000000000002")
# A's side first: confirming the worksheet reconciles its row into an
# internal single (not yet eligible for the intercompany balance).
status, _, data = self.upload_for_a(
ccb_bytes([ccb_row("6222000000000001", "6222000000000002", expense="100.00")]),
filename="A方.xlsx",
)
self.assertEqual(200, status, data)
batch_a = as_json(data)["batch_id"]
status, _, data = self.confirm(self.cashier_a, batch_a, ["正常流水"])
self.assertEqual(200, status, data)
self.assertEqual(1, as_json(data)["matching"]["created_events"])
connection = connect(self.db_path)
try:
row_a = connection.execute(
"""
SELECT r.id FROM source_rows r
JOIN sheet_batches s ON s.id = r.sheet_batch_id
WHERE s.import_batch_id = ?
""",
(batch_a,),
).fetchone()
status_a = connection.execute(
"""
SELECT d.classification FROM transfer_observation_claims c
JOIN transfer_match_decisions d ON d.id = c.decision_id
WHERE c.source_row_id = ?
""",
(row_a["id"],),
).fetchone()
self.assertEqual("internal_single", status_a["classification"])
finally:
connection.close()
# B's side arrives later: the second confirmation merges both into one
# paired intercompany event without duplicating the first.
status, _, data = cashier_b.post_multipart(
"/api/parse", {}, "B方.xlsx",
ccb_bytes([ccb_row("6222000000000002", "6222000000000001", income="100.00")]),
)
self.assertEqual(200, status, data)
batch_b = as_json(data)["batch_id"]
status, _, data = self.confirm(cashier_b, batch_b, ["正常流水"])
self.assertEqual(200, status, data)
connection = connect(self.db_path)
try:
eligible = connection.execute(
"SELECT * FROM eligible_intercompany_events"
).fetchall()
self.assertEqual(1, len(eligible))
self.assertEqual("100.00", eligible[0]["amount"])
self.assertEqual("paired", eligible[0]["pairing"])
# Exactly one current intercompany decision for the pair.
self.assertEqual(1, connection.execute(
"""
SELECT COUNT(*) AS n FROM current_transfer_decisions c
JOIN transfer_match_decisions d ON d.id = c.decision_id
WHERE d.classification = 'intercompany'
"""
).fetchone()["n"])
finally:
connection.close()
def test_confirm_rolls_back_when_matching_fails(self) -> None:
account_a = self._approve_test_account(self.cashier_a, "6222000000000003")
status, _, data = self.upload_for_a(
ccb_bytes([ccb_row("6222000000000003", "6222000000000004", expense="50.00")]),
filename="回滚.xlsx",
)
batch_id = as_json(data)["batch_id"]
with mock.patch(
"bank_importer.importing.matching.reconcile_rows",
side_effect=RuntimeError("simulated matching failure"),
):
status, _, data = self.confirm(self.cashier_a, batch_id, ["正常流水"])
self.assertEqual(500, status, data)
# The worksheet confirmation was rolled back with the failed matching.
sheet = self.batch_sheets(self.cashier_a, batch_id)[0]
self.assertEqual("pending", sheet["review_status"])
connection = connect(self.db_path)
try:
# No decision or claim was left behind for this batch's rows.
claims = connection.execute(
"""
SELECT COUNT(*) AS n FROM transfer_observation_claims c
JOIN source_rows r ON r.id = c.source_row_id
JOIN sheet_batches s ON s.id = r.sheet_batch_id
WHERE s.import_batch_id = ?
""",
(batch_id,),
).fetchone()["n"]
self.assertEqual(0, claims)
finally:
connection.close()
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+865
View File
@@ -0,0 +1,865 @@
"""Matching engine tests: bilateral ordering, cross-day windows, ambiguity,
same-company transfers, personal transit mappings, manual decisions, locking,
idempotency, concurrency, Decimal precision and B-44 eligibility."""
from __future__ import annotations
from decimal import Decimal
from pathlib import Path
import sqlite3
import tempfile
import threading
import unittest
from bank_importer import auth, matching, master_data, personal_transit
from bank_importer.db import connect, migrate, utc_now
class MatchingBase(unittest.TestCase):
def setUp(self) -> None:
self.temp_dir = tempfile.TemporaryDirectory()
self.addCleanup(self.temp_dir.cleanup)
root = Path(self.temp_dir.name)
self.db_path = root / "app.db"
self.connection = connect(self.db_path)
self.addCleanup(self.connection.close)
migrate(self.connection)
self.admin = self._admin()
self.company_a = self._company("甲公司")
self.company_b = self._company("乙公司")
self.company_c = self._company("丙公司")
self.account_a = self._approved_account(self.company_a, "6222000000000001")
self.account_b = self._approved_account(self.company_b, "6222000000000002")
self.account_a2 = self._approved_account(self.company_a, "6222000000000003")
self.account_b2 = self._approved_account(self.company_b, "6222000000000004")
def _admin(self):
auth.create_user(self.connection, "admin-u", "AdminPass123", "admin")
return self.connection.execute(
"SELECT * FROM users WHERE username = 'admin-u'"
).fetchone()
def _company(self, name: str) -> int:
with self.connection:
cursor = self.connection.execute(
"INSERT INTO companies (name, created_at, updated_at) VALUES (?, ?, ?)",
(name, utc_now(), utc_now()),
)
return int(cursor.lastrowid)
def _approved_account(self, company_id: int, number: str, start: str = "2026-01-01"):
account = master_data.submit_bank_account(
self.connection, company_id=company_id, bank_name="中信银行",
account_type="基本户", account_number=number, start_date=start,
actor=None,
)
return master_data.review_bank_account(
self.connection, account["id"], "approve", None, self.admin,
effective_from=start,
)
def add_row(
self,
company_id: int,
*,
own_account: str,
own_name: str = "测试公司",
cp_account: str | None = None,
cp_name: str | None = None,
income: str = "0",
expense: str = "0",
at: str = "2026-01-05T10:00:00",
currency: str = "CNY",
reference: str | None = None,
summary: str | None = None,
sheet: str = "流水",
) -> int:
with self.connection:
cursor = self.connection.execute(
"""
INSERT INTO source_files (sha256, original_filename, size_bytes, storage_path, created_at)
VALUES (?, '测试.xlsx', 1, 'data/files/测试.xlsx', ?)
""",
(utc_now(), utc_now()),
)
source_file_id = int(cursor.lastrowid)
cursor = self.connection.execute(
"""
INSERT INTO import_batches (source_file_id, status, company_id, created_at, updated_at)
VALUES (?, 'parsing', ?, ?, ?)
""",
(source_file_id, company_id, utc_now(), utc_now()),
)
batch_id = int(cursor.lastrowid)
cursor = self.connection.execute(
"""
INSERT INTO sheet_batches (
import_batch_id, sheet_name, bank_name, template_id, template_version,
header_row, own_account, own_name, period_start, period_end,
transaction_count, warnings, created_at
) VALUES (?, ?, '测试银行', 'test-v1', 1, 1, NULL, NULL, NULL, NULL, 1, '[]', ?)
""",
(batch_id, sheet, utc_now()),
)
sheet_batch_id = int(cursor.lastrowid)
self.connection.execute(
"""
INSERT INTO sheet_reviews (
import_batch_id, sheet_name, outcome, sheet_batch_id,
review_status, created_at
) VALUES (?, ?, 'parsed', ?, 'confirmed', ?)
""",
(batch_id, sheet, sheet_batch_id, utc_now()),
)
cursor = self.connection.execute(
"""
INSERT INTO source_rows (
sheet_batch_id, source_row, transaction_at, income, expense, balance,
own_account, own_name, counterparty_account, counterparty_name,
counterparty_bank, summary, purpose, reference, currency, created_at
) VALUES (?, 1, ?, ?, ?, NULL, ?, ?, ?, ?, NULL, ?, NULL, ?, ?, ?)
""",
(
sheet_batch_id, at, income, expense, own_account, own_name,
cp_account, cp_name, summary, reference, currency, utc_now(),
),
)
return int(cursor.lastrowid)
def current(self, row_id: int):
return self.connection.execute(
"""
SELECT d.classification, d.pairing, d.amount, d.currency, d.effective_at,
d.mode, d.locked, d.revision
FROM transfer_observation_claims c
JOIN transfer_match_decisions d ON d.id = c.decision_id
WHERE c.source_row_id = ?
""",
(row_id,),
).fetchone()
def _event_of(self, row_id: int) -> int:
return self.connection.execute(
"SELECT event_id FROM transfer_observation_claims WHERE source_row_id = ?",
(row_id,),
).fetchone()["event_id"]
def eligible(self) -> list[sqlite3.Row]:
return matching.eligible_intercompany_events(self.connection)
def event_count(self) -> int:
return self.connection.execute(
"SELECT COUNT(*) AS n FROM canonical_transfer_events"
).fetchone()["n"]
def decision_count(self) -> int:
return self.connection.execute(
"SELECT COUNT(*) AS n FROM transfer_match_decisions"
).fetchone()["n"]
class BilateralOrderTests(MatchingBase):
def test_a_outgoing_then_b_incoming_pairs_once(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00", at="2026-01-05T10:00:00",
)
matching.reconcile_rows(self.connection, [row_a])
self.assertEqual("internal_single", self.current(row_a)["classification"])
# A single internal observation must not enter the B-44 balance yet.
self.assertEqual([], self.eligible())
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00", at="2026-01-05T11:00:00",
)
matching.reconcile_rows(self.connection, [row_b])
for row_id in (row_a, row_b):
self.assertEqual("intercompany", self.current(row_id)["classification"])
self.assertEqual("paired", self.current(row_id)["pairing"])
eligible = self.eligible()
self.assertEqual(1, len(eligible))
self.assertEqual(Decimal("100.00"), Decimal(eligible[0]["amount"]))
self.assertEqual("CNY", eligible[0]["currency"])
self.assertEqual(self.company_a, eligible[0]["payer_company_id"])
self.assertEqual(self.company_b, eligible[0]["payee_company_id"])
self.assertEqual("paired", eligible[0]["pairing"])
# One event, one paired decision for both rows.
self.assertEqual(1, self.event_count())
def test_b_incoming_then_a_outgoing_same_result(self) -> None:
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00", at="2026-01-05T11:00:00",
)
matching.reconcile_rows(self.connection, [row_b])
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00", at="2026-01-05T10:00:00",
)
matching.reconcile_rows(self.connection, [row_a])
eligible = self.eligible()
self.assertEqual(1, len(eligible))
self.assertEqual("100.00", eligible[0]["amount"])
# Economic date is the payer's outgoing posting time, import-order free.
self.assertEqual("2026-01-05T10:00:00", eligible[0]["effective_at"])
self.assertEqual(self.company_a, eligible[0]["payer_company_id"])
def test_same_batch_both_sides_pairs_once(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00",
)
matching.reconcile_rows(self.connection, [row_b, row_a])
eligible = self.eligible()
self.assertEqual(1, len(eligible))
self.assertEqual("paired", self.current(row_a)["pairing"])
# One event, one auto decision carrying both observations.
self.assertEqual(1, self.event_count())
self.assertEqual(1, self.decision_count())
def test_repeated_reconcile_is_idempotent(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00",
)
matching.reconcile_rows(self.connection, [row_a, row_b])
before_decisions = self.decision_count()
stats = matching.reconcile_rows(self.connection, [row_a, row_b])
self.assertEqual(self.decision_count(), before_decisions)
self.assertEqual(2, stats["unchanged"])
self.assertEqual(0, stats["created_events"])
self.assertEqual(0, stats["updated_events"])
class DateWindowTests(MatchingBase):
def test_m1_same_reference_within_3_days(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
at="2026-01-01T09:00:00", reference="R1001",
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00",
at="2026-01-04T15:00:00", reference="R1001",
)
matching.reconcile_rows(self.connection, [row_a, row_b])
self.assertEqual("matched", matching.exposed_status(self.current(row_a)))
candidate = self.connection.execute(
"SELECT rule_tier FROM transfer_match_candidates WHERE accepted = 1"
).fetchone()
self.assertEqual("M1", candidate["rule_tier"])
def test_m2_exact_mirror_same_day_without_reference(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00",
)
matching.reconcile_rows(self.connection, [row_a, row_b])
self.assertEqual("matched", matching.exposed_status(self.current(row_a)))
candidate = self.connection.execute(
"SELECT rule_tier FROM transfer_match_candidates WHERE accepted = 1"
).fetchone()
self.assertEqual("M2", candidate["rule_tier"])
def test_over_window_goes_to_review(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
at="2026-01-01T09:00:00", reference="R1001",
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00",
at="2026-01-05T15:00:00", reference="R1001",
)
matching.reconcile_rows(self.connection, [row_a, row_b])
self.assertEqual("needs_review", matching.exposed_status(self.current(row_a)))
self.assertEqual([], self.eligible())
def test_reference_conflict_goes_to_review(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
reference="R-A",
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00",
reference="R-B",
)
matching.reconcile_rows(self.connection, [row_a, row_b])
self.assertEqual("needs_review", matching.exposed_status(self.current(row_a)))
class AmbiguityTests(MatchingBase):
def test_alias_across_companies_goes_to_review(self) -> None:
# The same observed number is an account alias of TWO companies; the
# counterparty cannot be uniquely decided, so it must go to review.
company_c_account = self._approved_account(self.company_c, "6222000000000005")
master_data.add_alias(
self.connection, self.account_b["id"], "account", "770077007700",
actor=self.admin,
)
master_data.add_alias(
self.connection, company_c_account["id"], "account", "770077007700",
actor=self.admin,
)
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="770077007700", expense="100.00",
)
matching.reconcile_rows(self.connection, [row_a])
self.assertEqual("needs_review", matching.exposed_status(self.current(row_a)))
self.assertEqual([], self.eligible())
def test_two_same_tier_candidates_go_to_review(self) -> None:
# Two identical incoming observations mirror the outgoing row exactly;
# no deterministic rule may pick one, so the pair goes to review.
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
)
row_b1 = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00",
)
row_b2 = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00",
)
matching.reconcile_rows(self.connection, [row_a, row_b1, row_b2])
# The ambiguous side stays in review; the two identical observations
# may not be auto-grabbed by a processing-order tie-break.
self.assertEqual("needs_review", matching.exposed_status(self.current(row_a)))
self.assertEqual("internal_single", matching.exposed_status(self.current(row_b1)))
self.assertEqual("internal_single", matching.exposed_status(self.current(row_b2)))
self.assertEqual([], self.eligible())
def test_different_amounts_never_match(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.01",
)
matching.reconcile_rows(self.connection, [row_a, row_b])
self.assertEqual("internal_single", matching.exposed_status(self.current(row_a)))
self.assertEqual("internal_single", matching.exposed_status(self.current(row_b)))
self.assertEqual([], self.eligible())
class SameCompanyTests(MatchingBase):
def test_same_company_transfer_excluded_from_intercompany(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000003", expense="50.00",
)
row_b = self.add_row(
self.company_a, own_account="6222000000000003",
cp_account="6222000000000001", income="50.00",
)
matching.reconcile_rows(self.connection, [row_a, row_b])
self.assertEqual("same_company_transfer", matching.exposed_status(self.current(row_a)))
self.assertEqual("same_company_transfer", matching.exposed_status(self.current(row_b)))
self.assertEqual([], self.eligible())
# The cash trail is kept: both rows remain claimed by one event.
self.assertEqual(1, self.event_count())
def test_same_company_single_observation_classified_immediately(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000003", expense="50.00",
)
matching.reconcile_rows(self.connection, [row_a])
self.assertEqual("same_company_transfer", matching.exposed_status(self.current(row_a)))
class UnresolvedAndExternalTests(MatchingBase):
def test_unknown_counterparty_stays_unresolved(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="9999999999999999", expense="100.00",
)
matching.reconcile_rows(self.connection, [row_a])
self.assertEqual("unresolved", matching.exposed_status(self.current(row_a)))
self.assertEqual([], self.eligible())
unresolved = matching.unresolved_amounts(self.connection, self.company_a)
self.assertEqual(1, len(unresolved))
self.assertEqual("100.00", unresolved[0]["amount"])
def test_internal_single_counts_as_unresolved_amount(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
)
matching.reconcile_rows(self.connection, [row_a])
self.assertEqual("internal_single", matching.exposed_status(self.current(row_a)))
self.assertEqual([], self.eligible())
unresolved = matching.unresolved_amounts(self.connection, self.company_a)
self.assertEqual(1, len(unresolved))
self.assertEqual("100.00", unresolved[0]["amount"])
def test_admin_confirm_single_becomes_eligible(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
)
matching.reconcile_rows(self.connection, [row_a])
current = self.current(row_a)
result = matching.apply_manual_decision(
self.connection, self._event_of(row_a),
"assign_participant",
reason="对方公司函证确认",
expected_revision=current["revision"],
request_key="assign-1",
actor=self.admin,
participant={"role": "payee", "company_id": self.company_b},
)
self.assertEqual("intercompany", result["classification"])
self.assertTrue(result["locked"])
eligible = self.eligible()
self.assertEqual(1, len(eligible))
self.assertEqual("single", eligible[0]["pairing"])
# The manual confirmation is skipped by auto reconcile.
decisions_before = self.decision_count()
stats = matching.reconcile_rows(self.connection, [row_a])
self.assertEqual(1, stats["skipped_locked"])
self.assertEqual(decisions_before, self.decision_count())
def test_mark_external_requires_admin_and_excludes(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="9999999999999999", expense="100.00",
)
matching.reconcile_rows(self.connection, [row_a])
event_id = self._event_of(row_a)
result = matching.apply_manual_decision(
self.connection, event_id, "mark_external",
reason="经核实为外部供应商付款",
expected_revision=self.current(row_a)["revision"],
request_key=None, actor=self.admin,
)
self.assertEqual("external", result["classification"])
self.assertEqual([], self.eligible())
def test_reverse_frees_claims_for_reconcile(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00",
)
matching.reconcile_rows(self.connection, [row_a, row_b])
event_id = self._event_of(row_a)
decisions_before = self.decision_count()
result = matching.apply_manual_decision(
self.connection, event_id, "reverse",
reason="配对依据有误,需要重新匹配",
expected_revision=self.current(row_a)["revision"],
request_key="rev-1", actor=self.admin,
)
self.assertEqual(decisions_before + 1, self.decision_count())
# Claims and current pointer are gone; B-44 sees nothing.
self.assertIsNone(self.connection.execute(
"SELECT 1 FROM transfer_observation_claims WHERE source_row_id = ?",
(row_a,),
).fetchone())
self.assertEqual([], self.eligible())
# A re-run is allowed and re-derives deterministically.
matching.reconcile_rows(self.connection, [row_a])
self.assertEqual("matched", matching.exposed_status(self.current(row_a)))
class PersonalTransitTests(MatchingBase):
def _submit_mapping(self, company_id: int, number: str, direction: str) -> int:
mapping = personal_transit.submit_mapping(
self.connection,
account_number=number,
account_name="张个人",
represented_company_id=company_id,
allowed_direction=direction,
effective_from="2026-01-01",
actor=self.admin,
)
approved = personal_transit.review_mapping(
self.connection, mapping["id"], "approve", None, self.admin,
effective_from="2026-01-01",
)
return approved["id"]
def test_personal_mapping_resolves_counterparty(self) -> None:
self._submit_mapping(self.company_b, "880088008800", "incoming")
# A pays to the personal account -> the counterparty represents B.
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="880088008800", expense="100.00",
)
matching.reconcile_rows(self.connection, [row_a])
self.assertEqual("internal_single", matching.exposed_status(self.current(row_a)))
unresolved = matching.unresolved_amounts(self.connection, self.company_a)
self.assertEqual(1, len(unresolved))
def test_personal_mapping_direction_mismatch_unresolved(self) -> None:
self._submit_mapping(self.company_b, "880088008800", "incoming")
# An outgoing row to the personal account is NOT covered by an
# incoming-only mapping.
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="880088008800", expense="100.00",
)
# reverse direction check: make it incoming via income
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="880088008800", income="100.00",
)
matching.reconcile_rows(self.connection, [row_a])
matching.reconcile_rows(self.connection, [row_b])
self.assertEqual("unresolved", matching.exposed_status(self.current(row_b)))
def test_unapproved_mapping_never_resolves(self) -> None:
personal_transit.submit_mapping(
self.connection,
account_number="880088008801",
account_name="李个人",
represented_company_id=self.company_b,
allowed_direction="both",
actor=self.admin,
)
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="880088008801", expense="100.00",
)
matching.reconcile_rows(self.connection, [row_a])
self.assertEqual("unresolved", matching.exposed_status(self.current(row_a)))
def test_mapping_effective_window_and_resubmit_reuse(self) -> None:
mapping = personal_transit.submit_mapping(
self.connection,
account_number="880088008802",
account_name="王个人",
represented_company_id=self.company_b,
allowed_direction="both",
effective_from="2026-03-01",
actor=self.admin,
)
personal_transit.review_mapping(
self.connection, mapping["id"], "approve", None, self.admin,
effective_from="2026-03-01",
)
# Outside the effective window the mapping does not resolve.
row_before = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="880088008802", expense="100.00", at="2026-02-20T10:00:00",
)
matching.reconcile_rows(self.connection, [row_before])
self.assertEqual("unresolved", matching.exposed_status(self.current(row_before)))
# Inside the window it does.
row_inside = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="880088008802", expense="100.00", at="2026-03-05T10:00:00",
)
matching.reconcile_rows(self.connection, [row_inside])
self.assertEqual("internal_single", matching.exposed_status(self.current(row_inside)))
# Returned mappings reopen on resubmission reusing the same row.
returned = personal_transit.submit_mapping(
self.connection,
account_number="880088008803",
account_name="赵个人",
represented_company_id=self.company_b,
allowed_direction="both",
actor=self.admin,
)
personal_transit.review_mapping(
self.connection, returned["id"], "return", "资料待补充", self.admin,
)
resubmitted = personal_transit.submit_mapping(
self.connection,
account_number="8800 8800 8803",
account_name="赵个人",
represented_company_id=self.company_b,
allowed_direction="incoming",
actor=self.admin,
)
self.assertEqual(returned["id"], resubmitted["id"])
self.assertEqual("pending", resubmitted["status"])
class ResolutionTests(MatchingBase):
def test_name_alias_never_resolves_alone(self) -> None:
master_data.add_alias(
self.connection, self.account_b["id"], "name", "乙公司贸易部",
actor=self.admin,
)
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account=None, cp_name="乙公司贸易部", expense="100.00",
)
matching.reconcile_rows(self.connection, [row_a])
self.assertEqual("unresolved", matching.exposed_status(self.current(row_a)))
def test_own_account_conflict_with_upload_account_goes_to_review(self) -> None:
# own_account belongs to company B while the batch is uploaded for A.
row = self.add_row(
self.company_a, own_account="6222000000000002",
cp_account="6222000000000001", expense="100.00",
)
# Simulate an upload account pointing at A.
self.connection.execute(
"""
UPDATE import_batches SET upload_bank_account_id = ?
WHERE id = (SELECT b.id FROM import_batches b
JOIN sheet_batches s ON s.import_batch_id = b.id
JOIN source_rows r ON r.sheet_batch_id = s.id
WHERE r.id = ?)
""",
(self.account_a["id"], row),
)
matching.reconcile_rows(self.connection, [row])
self.assertEqual("needs_review", matching.exposed_status(self.current(row)))
def test_upload_account_fallback_resolves_own(self) -> None:
row = self.add_row(
self.company_a, own_account=None,
cp_account="6222000000000002", expense="100.00",
)
self.connection.execute(
"""
UPDATE import_batches SET upload_bank_account_id = ?
WHERE id = (SELECT b.id FROM import_batches b
JOIN sheet_batches s ON s.import_batch_id = b.id
JOIN source_rows r ON r.sheet_batch_id = s.id
WHERE r.id = ?)
""",
(self.account_a["id"], row),
)
matching.reconcile_rows(self.connection, [row])
self.assertEqual("internal_single", matching.exposed_status(self.current(row)))
def test_alias_mirror_m3_pairs(self) -> None:
# A's own account has an account alias; B's statement references the
# alias, so the mirror is proven through the approved alias (M3).
master_data.add_alias(
self.connection, self.account_a["id"], "account", "770077007700",
actor=self.admin,
)
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00", summary="货款",
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="770077007700", income="100.00", summary="货款",
)
matching.reconcile_rows(self.connection, [row_a, row_b])
self.assertEqual("matched", matching.exposed_status(self.current(row_a)))
candidate = self.connection.execute(
"SELECT rule_tier FROM transfer_match_candidates WHERE accepted = 1"
).fetchone()
self.assertEqual("M3", candidate["rule_tier"])
class ManualLinkTests(MatchingBase):
def test_manual_link_locks_pair(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00",
)
matching.reconcile_rows(self.connection, [row_a])
matching.reconcile_rows(self.connection, [row_b])
event_id = self._event_of(row_a)
current_revision = self.current(row_a)["revision"]
result = matching.apply_manual_decision(
self.connection, event_id, "link_rows",
reason="人工核对后确认是同一笔",
expected_revision=current_revision,
request_key="link-1", actor=self.admin,
source_row_ids=[row_a, row_b],
)
self.assertTrue(result["locked"])
self.assertEqual("paired", result["pairing"])
# Auto reconcile never touches locked decisions.
stats = matching.reconcile_rows(self.connection, [row_a, row_b])
self.assertEqual(2, stats["skipped_locked"])
def test_manual_decision_requires_reason(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
)
matching.reconcile_rows(self.connection, [row_a])
with self.assertRaises(matching.MatchInputError):
matching.apply_manual_decision(
self.connection, self._event_of(row_a), "reverse",
reason=" ", expected_revision=None, request_key=None,
actor=self.admin,
)
def test_stale_expected_revision_conflicts(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
)
matching.reconcile_rows(self.connection, [row_a])
event_id = self._event_of(row_a)
with self.assertRaises(matching.MatchConflictError):
matching.apply_manual_decision(
self.connection, event_id, "reverse",
reason="测试", expected_revision=999, request_key=None,
actor=self.admin,
)
class DecimalPrecisionTests(MatchingBase):
def test_equivalent_decimal_strings_match(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.0",
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00",
)
matching.reconcile_rows(self.connection, [row_a, row_b])
self.assertEqual("matched", matching.exposed_status(self.current(row_a)))
def test_cent_difference_never_matches(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.01",
)
matching.reconcile_rows(self.connection, [row_a, row_b])
self.assertEqual("internal_single", matching.exposed_status(self.current(row_a)))
def test_large_amount_precision(self) -> None:
amount = "99999999999999999999.99"
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense=amount,
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income=amount,
)
matching.reconcile_rows(self.connection, [row_a, row_b])
eligible = self.eligible()
self.assertEqual(1, len(eligible))
self.assertEqual(Decimal(amount), Decimal(eligible[0]["amount"]))
def test_currency_mismatch_never_matches(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00", currency="CNY",
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00", currency="USD",
)
matching.reconcile_rows(self.connection, [row_a, row_b])
self.assertEqual("internal_single", matching.exposed_status(self.current(row_a)))
class ProjectionRebuildTests(MatchingBase):
def test_rebuild_matches_current_projection(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00",
)
matching.reconcile_rows(self.connection, [row_a, row_b])
before = [
(item["event_id"], item["decision_id"])
for item in self.connection.execute(
"SELECT * FROM current_transfer_decisions"
).fetchall()
]
claims_before = [
(item["source_row_id"], item["event_id"], item["decision_id"])
for item in self.connection.execute(
"SELECT * FROM transfer_observation_claims"
).fetchall()
]
matching.rebuild_current_projection(self.connection)
after = [
(item["event_id"], item["decision_id"])
for item in self.connection.execute(
"SELECT * FROM current_transfer_decisions"
).fetchall()
]
claims_after = [
(item["source_row_id"], item["event_id"], item["decision_id"])
for item in self.connection.execute(
"SELECT * FROM transfer_observation_claims"
).fetchall()
]
self.assertEqual(sorted(before), sorted(after))
self.assertEqual(sorted(claims_before), sorted(claims_after))
class ConcurrentReconcileTests(MatchingBase):
def test_concurrent_reconcile_creates_one_event(self) -> None:
row_a = self.add_row(
self.company_a, own_account="6222000000000001",
cp_account="6222000000000002", expense="100.00",
)
row_b = self.add_row(
self.company_b, own_account="6222000000000002",
cp_account="6222000000000001", income="100.00",
)
errors: list[Exception] = []
def run() -> None:
db = connect(self.db_path)
try:
matching.reconcile_rows(db, [row_a, row_b])
except Exception as exc: # pragma: no cover
errors.append(exc)
finally:
db.close()
threads = [threading.Thread(target=run) for _ in range(2)]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
self.assertEqual([], errors)
self.assertEqual(1, self.event_count())
eligible = self.eligible()
self.assertEqual(1, len(eligible))
claims = self.connection.execute(
"SELECT COUNT(*) AS n FROM transfer_observation_claims"
).fetchone()["n"]
self.assertEqual(2, claims)
if __name__ == "__main__":
unittest.main()
+463
View File
@@ -0,0 +1,463 @@
"""HTTP integration tests for the canonical transfer event APIs (B-43).
Covers the admin event list/detail, manual decisions and reconcile endpoints,
personal transit mapping workflow, upload-account persistence, and the
company-side read scoping with masked counterparty evidence. Uses a real
``ThreadingHTTPServer`` like ``test_server_auth``.
"""
from __future__ import annotations
import io
import json
import os
from pathlib import Path
import tempfile
import threading
import unittest
from openpyxl import Workbook
from bank_importer.db import connect, migrate
import server
from test_server_auth import Client, as_json
BOOTSTRAP_PASSWORD = "BootAdmin123"
ADMIN_PASSWORD = "AdminPass123"
CASHIER_PASSWORD = "Cashier123"
CCB_HEADER = [
"客户账号", "账户名称", "交易时间", "借方发生额(支取)", "贷方发生额(收入)",
"余额", "币种", "对方户名", "对方账号", "对方开户机构", "摘要", "备注",
]
ACCOUNT_A = "6222000000000001"
ACCOUNT_B = "6222000000000002"
ACCOUNT_C = "6222000000000003"
def workbook_bytes(rows) -> bytes:
workbook = Workbook()
sheet = workbook.active
sheet.title = "正常流水"
sheet.append(CCB_HEADER)
for row in rows:
sheet.append(row)
buffer = io.BytesIO()
workbook.save(buffer)
return buffer.getvalue()
def outgoing(own: str, cp: str, amount: str, at: str = "2026-01-05 10:00:00"):
return [own, "测试公司", at, amount, "", "50000.00", "RMB", "对方", cp, "某银行", "货款", ""]
def incoming(own: str, cp: str, amount: str, at: str = "2026-01-05 11:00:00"):
return [own, "测试公司", at, "", amount, "50000.00", "RMB", "对方", cp, "某银行", "收款", ""]
class MatchingApiTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.temp_dir = tempfile.TemporaryDirectory()
root = Path(cls.temp_dir.name)
cls.db_path = root / "app.db"
cls.storage = root / "files"
cls._old_db_path = server.DB_PATH
cls._old_storage = server.STORAGE_DIR
server.DB_PATH = cls.db_path
server.STORAGE_DIR = cls.storage
os.environ["APP_BOOTSTRAP_ADMIN_PASSWORD"] = BOOTSTRAP_PASSWORD
connection = connect(cls.db_path)
migrate(connection)
assert server.ensure_bootstrap_admin(connection) is None
connection.close()
class QuietHandler(server.AppHandler):
def log_message(self, *args) -> None:
pass
cls.httpd = server.ThreadingHTTPServer(("127.0.0.1", 0), QuietHandler)
cls.port = cls.httpd.server_address[1]
cls.thread = threading.Thread(target=cls.httpd.serve_forever, daemon=True)
cls.thread.start()
cls.admin = Client("127.0.0.1", cls.port)
status, _, data = cls.admin.post_json(
"/api/login",
{"username": "group-admin", "password": BOOTSTRAP_PASSWORD, "portal": "admin"},
)
assert status == 200, data
status, _, data = cls.admin.post_json(
"/api/password/change",
{"old_password": BOOTSTRAP_PASSWORD, "new_password": ADMIN_PASSWORD},
)
assert status == 200, data
cls.company_a = cls._create_company("甲公司", "cashier-a")
cls.company_b = cls._create_company("乙公司", "cashier-b")
cls.company_c = cls._create_company("丙公司", "cashier-c")
cls.cashier_a = cls._login_company("cashier-a", cls.company_a)
cls.cashier_b = cls._login_company("cashier-b", cls.company_b)
cls.cashier_c = cls._login_company("cashier-c", cls.company_c)
cls.account_a = cls._approve_account(cls.company_a, ACCOUNT_A)
cls.account_b = cls._approve_account(cls.company_b, ACCOUNT_B)
cls.account_c = cls._approve_account(cls.company_c, ACCOUNT_C)
# One A<->B matched event and one B<->C matched event.
cls._upload_and_confirm(cls.cashier_a, cls.company_a, [outgoing(ACCOUNT_A, ACCOUNT_B, "100.00")])
cls._upload_and_confirm(cls.cashier_b, cls.company_b, [incoming(ACCOUNT_B, ACCOUNT_A, "100.00")])
cls._upload_and_confirm(cls.cashier_b, cls.company_b, [outgoing(ACCOUNT_B, ACCOUNT_C, "200.00")])
cls._upload_and_confirm(cls.cashier_c, cls.company_c, [incoming(ACCOUNT_C, ACCOUNT_B, "200.00")])
@classmethod
def _create_company(cls, name: str, username: str) -> int:
status, _, data = cls.admin.post_json(
"/api/admin/companies",
{"name": name, "username": username},
)
assert status == 200, data
cls.initial_passwords.setdefault(username, as_json(data)["initial_password"])
return as_json(data)["company_id"]
initial_passwords: dict[str, str] = {}
@classmethod
def _login_company(cls, username: str, company_id: int) -> Client:
client = Client("127.0.0.1", cls.port)
initial = cls.initial_passwords[username]
status, _, data = client.post_json(
"/api/login", {"username": username, "password": initial, "portal": "company"}
)
assert status == 200, data
status, _, data = client.post_json(
"/api/password/change",
{"old_password": initial, "new_password": CASHIER_PASSWORD},
)
assert status == 200, data
return client
@classmethod
def _approve_account(cls, company_id: int, number: str) -> int:
# Build the account through company submission + admin review.
company_client = {
cls.company_a: cls.cashier_a,
cls.company_b: cls.cashier_b,
cls.company_c: cls.cashier_c,
}[company_id]
status, _, data = company_client.post_json(
"/api/company/accounts",
{"bank_name": "中信银行", "account_type": "基本户",
"account_number": number, "start_date": "2026-01-01"},
)
assert status == 200, data
account_id = as_json(data)["account"]["id"]
status, _, data = cls.admin.post_json(
f"/api/admin/accounts/{account_id}/review",
{"decision": "approve", "reason": "测试启用",
"effective_from": "2026-01-01"},
)
assert status == 200, data
return account_id
@classmethod
def _upload_and_confirm(cls, client, company_id: int, rows) -> int:
content = workbook_bytes(rows)
status, _, data = cls.admin.post_multipart(
"/api/parse", {"company_id": str(company_id)}, "账单.xlsx", content
)
assert status == 200, data
batch_id = as_json(data)["batch_id"]
status, _, data = client.get(f"/api/batches/{batch_id}/sheets")
assert status == 200, data
names = [s["sheet_name"] for s in as_json(data)["sheets"] if s["outcome"] == "parsed"]
status, _, data = client.post_json(
f"/api/batches/{batch_id}/confirm", {"sheets": names}
)
assert status == 200, data
return batch_id
@classmethod
def tearDownClass(cls) -> None:
cls.httpd.shutdown()
cls.httpd.server_close()
server.DB_PATH = cls._old_db_path
server.STORAGE_DIR = cls._old_storage
os.environ.pop("APP_BOOTSTRAP_ADMIN_PASSWORD", None)
cls.temp_dir.cleanup()
def events(self, client) -> list[dict]:
status, _, data = client.get("/api/admin/transfer-events")
self.assertEqual(200, status, data)
return as_json(data)["events"]
def company_events(self, client) -> list[dict]:
status, _, data = client.get("/api/company/transfer-events")
self.assertEqual(200, status, data)
return as_json(data)["events"]
# ------------------------------------------------------------------
# Admin event views
# ------------------------------------------------------------------
def test_admin_lists_matched_events(self) -> None:
events = self.events(self.admin)
matched = [e for e in events if e["status"] == "matched"]
self.assertGreaterEqual(len(matched), 2)
by_amount = {e["amount"]: e for e in matched}
self.assertEqual(self.company_a, by_amount["100.00"]["payer_company_id"])
self.assertEqual(self.company_b, by_amount["100.00"]["payee_company_id"])
self.assertEqual("paired", by_amount["100.00"]["pairing"])
self.assertEqual(2, by_amount["100.00"]["evidence_count"])
def test_admin_event_detail_has_history_and_observations(self) -> None:
events = self.events(self.admin)
event = next(e for e in events if e["status"] == "matched")
status, _, data = self.admin.get(f"/api/admin/transfer-events/{event['event_id']}")
self.assertEqual(200, status, data)
detail = as_json(data)["event"]
self.assertEqual("matched", detail["status"])
self.assertEqual(2, len(detail["observations"]))
self.assertTrue(detail["history"])
self.assertTrue(detail["candidates"])
def test_admin_match_exceptions(self) -> None:
status, _, data = self.admin.get("/api/admin/match-exceptions")
self.assertEqual(200, status, data)
self.assertIsInstance(as_json(data)["exceptions"], list)
# ------------------------------------------------------------------
# Admin manual decisions and reconcile via API
# ------------------------------------------------------------------
def test_admin_reconcile_by_batch_is_idempotent(self) -> None:
# Re-running reconcile on an already settled batch changes nothing.
status, _, data = self.admin.post_json(
"/api/admin/transfer-events/reconcile", {"batch_id": 1}
)
self.assertEqual(200, status, data)
result = as_json(data)["matching"]
self.assertEqual(0, result["created_events"])
self.assertEqual(0, result["updated_events"])
def test_admin_confirm_single_and_reverse_via_api(self) -> None:
# A uploads its side only -> internal_single.
status, _, data = self.admin.post_multipart(
"/api/parse", {"company_id": str(self.company_a)},
"单边.xlsx", workbook_bytes([outgoing(ACCOUNT_A, ACCOUNT_B, "300.00", "2026-02-01 10:00:00")]),
)
self.assertEqual(200, status, data)
batch_id = as_json(data)["batch_id"]
status, _, data = self.cashier_a.get(f"/api/batches/{batch_id}/sheets")
names = [s["sheet_name"] for s in as_json(data)["sheets"] if s["outcome"] == "parsed"]
status, _, data = self.cashier_a.post_json(
f"/api/batches/{batch_id}/confirm", {"sheets": names}
)
self.assertEqual(200, status, data)
events = self.events(self.admin)
single = next(e for e in events if e["status"] == "internal_single" and e["amount"] == "300.00")
detail = self._admin_detail(single["event_id"])
revision = detail["revision"]
# Company users cannot write manual decisions.
status, _, data = self.cashier_a.post_json(
f"/api/admin/transfer-events/{single['event_id']}/decisions",
{"action": "reverse", "reason": "不应允许", "expected_revision": revision},
)
self.assertEqual(403, status, data)
# Admin confirms the single event based on evidence.
status, _, data = self.admin.post_json(
f"/api/admin/transfer-events/{single['event_id']}/decisions",
{"action": "assign_participant", "reason": "函证确认",
"expected_revision": revision, "request_key": "confirm-300",
"participant": {"role": "payee", "company_id": self.company_b}},
)
self.assertEqual(200, status, data)
decision = as_json(data)["decision"]
self.assertEqual("intercompany", decision["classification"])
self.assertTrue(decision["locked"])
# Replaying the same request key is idempotent.
status, _, data = self.admin.post_json(
f"/api/admin/transfer-events/{single['event_id']}/decisions",
{"action": "assign_participant", "reason": "函证确认",
"expected_revision": revision, "request_key": "confirm-300",
"participant": {"role": "payee", "company_id": self.company_b}},
)
self.assertEqual(200, status, data)
# Stale revision conflicts.
status, _, data = self.admin.post_json(
f"/api/admin/transfer-events/{single['event_id']}/decisions",
{"action": "reverse", "reason": "撤销", "expected_revision": revision},
)
self.assertEqual(409, status, data)
def _admin_detail(self, event_id: int) -> dict:
status, _, data = self.admin.get(f"/api/admin/transfer-events/{event_id}")
self.assertEqual(200, status, data)
return as_json(data)["event"]
# ------------------------------------------------------------------
# Company scope and masking
# ------------------------------------------------------------------
def test_company_sees_only_events_it_participates_in(self) -> None:
events = self.company_events(self.cashier_a)
self.assertTrue(events)
for event in events:
self.assertEqual(self.company_a, event["own_company_id"])
matched = [e for e in events if e["status"] == "matched"]
self.assertTrue(matched)
self.assertEqual(
{"乙公司"},
{e["counterparty_company_name"] for e in matched},
)
# B<->C event is invisible to A.
status, _, data = self.cashier_a.get("/api/company/transfer-events")
bc_events = [
e for e in as_json(data)["events"]
if e["counterparty_company_name"] == "丙公司"
]
self.assertEqual([], bc_events)
def test_company_detail_masks_counterparty_and_own_rows_only(self) -> None:
events = self.company_events(self.cashier_a)
event = next(e for e in events if e["status"] == "matched")
status, _, data = self.cashier_a.get(
f"/api/company/transfer-events/{event['event_id']}"
)
self.assertEqual(200, status, data)
detail = as_json(data)["event"]
self.assertEqual("乙公司", detail["counterparty"]["company_name"])
self.assertTrue(detail["counterparty"].get("account_number_masked", "").startswith("****"))
# The counterparty's full account number never leaves the server.
self.assertNotIn(ACCOUNT_B, json.dumps(detail, ensure_ascii=False))
# Only A's own observation rows are exposed.
self.assertEqual(1, len(detail["observations"]))
observation = detail["observations"][0]
self.assertTrue(observation["own_account_masked"].startswith("****"))
self.assertEqual(self.company_a, observation.get("batch_company_id"))
def test_company_cannot_read_event_it_does_not_participate_in(self) -> None:
# The B<->C event id comes from B's own list; A probing it returns 404.
status, _, data = self.cashier_b.get("/api/company/transfer-events")
bc_event_ids = [
e["event_id"] for e in as_json(data)["events"]
if e["counterparty_company_name"] == "丙公司"
]
self.assertTrue(bc_event_ids)
for event_id in bc_event_ids:
status, _, data = self.cashier_a.get(f"/api/company/transfer-events/{event_id}")
self.assertEqual(404, status, data)
status, _, data = self.cashier_a.get(f"/api/admin/transfer-events/{event_id}")
self.assertEqual(403, status, data)
def test_company_match_exceptions_scoped_to_own_company(self) -> None:
status, _, data = self.admin.post_multipart(
"/api/parse", {"company_id": str(self.company_a)},
"未决.xlsx",
workbook_bytes([outgoing(ACCOUNT_A, "9999999999999999", "77.00", "2026-02-05 10:00:00")]),
)
self.assertEqual(200, status, data)
batch_id = as_json(data)["batch_id"]
status, _, data = self.cashier_a.get(f"/api/batches/{batch_id}/sheets")
names = [s["sheet_name"] for s in as_json(data)["sheets"] if s["outcome"] == "parsed"]
status, _, data = self.cashier_a.post_json(
f"/api/batches/{batch_id}/confirm", {"sheets": names}
)
self.assertEqual(200, status, data)
status, _, data = self.cashier_a.get("/api/company/match-exceptions")
self.assertEqual(200, status, data)
mine = [e for e in as_json(data)["exceptions"] if e["amount"] == "77.00"]
self.assertEqual(1, len(mine))
self.assertEqual("unresolved", mine[0]["status"])
status, _, data = self.cashier_b.get("/api/company/match-exceptions")
self.assertEqual(200, status, data)
self.assertEqual(
[],
[e for e in as_json(data)["exceptions"] if e["amount"] == "77.00"],
)
def test_company_forbidden_on_admin_event_endpoints(self) -> None:
for call in (
lambda: self.cashier_a.get("/api/admin/transfer-events"),
lambda: self.cashier_a.get("/api/admin/match-exceptions"),
lambda: self.cashier_a.post_json(
"/api/admin/transfer-events/reconcile", {"batch_id": 1}
),
lambda: self.cashier_a.post_json(
"/api/admin/personal-transit-mappings", {}
),
lambda: self.cashier_a.get("/api/admin/personal-transit-mappings"),
):
status, _, data = call()
self.assertEqual(403, status, data)
# ------------------------------------------------------------------
# Personal transit mappings
# ------------------------------------------------------------------
def test_personal_transit_mapping_workflow(self) -> None:
status, _, data = self.admin.post_json(
"/api/admin/personal-transit-mappings",
{"account_number": "880088008800", "account_name": "张个人",
"represented_company_id": self.company_b,
"allowed_direction": "incoming", "effective_from": "2026-01-01"},
)
self.assertEqual(200, status, data)
mapping_id = as_json(data)["mapping"]["id"]
self.assertEqual("pending", as_json(data)["mapping"]["status"])
status, _, data = self.admin.post_json(
f"/api/admin/personal-transit-mappings/{mapping_id}/review",
{"decision": "approve", "reason": "资料齐全", "effective_from": "2026-01-01"},
)
self.assertEqual(200, status, data)
self.assertEqual("active", as_json(data)["mapping"]["status"])
status, _, data = self.admin.get("/api/admin/personal-transit-mappings")
self.assertEqual(200, status, data)
mappings = as_json(data)["mappings"]
self.assertTrue(any(m["id"] == mapping_id for m in mappings))
# Duplicate account number conflicts.
status, _, data = self.admin.post_json(
"/api/admin/personal-transit-mappings",
{"account_number": "880088008800", "account_name": "张个人",
"represented_company_id": self.company_b, "allowed_direction": "both"},
)
self.assertEqual(409, status, data)
# ------------------------------------------------------------------
# Upload account persistence
# ------------------------------------------------------------------
def test_upload_persists_approved_bank_account(self) -> None:
content = workbook_bytes([outgoing(ACCOUNT_A, ACCOUNT_B, "55.00", "2026-03-01 10:00:00")])
status, _, data = self.cashier_a.post_multipart(
"/api/parse", {"bank_account_id": str(self.account_a)},
"带账户.xlsx", content,
)
self.assertEqual(200, status, data)
batch_id = as_json(data)["batch_id"]
connection = connect(self.db_path)
try:
row = connection.execute(
"SELECT upload_bank_account_id FROM import_batches WHERE id = ?",
(batch_id,),
).fetchone()
finally:
connection.close()
self.assertEqual(self.account_a, row["upload_bank_account_id"])
if __name__ == "__main__":
unittest.main()
+127 -4
View File
@@ -41,7 +41,7 @@ class PersistenceTestCase(unittest.TestCase):
class MigrationTests(PersistenceTestCase): class MigrationTests(PersistenceTestCase):
def test_migrate_creates_schema_and_is_idempotent(self) -> None: def test_migrate_creates_schema_and_is_idempotent(self) -> None:
first = applied_versions(self.connection) first = applied_versions(self.connection)
self.assertEqual([1, 2, 3, 4], first) self.assertEqual([1, 2, 3, 4, 5], first)
self.assertEqual([], migrate(self.connection)) self.assertEqual([], migrate(self.connection))
self.assertEqual(first, applied_versions(self.connection)) self.assertEqual(first, applied_versions(self.connection))
tables = { tables = {
@@ -65,19 +65,142 @@ class MigrationTests(PersistenceTestCase):
"sessions", "sessions",
"login_attempts", "login_attempts",
"audit_log", "audit_log",
"personal_transit_mappings",
"canonical_transfer_events",
"transfer_match_decisions",
"transfer_decision_observations",
"transfer_decision_participants",
"transfer_match_candidates",
"current_transfer_decisions",
"transfer_observation_claims",
"schema_migrations", "schema_migrations",
): ):
self.assertIn(table, tables) self.assertIn(table, tables)
def test_rollback_removes_schema_and_forward_rebuilds_it(self) -> None: def test_rollback_removes_schema_and_forward_rebuilds_it(self) -> None:
self.assertEqual([4, 3, 2, 1], rollback(self.connection, 0)) self.assertEqual([5, 4, 3, 2, 1], rollback(self.connection, 0))
self.assertEqual([], applied_versions(self.connection)) self.assertEqual([], applied_versions(self.connection))
remaining = self.connection.execute( remaining = self.connection.execute(
"SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'source_rows'" "SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'source_rows'"
).fetchone() ).fetchone()
self.assertIsNone(remaining) self.assertIsNone(remaining)
self.assertEqual([1, 2, 3, 4], migrate(self.connection)) self.assertEqual([1, 2, 3, 4, 5], migrate(self.connection))
self.assertEqual([1, 2, 3, 4], applied_versions(self.connection)) self.assertEqual([1, 2, 3, 4, 5], applied_versions(self.connection))
def test_rollback_to_4_keeps_bank_evidence_and_drops_event_layer(self) -> None:
self.import_sample()
row_count = self.connection.execute(
"SELECT COUNT(*) AS n FROM source_rows"
).fetchone()["n"]
self.assertGreater(row_count, 0)
self.assertEqual([5], rollback(self.connection, 4))
# The pre-migration evidence and schema are untouched.
self.assertEqual(
row_count,
self.connection.execute("SELECT COUNT(*) AS n FROM source_rows").fetchone()["n"],
)
self.assertEqual(
"parsed",
self.connection.execute(
"SELECT status FROM import_batches LIMIT 1"
).fetchone()["status"],
)
remaining = self.connection.execute(
"SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'transfer_match_decisions'"
).fetchone()
self.assertIsNone(remaining)
def test_event_layer_views_exist_after_migration(self) -> None:
view = self.connection.execute(
"SELECT name FROM sqlite_master WHERE type = 'view' AND name = 'eligible_intercompany_events'"
).fetchone()
self.assertIsNotNone(view)
def test_decision_log_immutability_triggers(self) -> None:
self.import_sample()
source_row_id = self.connection.execute(
"SELECT id FROM source_rows LIMIT 1"
).fetchone()["id"]
with self.connection:
self.connection.execute(
"INSERT INTO canonical_transfer_events (lifecycle, created_at) VALUES ('active', ?)",
(utc_now(),),
)
event_id = self.connection.execute(
"SELECT id FROM canonical_transfer_events LIMIT 1"
).fetchone()["id"]
self.connection.execute(
"""
INSERT INTO transfer_match_decisions (
event_id, revision, classification, pairing, mode, rule_version,
locked, created_at
) VALUES (?, 1, 'unresolved', 'not_applicable', 'auto', 'transfer-match-v1', 0, ?)
""",
(event_id, utc_now()),
)
decision_id = self.connection.execute(
"SELECT id FROM transfer_match_decisions LIMIT 1"
).fetchone()["id"]
self.connection.execute(
"""
INSERT INTO transfer_match_candidates (
decision_id, source_row_id, rule_tier, rule_version, created_at
) VALUES (?, ?, 'R1', 'transfer-match-v1', ?)
""",
(decision_id, source_row_id, utc_now()),
)
for statement in (
"UPDATE transfer_match_decisions SET classification = 'external'",
"DELETE FROM transfer_match_decisions",
"UPDATE transfer_match_candidates SET rule_tier = 'M1'",
"DELETE FROM transfer_match_candidates",
"DELETE FROM canonical_transfer_events",
):
with self.subTest(statement=statement):
with self.assertRaises(sqlite3.IntegrityError):
self.connection.execute(statement)
self.connection.rollback()
def test_observation_claims_source_row_is_unique(self) -> None:
self.import_sample()
source_row_id = self.connection.execute(
"SELECT id FROM source_rows LIMIT 1"
).fetchone()["id"]
with self.connection:
self.connection.execute(
"INSERT INTO canonical_transfer_events (lifecycle, created_at) VALUES ('active', ?)",
(utc_now(),),
)
event_id = self.connection.execute(
"SELECT id FROM canonical_transfer_events LIMIT 1"
).fetchone()["id"]
self.connection.execute(
"""
INSERT INTO transfer_match_decisions (
event_id, revision, classification, pairing, mode, locked, created_at
) VALUES (?, 1, 'unresolved', 'not_applicable', 'auto', 0, ?)
""",
(event_id, utc_now()),
)
decision_id = self.connection.execute(
"SELECT id FROM transfer_match_decisions LIMIT 1"
).fetchone()["id"]
self.connection.execute(
"""
INSERT INTO transfer_observation_claims (source_row_id, event_id, decision_id)
VALUES (?, ?, ?)
""",
(source_row_id, event_id, decision_id),
)
with self.assertRaises(sqlite3.IntegrityError):
self.connection.execute(
"""
INSERT INTO transfer_observation_claims (source_row_id, event_id, decision_id)
VALUES (?, ?, ?)
""",
(source_row_id, event_id, decision_id),
)
self.connection.rollback()
class ImportPersistenceTests(PersistenceTestCase): class ImportPersistenceTests(PersistenceTestCase):