Files
ccdi/docs/plans/backend/2026-03-20-lsfx-mock-phase2-random-hit-backend-implementation.md

574 lines
20 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# LSFX Mock Phase 2 Random Hit Backend Implementation Plan
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** 改造 `lsfx-mock-server`,在保留第一期稳定随机命中方案的前提下,为第二期规则补齐稳定随机命中计划、最小流水样本和幂等数据库基线,使兰溪本地接口取数入库后可命中新增加的第二期真实规则。
**Architecture:** 保持现有 `FileService -> StatementService -> 缓存分页` 主链路不变,不新增兼容性双轨。`FileService` 只负责生成并持久化第二期规则命中计划,`statement_rule_samples.py` 只负责装配可由银行流水驱动的第二期样本,新增的 `phase2_baseline_service.py` 负责幂等补齐采购与资产事实基线;主工程继续沿用现有真实 SQL 打标链路,不加联调补丁。
**Tech Stack:** Python 3, FastAPI, pytest, PyMySQL, MySQL, Bash
---
## File Structure
- `lsfx-mock-server/services/file_service.py`: 在 `FileRecord` 中新增第二期命中计划字段,并让上传与拉取行内流水链路都能持久化第二期规则子集。
- `lsfx-mock-server/services/statement_rule_samples.py`: 按规则代码补齐第二期流水样本 builder保证每条规则只生成最小命中样本。
- `lsfx-mock-server/services/statement_service.py`: 读取第二期规则命中计划,装配第二期流水样本并保持缓存稳定性。
- `lsfx-mock-server/services/phase2_baseline_service.py`: 基于项目配置中的数据库连接信息,幂等写入第二期采购、资产与低收入事实基线。
- `lsfx-mock-server/tests/test_file_service.py`: 锁定第二期命中计划生成与持久化语义。
- `lsfx-mock-server/tests/test_statement_service.py`: 锁定第二期样本装配、互斥规则隔离与缓存稳定性。
- `lsfx-mock-server/tests/test_phase2_baseline_service.py`: 锁定第二期数据库基线写入的幂等性与命中前提。
- `lsfx-mock-server/tests/integration/test_full_workflow.py`: 验证 `getJZFileOrZjrcuFile -> getBSByLogId` 端到端链路在第二期下仍保持稳定。
- `sql/migration/2026-03-20-lsfx-mock-phase2-hit-baseline.sql`: 固化第二期采购、资产与低收入事实基线的最小 SQL。
- `docs/reports/implementation/2026-03-20-lsfx-mock-phase2-random-hit-backend-record.md`: 记录本次后端实施范围、规则分层与落地结果。
- `docs/tests/records/2026-03-20-lsfx-mock-phase2-random-hit-backend-verification.md`: 记录测试命令、数据库核验和端到端结果。
### Task 1: 在 FileService 中持久化第二期稳定随机命中计划
**Files:**
- Modify: `lsfx-mock-server/services/file_service.py`
- Modify: `lsfx-mock-server/tests/test_file_service.py`
- Reference: `docs/design/2026-03-20-lsfx-mock-phase2-random-hit-design.md`
- [ ] **Step 1: Write the failing test**
`lsfx-mock-server/tests/test_file_service.py` 中先补两条失败用例,锁定“同一 `logId` 第二期命中计划稳定”和“`fetch_inner_flow()` 会把第二期命中计划落到 `FileRecord`”:
```python
def test_build_rule_hit_plan_should_include_phase2_rule_sets():
service = FileService(staff_identity_repository=FakeStaffIdentityRepository())
plan1 = service._build_rule_hit_plan(20001)
plan2 = service._build_rule_hit_plan(20001)
assert plan1 == plan2
assert 2 <= len(plan1["phase2_statement_hit_rules"]) <= 4
assert 2 <= len(plan1["phase2_baseline_hit_rules"]) <= 4
def test_fetch_inner_flow_should_persist_phase2_rule_hit_plan(monkeypatch):
service = FileService(staff_identity_repository=FakeStaffIdentityRepository())
monkeypatch.setattr(
service,
"_build_rule_hit_plan",
lambda log_id: {
"large_transaction_hit_rules": [],
"phase1_hit_rules": [],
"phase2_statement_hit_rules": [
"MULTI_PARTY_GAMBLING_TRANSFER",
"SALARY_QUICK_TRANSFER",
],
"phase2_baseline_hit_rules": [
"SUPPLIER_CONCENTRATION",
"HOUSE_REGISTRATION_MISMATCH",
],
},
)
response = service.fetch_inner_flow(
{
"groupId": 1001,
"customerNo": "phase2_customer_001",
"dataChannelCode": "test_code",
"requestDateId": 20240101,
"dataStartDateId": 20240101,
"dataEndDateId": 20240131,
"uploadUserId": 902001,
}
)
log_id = response["data"][0]
record = service.file_records[log_id]
assert record.phase2_statement_hit_rules == [
"MULTI_PARTY_GAMBLING_TRANSFER",
"SALARY_QUICK_TRANSFER",
]
assert record.phase2_baseline_hit_rules == [
"SUPPLIER_CONCENTRATION",
"HOUSE_REGISTRATION_MISMATCH",
]
```
- [ ] **Step 2: Run test to verify it fails**
Run:
```bash
cd lsfx-mock-server
python3 -m pytest tests/test_file_service.py -k "phase2_rule_hit_plan" -v
```
Expected:
- `FAIL`
- 原因是 `FileRecord` 尚未保存第二期规则命中计划,`_build_rule_hit_plan()` 也未生成第二期规则子集
- [ ] **Step 3: Write minimal implementation**
`lsfx-mock-server/services/file_service.py` 中只做最小改动:
1.`FileRecord` 新增两个字段:
```python
phase2_statement_hit_rules: List[str] = field(default_factory=list)
phase2_baseline_hit_rules: List[str] = field(default_factory=list)
```
2. 定义第二期规则池,按设计分层:
```python
PHASE2_STATEMENT_RULE_CODES = [
"LOW_INCOME_RELATIVE_LARGE_TRANSACTION",
"MULTI_PARTY_GAMBLING_TRANSFER",
"MONTHLY_FIXED_INCOME",
"FIXED_COUNTERPARTY_TRANSFER",
"SALARY_QUICK_TRANSFER",
"SALARY_UNUSED",
]
PHASE2_BASELINE_RULE_CODES = [
"HOUSE_REGISTRATION_MISMATCH",
"PROPERTY_FEE_REGISTRATION_MISMATCH",
"TAX_ASSET_REGISTRATION_MISMATCH",
"SUPPLIER_CONCENTRATION",
]
```
3. 复用现有 `_pick_rule_subset()`,在 `_build_rule_hit_plan()` 中新增第二期两组子集。
4.`_create_file_record()``upload_file()``fetch_inner_flow()` 中都透传并保存第二期规则计划。
- [ ] **Step 4: Run test to verify it passes**
Run:
```bash
cd lsfx-mock-server
python3 -m pytest tests/test_file_service.py -k "phase2_rule_hit_plan" -v
```
Expected:
- `PASS`
- 第二期规则计划已按 `logId` 稳定随机生成并持久化
- [ ] **Step 5: Commit**
```bash
git add lsfx-mock-server/services/file_service.py lsfx-mock-server/tests/test_file_service.py
git commit -m "补充第二期Mock规则命中计划"
```
### Task 2: 新增第二期数据库基线服务并幂等补齐事实输入
**Files:**
- Create: `lsfx-mock-server/services/phase2_baseline_service.py`
- Create: `lsfx-mock-server/tests/test_phase2_baseline_service.py`
- Create: `sql/migration/2026-03-20-lsfx-mock-phase2-hit-baseline.sql`
- Reference: `lsfx-mock-server/config/settings.py`
- Reference: `docs/design/2026-03-20-lsfx-mock-phase2-random-hit-design.md`
- [ ] **Step 1: Write the failing test**
先在 `lsfx-mock-server/tests/test_phase2_baseline_service.py` 中补最小失败用例,锁定“抽中第二期基线规则时会生成幂等 SQL”与“互斥/无关规则不会写脏数据”:
```python
def test_apply_phase2_baselines_should_return_idempotent_sql_plan():
service = Phase2BaselineService()
sql_plan = service.build_sql_plan(
staff_id_card="330101198801010011",
family_id_cards=["330101199001010022"],
baseline_rule_codes=[
"SUPPLIER_CONCENTRATION",
"HOUSE_REGISTRATION_MISMATCH",
],
)
assert any("LSFXMOCKP2PUR" in sql for sql in sql_plan)
assert any("ccdi_asset_info" in sql for sql in sql_plan)
assert all("DELETE" in sql or "INSERT" in sql for sql in sql_plan)
def test_apply_phase2_baselines_should_skip_unselected_rules():
service = Phase2BaselineService()
sql_plan = service.build_sql_plan(
staff_id_card="330101198801010011",
family_id_cards=[],
baseline_rule_codes=["SUPPLIER_CONCENTRATION"],
)
assert any("ccdi_purchase_transaction" in sql for sql in sql_plan)
assert not any("ccdi_asset_info" in sql for sql in sql_plan)
```
- [ ] **Step 2: Run test to verify it fails**
Run:
```bash
cd lsfx-mock-server
python3 -m pytest tests/test_phase2_baseline_service.py -v
```
Expected:
- `FAIL`
- 原因是 `Phase2BaselineService` 与第二期幂等基线 SQL 规划尚不存在
- [ ] **Step 3: Write minimal implementation**
`lsfx-mock-server/services/phase2_baseline_service.py` 中新增最小服务:
1. 复用 `config/settings.py` 中的数据库连接配置。
2. 提供两个入口:
```python
def build_sql_plan(self, staff_id_card: str, family_id_cards: List[str], baseline_rule_codes: List[str]) -> List[str]:
...
def apply(self, staff_id_card: str, family_id_cards: List[str], baseline_rule_codes: List[str]) -> None:
...
```
3. 对四类第二期基线规则分别输出幂等 SQL
- `SUPPLIER_CONCENTRATION`:固定采购业务主键,先删后插;
- `HOUSE_REGISTRATION_MISMATCH` / `PROPERTY_FEE_REGISTRATION_MISMATCH` / `TAX_ASSET_REGISTRATION_MISMATCH`:按固定资产标识清理并重建“故意不匹配”资产事实;
4. 将稳定 SQL 同步沉淀到 `sql/migration/2026-03-20-lsfx-mock-phase2-hit-baseline.sql`,便于单独重放和排障。
- [ ] **Step 4: Run test to verify it passes**
Run:
```bash
cd lsfx-mock-server
python3 -m pytest tests/test_phase2_baseline_service.py -v
```
Expected:
- `PASS`
- 第二期数据库基线服务可生成幂等 SQL 计划
- [ ] **Step 5: Commit**
```bash
git add lsfx-mock-server/services/phase2_baseline_service.py lsfx-mock-server/tests/test_phase2_baseline_service.py sql/migration/2026-03-20-lsfx-mock-phase2-hit-baseline.sql
git commit -m "补充第二期Mock基线编排服务"
```
### Task 3: 按规则代码补齐第二期流水样本并接入 StatementService
**Files:**
- Modify: `lsfx-mock-server/services/statement_rule_samples.py`
- Modify: `lsfx-mock-server/services/statement_service.py`
- Modify: `lsfx-mock-server/tests/test_statement_service.py`
- [ ] **Step 1: Write the failing test**
`lsfx-mock-server/tests/test_statement_service.py` 中补两组失败用例,锁定“只装配被选中的第二期规则样本”和“互斥工资类规则不落在同一员工对象上”:
```python
def test_build_seed_statements_for_rule_plan_should_only_include_requested_phase2_rules():
plan = {
"large_transaction_hit_rules": [],
"phase1_hit_rules": [],
"phase2_statement_hit_rules": [
"MULTI_PARTY_GAMBLING_TRANSFER",
"SALARY_QUICK_TRANSFER",
],
"phase2_baseline_hit_rules": [],
}
statements = build_seed_statements_for_rule_plan(
group_id=1000,
log_id=30001,
rule_plan=plan,
)
assert any(item["userMemo"] == "工资入账" for item in statements)
assert any(item["customerName"] == "欢乐游戏科技有限公司" for item in statements)
assert not any(item["userMemo"] == "季度稳定兼职收入" for item in statements)
def test_salary_quick_transfer_and_salary_unused_should_use_different_identity_groups():
plan = {
"large_transaction_hit_rules": [],
"phase1_hit_rules": [],
"phase2_statement_hit_rules": [
"SALARY_QUICK_TRANSFER",
"SALARY_UNUSED",
],
"phase2_baseline_hit_rules": [],
}
statements = build_seed_statements_for_rule_plan(
group_id=1000,
log_id=30001,
rule_plan=plan,
)
salary_id_cards = {
item["cretNo"]
for item in statements
if item["userMemo"] == "工资入账"
}
assert len(salary_id_cards) >= 2
```
- [ ] **Step 2: Run test to verify it fails**
Run:
```bash
cd lsfx-mock-server
python3 -m pytest tests/test_statement_service.py -k "requested_phase2_rules or salary_quick_transfer_and_salary_unused" -v
```
Expected:
- `FAIL`
- 原因是当前样本模块还没有第二期 builder 与互斥规则隔离能力
- [ ] **Step 3: Write minimal implementation**
`lsfx-mock-server/services/statement_rule_samples.py` 中补齐第二期样本 builder 与映射:
```python
PHASE2_STATEMENT_RULE_BUILDERS = {
"LOW_INCOME_RELATIVE_LARGE_TRANSACTION": build_low_income_relative_large_transaction_samples,
"MULTI_PARTY_GAMBLING_TRANSFER": build_multi_party_gambling_transfer_samples,
"MONTHLY_FIXED_INCOME": build_monthly_fixed_income_samples,
"FIXED_COUNTERPARTY_TRANSFER": build_fixed_counterparty_transfer_samples,
"SALARY_QUICK_TRANSFER": build_salary_quick_transfer_samples,
"SALARY_UNUSED": build_salary_unused_samples,
}
```
实现要求:
- `MULTI_PARTY_GAMBLING_TRANSFER`:同一员工、同一天、多个对手方、区间金额;
- `MONTHLY_FIXED_INCOME`:连续 3 至 4 个月固定转入,排除工资代发;
- `FIXED_COUNTERPARTY_TRANSFER`:固定对手方、季度稳定区间金额;
- `SALARY_QUICK_TRANSFER`:工资入账后 24 小时内大额转出;
- `SALARY_UNUSED`:工资入账后 30 天无有效使用记录;
- `SALARY_QUICK_TRANSFER``SALARY_UNUSED` 必须使用不同 identity group。
同时在 `lsfx-mock-server/services/statement_service.py` 中:
1. 读取 `record.phase2_statement_hit_rules`
2. 把第二期样本装配进 `build_seed_statements_for_rule_plan(...)`
3. 保持总数 `FIXED_TOTAL_COUNT = 200`、ID 分配和缓存语义不变。
- [ ] **Step 4: Run test to verify it passes**
Run:
```bash
cd lsfx-mock-server
python3 -m pytest tests/test_statement_service.py -k "requested_phase2_rules or salary_quick_transfer_and_salary_unused" -v
```
Expected:
- `PASS`
- 第二期样本已按规则子集装配,工资类互斥规则已隔离
- [ ] **Step 5: Commit**
```bash
git add lsfx-mock-server/services/statement_rule_samples.py lsfx-mock-server/services/statement_service.py lsfx-mock-server/tests/test_statement_service.py
git commit -m "补充第二期Mock流水样本生成"
```
### Task 4: 在拉取链路中接通第二期基线写入并补集成回归
**Files:**
- Modify: `lsfx-mock-server/services/file_service.py`
- Modify: `lsfx-mock-server/services/statement_service.py`
- Modify: `lsfx-mock-server/tests/integration/test_full_workflow.py`
- Modify: `lsfx-mock-server/tests/test_file_service.py`
- Reference: `lsfx-mock-server/routers/api.py`
- [ ] **Step 1: Write the failing test**
`lsfx-mock-server/tests/integration/test_full_workflow.py` 中补失败用例,锁定“同一 `logId` 抽中第二期基线规则时,获取流水前已补齐对应数据库事实”:
```python
def test_inner_flow_should_apply_phase2_baselines_before_get_bank_statement(client, monkeypatch):
from routers.api import file_service
applied = {}
def fake_apply(**kwargs):
applied["called"] = True
applied["baseline_rule_codes"] = kwargs["baseline_rule_codes"]
monkeypatch.setattr(file_service.phase2_baseline_service, "apply", fake_apply)
monkeypatch.setattr(
file_service,
"_build_rule_hit_plan",
lambda log_id: {
"large_transaction_hit_rules": [],
"phase1_hit_rules": [],
"phase2_statement_hit_rules": ["MONTHLY_FIXED_INCOME"],
"phase2_baseline_hit_rules": ["SUPPLIER_CONCENTRATION"],
},
)
response = client.post(
"/watson/api/project/getJZFileOrZjrcuFile",
data={
"groupId": 1001,
"customerNo": "phase2_customer",
"dataChannelCode": "channel_code",
"requestDateId": 20240101,
"dataStartDateId": 20240101,
"dataEndDateId": 20240131,
"uploadUserId": 902001,
},
)
assert response.status_code == 200
assert applied["called"] is True
assert applied["baseline_rule_codes"] == ["SUPPLIER_CONCENTRATION"]
```
- [ ] **Step 2: Run test to verify it fails**
Run:
```bash
cd lsfx-mock-server
python3 -m pytest tests/integration/test_full_workflow.py -k "apply_phase2_baselines" -v
```
Expected:
- `FAIL`
- 原因是当前 `fetch_inner_flow()` 链路还没有接通第二期基线服务
- [ ] **Step 3: Write minimal implementation**
`lsfx-mock-server/services/file_service.py` 中:
1.`__init__()` 中注入 `phase2_baseline_service`
2.`fetch_inner_flow()``upload_file()` 创建 `FileRecord` 后、返回响应前,根据当前记录的 `phase2_baseline_hit_rules` 调用:
```python
self.phase2_baseline_service.apply(
staff_id_card=file_record.staff_id_card,
family_id_cards=file_record.family_id_cards,
baseline_rule_codes=file_record.phase2_baseline_hit_rules,
)
```
要求:
- 只对当前 `logId` 命中的第二期基线规则写入;
- 不因空规则集报错;
- 基线写入异常直接暴露,避免产生“流水有了但基线未写”的假成功状态。
- [ ] **Step 4: Run test to verify it passes**
Run:
```bash
cd lsfx-mock-server
python3 -m pytest tests/integration/test_full_workflow.py -k "apply_phase2_baselines" -v
```
Expected:
- `PASS`
- 第二期基线写入已在拉取链路中接通
- [ ] **Step 5: Commit**
```bash
git add lsfx-mock-server/services/file_service.py lsfx-mock-server/tests/integration/test_full_workflow.py
git commit -m "接通第二期Mock基线写入链路"
```
### Task 5: 完成第二期回归、数据库核验与实施记录
**Files:**
- Modify: `docs/reports/implementation/2026-03-20-lsfx-mock-phase2-random-hit-design-record.md`
- Create: `docs/reports/implementation/2026-03-20-lsfx-mock-phase2-random-hit-backend-record.md`
- Create: `docs/tests/records/2026-03-20-lsfx-mock-phase2-random-hit-backend-verification.md`
- Test: `lsfx-mock-server/tests/test_file_service.py`
- Test: `lsfx-mock-server/tests/test_statement_service.py`
- Test: `lsfx-mock-server/tests/test_phase2_baseline_service.py`
- Test: `lsfx-mock-server/tests/integration/test_full_workflow.py`
- [ ] **Step 1: Run focused and full regression**
Run:
```bash
cd lsfx-mock-server
python3 -m pytest tests/test_file_service.py -k "phase2_rule_hit_plan" -v
python3 -m pytest tests/test_phase2_baseline_service.py -v
python3 -m pytest tests/test_statement_service.py -k "phase2 or salary_quick_transfer_and_salary_unused" -v
python3 -m pytest tests/integration/test_full_workflow.py -k "phase2" -v
python3 -m pytest tests/test_file_service.py tests/test_statement_service.py tests/test_phase2_baseline_service.py tests/test_api.py tests/integration/test_full_workflow.py -v
```
Expected:
- 聚焦用例全部 `PASS`
- 全量回归 `PASS`
- [ ] **Step 2: Verify database baselines**
Run:
```bash
bin/mysql_utf8_exec.sh sql/migration/2026-03-20-lsfx-mock-phase2-hit-baseline.sql
```
再使用只读 SQL 核验:
```sql
SELECT purchase_id, supplier_name, actual_amount
FROM ccdi_purchase_transaction
WHERE purchase_id LIKE 'LSFXMOCKP2PUR%';
SELECT asset_name, person_id, asset_main_type, asset_status
FROM ccdi_asset_info
WHERE asset_name LIKE 'LSFX Mock P2%';
```
Expected:
- 第二期采购与资产基线存在
- 重跑 SQL 后结果仍稳定,无重复脏数据
- [ ] **Step 3: Write implementation record**
`docs/reports/implementation/2026-03-20-lsfx-mock-phase2-random-hit-backend-record.md` 中记录:
- 第二期规则如何拆分为流水样本与数据库基线两层
- `FileService``StatementService``Phase2BaselineService` 的职责边界
- 互斥工资规则的样本隔离策略
- 幂等 SQL 方案与数据库基线范围
- [ ] **Step 4: Write verification record**
`docs/tests/records/2026-03-20-lsfx-mock-phase2-random-hit-backend-verification.md` 中记录:
- pytest 执行命令与结果摘要
- SQL 执行与核验结果
- 端到端接口链路结果
- 结论与环境清理情况
- [ ] **Step 5: Commit**
```bash
git add docs/reports/implementation/2026-03-20-lsfx-mock-phase2-random-hit-design-record.md docs/reports/implementation/2026-03-20-lsfx-mock-phase2-random-hit-backend-record.md docs/tests/records/2026-03-20-lsfx-mock-phase2-random-hit-backend-verification.md
git add lsfx-mock-server/tests/test_file_service.py lsfx-mock-server/tests/test_statement_service.py lsfx-mock-server/tests/test_phase2_baseline_service.py lsfx-mock-server/tests/integration/test_full_workflow.py
git commit -m "完成第二期Mock随机命中回归验证"
```