ccdi/docs/plans/2026-03-09-bank-statement-duplicate-check.md

# 银行流水重复校验 Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** 为 `fetchAndSaveBankStatements(...)` 增加基于 `project_id + LE_ACCOUNT_NO + ACCOUNTING_DATE_ID + AMOUNT_DR + AMOUNT_CR` 的数据库级重复校验，确保重复流水跳过插入且保留原数据不变。

**Architecture:** 先清理测试库中的异常行和重复行，再为 `ccdi_bank_statement` 增加新唯一键。业务代码只负责对 `LE_ACCOUNT_NO` 做轻量标准化，并将批量插入改为 no-op upsert；真实数据库错误仍按现有异步文件处理链路向上抛出。

**Tech Stack:** Java 21, Spring Boot 3, MyBatis Plus, MySQL, JUnit 5, Mockito

---

### Task 1: 固化新去重键的标准化失败测试

**Files:**
- Modify: `ccdi-project/src/test/java/com/ruoyi/ccdi/project/service/impl/CcdiFileUploadServiceImplTest.java`
- Modify: `ccdi-project/src/main/java/com/ruoyi/ccdi/project/service/impl/CcdiFileUploadServiceImpl.java`

**Step 1: Write the failing test**

在现有 `CcdiFileUploadServiceImplTest` 中新增一个聚焦标准化的测试。通过 mock `lsfxClient.getBankStatement(...)` 返回一条带空白账号的流水，并捕获传给 `bankStatementMapper.insertBatch(...)` 的实体。

```java
@Test
void fetchAndSaveBankStatements_shouldTrimLeAccountNoBeforeInsert() {
    // arrange
    // leAccountNo = " 62220001 "
    // accountingDateId = 20260310
    // amountDr = new BigDecimal("100.00")
    // amountCr = new BigDecimal("0.00")
    // act
    // assert
    verify(bankStatementMapper).insertBatch(argThat(list ->
        "62220001".equals(list.get(0).getLeAccountNo())));
}
```

**Step 2: Run test to verify it fails**

Run:

```bash
mvn test -pl ccdi-project -Dtest=CcdiFileUploadServiceImplTest#fetchAndSaveBankStatements_shouldTrimLeAccountNoBeforeInsert
```

Expected:

- FAIL
- 失败原因体现当前实现尚未对 `LE_ACCOUNT_NO` 做 `trim`

**Step 3: Write the minimal implementation**

在 `CcdiFileUploadServiceImpl` 内增加最小辅助方法：

```java
private String trimAccountNo(String value) {
    return value == null ? null : value.trim();
}

private void normalizeDedupFields(CcdiBankStatement statement) {
    statement.setLeAccountNo(trimAccountNo(statement.getLeAccountNo()));
}
```

并在 `fromResponse(...)` 结果加入批次列表前调用。

**Step 4: Run test to verify it passes**

Run:

```bash
mvn test -pl ccdi-project -Dtest=CcdiFileUploadServiceImplTest#fetchAndSaveBankStatements_shouldTrimLeAccountNoBeforeInsert
```

Expected:

- PASS

**Step 5: Commit**

```bash
git add ccdi-project/src/test/java/com/ruoyi/ccdi/project/service/impl/CcdiFileUploadServiceImplTest.java ccdi-project/src/main/java/com/ruoyi/ccdi/project/service/impl/CcdiFileUploadServiceImpl.java
git commit -m "test(ccdi-project): cover bank statement account no normalization"
```

### Task 2: 用失败测试锁定“重复不失败”的服务层语义

**Files:**
- Modify: `ccdi-project/src/test/java/com/ruoyi/ccdi/project/service/impl/CcdiFileUploadServiceImplTest.java`
- Modify: `ccdi-project/src/main/java/com/ruoyi/ccdi/project/service/impl/CcdiFileUploadServiceImpl.java`

**Step 1: Write the failing test**

新增一个测试，模拟 Mapper 在重复导入场景下不抛异常，并验证上传记录最终不会因为重复而进入 `parsed_failed`。

```java
@Test
void processFileAsync_shouldNotFailWhenDuplicateStatementsAreSkipped() {
    // mock upload success / parse success
    // mock bankStatementMapper.insertBatch(...) 返回“未报错的重复跳过结果”
    // assert recordMapper 不会收到 parsed_failed
}
```

**Step 2: Run test to verify it fails**

Run:

```bash
mvn test -pl ccdi-project -Dtest=CcdiFileUploadServiceImplTest#processFileAsync_shouldNotFailWhenDuplicateStatementsAreSkipped
```

Expected:

- FAIL
- 失败点应体现当前实现还没有为重复跳过设计明确语义或日志

**Step 3: Write minimal implementation**

只做本任务所需的最小改动：

- 在 `fetchAndSaveBankStatements(...)` 中补充接口返回数、尝试写入数的日志
- 保持 duplicate 场景不抛异常
- 不调整其他与重复无关的异常路径

示例日志：

```java
log.info("【文件上传】流水入库完成: fetchedCount={}, attemptedCount={}, insertedCount={}, duplicateCount={}",
    fetchedCount, attemptedCount, insertedCount, duplicateCount);
```

**Step 4: Run the test to verify it passes**

Run:

```bash
mvn test -pl ccdi-project -Dtest=CcdiFileUploadServiceImplTest#processFileAsync_shouldNotFailWhenDuplicateStatementsAreSkipped
```

Expected:

- PASS

**Step 5: Commit**

```bash
git add ccdi-project/src/test/java/com/ruoyi/ccdi/project/service/impl/CcdiFileUploadServiceImplTest.java ccdi-project/src/main/java/com/ruoyi/ccdi/project/service/impl/CcdiFileUploadServiceImpl.java
git commit -m "test(ccdi-project): keep duplicate bank statements from failing uploads"
```

### Task 3: 为测试库编写数据清洗和唯一键迁移脚本

**Files:**
- Create: `assets/database/2026-03-10-bank-statement-dedup.sql`
- Modify: `assets/对接流水分析/ccdi_bank_statement.md`

**Step 1: Write the migration script**

创建测试库迁移脚本，包含以下 SQL：

```sql
DELETE FROM ccdi_bank_statement
WHERE project_id IS NULL
   OR LE_ACCOUNT_NO IS NULL
   OR ACCOUNTING_DATE_ID IS NULL;

UPDATE ccdi_bank_statement
SET LE_ACCOUNT_NO = TRIM(LE_ACCOUNT_NO);

DELETE t1
FROM ccdi_bank_statement t1
JOIN ccdi_bank_statement t2
  ON t1.bank_statement_id > t2.bank_statement_id
 AND t1.project_id = t2.project_id
 AND t1.LE_ACCOUNT_NO = t2.LE_ACCOUNT_NO
 AND t1.ACCOUNTING_DATE_ID = t2.ACCOUNTING_DATE_ID
 AND t1.AMOUNT_DR = t2.AMOUNT_DR
 AND t1.AMOUNT_CR = t2.AMOUNT_CR;

ALTER TABLE ccdi_bank_statement
    MODIFY COLUMN project_id bigint(20) NOT NULL COMMENT '关联项目ID',
    MODIFY COLUMN LE_ACCOUNT_NO varchar(240) NOT NULL DEFAULT '' COMMENT '企业银行账号',
    MODIFY COLUMN ACCOUNTING_DATE_ID int(11) NOT NULL COMMENT '账号日期ID';

ALTER TABLE ccdi_bank_statement
    ADD UNIQUE KEY uk_bank_statement_dedup (
        project_id,
        LE_ACCOUNT_NO,
        ACCOUNTING_DATE_ID,
        AMOUNT_DR,
        AMOUNT_CR
    );
```

**Step 2: Manually run the migration in the local test database**

Run:

```bash
mysql -u <user> -p <database> < assets/database/2026-03-10-bank-statement-dedup.sql
```

Expected:

- SQL 全部执行成功
- `SHOW INDEX FROM ccdi_bank_statement;` 能看到 `uk_bank_statement_dedup`

**Step 3: Update the bank statement schema note**

同步更新 `assets/对接流水分析/ccdi_bank_statement.md`，补齐：

- `project_id`
- 新唯一键说明
- `project_id` / `LE_ACCOUNT_NO` / `ACCOUNTING_DATE_ID` 的非空语义

**Step 4: Verify migration result**

Run:

```bash
mysql -u <user> -p -e "SHOW CREATE TABLE ccdi_bank_statement\G"
```

Expected:

- 表结构包含新的唯一键
- `project_id`、`LE_ACCOUNT_NO`、`ACCOUNTING_DATE_ID` 为 `NOT NULL`

**Step 5: Commit**

```bash
git add assets/database/2026-03-10-bank-statement-dedup.sql assets/对接流水分析/ccdi_bank_statement.md
git commit -m "feat(database): add bank statement dedup unique key"
```

### Task 4: 将 Mapper 批量插入改为 no-op upsert

**Files:**
- Modify: `ccdi-project/src/main/java/com/ruoyi/ccdi/project/mapper/CcdiBankStatementMapper.java`
- Modify: `ccdi-project/src/main/resources/mapper/ccdi/project/CcdiBankStatementMapper.xml`

**Step 1: Write the failing verification case**

先在本地 MySQL 中准备两次相同业务键的数据，第二次执行当前批量插入 SQL，确认现状会抛唯一键冲突。

Run:

```bash
mvn test -pl ccdi-project -Dtest=CcdiFileUploadServiceImplTest#processFileAsync_shouldNotFailWhenDuplicateStatementsAreSkipped
```

Expected:

- FAIL 或本地重复场景仍未被正确跳过

**Step 2: Change the insert SQL**

将 XML 中的批量插入改为：

```xml
<insert id="insertBatch" parameterType="java.util.List">
    insert into ccdi_bank_statement (...)
    values
    <foreach collection="list" item="item" separator=",">
        (...)
    </foreach>
    on duplicate key update
        bank_statement_id = bank_statement_id
</insert>
```

**Step 3: Keep mapper signature unchanged**

保持 `CcdiBankStatementMapper.insertBatch(@Param("list") List<CcdiBankStatement> list)` 不变，避免扩大调用面。

**Step 4: Run the targeted test suite**

Run:

```bash
mvn test -pl ccdi-project -Dtest=CcdiFileUploadServiceImplTest
```

Expected:

- PASS

**Step 5: Manual duplicate verification**

在本地测试库重复导入同一批数据后执行：

```bash
mysql -u <user> -p -e "SELECT COUNT(*) FROM ccdi_bank_statement WHERE project_id = <projectId> AND LE_ACCOUNT_NO = '62220001' AND ACCOUNTING_DATE_ID = 20260310 AND AMOUNT_DR = 100.00 AND AMOUNT_CR = 0.00;"
```

Expected:

- 结果始终为 `1`

**Step 6: Commit**

```bash
git add ccdi-project/src/main/java/com/ruoyi/ccdi/project/mapper/CcdiBankStatementMapper.java ccdi-project/src/main/resources/mapper/ccdi/project/CcdiBankStatementMapper.xml
git commit -m "feat(ccdi-project): skip duplicate bank statements on insert"
```

### Task 5: 校准日志计数并验证 MySQL 受影响行数语义

**Files:**
- Modify: `ccdi-project/src/main/java/com/ruoyi/ccdi/project/service/impl/CcdiFileUploadServiceImpl.java`
- Modify: `ccdi-project/src/test/java/com/ruoyi/ccdi/project/service/impl/CcdiFileUploadServiceImplTest.java`

**Step 1: Add a temporary probe in the service**

在开发阶段先打印：

```java
log.info("【文件上传】dedup probe: batchSize={}, mapperAffectedRows={}", batchList.size(), affectedRows);
```

**Step 2: Run a duplicate import manually**

使用相同测试文件连续导入两次，并观察第二次日志。

Expected:

- 能确认 duplicate 分支下 MyBatis/JDBC 返回的 `affectedRows` 语义

**Step 3: Finalize the counting logic**

如果实测 duplicate 返回 `0`，则直接落正式逻辑：

```java
insertedCount += affectedRows;
duplicateCount += batchSize - affectedRows;
```

如果实测不稳定，则不要伪造精确计数，改为保守日志：

```java
log.info("【文件上传】流水入库完成: fetchedCount={}, attemptedCount={}", fetchedCount, attemptedCount);
```

**Step 4: Run the test suite**

Run:

```bash
mvn test -pl ccdi-project -Dtest=CcdiFileUploadServiceImplTest
```

Expected:

- PASS

**Step 5: Remove temporary probe and commit**

```bash
git add ccdi-project/src/main/java/com/ruoyi/ccdi/project/service/impl/CcdiFileUploadServiceImpl.java ccdi-project/src/test/java/com/ruoyi/ccdi/project/service/impl/CcdiFileUploadServiceImplTest.java
git commit -m "refactor(ccdi-project): finalize bank statement dedup logging"
```

### Task 6: 做最终回归验证并整理交付

**Files:**
- Review: `ccdi-project/src/main/java/com/ruoyi/ccdi/project/service/impl/CcdiFileUploadServiceImpl.java`
- Review: `ccdi-project/src/main/resources/mapper/ccdi/project/CcdiBankStatementMapper.xml`
- Review: `assets/database/2026-03-10-bank-statement-dedup.sql`
- Review: `assets/对接流水分析/ccdi_bank_statement.md`

**Step 1: Run automated tests**

Run:

```bash
mvn test -pl ccdi-project -Dtest=CcdiFileUploadServiceImplTest
```

Expected:

- PASS

**Step 2: Run a focused compile**

Run:

```bash
mvn clean compile -pl ccdi-project -am
```

Expected:

- BUILD SUCCESS

**Step 3: Execute manual acceptance checks**

验证以下结果：

- 首次导入：数据写入成功
- 第二次导入同一数据：原记录不变，数量不增加
- 非重复数据再次导入：仅新增新记录
- 制造非唯一键类 SQL 错误：上传记录进入 `parsed_failed`

**Step 4: Prepare the delivery summary**

总结：

- 唯一键是否已生效
- 重复导入是否跳过
- 原数据是否保持不变
- 日志计数是否为精确值还是保守值

**Step 5: Commit**

```bash
git status --short
git add <verified files>
git commit -m "docs: finalize bank statement dedup verification notes"
```