调整项目人数按流水证件号统计
This commit is contained in:
@@ -117,37 +117,12 @@ PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN"
|
||||
</resultMap>
|
||||
|
||||
<select id="countMatchedStaffCountByProjectId" resultType="java.lang.Integer">
|
||||
select count(distinct scope_staff.id_card)
|
||||
from (
|
||||
select staff.id_card
|
||||
from ccdi_bank_statement bs
|
||||
inner join ccdi_base_staff staff on staff.id_card = trim(bs.cret_no)
|
||||
where bs.project_id = #{projectId}
|
||||
and bs.cret_no is not null
|
||||
and trim(bs.cret_no) != ''
|
||||
union
|
||||
select family_staff.id_card
|
||||
from ccdi_bank_statement bs
|
||||
inner join ccdi_staff_fmy_relation relation
|
||||
on relation.relation_cert_no = trim(bs.cret_no)
|
||||
and relation.status = 1
|
||||
inner join ccdi_base_staff family_staff
|
||||
on family_staff.id_card = relation.person_id
|
||||
where bs.project_id = #{projectId}
|
||||
and bs.cret_no is not null
|
||||
and trim(bs.cret_no) != ''
|
||||
union
|
||||
select account_staff.id_card
|
||||
from ccdi_bank_statement bs
|
||||
inner join ccdi_account_info account
|
||||
on trim(account.account_no) = trim(bs.LE_ACCOUNT_NO)
|
||||
and account.owner_type = 'EMPLOYEE'
|
||||
inner join ccdi_base_staff account_staff
|
||||
on account_staff.id_card = account.owner_id
|
||||
where bs.project_id = #{projectId}
|
||||
and bs.LE_ACCOUNT_NO is not null
|
||||
and trim(bs.LE_ACCOUNT_NO) != ''
|
||||
) scope_staff
|
||||
select count(distinct trim(bs.cret_no))
|
||||
from ccdi_bank_statement bs
|
||||
inner join ccdi_base_staff staff on staff.id_card = trim(bs.cret_no)
|
||||
where bs.project_id = #{projectId}
|
||||
and bs.cret_no is not null
|
||||
and trim(bs.cret_no) != ''
|
||||
</select>
|
||||
|
||||
<sql id="parsedTrxDateExpr">
|
||||
|
||||
@@ -221,18 +221,16 @@ class CcdiBankStatementMapperXmlTest {
|
||||
}
|
||||
|
||||
@Test
|
||||
void targetCount_shouldUseResolvedProjectEmployeeScope() throws Exception {
|
||||
void targetCount_shouldOnlyUseStatementCretNoMatchedStaff() throws Exception {
|
||||
try (InputStream inputStream = getClass().getClassLoader().getResourceAsStream(RESOURCE)) {
|
||||
String xml = new String(inputStream.readAllBytes(), StandardCharsets.UTF_8);
|
||||
String selectSql = extractSelect(xml, "countMatchedStaffCountByProjectId");
|
||||
|
||||
assertTrue(selectSql.contains("select count(distinct scope_staff.id_card)"), selectSql);
|
||||
assertTrue(selectSql.contains("select count(distinct trim(bs.cret_no))"), selectSql);
|
||||
assertTrue(selectSql.contains("inner join ccdi_base_staff staff on staff.id_card = trim(bs.cret_no)"), selectSql);
|
||||
assertTrue(selectSql.contains("inner join ccdi_staff_fmy_relation relation"), selectSql);
|
||||
assertTrue(selectSql.contains("family_staff.id_card = relation.person_id"), selectSql);
|
||||
assertTrue(selectSql.contains("inner join ccdi_account_info account"), selectSql);
|
||||
assertTrue(selectSql.contains("account.owner_type = 'EMPLOYEE'"), selectSql);
|
||||
assertTrue(selectSql.contains("account_staff.id_card = account.owner_id"), selectSql);
|
||||
assertFalse(selectSql.contains("ccdi_staff_fmy_relation"), selectSql);
|
||||
assertFalse(selectSql.contains("ccdi_account_info"), selectSql);
|
||||
assertFalse(selectSql.contains("LE_ACCOUNT_NO"), selectSql);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,221 @@
|
||||
# 生产流水上传后卡片人数异常排查方案
|
||||
|
||||
## 背景
|
||||
|
||||
- 问题现象:生产环境上传了一个人的流水后,结果总览卡片仍显示两个人。
|
||||
- 排查目标:确认卡片显示的“两个人”来自项目总人数、风险人员数,还是风险模型卡片命中人数,并定位数据来源是否符合当前项目真实流水范围。
|
||||
- 本方案只做排查,不直接修改生产数据。
|
||||
|
||||
## 已确认的数据链路
|
||||
|
||||
1. 前端结果总览页面 `PreliminaryCheck.vue` 会并行请求:
|
||||
- `/ccdi/project/overview/dashboard`
|
||||
- `/ccdi/project/overview/risk-people`
|
||||
- `/ccdi/project/overview/risk-models/cards`
|
||||
- `/ccdi/project/overview/suspicious-transactions`
|
||||
- `/ccdi/project/overview/employee-credit-negative`
|
||||
|
||||
2. 顶部统计卡片 `总人数 / 高风险 / 中风险 / 低风险 / 无风险人员` 来自 `ccdi_project`:
|
||||
- `target_count`
|
||||
- `high_risk_count`
|
||||
- `medium_risk_count`
|
||||
- `low_risk_count`
|
||||
|
||||
3. 风险模型卡片的人数来自 `ccdi_project_overview_employee_result`:
|
||||
- 按 `staff_id_card` 去重统计模型命中人员数。
|
||||
|
||||
4. 上传流水成功后,系统会:
|
||||
- 将流水写入 `ccdi_bank_statement`
|
||||
- 根据项目内已入库流水重新计算 `ccdi_project.target_count`
|
||||
- 批处理完成后触发流水标签重算
|
||||
- 重算完成后刷新 `ccdi_project_overview_employee_result` 和风险人数统计
|
||||
|
||||
5. 经业务确认,项目总人数口径应为“当前项目内所有已入库流水的 `cret_no` 直接匹配员工主数据后的去重人数”。不纳入家属关系归属员工,也不纳入本方账号账户库归属员工。
|
||||
|
||||
## 排查步骤
|
||||
|
||||
### 1. 确认异常卡片类型
|
||||
|
||||
先在浏览器 Network 中打开结果总览页面,记录以下接口返回:
|
||||
|
||||
```text
|
||||
GET /ccdi/project/overview/dashboard?projectId={projectId}
|
||||
GET /ccdi/project/overview/risk-models/cards?projectId={projectId}
|
||||
GET /ccdi/project/overview/risk-people?projectId={projectId}&pageNum=1&pageSize=5
|
||||
```
|
||||
|
||||
判断口径:
|
||||
|
||||
- 如果 `dashboard.stats` 里的 `people` 为 2,问题在 `ccdi_project.target_count` 或项目流水范围。
|
||||
- 如果某个 `risk-models/cards.cardList[].peopleCount` 为 2,问题在 `ccdi_project_overview_employee_result` 或标签重算结果。
|
||||
- 如果 `risk-people.rows` 有 2 条,问题在风险人员汇总结果。
|
||||
|
||||
### 2. 核对项目基础统计字段
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
project_id,
|
||||
project_name,
|
||||
status,
|
||||
target_count,
|
||||
high_risk_count,
|
||||
medium_risk_count,
|
||||
low_risk_count,
|
||||
update_time
|
||||
FROM ccdi_project
|
||||
WHERE project_id = {projectId}
|
||||
AND del_flag = '0';
|
||||
```
|
||||
|
||||
结论判断:
|
||||
|
||||
- `target_count = 2`:继续查项目内 `cret_no` 实际匹配到的员工范围。
|
||||
- `target_count = 1` 但页面显示 2:继续查接口缓存、浏览器缓存或前端请求的 `projectId` 是否错误。
|
||||
|
||||
### 3. 核对上传记录是否只剩一个人的文件
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
id,
|
||||
project_id,
|
||||
log_id,
|
||||
file_name,
|
||||
file_status,
|
||||
enterprise_names,
|
||||
account_nos,
|
||||
error_message,
|
||||
upload_time,
|
||||
upload_user
|
||||
FROM ccdi_file_upload_record
|
||||
WHERE project_id = {projectId}
|
||||
ORDER BY upload_time DESC, id DESC;
|
||||
```
|
||||
|
||||
重点看:
|
||||
|
||||
- 是否存在历史上传成功文件仍在该项目下。
|
||||
- 是否存在本次上传文件之外的 `parsed_success` 记录。
|
||||
- `enterprise_names`、`account_nos` 是否出现多个主体或多个本方账号。
|
||||
|
||||
### 4. 核对项目内流水 cret_no 实际匹配了几个人
|
||||
|
||||
先看项目流水批次和本方主体:
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
batch_id,
|
||||
COUNT(*) AS statement_count,
|
||||
GROUP_CONCAT(DISTINCT TRIM(LE_ACCOUNT_NAME) ORDER BY TRIM(LE_ACCOUNT_NAME) SEPARATOR '、') AS le_names,
|
||||
GROUP_CONCAT(DISTINCT TRIM(LE_ACCOUNT_NO) ORDER BY TRIM(LE_ACCOUNT_NO) SEPARATOR '、') AS le_accounts,
|
||||
GROUP_CONCAT(DISTINCT TRIM(cret_no) ORDER BY TRIM(cret_no) SEPARATOR '、') AS cret_nos
|
||||
FROM ccdi_bank_statement
|
||||
WHERE project_id = {projectId}
|
||||
GROUP BY batch_id
|
||||
ORDER BY batch_id DESC;
|
||||
```
|
||||
|
||||
再按总人数口径展开员工范围:
|
||||
|
||||
```sql
|
||||
SELECT DISTINCT
|
||||
TRIM(bs.cret_no) AS id_card,
|
||||
staff.name,
|
||||
staff.staff_id
|
||||
FROM ccdi_bank_statement bs
|
||||
INNER JOIN ccdi_base_staff staff
|
||||
ON staff.id_card = TRIM(bs.cret_no)
|
||||
WHERE bs.project_id = {projectId}
|
||||
AND bs.cret_no IS NOT NULL
|
||||
AND TRIM(bs.cret_no) != ''
|
||||
ORDER BY id_card;
|
||||
```
|
||||
|
||||
结论判断:
|
||||
|
||||
- 如果这里确实查出 2 个 `id_card`,页面显示 2 符合当前口径,继续确认多出的人员来自哪个批次或哪条流水的 `cret_no`。
|
||||
- 如果这里只查出 1 个 `id_card`,但 `ccdi_project.target_count = 2`,说明项目统计字段未刷新或刷新失败。
|
||||
|
||||
### 5. 核对风险人员汇总是否仍有两个人
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
project_id,
|
||||
staff_id_card,
|
||||
staff_name,
|
||||
risk_level_code,
|
||||
rule_count,
|
||||
model_count,
|
||||
hit_count,
|
||||
model_codes_csv,
|
||||
update_time
|
||||
FROM ccdi_project_overview_employee_result
|
||||
WHERE project_id = {projectId}
|
||||
ORDER BY staff_id_card;
|
||||
```
|
||||
|
||||
如果这里有 2 条,再查标签原始结果:
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
tr.project_id,
|
||||
tr.object_type,
|
||||
tr.object_key,
|
||||
tr.bank_statement_id,
|
||||
tr.model_code,
|
||||
tr.rule_code,
|
||||
bs.batch_id,
|
||||
bs.cret_no,
|
||||
bs.LE_ACCOUNT_NO,
|
||||
bs.LE_ACCOUNT_NAME
|
||||
FROM ccdi_bank_statement_tag_result tr
|
||||
LEFT JOIN ccdi_bank_statement bs
|
||||
ON bs.bank_statement_id = tr.bank_statement_id
|
||||
WHERE tr.project_id = {projectId}
|
||||
ORDER BY tr.id DESC
|
||||
LIMIT 200;
|
||||
```
|
||||
|
||||
结论判断:
|
||||
|
||||
- 汇总表有 2 人且标签结果能追溯到 2 人:需要确认多出人员是否来自历史流水、家属归属或账号归属。
|
||||
- 汇总表有 2 人但标签结果已不是 2 人:说明 `ccdi_project_overview_employee_result` 未随最新重算刷新。
|
||||
|
||||
### 6. 核对打标任务是否完成
|
||||
|
||||
从日志中按 `projectId` 搜索:
|
||||
|
||||
```text
|
||||
【流水标签】自动重算任务已异步提交
|
||||
【流水标签】任务执行成功
|
||||
【流水标签】任务执行失败
|
||||
【流水标签】任务执行结束
|
||||
【文件上传】批处理完成,准备触发自动重算
|
||||
```
|
||||
|
||||
结论判断:
|
||||
|
||||
- 上传成功但没有自动重算提交日志:排查上传批处理完成回调。
|
||||
- 有提交但任务失败:优先处理失败原因。
|
||||
- 任务成功但汇总未更新:排查 `refreshOverviewEmployeeResults` 是否执行完成。
|
||||
|
||||
## 最可能原因排序
|
||||
|
||||
1. 项目下仍存在历史成功上传流水,本次只上传了一个人的流水,但项目统计按项目全量流水的 `cret_no` 计算,所以仍显示两个人。
|
||||
2. 本次文件中只有一个主体,但流水明细里存在两个不同的有效 `cret_no`。
|
||||
3. 上传后 `target_count` 已变为 1,但风险汇总表 `ccdi_project_overview_employee_result` 仍是旧数据,导致风险模型卡片或风险人员列表显示 2。
|
||||
4. 自动打标重算失败或尚未完成,页面刷新时读到了重算前的风险统计。
|
||||
5. 前端打开的不是预期项目,或结果总览接口请求的 `projectId` 与上传页面项目不一致。
|
||||
|
||||
## 排查结束判定
|
||||
|
||||
满足以下任一条件即可闭环:
|
||||
|
||||
- 查明第二个人来自项目内哪条流水的 `cret_no`,并能定位到具体 `batch_id`、`log_id`、文件名或身份证号。
|
||||
- 查明统计字段或汇总表未刷新,并能定位到上传后刷新目标人数或自动打标重算的失败日志。
|
||||
- 查明页面请求了错误项目,接口返回和数据库统计一致。
|
||||
|
||||
## 后续处理原则
|
||||
|
||||
- 如果第二个人来自历史上传文件,需要先确认业务上是否应删除该文件;删除必须走页面或正式删除接口,让系统同步清理流水并触发重算。
|
||||
- 如果第二个人只来自账号库或家属关系,而流水 `cret_no` 未直接匹配该员工,则不应计入项目总人数。
|
||||
- 如果是统计刷新或重算失败,先保留现场日志和相关 SQL 查询结果,再进入缺陷修复流程。
|
||||
@@ -0,0 +1,79 @@
|
||||
# 项目人数按流水 cret_no 统计实施记录
|
||||
|
||||
## 背景
|
||||
|
||||
- 生产反馈:同一项目上传一个人的流水后,结果总览卡片仍显示两个人。
|
||||
- 业务确认:项目里的人数只按照流水中的 `cret_no` 计算。
|
||||
- 本次只调整 `ccdi_project.target_count` 的统计口径,不调整风险人员、风险模型命中人员的归属与展示逻辑。
|
||||
|
||||
## 修改内容
|
||||
|
||||
1. 调整运行时统计 SQL:
|
||||
- 文件:`ccdi-project/src/main/resources/mapper/ccdi/project/CcdiBankStatementMapper.xml`
|
||||
- 方法:`countMatchedStaffCountByProjectId`
|
||||
- 新口径:仅统计 `ccdi_bank_statement.cret_no` 直接匹配 `ccdi_base_staff.id_card` 后的去重人数。
|
||||
- 移除口径:不再纳入家属关系归属员工,不再纳入本方账号账户库归属员工。
|
||||
|
||||
2. 新增生产回填脚本:
|
||||
- 文件:`sql/migration/2026-06-02-fix-project-target-count-cret-no-only.sql`
|
||||
- 作用:按新口径回写既有项目的 `ccdi_project.target_count`。
|
||||
|
||||
3. 补充测试约束:
|
||||
- `CcdiBankStatementMapperXmlTest.targetCount_shouldOnlyUseStatementCretNoMatchedStaff`
|
||||
|
||||
## 影响范围
|
||||
|
||||
- 结果总览顶部“总人数”卡片。
|
||||
- 项目列表中的目标人数。
|
||||
- 上传成功、删除上传记录、历史导入、标签重算后写回的 `target_count`。
|
||||
- 生产既有项目需要执行新增迁移脚本,否则已存在的 `target_count` 不会自动立即回写。
|
||||
|
||||
## 不变范围
|
||||
|
||||
- 风险人员列表仍按打标结果归属员工展示。
|
||||
- 风险模型卡片 `peopleCount` 仍来自 `ccdi_project_overview_employee_result`。
|
||||
- 涉疑交易、负面征信、异常账户等结果总览子模块不在本次修改范围。
|
||||
|
||||
## 验证情况
|
||||
|
||||
- 已尝试运行:
|
||||
- `mvn -pl ccdi-project -Dtest=CcdiBankStatementMapperXmlTest#targetCount_shouldOnlyUseStatementCretNoMatchedStaff test`
|
||||
- 结果:测试编译阶段被既有无关错误阻断,目标测试未执行到断言。
|
||||
- 阻断点包括:
|
||||
- `CcdiBankStatementTest` 引用的 `BankStatementItem#setCustomerCertNo`、`setCustomerSocialCreditCode` 不存在。
|
||||
- `CcdiFileUploadServiceImplTest` 中多处仍按旧的 `LsfxAnalysisClient.uploadFile(Integer, Object, String)` 签名编写。
|
||||
- 已通过静态校验确认:
|
||||
- 运行时 `countMatchedStaffCountByProjectId` 只统计 `ccdi_bank_statement.cret_no`。
|
||||
- 新增生产回填脚本不包含 `ccdi_staff_fmy_relation`、`ccdi_account_info`、`LE_ACCOUNT_NO`。
|
||||
- 已通过源码编译:
|
||||
- `mvn -pl ccdi-project -DskipTests compile`
|
||||
|
||||
## 生产执行提醒
|
||||
|
||||
- 执行包含中文注释的 SQL 脚本时,按项目规则使用:
|
||||
|
||||
```bash
|
||||
bin/mysql_utf8_exec.sh sql/migration/2026-06-02-fix-project-target-count-cret-no-only.sql
|
||||
```
|
||||
|
||||
- 执行后可用以下 SQL 核对单个项目:
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
project.project_id,
|
||||
project.target_count,
|
||||
stats.target_count AS expected_target_count
|
||||
FROM ccdi_project project
|
||||
LEFT JOIN (
|
||||
SELECT
|
||||
bs.project_id,
|
||||
COUNT(DISTINCT TRIM(bs.cret_no)) AS target_count
|
||||
FROM ccdi_bank_statement bs
|
||||
INNER JOIN ccdi_base_staff staff
|
||||
ON staff.id_card = TRIM(bs.cret_no)
|
||||
WHERE bs.cret_no IS NOT NULL
|
||||
AND TRIM(bs.cret_no) != ''
|
||||
GROUP BY bs.project_id
|
||||
) stats ON stats.project_id = project.project_id
|
||||
WHERE project.project_id = {projectId};
|
||||
```
|
||||
@@ -0,0 +1,17 @@
|
||||
-- 修复项目总人数统计口径:仅按项目流水 cret_no 直接匹配员工主数据后的去重人数回写。
|
||||
UPDATE ccdi_project project
|
||||
LEFT JOIN (
|
||||
SELECT
|
||||
bs.project_id,
|
||||
COUNT(DISTINCT TRIM(bs.cret_no)) AS target_count
|
||||
FROM ccdi_bank_statement bs
|
||||
INNER JOIN ccdi_base_staff staff
|
||||
ON staff.id_card = TRIM(bs.cret_no)
|
||||
WHERE bs.cret_no IS NOT NULL
|
||||
AND TRIM(bs.cret_no) != ''
|
||||
GROUP BY bs.project_id
|
||||
) stats ON stats.project_id = project.project_id
|
||||
SET project.target_count = COALESCE(stats.target_count, 0),
|
||||
project.update_by = 'system',
|
||||
project.update_time = NOW()
|
||||
WHERE project.del_flag = '0';
|
||||
Reference in New Issue
Block a user