多源知识融合与应用

本书围绕多源知识融合技术展开，系统地介绍了多源知识融合的基本概念、关键技 术、应用场景和发展趋势。书中涵盖当前主流的多模态数据处理技术，这些技术能够实 现跨模态信息检索，从而消除不同数据源之间的语义隔阂，促进知识的互通与共享。 本书首先详细讲解了文本、图像、信号和视频等不同模态数据的知识获取方法；然 后重点探讨了多模态数据的语义表示与检索，以及多模态知识图谱的融合方法；最后探 讨了多源知识融合技术在推荐系统、知识问答、辅助决策等前沿领域的应用及相关技术 挑战与未来展望，并通过实际案例展示了多源知识融合技术如何赋能领域应用。

第1 章 多源知识融合概述···························································.001
1.1 多源知识融合的应用背景·················································.001
1.2 多源知识融合的相关技术·················································.002
1.3 多源知识融合技术的应用·················································.004
1.4 知识融合技术的发展前景·················································.005
第2 章 文本数据的知识获取························································.007
2.1 文本知识抽取任务的定义·················································.007
2.1.1 实体识别的定义····················································.007
2.1.2 关系抽取的定义····················································.008
2.2 文本知识抽取的常用方法·················································.008
2.2.1 实体识别的常用方法··············································.008
2.2.2 关系抽取的常用方法··············································.013
2.3 常用数据集···································································.018
2.3.1 实体识别数据集····················································.018
2.3.2 关系抽取数据集····················································.019
2.4 技术挑战与未来展望·······················································.020
2.5 本章小结······································································.021
参考文献············································································.021
第3 章 图像数据的知识获取························································.025
3.1 图像知识表征方法··························································.025
3.1.1 传统的图像知识表征方法········································.026
3.1.2 基于深度神经网络的图像知识表征方法·······················.027
3.2 图像知识抽取任务··························································.029
3.2.1 目标检测·····························································.029
3.2.2 关键点检测··························································.033
3.2.3 图像分割·····························································.035
3.2.4 图像生成·····························································.037
3.3 常用数据集···································································.039
3.3.1 ImageNet 数据集····················································.039
3.3.2 COCO 数据集·······················································.039
3.3.3 Open Images 数据集················································.040
3.3.4 MSTAR 数据集·····················································.040
3.4 技术挑战与未来展望·······················································.040
3.5 本章小结······································································.041
参考文献············································································.041
第4 章 信号数据的知识获取························································.045
4.1 信号数据的定义及特点····················································.045
4.2 信号数据知识表征··························································.047
4.2.1 传统的信号数据知识表征方法···································.047
4.2.2 基于深度神经网络的信号数据知识表征方法·················.050
4.3 信号数据知识抽取任务····················································.054
4.3.1 信号分类·····························································.054
4.3.2 信号生成·····························································.056
4.3.3 长期预测·····························································.056
4.3.4 异常检测·····························································.057
4.3.5 语音识别·····························································.057
4.4 常用数据集···································································.059
4.4.1 UCI-HAR 数据集···················································.059
4.4.2 SEED 数据集························································.059
4.4.3 ETT 数据集··························································.060
4.4.4 SWaT 数据集························································.060
4.4.5 LibriSpeech 数据集·················································.060
4.4.6 MTAD 数据集·······················································.060
4.5 技术挑战与未来展望·······················································.060
4.6 本章小结······································································.061
参考文献············································································.061
第5 章 视频数据的知识获取························································.063
5.1 视频知识的含义·····························································.063
5.2 视频内容知识抽取任务····················································.064
5.2.1 时序动作分类·······················································.064
5.2.2 时序动作检测·······················································.066
5.3 高层语义分析任务··························································.068
5.3.1 视频摘要生成与事件高亮片段检测·····························.068
5.3.2 视频中的人物重识别与检索······································.071
5.3.3 视频中的人物关系识别···········································.072
5.3.4 视频中的人物微表情分析········································.074
5.3.5 视频中的片段检索·················································.075
5.4 常用数据集···································································.077
5.4.1 视频动作分类数据集··············································.077
5.4.2 时序动作检测数据集··············································.077
5.4.3 视频检索数据集····················································.078
5.5 技术挑战与未来展望·······················································.079
5.6 本章小结······································································.079
参考文献············································································.080
第6 章 多模态数据的语义表示与检索············································.082
6.1 跨模态检索任务的核心概念··············································.082
6.2 多模态数据的语义表示方法··············································.084
6.2.1 实值表示学习·······················································.084
6.2.2 二值表示学习·······················································.085
6.3 跨模态检索方法·····························································.086
6.3.1 基于传统方法的跨模态检索······································.086
6.3.2 基于深度学习的跨模态检索······································.089
6.4 跨模态检索的数据集·······················································.102
6.4.1 NUS-WIDE 数据集·················································.102
6.4.2 COCO 数据集·······················································.102
6.4.3 Flickr30k 数据集····················································.103
6.4.4 MUGE 数据集·······················································.103
6.4.5 WuDaoMM 数据集·················································.103
6.5 跨模态检索的评估标准····················································.104
6.5.1 mAP···································································.104
6.5.2 Precision-Recall ·····················································.104
6.5.3 Precision-TopK ······················································.105
6.5.4 Recall@K ····························································.106
6.6 跨模态检索任务的典型应用··············································.106
6.6.1 跨模态食谱检索····················································.106
6.6.2 跨模态人脸检索····················································.107
6.7 技术挑战与未来展望·······················································.108
6.8 本章小结······································································.108
参考文献············································································.109
第7 章 多模态知识图谱的融合·····················································.111
7.1 多源知识图谱融合的定义·················································.111
7.2 多模态知识图谱融合方法·················································.112
7.2.1 本体匹配·····························································.112
7.2.2 实体对齐·····························································.113
7.2.3 实体链接·····························································.115
7.2.4 真值发现·····························································.116
7.3 工具软件和评测数据集····················································.117
7.4 技术挑战与未来展望·······················································.118
7.5 本章小结······································································.119
参考文献············································································.119
第8 章 基于多模态知识的推荐系统···············································.122
8.1 推荐系统和多模态推荐····················································.122
8.1.1 推荐系统的任务定义··············································.122
8.1.2 多模态推荐的任务定义···········································.123
8.2 多模态推荐的特征提取····················································.124
8.3 基于矩阵分解的多模态推荐··············································.126
8.4 基于注意力网络的多模态推荐···········································.127
8.5 基于图神经网络的多模态推荐···········································.128
8.5.1 基于异质图融合的多模态推荐···································.129
8.5.2 基于同质图融合的多模态推荐···································.129
8.6 多模态推荐的模态融合····················································.131
8.6.1 早期融合·····························································.131
8.6.2 晚期融合·····························································.132
8.6.3 中间融合·····························································.132
8.7 多模态推荐的常用数据集·················································.132
8.8 技术挑战与未来展望·······················································.133
8.9 本章小结······································································.134
参考文献············································································.134
第9 章 知识问答系统·································································.137
9.1 基于流水线方法的知识问答系统········································.137
9.1.1 多源问题解析和查询生成········································.138
9.1.2 信息检索与答案生成··············································.141
9.2 基于端到端方法的知识问答系统········································.144
9.2.1 基于表示学习的方法··············································.145
9.2.2 基于深度学习的方法··············································.147
9.2.3 基于知识的回复改写方法········································.155
9.3 知识问答系统的应用·······················································.157
9.3.1 多轮对话系统的设计方案········································.157
9.3.2 多轮对话系统的架构··············································.159
9.3.3 多轮对话系统应用示例···········································.160
9.4 知识问答系统的常用数据集··············································.164
9.5 技术挑战与未来展望·······················································.165
9.6 本章小结······································································.166
参考文献············································································.166
第10 章 基于多源知识的辅助决策系统··········································.169
10.1 基于多源知识的推理与决策·············································.169
10.1.1 基于规则的推理···················································.169
10.1.2 基于表示学习的推理·············································.173
10.1.3 基于知识的时空数据挖掘计算·································.175
10.1.4 基于时空知识图谱的异常挖掘·································.177
10.2 辅助决策系统在军事领域的应用实例·································.180
10.2.1 态势分析与预警···················································.181
10.2.2 敌我事件发展趋势分析··········································.182
10.2.3 重点目标全维画像与意图分析·································.182
10.2.4 基于多源知识图谱的情报分析·································.185
10.3 技术挑战与未来展望······················································.187
10.4 本章小结····································································.188
参考文献············································································.18