人工智能数据素养

数据作为一种新型生产要素，在未来的社会发展过程中将扮演越来越重要的角色，提升数据素养将有助于促进中国人工智能后备人才的高质量发展。本书以人工智能下的大数据时代为背景，从数据素养、数据分析基础、统计分析、机器学习多个维度全面系统地介绍了如何探索数据、整理数据并分析数据。本书没有给出晦涩难懂的数学公式，也不涉及复杂烦琐的程序代码，而是在阐述基本原理的基础上，辅以简洁的Python 程序，让读者能够快速入门，提升个人的数据综合素养。

"孙越，上海外国语大学附属龙岗学校校长，长期从事教育信息化工作和智慧校园建设。目前担任中国发明协会中小学创造教育分会常务理事，深圳教育学会教育信息化和人工智能专委会副理事长，香港中文大学（深圳）特聘导师。同时获评上海市“普教系统名校长名师培养工程——攻关计划”人才、上海市教育科研专家库专家、深圳市龙岗区很好校长、深圳市龙岗区“名师工作室”主持人等。
龚超，日本工学博士，清华大学日本研究中心主任助理，深圳清华大学研究院下一代互联网研发中心核心成员，未来基因（北京）人工智能研究院首席专家，教育部教育信息化教学应用实践共同体项目特聘专家。中国高科技产业化研究会理事，中国人工智能学会中小学工作委员会委员，中国自动化学会普及工作委员会委员。研究方向为人工智能优化算法，人工智能在数字化转型中的应用等。多家500强企业数字化转型领域高级顾问，在国内外期刊上发表文章共计60余篇。
袁中果，中国人民大学附属中学信息技术教研组组长，中国人工智能学会中小学工作委员会秘书长，中国自动化学会普及工作委员会副主任委员，北京市特级教师，中国人民大学博士，海淀区名师工作站导师，海淀区优秀种子教师工作站实践导师，海淀区督学。"

第1 章 人工智能下的大数据时代 ...................................................................................... 001
1.1 大数据时代和人工智能 ····································································.001
1.1.1 一切皆为数据 ······································································.001
1.1.2 数据高速增长时代 ································································.002
1.1.3 利用人工智能掘金大数据························································.003
1.2 人工智能三要素 ·············································································.004
1.2.1 数据――AI 之源 ···································································.005
1.2.2 算法――AI 之核 ···································································.006
1.2.3 算力――AI 之驱 ···································································.007
1.3 数据素养 ······················································································.007
1.3.1 何为数据素养 ······································································.007
1.3.2 数据素养为何重要 ································································.010
1.3.3 如何提升数据素养 ································································.011
1.4 本章小结 ······················································································.012
第2 章 Python 数据分析基础 ............................................................................................. 013
2.1 Python 基础 ··················································································.013
2.1.1 Python 简介 ·········································································.013
2.1.2 Python 数据类型 ···································································.017
2.1.3 常用的操作、函数和方法························································.021
2.1.4 列表、元组、字典 ································································.024
2.1.5 顺序结构 ············································································.027
2.1.6 分支结构 ············································································.027
2.1.7 循环结构 ············································································.030
2.2 Python 数据分析环境 ······································································.032
2.2.1 使用pip 安装数据分析相关库 ··················································.032
2.2.2 安装Anaconda ·····································································.033
2.3 Python 数据分析相关库 ···································································.033
2.3.1 NumPy 库 ··········································································.033
2.3.2 Matplotlib 库 ·······································································.034
2.3.3 SciPy 库·············································································.035
2.3.4 Pandas 库 ···········································································.036
2.3.5 xlrd 库 ···············································································.036
2.3.6 PyMySQL 库 ·······································································.037
2.3.7 其他数据分析相关库 ·····························································.037
2.4 本章小结 ·····················································································.038
第3 章 Jupyter 环境的使用 ................................................................................................. 039
3.1 Jupyter Notebook 概述 ·····································································.039
3.1.1 Jupyter Notebook 简介及优点 ···················································.039
3.1.2 Jupyter Notebook 开发环境的搭建 ·············································.039
3.1.3 使用pip 命令安装 ································································.044
3.2 认识Jupyter Notebook ·····································································.044
3.2.1 认识Files、Running、Clusters 页面 ··········································.044
3.2.2 认识Jupyter Notebook 的主页面 ···············································.046
3.3 新建、运行、保存Jupyter Notebook 文件 ·············································.048
3.3.1 新建一个Jupyter Notebook ······················································.048
3.3.2 运行代码 ···········································································.049
3.3.3 重命名Jupyter Notebook 文件 ··················································.049
3.3.4 保存Jupyter Notebook 文件 ·····················································.050
3.4 处理不同类型的数据 ······································································.050
3.4.1 处理txt 文件 ·······································································.050
3.4.2 处理CSV 文件 ····································································.052
3.4.3 处理Excel 文件 ···································································.053
3.4.4 处理sql 文件 ······································································.053
3.5 在Markdown 中使用LaTeX 输入数学公式 ············································.054
3.5.1 使用LaTeX 输入一个数学公式 ·················································.054
3.5.2 LaTeX 的2 种公式格式 ··························································.055
3.5.3 常用数学公式的写法 ·····························································.056
3.6 Jupyter Notebook 应用实例解析 ··························································.058
3.6.1 实例1：能力六维雷达图的绘制 ···············································.058
3.6.2 实例2：词频统计 ·································································.059
3.7 本章小结 ······················································································.060
第4 章 探索数据 .................................................................................................................. 062
4.1 走进数据的世界 ·············································································.062
4.1.1 定义数据 ············································································.062
4.1.2 数据的分类 ·········································································.063
4.1.3 深挖数据的4 种能力 ·····························································.065
4.1.4 善用指标分析问题 ································································.067
4.2 数据的评估 ···················································································.069
4.2.1 指标真的可靠吗 ···································································.069
4.2.2 统计数据会“说谎” ·····························································.071
4.3 数据怎么用 ···················································································.072
4.3.1 数据清洗 ············································································.072
4.3.2 数据的标准化 ······································································.076
4.4 本章小结 ······················································································.078
第5 章 描述统计 .................................................................................................................. 079
5.1 数据集中趋势 ················································································.079
5.1.1 均值的定义与应用 ································································.079
5.1.2 中位数的定义与应用 ·····························································.081
5.1.3 众数的定义与应用 ································································.083
5.1.4 案例分析 ············································································.085
5.2 数据离散程度 ················································································.087
5.2.1 极差的定义与应用 ································································.088
5.2.2 方差的定义与应用 ································································.090
5.3 本章小结 ·····················································································.091
第6 章 推断统计 ................................................................................................................... 092
6.1 基础知识要点 ···············································································.092
6.1.1 排列与组合 ········································································.092
6.1.2 随机事件及其概率 ································································.095
6.2 概率分布及其特征 ·········································································.095
6.2.1 二项分布 ···········································································.096
6.2.2 正态分布 ···········································································.098
6.3 统计量 ························································································.104
6.3.1 总体与样本 ········································································.105
6.3.2 参数估计 ···········································································.109
6.3.3 假设检验 ···········································································.112
6.4 本章小结 ·····················································································.116
第7 章 数据可视化 ............................................................................................................... 117
7.1 什么是数据可视化 ·········································································.117
7.1.1 数据可视化的定义和意义 ·······················································.117
7.1.2 数据可视化的发展历史 ··························································.118
7.2 图形对象与元素 ············································································.119
7.2.1 如何建立坐标系···································································.121
7.2.2 如何设置坐标轴的文本和图例 ·················································.122
7.3 可视化色彩的运用原理 ···································································.123
7.3.1 RGB 颜色模式 ·····································································.123
7.3.2 HSL 颜色模式 ·····································································.124
7.3.3 颜色搭配的技巧和案例 ··························································.124
7.4 图表的基本类型 ············································································.126
7.4.1 如何绘制柱形图···································································.126
7.4.2 如何绘制散点图···································································.127
7.4.3 如何绘制饼形图···································································.129
7.4.4 如何绘制折线图 ···································································.130
7.5 数据分析及可视化案例 ····································································.132
7.5.1 数据可视化经典案例 ·····························································.132
7.5.2 非结构化数据的可视化案例 ·····················································.132
7.6 常见的数据可视化流程 ····································································.133
7.7 本章小结 ······················································································.133
第8 章 NumPy 数组 ............................................................................................................ 134
8.1 NumPy 库简介 ···············································································.134
8.2 NumPy 数组的生成 ·········································································.134
8.2.1 生成一般数组 ······································································.135
8.2.2 生成特殊数组 ······································································.136
8.2.3 生成随机数组 ······································································.138
8.3 NumPy 数组基础 ············································································.140
8.3.1 NumPy 数组的基本属性 ··························································.140
8.3.2 数组索引：获取单个元素························································.141
8.3.3 数组切片：获取子数组 ··························································.141
8.4 NumPy 数组重塑 ············································································.143
8.4.1 NumPy 数组的变形 ·······························································.144
8.4.2 NumPy 数组的转置和换轴 ·······················································.144
8.4.3 NumPy 数组的拼接与分裂 ·······················································.146
8.5 NumPy 库中的线性代数 ···································································.147
8.5.1 矩阵乘法 ············································································.147
8.5.2 行列式 ···············································································.148
8.5.3 求线性方程的解 ···································································.148
8.6 通用函数 ······················································································.150
8.6.1 一元通用函数 ······································································.150
8.6.2 二元通用函数 ······································································.150
8.6.3 广播 ··················································································.152
8.7 常用的数据分析函数 ·······································································.154
8.7.1 条件函数 ············································································.154
8.7.2 聚合函数 ···········································································.154
8.7.3 快速排序 ···········································································.156
8.7.4 唯一值与其他集合逻辑 ··························································.157
8.8 本章小结 ·····················································································.158
第9 章 时间序列数据 ........................................................................................................... 159
9.1 时间序列的定义及分类 ···································································.159
9.1.1 时间序列的定义···································································.159
9.1.2 时间序列的分类···································································.159
9.2 时间序列的描述性分析 ···································································.162
9.2.1 图形描述 ···········································································.162
9.2.2 增长率分析 ········································································.163
9.3 时间序列的预测 ············································································.164
9.3.1 确定时间序列成分 ································································.165
9.3.2 选择预测方法 ·····································································.166
9.3.3 预测方法评估 ·····································································.167
9.4 平稳时间序列的预测 ······································································.169
9.4.1 简单平均法 ········································································.169
9.4.2 移动平均法 ········································································.170
9.4.3 指数平滑法 ········································································.171
9.5 趋势型和复合型时间序列的预测 ························································.172
9.5.1 线性趋势预测 ·····································································.172
9.5.2 非线性趋势预测···································································.173
9.5.3 复合型时间序列的分解预测 ····················································.174
9.6 使用Python 处理时间序列数据 ··························································.174
9.6.1 时间序列数据处理工具的选择 ·················································.175
9.6.2 时间序列数据的导入 ·····························································.175
9.6.3 时间序列数据预处理 ·····························································.176
9.6.4 时间序列数据处理 ································································.177
9.7 本章小结 ·····················································································.185
第10 章 文本数据 ................................................................................................................ 186
10.1 文本数据的导入 ···········································································.186
10.1.1 文本数据与自然语言处理 ······················································.186
10.1.2 分词 ················································································.187
10.2 文本数据的处理 ···········································································.189
10.2.1 文本特征初探 ·····································································.189
10.2.2 文本信息的提取 ··································································.194
10.2.3 文本向量化 ········································································.198
10.3 文本分析的应用 ···········································································.200
10.3.1 文本分类 ···········································································.200
10.3.2 文本情感分析 ·····································································.201
10.4 本章小结 ····················································································.204
第11 章 回归分析 ................................................................................................................ 205
11.1 叩响人工智能之门 ·········································································.205
11.1.1 人工智能与机器学习 ····························································.205
11.1.2 工欲善其事，必先利其器 ······················································.208
11.1.3 算法，该“出道”了 ····························································.208
11.2 万朝归宗：线性回归 ······································································.209
11.2.1 一元之道 ···········································································.210
11.2.2 从一元到多元 ·····································································.216
11.2.3 学习和工作中的线性回归 ······················································.218
11.3 回归增强术 ·················································································.221
11.3.1 非线性回归 ········································································.221
11.3.2 可分类的回归 ·····································································.224
11.3.3 能降维的回归 ·····································································.227
11.4 本章小结 ····················································································.228
第12 章 聚类分析 ................................................................................................................ 229
12.1 数据之眼看聚类 ···········································································.229
12.1.1 什么是聚类 ·······································································.229
12.1.2 人工智能的未来：无监督学习 ···············································.231
12.1.3 距离产生美 ·······································································.232
12.2 K 均值聚类 ·················································································.234
12.2.1 K 均值聚类的思想 ······························································.234
12.2.2 抽丝剥茧K 均值 ·································································.234
12.2.3 鸢尾花的K 均值聚类 ···························································.238
12.3 案例：数据下的省、区、市 ····························································.240
12.3.1 提出问题 ··········································································.240
12.3.2 数据获取与处理 ·································································.241
12.3.3 建模分析与结果 ·································································.242
12.4 本章小结 ···················································································.244
第13 章 数据素养综合案例 ................................................................................................ 245
13.1 综合案例1：利用人工智能爬取大数据，轻松掌握股市动态 ·························.245
13.1.1 认识人工智能网络爬虫 ························································.245
13.1.2 爬取股市大数据，分析需求 ··················································.246
13.1.3 爬取股市大数据案例 ···························································.246
13.2 综合案例2：人工智能数据――体型分析 ···········································.251
13.2.1 K 最邻近分类算法原理·························································.251
13.2.2 使用KNN 分类算法对体型进行分类的案例 ·······························.253
13.3 其他案例集 ·················································································.254
13.3.1 计算生肖和星座 ·································································.254
13.3.2 猜数游戏 ··········································································.255
13.3.3 二维列表排序 ····································································.255
13.3.4 学生信息录入 ····································································.255
13.3.5 打印回文素数（合数） ························································.256
13.3.6 数据库加密 ·······································································.256
13.3.7 计算圆台的体积和表面积 ·····················································.256
13.4 本章小结 ···················································································.257