-
Notifications
You must be signed in to change notification settings - Fork 5
/
kudu.txt
106 lines (106 loc) · 11.2 KB
/
kudu.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
前言 ............................................................................... XIII
第 1 章 为什么会有 Kudu ..................................................1
Kudu 为什么重要 .................................................................. 1
易用性驱动接纳度 .................................................................................. 2
新的应用场景 ......................................................................................... 5
物联网 ...........................................................................................................5
现有的实时分析方案 .....................................................................................7
实时处理 .....................................................................................................13
硬件环境 ............................................................................................... 15
Kudu 在大数据生态中的独特位置 ........................................................ 17
与其他生态的组件对比 .......................................................................19
与大数据组件对比——HDFS、HBase 和 Cassandra ..................................24
小结 ...................................................................................................... 26
第 2 章 Kudu 简介 ..........................................................27
Kudu 的高层设计 .................................................................................. 29
Kudu 中的角色 ............................................................................................29
master 服务器 ..............................................................................................31
tablet 服务器................................................................................................32
Kudu 中的概念与机制 .......................................................................... 42
热点 .............................................................................................................42
分区 .............................................................................................................44
第 3 章 安装与运行 .........................................................49
安装 ...................................................................................................... 49
使用 Kudu Quickstart VM ...........................................................................49
使用 Cloudera Manager ...............................................................................51
从源代码构建 ..............................................................................................52
软件包 .........................................................................................................53
Cloudera Quickstart VM ..............................................................................53
快速安装 ：3 分钟或者更短 .................................................................. 54
小结 ...................................................................................................... 58
第 4 章 Kudu 的管理 .......................................................59
为 Kudu 做规划 ..................................................................................... 59
master 服务器和 tablet 服务器 .....................................................................60
预写日志 .....................................................................................................65
数据服务器和存储 ......................................................................................68
复制策略（replication strategy） ..................................................................69
部署时的注意事项 ：是采用新集群还是现有集群 ................................ 70
全新的仅有 Kudu 的集群 ............................................................................70
全新的包含 Kudu 的 Hadoop 集群 ..............................................................71
在现有的 Hadoop 集群中添加 Kudu ...........................................................77
tablet 服务器和 master 服务器的 Web UI .............................................. 81
master 服务器 UI 和 tablet 服务器 UI ..........................................................82
master 服务器 UI .........................................................................................83
tablet 服务器 UI ...........................................................................................83
Kudu 命令行接口 .................................................................................. 84
集群 .............................................................................................................84
文件 .....................................................................................................86
tablet 副本 ...................................................................................................92
与 Raft 一致性相关的元数据.....................................................................106
添加和删除 tablet 服务器 .................................................................... 107
添加 tablet 服务器 .....................................................................................107
删除 tablet 服务器 .....................................................................................108
安全 .................................................................................................... 109
一个简单的类比 ........................................................................................ 110
Kudu 的安全功能 ...................................................................................... 112
基本的性能调优 .................................................................................. 117
Kudu 的内存限制 ...................................................................................... 117
维护管理器的线程 .................................................................................... 118
监控性能 ................................................................................................... 119
未雨绸缪，远离麻烦 .......................................................................... 119
避免耗尽磁盘空间 .................................................................................... 119
容忍磁盘故障 ............................................................................................120
备份 ...........................................................................................................120
小结 .................................................................................................... 121
第 5 章 Kudu 常用的开发接口 .......................................123
客户端 API .......................................................................................... 124
Kudu Client（客户端）...............................................................................124
Kudu Table ................................................................................................125
Kudu DDL .................................................................................................125
Kudu 扫描器（Scanner）读取模式 ...........................................................126
C API .............................................................................................. 127
Python API .......................................................................................... 130
准备 Python 开发环境 ...............................................................................131
使用 Python 开发 Kudu 应用 .....................................................................131
Java ..................................................................................................... 135
Java 应用 ...................................................................................................137
Spark ................................................................................................... 140
在 Impala 中使用 Kudu ....................................................................... 145
第 6 章 表和模式设计 ...................................................149
模式设计基础 ..................................................................................... 150
在线事务处理 / 在线分析处理混合的模式设计 .................................. 151
Lambda 架构 .............................................................................................151
OLTP/OLAP 拆分 ......................................................................................152
主键和列的设计 .................................................................................. 153
列模式的其他注意事项 .............................................................................154
分区的基础知识 .................................................................................. 160
范围分区 ...................................................................................................161
哈希分区 ...................................................................................................161
模式的更改 ......................................................................................... 162
*实践和提示 .................................................................................. 163
分区 ...........................................................................................................163
大对象 .......................................................................................................164
decimal（十进制数）..................................................................................164
不重复的字符串 ........................................................................................165
压缩 ...........................................................................................................165
对象的命名 ................................................................................................165
列的数量 ...................................................................................................165
二进制类型 ................................................................................................166
网络包示例 ......................................................................................... 166
小结 .................................................................................................... 168
第 7 章 Kudu 用例 ........................................................169
实时物联网分析 .................................................................................. 169
预测建模 ............................................................................................. 173
多平台混合方案 .................................................................................. 176
关于作者 ........................................................................180
封面图片 ........................................................................182