“Flink安装配置”的版本间的差异
(创建页面,内容为“==下载地址== https://flink.apache.org/zh/downloads.html https://archive.apache.org/dist/flink/ 版本:flink-1.10.1 ==本地Flink多节点配置== ===删除…”) |
|||
(未显示同一用户的2个中间版本) | |||
第16行: | 第16行: | ||
ssh slave3 rm -rf /usr/local/flink-1.9.2/ | ssh slave3 rm -rf /usr/local/flink-1.9.2/ | ||
− | === | + | ===下载解压flink安装包=== |
将flink-1.10.1-bin-scala_2.11.tgz通过新建文件传输上传到master。 | 将flink-1.10.1-bin-scala_2.11.tgz通过新建文件传输上传到master。 | ||
tar -zxf /opt/flink-1.10.1-bin-scala_2.11.tgz -C /usr/local | tar -zxf /opt/flink-1.10.1-bin-scala_2.11.tgz -C /usr/local | ||
− | + | ||
− | export FLINK_HOME=/usr/local/flink-1.10.1 | + | vim /etc/profile |
− | export PATH=$FLINK_HOME/bin:$PATH | + | |
+ | <nowiki>export FLINK_HOME=/usr/local/flink-1.10.1 | ||
+ | export PATH=$FLINK_HOME/bin:$PATH</nowiki> | ||
source /etc/profile | source /etc/profile | ||
+ | |||
mkdir -p /var/log/flink/tmp | mkdir -p /var/log/flink/tmp | ||
+ | |||
+ | ===修改配置文件=== | ||
+ | /usr/local/flink-1.10.1/conf/flink-conf.yaml | ||
+ | |||
+ | <nowiki>jobmanager.rpc.address: master | ||
+ | jobmanager.heap.size: 512m | ||
+ | #taskmanager.memory.process.size: 768m | ||
+ | taskmanager.memory.flink.size: 1024m | ||
+ | taskmanager.numberOfTaskSlots: 1 | ||
+ | parallelism.default: 1 | ||
+ | rest.port: 8081 | ||
+ | rest.address: master | ||
+ | rest.bind-address: master | ||
+ | io.tmp.dirs: /var/log/flink/tmp | ||
+ | </nowiki> | ||
+ | |||
+ | /usr/local/flink-1.10.1/conf/slaves | ||
+ | |||
+ | <nowiki>slave1 | ||
+ | slave2 | ||
+ | slave3</nowiki> | ||
+ | |||
+ | |||
+ | /usr/local/flink-1.10.1/conf/masters | ||
+ | |||
+ | master:8081 | ||
+ | |||
+ | ===向slave节点分发安装文件=== | ||
+ | scp -r /usr/local/flink-1.10.1 slave1:/usr/local | ||
+ | |||
+ | scp -r /usr/local/flink-1.10.1 slave2:/usr/local | ||
+ | |||
+ | scp -r /usr/local/flink-1.10.1 slave3:/usr/local | ||
+ | |||
+ | 在各slave节点: | ||
+ | |||
+ | vim /etc/profile | ||
+ | |||
+ | <nowiki>export FLINK_HOME=/usr/local/flink-1.10.1 | ||
+ | export PATH=$FLINK_HOME/bin:$PATH</nowiki> | ||
+ | |||
+ | source /etc/profile | ||
+ | |||
+ | mkdir -p /var/log/flink/tmp | ||
+ | |||
+ | ==启动与停止== | ||
+ | ===启动flink集群=== | ||
+ | /usr/local/flink-1.10.1/bin/start-cluster.sh | ||
+ | |||
+ | <nowiki>Starting cluster. | ||
+ | Starting standalonesession daemon on host master.centos.com. | ||
+ | Starting taskexecutor daemon on host slave1.centos.com. | ||
+ | Starting taskexecutor daemon on host slave2.centos.com. | ||
+ | Starting taskexecutor daemon on host slave3.centos.com.</nowiki> | ||
+ | |||
+ | 查看进程jps | ||
+ | [root@master flink-1.10.1]# jps | ||
+ | |||
+ | <nowiki>10753 StandaloneSessionClusterEntrypoint | ||
+ | 3172 QuorumPeerMain | ||
+ | 3495 Kafka | ||
+ | 10812 Jps</nowiki> | ||
+ | |||
+ | [root@master flink-1.10.1]# ssh slave1 jps | ||
+ | |||
+ | <nowiki>3637 Kafka | ||
+ | 7356 TaskManagerRunner | ||
+ | 7404 Jps</nowiki> | ||
+ | |||
+ | [root@master flink-1.10.1]# ssh slave2 jps | ||
+ | |||
+ | <nowiki>3188 Kafka | ||
+ | 5130 TaskManagerRunner | ||
+ | 5178 Jps | ||
+ | 2846 QuorumPeerMain</nowiki> | ||
+ | |||
+ | [root@master flink-1.10.1]# ssh slave3 jps | ||
+ | |||
+ | <nowiki>2636 Jps | ||
+ | 2589 TaskManagerRunner</nowiki> | ||
+ | |||
+ | ===浏览器查看=== | ||
+ | master:8081,相关配置可以查看到。 | ||
+ | |||
+ | 或http://192.168.128.130:8081 | ||
+ | |||
+ | [[文件:bd20-6-7.png|600px]] | ||
+ | |||
+ | ===关闭flink集群=== | ||
+ | [root@master flink-1.10.1]# /usr/local/flink-1.10.1/bin/stop-cluster.sh | ||
+ | |||
+ | <nowiki>Stopping taskexecutor daemon (pid: 7356) on host slave1.centos.com. | ||
+ | Stopping taskexecutor daemon (pid: 5130) on host slave2.centos.com. | ||
+ | Stopping taskexecutor daemon (pid: 2589) on host slave3.centos.com. | ||
+ | Stopping standalonesession daemon (pid: 10753) on host master.centos.com.</nowiki> |
2020年8月8日 (六) 07:05的最新版本
目录
下载地址
https://flink.apache.org/zh/downloads.html
https://archive.apache.org/dist/flink/
版本:flink-1.10.1
本地Flink多节点配置
删除旧版本
1.若用提供的虚拟机的话要先在master节点执行
ssh slave1 rm -rf /usr/local/flink-1.9.2/
ssh slave2 rm -rf /usr/local/flink-1.9.2/
ssh slave3 rm -rf /usr/local/flink-1.9.2/
下载解压flink安装包
将flink-1.10.1-bin-scala_2.11.tgz通过新建文件传输上传到master。
tar -zxf /opt/flink-1.10.1-bin-scala_2.11.tgz -C /usr/local
vim /etc/profile
export FLINK_HOME=/usr/local/flink-1.10.1 export PATH=$FLINK_HOME/bin:$PATH
source /etc/profile
mkdir -p /var/log/flink/tmp
修改配置文件
/usr/local/flink-1.10.1/conf/flink-conf.yaml
jobmanager.rpc.address: master jobmanager.heap.size: 512m #taskmanager.memory.process.size: 768m taskmanager.memory.flink.size: 1024m taskmanager.numberOfTaskSlots: 1 parallelism.default: 1 rest.port: 8081 rest.address: master rest.bind-address: master io.tmp.dirs: /var/log/flink/tmp
/usr/local/flink-1.10.1/conf/slaves
slave1 slave2 slave3
/usr/local/flink-1.10.1/conf/masters
master:8081
向slave节点分发安装文件
scp -r /usr/local/flink-1.10.1 slave1:/usr/local
scp -r /usr/local/flink-1.10.1 slave2:/usr/local
scp -r /usr/local/flink-1.10.1 slave3:/usr/local
在各slave节点:
vim /etc/profile
export FLINK_HOME=/usr/local/flink-1.10.1 export PATH=$FLINK_HOME/bin:$PATH
source /etc/profile
mkdir -p /var/log/flink/tmp
启动与停止
启动flink集群
/usr/local/flink-1.10.1/bin/start-cluster.sh
Starting cluster. Starting standalonesession daemon on host master.centos.com. Starting taskexecutor daemon on host slave1.centos.com. Starting taskexecutor daemon on host slave2.centos.com. Starting taskexecutor daemon on host slave3.centos.com.
查看进程jps [root@master flink-1.10.1]# jps
10753 StandaloneSessionClusterEntrypoint 3172 QuorumPeerMain 3495 Kafka 10812 Jps
[root@master flink-1.10.1]# ssh slave1 jps
3637 Kafka 7356 TaskManagerRunner 7404 Jps
[root@master flink-1.10.1]# ssh slave2 jps
3188 Kafka 5130 TaskManagerRunner 5178 Jps 2846 QuorumPeerMain
[root@master flink-1.10.1]# ssh slave3 jps
2636 Jps 2589 TaskManagerRunner
浏览器查看
master:8081,相关配置可以查看到。
或http://192.168.128.130:8081
关闭flink集群
[root@master flink-1.10.1]# /usr/local/flink-1.10.1/bin/stop-cluster.sh
Stopping taskexecutor daemon (pid: 7356) on host slave1.centos.com. Stopping taskexecutor daemon (pid: 5130) on host slave2.centos.com. Stopping taskexecutor daemon (pid: 2589) on host slave3.centos.com. Stopping standalonesession daemon (pid: 10753) on host master.centos.com.