kafka集群部署（最新版本kafka-2.13-3.1.0）

1、kafka集群部署（最新版本kafka_2.13-3.1.0）集群環(huán)境：節(jié)點(diǎn)名ip地址OS版本部署服務(wù)broker信息kafka011CentOS Linux release 7.9.2009 (Core)jdk、zookeeper、kafkabroker : 0kafka022CentOS Linux release 7.9.2009 (Core)jdk、zookeeper、kafkabroker : 1kafka032CentOS Linux release 7.9.2009 (Core)jdk、zookeeper、kafkabroker : 2軟件版本：jdk 1.8 版本：https:

2、/ HYPERLINK /java/technologies/downloads/#java8 /java/technologies/downloads/#java8zookeeper 3.7.0 版本: /zookeeper/zookeeper-3.7.0/apache-zookeeper-3.7.0-bin.tar.gzkafka 2.13-3.1.0 版本: /kafka/3.1.0/kafka_2.13-3.1.0.tgz開(kāi)始部署：、部署jdk解壓并重命名服務(wù)cd /opt/software/tar -zxvf jdk-8u321-linux-x64.tar.gz mv jdk1.8.

3、0_321 /usr/local/jdk1.8.0添加環(huán)境變量vi /etc/profile# 在件尾部追加以下內(nèi)容export JAVA_HOME=/usr/local/jdk1.8.0 export JRE_HOME=$JAVA_HOME/jreexport CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/libexport PATH=$JAVA_HOME/bin:$PATH重載環(huán)境變量，使新配置效source /etc/profile驗(yàn)證java -version繼續(xù)按照以上法部署另外2臺(tái)服務(wù)器、部署zookeeper在線下載安裝包c(diǎn)d /opt/softwar

4、e/wget /zookeeper/zookeeper-3.7.0/apache-zookeeper-3.7.0-bin.tar.gz -no-check-certificate解壓并重命名服務(wù)tar -zxvf apache-zookeeper-3.7.0-bin.tar.gzmv apache-zookeeper-3.7.0-bin /usr/local/zookeeper-3.7.0創(chuàng)建zookeeper服務(wù)的data錄mkdir /usr/local/zookeeper-3.7.0/data創(chuàng)建myid件（zookeeper于惟標(biāo)識(shí)的id）echo 1 /usr/local/zooke

5、eper-3.7.0/data/myid成zoo.cfg配置件cp /usr/local/zookeeper-3.7.0/conf/zoo_sample.cfg /usr/local/zookeeper-3.7.0/conf/zoo.cfg修改zoo.cfg配置件vi /usr/local/zookeeper-3.7.0/conf/zoo.cfgtickTime=2000initLimit=10 syncLimit=5dataDir=/usr/local/zookeeper-3.7.0/data clientPort=2181server.1=0:2888:3888 server.2=1:28

6、88:3888 server.3=2:2888:3888使scp同步發(fā)送到另外2臺(tái)服務(wù)器scp -r /usr/local/zookeeper-3.7.0 root1:/usr/local/scp -r /usr/local/zookeeper-3.7.0 root2:/usr/local/配置另外2個(gè)節(jié)點(diǎn)的myid件1 服務(wù)器echo 2 /usr/local/zookeeper-3.7.0/data/myid2 服務(wù)器echo 3 /usr/local/zookeeper-3.7.0/data/myid最后，分別在3臺(tái)服務(wù)器上啟動(dòng)zk服務(wù)/usr/local/zookeeper-3.7.0

7、/bin/zkServer.sh start驗(yàn)證jps -m三、部署kafka下載軟件包并解壓、重命名服務(wù)cd /opt/software/wget /kafka/3.1.0/kafka_2.13-3.1.0.tgz tar -zxvf kafka_2.13-3.1.0.tgz -no-check-certificatemv kafka_2.13-3.1.0 /usr/local/kafka-2.13配置kafkavi /usr/local/kafka-2.13/config/pertieskafka01服務(wù)器broker.id=0 listeners=PLAINTEXT:/0:9092log

8、.dirs=/tmp/kafka-logszookeeper.connect=0:2181,1:2181,2:2181使scp同步發(fā)送到另外2臺(tái)服務(wù)器scp -r /usr/local/kafka-2.13 root1:/usr/local/scp -r /usr/local/kafka-2.13 root2:/usr/local/分別修改kafka02、kafka03的配置，如下kafka02服務(wù)器broker.id=1listeners=PLAINTEXT:/1:9092kafka03服務(wù)器broker.id=2listeners=PLAINTEXT:/2:9092分別在3臺(tái)服務(wù)器上，啟動(dòng)kafka集群cd /usr/local/kafka-2.13 & bin/kafka-server-start.sh config/perties &四、驗(yàn)證測(cè)試創(chuàng)建topicbin/kafka-topics.sh -create -topic quickstart-events -bootstrap-server 0:9092查看topicbin/kafka-topics.sh -describe -topic quickstart-events -bootstrap-server 0:9092產(chǎn)者：發(fā)送消息bin/kafka-console-producer.sh -topic qui