Zookeeper multiple-nodes cluster setup

trinhhieu668

In the previous post, i was overview about Zookeeper and explained how to it work (post). Zookeeper is core system, it provide an API to manage the status of application nodes in distributed environment,  Zookeeper is very important component so what happens when a server Zookeeper is died ? Maybe the whole system will stop working and this is serious. To resolve this problem then Zookeeper support it can be run as cluster go when leader node die all of the nodes will run “Algorithm election” and a new node will became leader. So Zookeeper cluster is high availability, and below i will present details how to setup Zookeeper on multiple-nodes.

Pre-requisites : you have some servers to setup, with me i have 4 server need to setup Zookeeper cluster

Node1 : 10.3.0.100
Node2 : 10.3.0.101
Node3 : 10.3.0.102
Node4 : 10.3.0.103

Step 1 : Add information servers to DNS hostname configuration

vi /etc/hosts

10.3.0.100 cloud1
10.3.0.101 cloud2
10.3.0.102 cloud3
10.3.0.103 cloud4

Step 2 : You need download Zookeeper stable version from page (Zookeeper homepage) and save into node1

wget http://mirror.downloadvn.com/apache/zookeeper/stable/zookeeper-3.4.10.tar.gz
tar -xvf zookeeper-3.4.10.tar.gz

Step 3 : Create path folder to store Zookeeper data (all nodes).

mkdir /data/hdfs/zookeeper

Step 4 : Setup memory use to run Zookeeper instance, create java.env to add the below configuration

vi $ZOOKEEPER_HOME/conf/java.env

Add this below content to file

export JAVA_OPTS="-Xms4096m -Xmx4096m"

(i was changed memory to run Zookeeper instance to 4GB)

Step 5 : Config zookeeper

cp zoo_sample.cfg zoo.cfg
vi zoo.cfg

add this below configuration to file

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/data/hdfs/zookeeper
clientPort=2181
maxClientCnxns=2000
server.1=cloud1:2888:3888
server.2=cloud2:2888:3888
server.3=cloud3:2888:3888
server.4=cloud4:2888:3888

Explain some parameters is used

maxClientCnxns : max clients number connection to node.
clientPort : current node listen on port 2181
dataDir : path to store data.
3888 : port listen connect from other nodes in cluster.
2888 : port listen if that is leader node.

Step 6 : Copy zookeeper folder to all of them

rsync -avz zookeeper-3.4.10/ cloud2:/data/zookeeper-3.4.10/
rsync -avz zookeeper-3.4.10/ cloud3:/data/zookeeper-3.4.10/
rsync -avz zookeeper-3.4.10/ cloud4:/data/zookeeper-3.4.10/

Step 7 : Config ID of node to run cluster

on cloud1

echo "1" >> /data/hdfs/zookeeper/myid

on cloud2

echo "2" >> /data/hdfs/zookeeper/myid

on cloud3

echo "3" >> /data/hdfs/zookeeper/myid

on cloud4

echo "4" >> /data/hdfs/zookeeper/myid

Step 8 : Start all instance with command

./bin/zkServer.sh start

Run command jps to check

Done, If you have any question about setup Zookeeper cluster please contact me (facebook or linked) we can discuss about it. Thank for reading.

 

 

Leave a comment