Apache Kafka - Installation Steps

Apache Kafka - Installation Steps

Following are the steps for installing Java on your machine.

Step 1 - Verifying Java Installation

Hopefully you have already installed java on your machine right now, so you just verify it using the following command.

$ java -version

If java is successfully installed on your machine, you could see the version of the installed Java.

Step 1.1 - Download JDK

If Java is not downloaded, please download the latest version of JDK by visiting the following link and download latest version.

http://www.oracle.com/technetwork/java/javase/downloads/index.html

Now the latest version is JDK 8u 60 and the file is “jdk-8u60-linux-x64.tar.gz”. Please download the file on your machine.

Step 1.2 - Extract Files

Generally, files being downloaded are stored in the downloads folder, verify it and extract the tar setup using the following commands.

$ cd /go/to/download/path $ tar -zxf jdk-8u60-linux-x64.gz

Step 1.3 - Move to Opt Directory

To make java available to all users, move the extracted java content to usr/local/java/ folder.

$ su password: (type password of root user) $ mkdir /opt/jdk $ mv jdk-1.8.0_60 /opt/jdk/

Step 1.4 - Set path

To set path and JAVA_HOME variables, add the following commands to ~/.bashrc file.

export JAVA_HOME =/usr/jdk/jdk-1.8.0_60 export PATH=$PATH:$JAVA_HOME/bin

Now apply all the changes into current running system.

$ source ~/.bashrc

Step 1.5 - Java Alternatives

Use the following command to change Java Alternatives.

update-alternatives --install /usr/bin/java java /opt/jdk/jdk1.8.0_60/bin/java 100

Step 1.6 − Now verify java using verification command (java -version) explained in Step 1.

Step 2 - ZooKeeper Framework Installation

Step 2.1 - Download ZooKeeper

To install ZooKeeper framework on your machine, visit the following link and download the latest version of ZooKeeper.

http://zookeeper.apache.org/releases.html

As of now, latest version of ZooKeeper is 3.4.6 (ZooKeeper-3.4.6.tar.gz).

Step 2.2 - Extract tar file

Extract tar file using the following command

$ cd opt/ $ tar -zxf zookeeper-3.4.6.tar.gz $ cd zookeeper-3.4.6 $ mkdir data

Step 2.3 - Create Configuration File

Open Configuration File named conf/zoo.cfg using the command vi “conf/zoo.cfg” and all the following parameters to set as starting point.

$ vi conf/zoo.cfg tickTime=2000 dataDir=/path/to/zookeeper/data clientPort=2181 initLimit=5 syncLimit=2

Once the configuration file has been saved successfully and return to terminal again, you can start the zookeeper server.

Step 2.4 - Start ZooKeeper Server

$ bin/zkServer.sh start

After executing this command, you will get a response as shown below −

$ JMX enabled by default $ Using config: /Users/../zookeeper-3.4.6/bin/../conf/zoo.cfg $ Starting zookeeper ... STARTED

Step 2.5 - Start CLI

$ bin/zkCli.sh

After typing the above command, you will be connected to the zookeeper server and will get the below response.

Connecting to localhost:2181 ................ ................ ................ Welcome to ZooKeeper! ................ ................ WATCHER:: WatchedEvent state:SyncConnected type: None path:null [zk: localhost:2181(CONNECTED) 0]

Step 2.6 - Stop Zookeeper Server

After connecting the server and performing all the operations, you can stop the zookeeper server with the following command −

$ bin/zkServer.sh stop

Now you have successfully installed Java and ZooKeeper on your machine. Let us see the steps to install Apache Kafka.

Step 3 - Apache Kafka Installation

Let us continue with the following steps to install Kafka on your machine.

Step 3.1 - Download Kafka

To install Kafka on your machine, click on the below link −

https://www.apache.org/dyn/closer.cgi?path=/kafka/0.9.0.0/kafka_2.11-0.9.0.0.tgz

Now the latest version i.e., – kafka_2.11_0.9.0.0.tgz will be downloaded onto your machine.

Step 3.2 - Extract the tar file

Extract the tar file using the following command −

$ cd opt/ $ tar -zxf kafka_2.11.0.9.0.0 tar.gz $ cd kafka_2.11.0.9.0.0

Now you have downloaded the latest version of Kafka on your machine.

Step 3.3 - Start Server

You can start the server by giving the following command −

$ bin/kafka-server-start.sh config/server.properties

After the server starts, you would see the below response on your screen −

$ bin/kafka-server-start.sh config/server.properties [2016-01-02 15:37:30,410] INFO KafkaConfig values: request.timeout.ms = 30000 log.roll.hours = 168 inter.broker.protocol.version = 0.9.0.X log.preallocate = false security.inter.broker.protocol = PLAINTEXT ……………………………………………. …………………………………………….

Step 4 - Stop the Server

After performing all the operations, you can stop the server using the following command −

$ bin/kafka-server-stop.sh config/server.properties

Now that we have already discussed the Kafka installation, we can learn how to perform basic operations on Kafka in the next chapter.