This is the repository for the examples of using Kafka streams covered in the blog posts:
- Kafka Streams - The Processor API
- Kafka Streams - The KStreams API
- Machine Learning with Kafka Streams
- Java 8
- Gradle
- kafka version kafka_2.11-0.10.1.0 see the section marked "Running a task on a particular version of Scala"
- The json-data-generator from ACES,Inc
tar -xvzf kafka_2.11-0.10.1.0.tgz
kafka-install-dir/bin/zookeeper-server-start.sh kafka-install-dir/conf/zookeeper.properties
kafka-install-dir/bin/kafka-server-start.sh kafka-install-dir/conf/server.properties
Download the latest json-data-generator release and follow the install instructions here
Clone or fork the repo
git clone git@github.com:bbejeck/kafka-streams
cd kafka-streams
Then copy the json config files to json generator conf directory
cp streaming-workflows/* <dir>/json-data-generator-1.2.0/conf
Create all the topics required by the examples
./bin/create-topics.sh /usr/local/kafka_2.11-0.10.1.0 localhost 2181
args are kafka home, zookeeper host and zookeeper port adjust accordingly
cd <dir>/json-data-generator-1.2.0/
java -jar json-data-generator-1.2.0 purchases-config.json
cd kafka-streams
./gradlew runPurchaseProcessor | runPurchaseStreams
cd <dir>/json-data-generator-1.2.0/
java -jar json-data-generator-1.2.0 stock-transactions-config.json
cd kafka-streams
./gradlew runStockProcessor | runStockStreams
rename src/main/resources/twitter-app.properties.template to twitter-app.properties
fill out the properties file with all the required values
cd kafka-streams
./gradlew runTwitterKstreamNLP
cd kafka_install-dir/bin
./kafka-console-consumer --topic [patterns|rewards|purchases] --zookeeper localhost:2181
cd kafka_install-dir/bin
./kafka-console-consumer --topic [stocks-out|transaction-summary] --zookeeper localhost:2181
cd kafka_install-dir/bin
./kafka-console-consumer --topic [english|french|spanish] --zookeeper localhost:2181