I am using kafka 0.8.1.1. I have a 3 node kafka cluster with some topics having around 5 partitions. I planned to increase the number of nodes to 5 in cluster and moving some partitions from existing topics to the new brokers.
Previous partition state:
broker1 : topic1 { partition 0 }
broker2 : topic1 { partition 1,2}
broker3 : topic1 { partition 3,4}
New intended state:
broker1 : topic1 { partition 0}
broker2 : topic1 { partition 1}
broker3 : topic1 { partition 3}
broker4 : topic1 { partition 4}
broker5 : topic1 { partition 2}
command which I used:
bin/kafka-reassign-partitions.sh --reassignment-json-file partitions-to-move.json --zookeeper zkserver1:2181,zkserver2:2181,zkserver3:2181 --execute
partitions-to-move.json
{
"partitions":[{"topic": "topic1","partition": 4,"replicas": [4] }],
"version":1
}
Error messages that I am seeing:
[2015-12-22 11:19:46,447] WARN [KafkaApi-2] Produce request with correlation id 2462842 from client kafka-node-client on partition [topic1,4] failed due to Partition [topic1,4] doesn't exist on 3 (kafka.server.KafkaApis)
I did search but couldn't find any relevant answers. Appreciate any guidance/help to sort this out.