what is key schema in schema registry?

Question

I don't have exact idea of key schema, that what it is, and why it must be used as key is auto-generated and we just pass a value(message).

For value, we pass a schema to the AVRO Serialiser and the serialiser gets it's schema id from schema registry and embeds the schema id with the value(message) we have passed(correct me if I am wrong). What happens to key?

Do we also need to pass a key schema? What is the importance of passing a key schema? And, how to pass a key schema?

Incest

Answer 1

Kafka messages are key/value pairs. What you set the key is up to you and the requirements of what you are implementing.

The message key is used for partition assignment. Typically you would key a message based on the processing you expect to do, and any strict ordering you want to impose on the data. For example, if you want to have multiple parallel processes in the same consumer group with each process receiving all records for a given customer, you would key on the customer ID.

Speck

Answer 2

Adding more info with @Robin's answer,

As each kafka message will have a key and value, key can be null or some primitive type value.

If you send a message with string type key and integer type value for topic T, Schema registry creates two subjects: T-key and T-value.

T-key will store the avro schema of the string type key. If there is no key attached to message(null type), it won't register any schema in schema registry.

Cloth

you said "If there is no key attached to message(null type), it won't register any schema in schema registry". What if, key is autogenerated? and any idea about key schema? – Incest

I read that producer itself chooses the partition to which it sends the message. Then it will itself attach the key(id of partition) to the message. Isn't it correct? – Incest

If you don't pass the key with kafka producer while publishing the messages, it picks random partitions becuase key is null. Kafka record key and partition-key are two different concepts. One partition can contain multiple keys based on the partitioning strategy. Please refer the link for more details Partition when no key is specified – Cloth

If I don't attach any key to my message, then will it finds it's partition first to get the key or it goes to get register in the schema registry. If, it goes to get register first then, it will get failed right? So, what will do first? Also, how to pass a key schema, is it necessary? – Incest