Scenario

Running the Confluent Schema Registry and Kafka Connect when this error appears:

org.apache.kafka.common.errors.SerializationException: Error retrieving Avro schema
io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Subject not found. error code: 40401

Reason

My Kafka Connector config had the following converter settings:

"key.converter": "io.confluent.connect.avro.AvroConverter"
"value.converter": "io.confluent.connect.avro.AvroConverter"

However, the producer on the topic was of type [String, GenericRecord], meaning the key for each message was a RAW string, not an Avro-schema-backed message of type string. The value of the message was an Avro GenericRecord.

The Connector was attempting to reach the Schema Registry to pull down the schema for the message key. The schema ID is located in the magic bytes of a Kafka message per the Confluent docs. The issue is the message I was sending was a byte array serialized RAW string not an Avro record. Therefore, the Connector choked on the conversion with an unhelpful message.

The solution is to change the key convertor to StringConverter instead of AvroConverter:

"key.converter": "org.apache.kafka.connect.storage.StringConverter"
"value.converter": "io.confluent.connect.avro.AvroConverter"