Common mistakes made when configuring multiple Kafka Connect workers
Explains common configuration errors when setting up multiple Kafka Connect workers in a distributed cluster, focusing on network and REST API settings.
Robin Moffatt is a Principal DevEx Engineer and seasoned conference speaker with 15+ years of experience presenting at top events like QCon, Devoxx, Kafka Summit, and Strata. He shares insights on developer experience, distributed systems, and cloud technologies through his blog, YouTube, and public talks.
617 articles from this blog
Explains common configuration errors when setting up multiple Kafka Connect workers in a distributed cluster, focusing on network and REST API settings.
A technical tutorial on streaming data from SQL Server to Snowflake using Kafka and Kafka Connect with Debezium for CDC.
A guide to deploying a custom Kafka Connect worker in Docker on Google Cloud Platform, for use with Confluent Cloud.
Fixing the 'Public Key Retrieval is not allowed' error when connecting Debezium to MySQL v8, including authentication protocol solutions.
A technical guide on integrating Kafka Connect and Debezium with Confluent Cloud for streaming data from MySQL.
How to manually skip bad records when using the Kafka Connect JDBC sink connector to prevent pipeline halts.
A guide to using Kafka Connect to stream data from Kafka topics to Elasticsearch 7, covering configuration, common errors, and schema options.
A tutorial on using kafkacat to copy data between Kafka clusters via Unix pipelines for quick, simple data replication.
A Developer Advocate shares practical advice on managing time, avoiding burnout, and maintaining well-being while traveling extensively for tech conferences and meetups.
A developer's travel itinerary for tech conferences and meetups focused on Apache Kafka and software architecture for late 2019.
A guide to resetting offsets for Kafka Connect source connectors by modifying data in the Kafka offset topic, using tools like kafkacat.
How to configure a Kafka Connect sink connector to start reading from the end of a topic instead of the beginning.
A guide on troubleshooting and resetting Kafka consumer group offsets when using Replicator to copy data between clusters.
A technical guide on manually migrating Alfred's clipboard history database between Macs using symbolic links and Dropbox.
An updated guide on creating technical diagrams using Concepts app on iPad, detailing a workflow for presentations and talks.
How to manually delete a Kafka Connect connector by writing a tombstone message to its internal config topic using kafkacat.
A technical guide on automating the restart of failed Kafka Connect tasks using bash scripts and the REST API.
A guide to securely externalizing secrets like passwords in Kafka Connect configuration using KIP-297 and the FileConfigProvider.
Explains how to delete a Kafka Connect connector by directly modifying its configuration topic when the REST API is unavailable.
A workaround technique for exploding arrays in ksqlDB using manual indexing, simulating EXPLODE/UNNEST functionality.