A quick and dirty way to monitor data arriving on Kafka
A hacky method to monitor Kafka data arrival using kafkacat and Telegram alerts when message timestamps exceed a threshold.
Robin Moffatt is a Principal DevEx Engineer and seasoned conference speaker with 15+ years of experience presenting at top events like QCon, Devoxx, Kafka Summit, and Strata. He shares insights on developer experience, distributed systems, and cloud technologies through his blog, YouTube, and public talks.
617 articles from this blog
A hacky method to monitor Kafka data arrival using kafkacat and Telegram alerts when message timestamps exceed a threshold.
Explores the future of tech conferences post-COVID, analyzing their value, online alternatives, and the tension between commercial and community-driven events.
A technical guide on capturing Wi-Fi trace data with a Raspberry Pi and streaming it to Apache Kafka using Confluent Cloud.
How to configure the Kafka Connect JDBC Sink connector's pk.fields setting to use a Kafka message's primitive key for UPSERT operations into a PostgreSQL table.
A developer advocate explores using Docker Compose files to deploy a multi-container demo application on AWS Elastic Container Service (ECS).
ksqlDB 0.7 adds support for primitive data types as message keys, simplifying data joins and reducing the need for pre-processing.
A quick fix for Fantastical or Mac Calendar not displaying Google Shared Calendars by adjusting sync settings.
A technical guide on configuring Kafka Connect to stream data from Kafka topics into InfluxDB, covering schema requirements and connector setup.
Explains a Kafka Connect ClassCastException error related to schemas and provides solutions for fixing JSON converter configuration.
A technical guide to monitoring Sonos device health by streaming diagnostics data through Kafka, ksqlDB, InfluxDB, and visualizing with Grafana.
How to fix the Java UnsupportedClassVersionError when building a Docker image for ksqlDB by switching to JDK 8.
Explains how to dynamically change Kafka Connect logging levels using its admin API, enabling detailed debugging without restarting workers.
A guide on writing effective conference talk abstracts to increase acceptance chances, based on experience reviewing hundreds of submissions.
A technical exploration of ksqlDB window start times, including creating streams, inserting test data, and querying windowed aggregations.
A technical guide on integrating RabbitMQ with Kafka using Kafka Connect, including setup, schema handling, and use cases.
A technical guide on using ksqlDB and MongoDB to analyze network syslog data from UniFi devices, including Kafka stream processing.
A technical guide on using ksqlDB and Kafka Connect to ingest and analyze Syslog data for detecting SSH attacks.
A technical guide on using SSH and Linux pipes to stream and copy a MongoDB collection from a remote server to a local Docker instance.
Troubleshooting a Kafka Connect 'Request timed out' error caused by a misconfigured offsets.topic.replication.factor setting.
A guide on how to use tcpdump for network troubleshooting within Docker containers by building a custom image and attaching to container networks.