🎄 Twelve Days of SMT 🎄 - Day 6: InsertField II
Day 6 of a series on Kafka Connect Single Message Transforms, focusing on using InsertField to add static values and Kafka metadata to messages.
Robin Moffatt is a Principal DevEx Engineer and seasoned conference speaker with 15+ years of experience presenting at top events like QCon, Devoxx, Kafka Summit, and Strata. He shares insights on developer experience, distributed systems, and cloud technologies through his blog, YouTube, and public talks.
617 articles from this blog
Day 6 of a series on Kafka Connect Single Message Transforms, focusing on using InsertField to add static values and Kafka metadata to messages.
Explains how to use Kafka Connect's MaskField Single Message Transform to mask sensitive data fields like credit card numbers during data ingestion.
Explains how to use Kafka Connect's RegExRouter SMT to rename topics for source connectors and target objects for sink connectors.
Explains how to use the Flatten Single Message Transform (SMT) in Kafka Connect to convert nested JSON data into a flat structure for database insertion.
Explains how to use Kafka Connect's ValueToKey and ExtractField Single Message Transforms to set message keys from data fields.
Explains how to use Kafka Connect's InsertField Single Message Transform (SMT) to add message timestamps to data sent to a sink.
A Developer Advocate reflects on the impact of the pandemic and the rise of virtual conferences on the DevRel role, sharing speaker insights and challenges.
A developer shares their home office workstation setup, including hardware, peripherals, and software tools used for development and remote talks.
Explains why the 'Replace Fonts' option is disabled in Apple Keynote and provides a simple solution to fix it.
A technical guide on using Kafka tombstone messages with Kafka Connect and ksqlDB to handle logical deletions from a PostgreSQL source to Elasticsearch.
Tutorial on streaming geolocation data from Kafka to Elasticsearch, covering mapping and data formatting using ksqlDB.
Explains how to model variable fields in ksqlDB using MAP instead of STRUCT for nested data, with practical examples.
A technical guide on streaming XML data from IBM MQ to Kafka, transforming it, and then loading it into MongoDB using Kafka Connect.
Explores using the Kafka Connect FilePulse connector to ingest and process XML data into Apache Kafka, including configuration and troubleshooting.
A technical guide on using Kafka Connect with Single Message Transform to ingest and process XML data into Apache Kafka.
A technical guide on using Unix pipelines with curl, xq, and kafkacat to ingest and transform XML data into JSON for Kafka.
Introduction to ingesting XML data into Kafka, covering challenges and approaches for handling XML schemas in Kafka topics.
Fixing abcde CD ripping error due to permissions by adding user to cdrom group or using chmod.
Troubleshooting guide for resolving 'Channel was blocked' errors when connecting a client to IBM MQ running in a Docker container.
A technical guide on setting Kafka message keys when piping JSON data from jq to kafkacat, using a separator character.