Fabric RTI 101: Handling Backpressure
Learn how to handle backpressure in real-time data systems with strategies like buffering, scaling, and prioritization.
Greg Low is a long-time Microsoft Data Platform MVP and Regional Director, sharing expert insights on SQL Server, Azure SQL, Microsoft Fabric, and real-time data architecture through practical articles and interviews.
34 articles from this blog
Learn how to handle backpressure in real-time data systems with strategies like buffering, scaling, and prioritization.
Explore cost considerations for streaming vs batch in Microsoft Fabric RTI, including compute, retention, and hybrid pipeline strategies.
Troubleshooting SQL Server error 'Failed to update the database because the database is read-only', caused by availability group failover and connection to wrong server.
An analysis of flat adverbs (e.g., 'drive safe') and the debate over the 'ly' suffix in modern English usage.
Learn best practices for handling retries in real-time streaming systems, including exponential backoff, idempotency, poison queues, and logging.
Learn how to optimize cost and performance in Microsoft Fabric RTI by selective streaming, pre-aggregation, sampling, and logical data tiering.
Explores surprising SQL Server behavior where TRY_CAST and TRY_CONVERT return non-NULL values for empty strings when casting to certain types, with a type-by-type analysis.
Explore SQL Server's CONCAT and CONCAT_WS functions for concatenating column values with delimiters, handling NULLs, and simplifying string aggregation.
Learn how to design fault-tolerant real-time data pipelines using checkpoints, partition replicas, stateless consumers, and failure testing.
An expert's guide to scaling SQL Server performance, focusing on reducing database calls and improving application design.
A detailed review of 'Causal Inference with Bayesian Networks', covering its content, practical examples, and suitability for data scientists.
Explains replay and reprocessing in Microsoft Fabric RTI, covering event persistence, Delta Lake time travel, and use cases like debugging and audits.
Many real-time data projects fail not because of the technology, but because of design assumptions and unnecessary complexity. One of the most common
An opinion piece on why IT teams should avoid hiring clones of themselves, emphasizing the value of diverse backgrounds and perspectives in tech and development.
Opinion piece criticizing overly restrictive IT censorship that blocks developers from doing their jobs, urging companies to treat staff as adults.
Security and governance are just as important in real-time streaming systems as they are in traditional batch pipelines — and in some cases, even more
Using SQL Server database snapshots to provide large unit testing data with quick restores, avoiding transaction rollback limitations.
Learn key design principles for building reliable and resilient real-time streaming systems, including redundancy, failover, durability, and monitoring.
Compares two methods for bulk reloading SQL Server clustered columnstore indexes, recommending drop/recreate for best performance and packing.
Learn key strategies for designing scalable real-time architectures, including partitioning, hot/cold path separation, bottleneck identification, and realistic load testing.