View
Semantic Search: Fwd: How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes
Fwd: How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes
Score 0.898
· Account tjphuhs@gmail.com
· 3/17/2026, 11:46:27 AM
Fwd: How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes from tjphuhs@gmail.com on 2026-03-17T15:46:27.000Z logical cluster. The Cruise Control topic was overridden to stay consistent across both broker sets, allowing Reddit to use Cruise Control to move data between EC2 and Kubernetes brokers. Cruise Control is a Kafka tool that automates the process of rebalancing data across brokers in a controlled, measured way. It was central to the actual movement of data during the migration.
Fwd: How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes
Score 0.897
· Account tjphuhs@gmail.com
· 3/17/2026, 11:46:27 AM
Fwd: How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes from tjphuhs@gmail.com on 2026-03-17T15:46:27.000Z an open-source platform for running and managing containerized applications. Instead of manually provisioning and maintaining individual servers, Kubernetes lets developers describe what should be running and handles deployment, scaling, and recovery automatically. Strimzi, on the other hand, is a project under the Cloud Native Computing Foundation that specifically lets you ru
Fwd: How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes
Score 0.891
· Account tjphuhs@gmail.com
· 3/17/2026, 11:46:27 AM
Fwd: How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes from tjphuhs@gmail.com on 2026-03-17T15:46:27.000Z was comparatively straightforward. Phase 6: Cleaning Up and Handing Off to Standard Strimzi After both the data plane and the control plane were fully running on Kubernetes, Reddit removed all the configuration overrides that the forked Strimzi operator had introduced. Control of the clusters was handed off to the standard, unmodified Strimzi operator. The EC2 infrastructure wa
Fwd: How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes
Score 0.890
· Account tjphuhs@gmail.com
· 3/17/2026, 11:46:27 AM
Fwd: How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes from tjphuhs@gmail.com on 2026-03-17T15:46:27.000Z and it meant that riskier changes had to wait until everything else was stable. Phase 1: Taking Control of the Naming Layer The first phase of the migration did not touch Kafka at all. Reddit introduced a DNS facade, which is a set of DNS records that act as an intermediate layer between client applications and the actual Kafka brokers. DNS is the system that translates human-r
Fwd: How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes
Score 0.886
· Account tjphuhs@gmail.com
· 3/17/2026, 11:46:27 AM
Fwd: How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes from tjphuhs@gmail.com on 2026-03-17T15:46:27.000Z analysis tools might miss. CodeRabbit reviews 1 million PRs every week across 3 million repositories and is used by 100 thousand Open-source projects. CodeRabbit is free for all open-source repo’s. Get Started Today The Reddit Engineering Team completed one of the most demanding infrastructure migrations in the company’s history. It moved its entire Apache Kafka fleet, comprisi
Fwd: How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes
Score 0.867
· Account tjphuhs@gmail.com
· 3/17/2026, 11:46:27 AM
Fwd: How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes from tjphuhs@gmail.com on 2026-03-17T15:46:27.000Z TJ Begin forwarded message: From: ByteByteGo <bytebytego@substack.com> Date: March 17, 2026 at 11:43:14 AM EDT To: tjphuhs@gmail.com Subject: How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes Reply-To: ByteByteGo <reply+357rw3&4iwoe6&&457f75760bc411accd07b85c686060610590a8b4419f12cd9695d552c12ad6be@mg1.substack.com> How Reddit Migrated Petabyte-Scale Kafka from EC
How Stripe Detects Fraudulent Transactions Within 100 ms
Score 0.629
· Account tjphuhs@gmail.com
· 4/27/2026, 11:03:53 PM
How Stripe Detects Fraudulent Transactions Within 100 ms from bytebytego@substack.com on 2026-04-28T03:03:53.000Z advanced, they moved to more complex architectures. Each jump produced an equivalent leap in model performance. The architecture preceding the current one was called Wide & Deep. It combined two models into an ensemble. The “wide” component was XGBoost, a gradient-boosted decision tree that works by combining many small decision trees into one powerful predictor. XGBoost excelled at
How Pinterest Built a Production MCP Ecosystem
Score 0.623
· Account tjphuhs@gmail.com
· 5/11/2026, 11:31:17 AM
How Pinterest Built a Production MCP Ecosystem from bytebytego@substack.com on 2026-05-11T15:31:17.000Z at how Pinterest designed that ecosystem and what they had to get right beyond the protocol itself. Disclaimer: This post is based on publicly shared details from the Pinterest Engineering Team. Please comment if you notice any inaccuracies. What is MCP Model Context Protocol (MCP) is an open-source standard that gives large language models a unified way to talk to external tools and data sour
How Pinterest Built a Production MCP Ecosystem
Score 0.623
· Account tjphuhs@gmail.com
· 5/11/2026, 11:31:17 AM
How Pinterest Built a Production MCP Ecosystem from bytebytego@substack.com on 2026-05-11T15:31:17.000Z to act on the user’s behalf. This produces a JWT (JSON Web Token), a small signed token that encodes the user’s identity and group memberships. That JWT travels with every subsequent request. Before a request reaches any MCP server, it passes through Envoy, a network proxy that sits in front of every service in Pinterest’s infrastructure. Envoy validates the JWT by checking the signature and e
How Pinterest Built a Production MCP Ecosystem
Score 0.619
· Account tjphuhs@gmail.com
· 5/11/2026, 11:31:17 AM
How Pinterest Built a Production MCP Ecosystem from bytebytego@substack.com on 2026-05-11T15:31:17.000Z engineering team responded by building a unified deployment pipeline. Teams define their tools, and the platform handles deployment, scaling, and infrastructure. This turned what had been a multi-day setup process into something where domain experts could focus entirely on their business logic. Without this investment, the bet around many small servers would have collapsed under its own operat
How Pinterest Built a Production MCP Ecosystem
Score 0.618
· Account tjphuhs@gmail.com
· 5/11/2026, 11:31:17 AM
How Pinterest Built a Production MCP Ecosystem from bytebytego@substack.com on 2026-05-11T15:31:17.000Z experimentation, but the so-called paved path at Pinterest is to write a server, deploy it to their cloud compute environment, and register it in their central catalog. Every tool call becomes a network request, which adds latency compared to a local server. However, centralizing servers in the cloud meant that Pinterest could apply consistent authentication, authorization, logging, and monito
How Pinterest Built a Production MCP Ecosystem
Score 0.616
· Account tjphuhs@gmail.com
· 5/11/2026, 11:31:17 AM
How Pinterest Built a Production MCP Ecosystem from bytebytego@substack.com on 2026-05-11T15:31:17.000Z data. Why two layers instead of one? Envoy’s policies are fast, network-level checks that block obviously unauthorized traffic before it reaches any application code. The tool-level decorators handle nuanced, business-logic-specific permissions that a network proxy is not equipped to reason about. Together, they provide defense in depth. Even if one layer has a misconfiguration, the other stil
"Why?"
Score 0.614
· Account tjphuhs@gmail.com
· 5/10/2026, 8:35:50 PM
"Why?" from noreply@redditmail.com on 2026-05-11T00:35:50.000Z r/MaineCoon Hide r/MaineCoon View More Posts This email was intended for u/FunDirection5847. Unsubscribe from daily digest messages, or visit your settings to manage what emails Reddit sends you. 5‌4‌8 M‌a‌rket S‌t., #16093, San Francisco, CA 9‌4‌1‌0‌4-5‌4‌0‌1
How Figma Upgraded Data Pipeline from Multi-Day Latency to Real-Time
Score 0.611
· Account tjphuhs@gmail.com
· 5/12/2026, 11:31:03 AM
How Figma Upgraded Data Pipeline from Multi-Day Latency to Real-Time from bytebytego@substack.com on 2026-05-12T15:31:03.000Z is based on publicly shared details from the Figma Engineering Team. Please comment if you notice any inaccuracies. When SELECT * Becomes Your Bottleneck Figma’s original data pipeline did what’s called a full sync. Every run copied the entire contents of a database table, regardless of how much had actually changed since the last run. If a table had ten million rows and
How Pinterest Built a Production MCP Ecosystem
Score 0.611
· Account tjphuhs@gmail.com
· 5/11/2026, 11:31:17 AM
How Pinterest Built a Production MCP Ecosystem from bytebytego@substack.com on 2026-05-11T15:31:17.000Z View this post on the web at https://blog.bytebytego.com/p/how-pinterest-built-a-production Agents need context. Ship the integrations that give it to them. (Sponsored) [ https://substack.com/redirect/87dc365a-cf87-4640-8889-81a07e071928?j=eyJ1IjoiNGl3b2U2In0.sVDxRtmZ85v8kfdamY0krRXGMy3p768BWtuZifRB-Zs ] The context that actually matters isn't in your database. It's in the tools your users liv
How Figma Upgraded Data Pipeline from Multi-Day Latency to Real-Time
Score 0.610
· Account tjphuhs@gmail.com
· 5/12/2026, 11:31:03 AM
How Figma Upgraded Data Pipeline from Multi-Day Latency to Real-Time from bytebytego@substack.com on 2026-05-12T15:31:03.000Z capture only what changed and apply those changes to the destination. The concept is simple, but the execution is not. Incremental Synchronization Incremental synchronization flips the model. Rather than asking “what does the whole table look like right now?” it asks “what changed since last time?” Only the inserts, updates, and deletes since the last sync get transferred
High Performance Rate Limiting at Databricks
Score 0.609
· Account tjphuhs@gmail.com
· 5/13/2026, 11:30:35 AM
High Performance Rate Limiting at Databricks from bytebytego@substack.com on 2026-05-13T15:30:35.000Z earlier rejection rate mechanism became unnecessary, and the team eventually converted every rate limit in the system to a token bucket. Three Coupled Decisions The Databricks story resolves into three decisions that depend on each other: The first is the algorithm, which determines how the counter behaves at the boundaries of time intervals. Fixed window, sliding window, and token bucket each p
"Why?"
Score 0.609
· Account tjphuhs@gmail.com
· 5/10/2026, 8:35:50 PM
"Why?" from noreply@redditmail.com on 2026-05-11T00:35:50.000Z 96 Reddit r/mainecoons: Why? Good thing I looked. ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏‌ ͏&zwn
How Pinterest Built a Production MCP Ecosystem
Score 0.607
· Account tjphuhs@gmail.com
· 5/11/2026, 11:31:17 AM
How Pinterest Built a Production MCP Ecosystem from bytebytego@substack.com on 2026-05-11T15:31:17.000Z At the ecosystem level, Pinterest tracks the number of registered servers and tools, invocation counts across all servers, and a north-star metric that rolls everything up into a single number. That number is the time saved. For each tool, server owners provide a “minutes saved per invocation” estimate, based on lightweight user feedback and comparison to the prior manual workflow. Multiplied
How Stripe Detects Fraudulent Transactions Within 100 ms
Score 0.607
· Account tjphuhs@gmail.com
· 4/27/2026, 11:03:53 PM
How Stripe Detects Fraudulent Transactions Within 100 ms from bytebytego@substack.com on 2026-04-28T03:03:53.000Z overnight jobs could now run multiple times in a single working day. Stripe is now exploring techniques that this architectural shift made possible, including multi-task learning, where a single model is trained to handle several related objectives simultaneously. [Live on May 6] Stop babysitting your agents (Sponsored) [ https://substack.com/redirect/a95b6182-9924-4322-9ece-ce3923bf
Fwd: Synergy Over Maximalism: The Part of Stacking Nobody Talks About
Score 0.605
· Account tjphuhs@gmail.com
· 3/25/2026, 12:17:01 PM
Fwd: Synergy Over Maximalism: The Part of Stacking Nobody Talks About from tjphuhs@gmail.com on 2026-03-25T16:17:01.000Z ---------- Forwarded message ---------- From: Derek from Peptide Price <derekpruski@substack.com> Date: Mar 25, 2026 at 11:46 AM -0400 To: tjphuhs@gmail.com Subject: Synergy Over Maximalism: The Part of Stacking Nobody Talks About > A lot of you have seen the energy protocol. > ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Fwd: Synergy Over Maximalism: The Part of Stacking Nobody Talks About
Score 0.605
· Account oc.tjphuhs@gmail.com
· 3/25/2026, 12:17:01 PM
Fwd: Synergy Over Maximalism: The Part of Stacking Nobody Talks About from tjphuhs@gmail.com on 2026-03-25T16:17:01.000Z ---------- Forwarded message ---------- From: Derek from Peptide Price <derekpruski@substack.com> Date: Mar 25, 2026 at 11:46 AM -0400 To: tjphuhs@gmail.com Subject: Synergy Over Maximalism: The Part of Stacking Nobody Talks About > A lot of you have seen the energy protocol. > ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
How Pinterest Built a Production MCP Ecosystem
Score 0.605
· Account tjphuhs@gmail.com
· 5/11/2026, 11:31:17 AM
How Pinterest Built a Production MCP Ecosystem from bytebytego@substack.com on 2026-05-11T15:31:17.000Z is invisible. Pinterest also embeds AI bots in its internal communication platform, and these bots expose MCP tools as well. Auth is handled through the registry API, just like the web interface. These bots support context-aware tool scoping, meaning certain MCP tools are restricted to certain channels. Spark MCP tools, for example, only appear in Airflow support channels. This keeps tool list
EP214: Claude Code vs. OpenClaw: 5 Design Dimensions
Score 0.601
· Account tjphuhs@gmail.com
· 5/9/2026, 11:31:10 AM
EP214: Claude Code vs. OpenClaw: 5 Design Dimensions from bytebytego@substack.com on 2026-05-09T15:31:10.000Z Git can’t know which version is correct, so a revert conflict is triggered. How to resolve it: 1. Run git revert C2 2. Git pauses when it hits the conflict 3. You manually fix the file 4. Stage it 5. Continue the revert Git then creates a new commit that cleanly undoes C2 while keeping C3 intact. Over to you: Have you ever hit a revert conflict at the worst possible moment? How did you r
"Why buy cat trees?!?"
Score 0.600
· Account tjphuhs@gmail.com
· 5/9/2026, 8:49:30 PM
"Why buy cat trees?!?" from noreply@redditmail.com on 2026-05-10T00:49:30.000Z upvotes • 2 comments Hide r/IKEA Hide r/IKEA View More Posts This email was intended for u/FunDirection5847. Unsubscribe from daily digest messages, or visit your settings to manage what emails Reddit sends you. 5‌4‌8 M‌a‌rket S‌t., #16093, San Francisco, CA 9‌4‌1‌0‌4-5‌4‌0‌1