Hussein Nasser

Shopify increased their write throughput by 50% moving one of their MySQL keys from UUIDv4 to ULID.

But How?

When you make a purchase, Shopify guarantees idempotency by having the client uniquely tagging the request using UUID. If you accidentally replay the request and they check if the request exists or not and no-op.

UUIDs (at least version 4) are random while order placements are ordered, this leads to thrashing pages in and out of the buffer pool as it fills up dramatically slowing down writes.

Shopify replaced their primary key to ULID which is ordered and timestamped so now requests are chronologically ordered and as a result requests in a given time period live almost in the same pages and recent request (which is the common query) live in tail pages keeping those hot in memory.

Also worth mentioning that UUIDv7 and 8 are ordered and work as well, this is specific to UUIDv4.

Watch my full coverage on this amazing change, truly show engineers knowing what they are doing

The topic of UUID is surfacing again so I thought I’d share this again

https://lnkd.in/gdUwpDEU


—-
Get my fundamentals of database engineering to learn low level database internals https://databases.win

In interviews, I like to gauge the candidate’s knowledge and interest in certain topics. I found that a good conversational question for backend engineers is this:

what happens when the web server crashes while processing a request which has already connected to a database server, started a database transaction and sent few queries but didn’t yet commit?

It is a beautiful question that can lead to exploring backend communication protocols and networking.

And depending on how that goes I may change the discussion towards database internals by flipping the question.

What happens when the database server crashes while it’s in a middle of a transaction sent from a webserver server (backend server still running)?

I purposely don’t specify the database or protocol, this provides vast room for exploration of possibilities and it is what I enjoy the most.

Code with fewer lines is not always simple code.\n\nFewer lines of code often indicate an abstraction.\n\nAbstractions are not inherently simple; they create the illusion of simplicity.\n\nThey don’t remove complexity—they hide it.\n\nThis hidden complexity can cause engineers to lower their guard.\n\nEventually, abstractions leak this complexity to end users in unexpected ways:\n\nOften in the form of performance issues, latency, or high CPU and memory usage.\n\nDeep down, we know this.\n\nWrap a large block of code in a function call, and the client code appears simple.\nBut we know it’s not.\n\nPut that function in a library and ship it.\nNow, only you know the true nature of the code.\nThe developers using it don’t.\n\nThose devs build more software and libraries on top of it, creating further illusions of simplicity.\n\nThe same applies to frameworks, client libraries, APIs or even languages.\n\nThe best performance comes from fully understanding the intricacies and complexities of a system—\nAnd tuning them to your specific use case.\n\nAbstractions are most powerful when we understand what lies behind them.\n\nUse them, but never let your guard down.