Sam Newman

So another detailed study into the effectiveness of AI productivity gains for software delivery, and another showing very modest improvements. The latest report from DX is showing around 10% improvement, inline with similar studies: https://lnkd.in/e6rP-wiT

I had hoped since the early studies showed similar numbers that maturity around the tooling and the growth of expertise in how to use these tools well would show a bigger improvement, but so far that doesn't seem to be the case.

To be clear, 10% is not negligible, and will certainly be noticed. However from a cost analysis perspective, this has to be taken into account. None of the firms providing AI coding assistants turn a profit, and are actually burning money at an astonishing rate. So either they have to find a way to deliver the current capabilities at a much lower price point, or they'll have to charge us a lot more to make their business viable. At which point, expect a lot more companies to do the sums on whether or not it's worth paying the price.

File under "software delivery isn't just typing".

"Never let a good crisis go to waste" - Winston Churchill (generally accepted).

Calling the recent AWS outage a crisis is overstating it, but amidst all the blame flinging and hand wringing about the interconnected nature of our connected world, it would be remiss not to use the problems experienced by AWS's venerable US-EAST-1 region as an opportunity for some reflection.

Much of the discourse has centered on "is everyone is too reliant on AWS?". This is an easy discussion to start, but misses important nuance. Really, the more vital discussion to have is "are we too reliant on a specific AWS region?".

If your critical functionality is hosted entirely out of a single AWS region, you are vulnerable to that region going down. If you rely on third-party services who are in themselves reliant on a single AWS region, you are at risk if that AWS region goes down.

In the wake of this, many people will push for multi-cloud approaches. This comes at significant cost, and a decision to do this is hedging against a different set of risks than the outage yesterday. Delivering the same solution on a multi-cloud basis (different vendors) is more expensive than delivering the same solution on a multi-region basis (same vendor).

With either the multi-vendor or single-vendor, multi-region approach, when coping with a region failure, you have to solve issues around latency, cost, and data replication. With multi-vendor, you also have to add the cost of working with another cloud vendor, and all that entails.

Fundamentally, as you increase isolation in any system, you increase cost and complexity. (Blatant plug: I've written this up as part of the "Distribute Across Failure Planes" pattern in new book: https://lnkd.in/eeGxGf7r - this chapter will be available in early access soon). Often, this increased cost and complexity is justified. The case doesn't always stack up though as your isolation increases - hence the reason why so many big names appear to have made a conscious decision to not deliver a multi-region solution.

This might be worth it to you. But it's important that you understand what risk you are hedging with these two different approaches. When it comes do dealing with outages, with a multi-region approach, you are hedging the risk of a single region failing. With a multi-cloud approach, you are hedging the risk of an entire vendor failing. There are also arguments in favour of multi-cloud to ensure you can be held hostage in terms of price increases and the like, but in this context that is a separate discussion.

If you are considering a multi-vendor approach for your system, you'll have to solve all the same problems as you do for a single vendor, multi-region setup. It might make sense to walk before you can run.

As for asking cloud vendors to make multi-region solutions easier to implement, just be careful what you wish for. That is definitely a topic for a separate post.

In the wake of AWS's US-EAST-1 outage earlier this week, we probably need to update Leslie Lamport's famous definition of distributed systems:

“A distributed system is one in which the failure of a computer you didn't even know existed can render your own 𝚌̶𝚘̶𝚖̶𝚙̶𝚞̶𝚝̶𝚎̶𝚛̶ mattress unusable.”

https://lnkd.in/e-rXYZJU

Talking confidently without knowing if they are right, frequently denying that they made mistakes, taking way too long to learn from what they got wrong, and often requiring you to change how you work to fit their needs.

If you really did treat your GenAI coding agent like a junior developer, you'd have fired them by now.