We made this hard: over-engineering the web - Josh Brody We made this hard: over-engineering the web | Josh Brody
Back

We made this hard: over-engineering the web

We made this hard

The job hasn’t changed in thirty years. A request comes in. You return some HTML. The browser draws it. Maybe you sprinkle some CSS on top so it doesn’t look like a 1996 GeoCities fever dream, or maybe you do so it does. That’s the whole gig.

Somewhere along the way we convinced ourselves this hard.

It isn’t. It really, really isn’t. And the people paying for our confusion are the ones who can least afford it: the restaurant owner who just wants a menu online, the babysitter trying to put up a contact form, and—maybe most painfully—the indie dev who’s been “almost ready to launch” for two years because they’re still picking between four different ways to host Postgres, which came to mind only having a nightmare post-engineering-blog post they saw on HackerNews.

I want to talk to both of those groups. This isn’t a slight against React or Kubernetes or any other tool that exists for a reason for companies in the 99th percentile of scale. It’s a screed against using them as the default. There’s a difference between “we needed this because we have a real problem” and “we used this because the FAANG blog post said they did.”

And their problems are your problems, just as my problems are your problems, right?

I had a website in 2004

I was ten. It ran on a 100MB shared hosting plan that cost something like four bucks a month. PHP, MySQL, cPanel when it was cool, and an unreasonable amount of confidence because I knew how to run a lemonade stand. It served real traffic! It didn’t fall over! I didn’t know what a load balancer was and it didn’t matter whatsoever, because nobody on earth needed me to have one.

The 2004 version of today’s $4/month would have been a small datacenter, just to serve HTML.

The hardware got absurdly, stupidly better. Disks got faster by something like four orders of magnitude. RAM got cheap (and then expensive so we can make squirrels with horns). CPUs got many. SQLite, running on a laptop, will outperform what most Fortune 500 companies considered “the database tier” in 2008.

And yet shipping a website is harder than it was when I was ten. That makes sense.

What actually changed (spoiler: not much)

Browsers got better. CSS got way better—grid and flexbox alone fixed about eighty percent of what used to require a table-based layout and tears (tables: still dope). HTTPS became free, easy, and automatic. HTTP/2 quietly fixed the things people used to pre-optimize around. Static site generators became a thing so we don’t have partials to include headers and footers. Postgres got good enough to be the only database most people will ever need. SQLite got fast enough that “just use SQLite” is, increasingly, the correct answer.

Almost every meaningful platform change in the last twenty years made things easier. The complexity isn’t coming from the platform.

It’s coming from us.

Web’s complexity industrial complex

Let me walk me through how a small business gets a website in 2026, according to an aspiring entrepreneur who makes widgets, and has a friend who is a junior software engineer at a vendor for Intuit.

WidgetCo person hires InuitVendorJuniorStar, “hey I need something.” “Sure I can do it! Wait what is it?”

Before even getting started, InuitVendorJuniorStar pulls up a Twitter thread they bookmarked because they saw someone was successful, and they’re certain that copying that recipe is how they too can be successful. That thread mentioned Next.js or Remix or whatever framework was hot four months ago and will be embarrassing eighteen months from now. They set up a build pipeline. They add TypeScript. They add ESLint, Prettier, Husky, lint-staged, a commit message linter because god forbid the commit messages be inconsistent. They deploy to Vercel because that’s what the tutorial said. They add a CDN because performance. They add analytics from three vendors. They add a cookie banner because of the analytics. They add a CMS because the client might want to edit text someday. They add a headless CMS because WordPress is so yesterday.

WidgetCo’s site is a restaurant menu. A fucking menu.

The build outputs more JavaScript than the entire site needs to function (it really just needs a hamburger menu), ships hydration logic for a page with zero interactivity, and takes six seconds to load (but it has an animation spinner!) on the kind of phone the actual customer is putting in their pocket. The original brief may as well have been: show people what’s on today’s menu, and add a static Google Maps embed.

The now-half-baked result (note: originally I put product here but changed to result) is a single-page application that requires a service worker to display the address, and it won’t give you directions unless you give it your navigation.location permissions.

This isn’t an exaggeration. I’ve seen it. You’ve seen it. We’ve all seen it. The babysitter’s site has a CDN because her boyfriend heard about one on Discord. The four-location pizza place has Kubernetes—I am not making this up, I have personally seen a pizza place running their WordPress instance with a link to Toast’s ordering system on Kubernetes—and somewhere a cloud provider’s quarterly numbers went up by less than a rounding error.

The patterns we cargo-culted from Netflix and Stripe and Facebook and Google and Amazon don’t apply. Netflix has a different problem. Stripe has a different problem.

The problem was “people should be able to find our phone number and hopefully find us on Google.”

That problem was solved in 1996.

The indie dev trap

Now the other audience:

If you’re an indie dev or a solo founder, your enemy isn’t Kubernetes. Your enemy is subtler. Your enemy is the version of you that opens a new tab and starts reading “Postgres on RDS vs. Supabase vs. Neon vs. self-hosted vs. Planetscale vs. Turso vs. hey-I-launched-this-this-morning-at-11am-on-a-Tuesday-along-side-everything-else-that-launches-at-11am-on-a-Tuesday.

Your enemy is the version of you that’s been “architecting” for six weeks and hasn’t had a single production user.

Here’s the pattern. You have an idea. You get excited. You start a project. And then, instead of building (or continuing to build) the thing, you start optimizing for a scale you do not have and will not have for a long time, possibly ever. You pick a database based on what might happen if you got hugged to death by Hacker News times fifty. You set up a queue for operations that take forty milliseconds. You build a microservice architecture for an app with three endpoints. You spend a week on the deploy pipeline. Then another week on the staging deploy pipeline. Then a weekend on the observability stack, because how would you know if it broke?

It hasn’t broken. In fact it hasn’t done anything because it hasn’t been used by a single human being other than you and maybe your mom. There is nothing to observe, and you don’t even know what to measure.

Pre-optimization feels like you’re doing something so you do it. It even feels like progress. You’re typing and asking reddit. You’re committing and solving problems. You’re learning new tools because you’re being a Real Engineer.

You ain’t doing any productive shit.

You are procrastinating in the most expensive possible way, because if you’re still architecting, you can’t be judged yet. And you’re probably afraid to be judged, because heaven forbid an early adopter finds a bug and tells you for free and continues to use your product anyway because it solves their problem. Procrastinating means nobody can tell you the idea was bad, nobody can tell you nobody wanted it (because you totally built without a customer). You’re safe in the architecture diagram. You’re safe in the stack-shopping loop. The moment you ship, the safety ends and you find out whether you built something anyone cares about.

That’s the real fear. The architecture is just where it hides.

I’ve watched people I like avoid shipping because they couldn’t decide on the right framework (even though they’ve mastered one!), the right database (for their boring-ass ACID transactions!), the right hosting setup (because they thought they needed high-availability off the rip!). The thing that would’ve taken them a weekend to put up on a $4 VPS turned into a year-long Ferret Dress-Up Championship in which the prize is also a ferret.

For whatever reason we refuse to accept that the scale problems we’re worrying about are good problems. They mean people use your fucking thing. You do not have those problems, and you probably won’t because you’re too busy with eighty tabs open comparing providers and reading anecdotes from people you don’t even know who probably thought they were smart by writing down what I learned when in fact they have the same paralysis as you do, or they have demand-driven scale problems and have solved it and now have time to write about it.

Nobody writes about it in the middle of it.

When you do have these scaling problems, you will have users (and hopefully revenue), you will have signal, you will have measure, and you will have time to fix them. And the fix will almost always be embarrassingly boring: add an index, throw money at a bigger box, change your cache key.

It will not be “I should’ve started with Kafka and k8s.”

You can scale a boring monolith on one VPS further than you think. A modest Rails or Django or Phoenix app on a single 4-core box serves tens of thousands of users without sweating (my boring Rails app served at least 33,004 paying users according to the government, and it was not simple). Stack Overflow was lean as hell for years when they mattered. The thing you’re scared of is rarer than you think and easier to solve than you imagine.

Marketing people marketing fear

The marketing teams are the real winners here.

Not the users of the product they market. The users who want pages that load. Not the small business owner—they want their phone to ring. Not the indie dev who hasn’t shipped because they’re still picking a stack.

Cloud providers benefit. Framework maintainers benefit from popularity. The conference circuit benefits. Engineers writing the complexity benefit—it’s more interesting work, it pays better, it looks better on a resume than “I deployed a Rails monolith to a single server and it’s been up for three years.”

I’m not saying any of this is a conspiracy or fraud. It’s not. But it’s an incentive structure, and the incentive structure says: build the technically impressive thing, talk about the technically impressive thing, hire people who can build the technically impressive thing.

The babysitter does not have a seat at that table. Neither does the indie dev who hasn’t shipped.

It’s worth saying out loud anyway. The reason we keep recommending complicated stacks isn’t that they’re correct for the situation. It’s that they’re correct for us.

What you probably actually need

Your taco spot needs cheap shared hosting so you can install WordPress with a click; maybe buy a theme from ThemeForest. If they need a email at their domain (they may sure as well be fine with a Gmail account) most hosting providers have Google Workspace addons so you can manage everything under one roof so they don’t get stuck using an IP with poor reputation by a company whose specialty is not email. So yes, $15/month or whatever; or you can go use Yandex or Zoho or whatever else.

For ninety-nine percent of websites and six-nines percent of pre-launch indie projects, the answer is humiliatingly simple:

  • A $6/month VPS or a small computer under your desk—seriously, a Mac mini under your desk handles a lot
  • Static HTML for the babysitter, or an app in whatever language you already know (Ruby, Python, PHP, Go, Elixir, doesn’t matter)
  • Postgres or SQLite on the same machine as the app, with periodic backups to some probably-S3-compatible storage and a well-tested script to restore that database. 100GB restores in like 10 minutes and you can probably stomach 10 minutes of downtime
  • Nginx or Caddy in front
  • Cloudflare’s free tier if you want a “CDN”, which—sure, why not, it’s free

That’s it. That stack will serve more traffic than you will get. It will be up at 3am save for the datacenter burning down. It will not page you. It will cost less than your coffee. You can build it in a weekend and forget about it for two years.

If you’re an indie dev: ship on this. Get users. Find out if anyone wants the thing. Then worry about the rest.

The real cases, briefly

There are real reasons to use the complicated stuff. Genuine multi-region latency requirements. Regulatory constraints that demand specific architectures. Team sizes where the coordination overhead of a monolith breaks down. Actual scale—the kind where you have so many users that the simple answer stops working.

These are real and they are rare. If you have to wonder whether you’re in this category, you aren’t.

And here’s the part that should be liberating: boring stacks don’t trap you. A Rails app on a single server is the easiest thing in the world to migrate out of, because it’s standard, because it’s understood, because every engineer you’ll ever hire has worked on one. The lock-in stories are about complex setups, not simple ones. You can always add complexity later. It’s much, much harder to remove it.

The job

A request comes in. You return some HTML. The browser draws it.

That’s the web. That’s been the web since 1995. Everything we’ve built on top is optional. Some of it’s even useful, but almost none of it is necessary for what most of us are actually building, and a striking amount of it is being deployed in situations where it makes things measurably worse.

Build the smallest thing that works. Ship it. The scale problems are the good ones—and you don’t have them yet.

Stay in the loop

Occasional essays on design, tools, and the craft of building things. No spam, unsubscribe anytime.

Ambient weather

The background of this site reflects the current weather and time of day in Saint Paul. The orbs shift in color and behavior based on what's happening outside my window.

Learn more about how this works