Posts for Architecture

Building systems beyond our ability to intellectually manage

Quote - Joe Campbell - October 4, 2017

One of the smartest articles on software development and what it means that I’ve read:

Software is different. Just by editing the text in a file somewhere, the same hunk of silicon can become an autopilot or an inventory-control system. This flexibility is software’s miracle, and its curse. Because it can be changed cheaply, software is constantly changed; and because it’s unmoored from anything physical—a program that is a thousand times more complex than another takes up the same actual space—it tends to grow without bound. “The problem,” Leveson wrote in a book, “is that we are attempting to build systems that are beyond our ability to intellectually manage.”

Our standard framework for thinking about engineering failures—reflected, for instance, in regulations for medical devices—was developed shortly after World War II, before the advent of software, for electromechanical systems. The idea was that you make something reliable by making its parts reliable (say, you build your engine to withstand 40,000 takeoff-and-landing cycles) and by planning for the breakdown of those parts (you have two engines). But software doesn’t break…[In the case of major failures such as with Intrado:] The software did exactly what it was told to do. In fact it did it perfectly. The reason it failed is that it was told to do the wrong thing. Software failures are failures of understanding, and of imagination…

This is the trouble with making things out of code, as opposed to something physical. “The complexity,” as Leveson puts it, “is invisible to the eye.”

James Somers on The Coming Software Apocalypse in the Atlantic.

Continue Reading

Continue Reading

A product is useless without a platform

Quote - Joe Campbell - September 26, 2017

I came across an accidentally publicly shared Google Plus post by Steve Yegge years ago in which he described one of Amazon’s turning points…

[One day Jeff Bezos who] makes ordinary control freaks look like stoned hippies…issued a mandate…

1) All teams will henceforth expose their data and functionality through service interfaces.

2) Teams must communicate with each other through these interfaces.

3) There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.

4) It doesn’t matter what technology they use. HTTP, Corba, Pubsub, custom protocols — doesn’t matter. Bezos doesn’t care.

5) All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.

6) Anyone who doesn’t do this will be fired…

There are without question pros and cons to the SOA approach, and some of the cons are pretty long. But overall it’s the right thing because SOA-driven design enables Platforms…

A product is useless without a platform, or more precisely and accurately, a platform-less product will always be replaced by an equivalent platform-ized product.

My excerpts are shit. Taking so much of the personal observations out of it. Read the whole thing. It’s an incredibly insightful piece of writing. A few years later, I read another angle of this from Brad Stone in his The Everything Store:

In the engineering department, employees were constantly trying to fix a technical infrastructure that was now an aging, sprawling mess. The company had outgrown the original framework devised by Shel Kaphan in the 1990s, the monolithic code base dubbed Obidos that for years was held together by what Amazon executive Werner Vogels later called “duct tape and WD40 engineering.” And when Amazon cloned its clunky code base to run the websites of Target and Borders, those deals were lucrative but they magnified the company’s infrastructure problems. Instead of fighting flames emanating from a single building, engineers often had to deal with a neighborhoodwide inferno.

Like a lot of other technology companies at the time, Amazon got an education in the wisdom of moving to a simpler and more flexible technology infrastructure, called service-oriented architecture. In this kind of framework, every feature and service is treated as an independent piece and each can easily be updated or replaced without breaking the whole.

Led by Amazon’s chief technology officer at the time, an avid pilot named Al Vermeulen, whom colleagues fondly called Al V., the company rebuilt its technology infrastructure as a series of these independent but interconnected parts. The awkward and extended transition to this new code base, one element of which Amazon called Gurupa (after a section of the Amazon river where the tributaries diverged), took over three years and caused all kinds of excruciating pain…

Brad Stone in The Everything Store (pp. 201-202). Little, Brown and Company. Kindle Edition.

This type of pivot is tough for any company to make. And it’s really only possible in a company that values technology. For in a company that doesn’t, the next day’s sales take priority.

Continue Reading