Get notified about Kevlin Henney

Sign up to a email when Kevlin Henney publishes a new video

right good morning so there's a bunch of
things that people don't generally say
as you left me with the legacy systems
thing I do there is the question why do
systems become legacy what are the
properties of a legacy system and this
is actually something I run in workshops
I ask people what what characterizes
legacy and I get people to list off
properties of legacy systems because
there are many things that qualify as
legacy we actually hide behind the word
different things to people using that
term can mean quite different things we
can be talking about a system that is
just using old technology that we don't
like we can use we can talk about the
word legacy to mean code that somebody
else wrote that was not me that's one of
the most popular definitions but one of
people sort of invariably end up with is
at some point there's a kind of sense of
size and I'm going to pick on small
today just of all the many things that
we could talk about I'm going to talk
about size issue sometimes that relates
to simplicity not always
so I'm pre-empting the fact that
sometimes people come up to me and say
yeah but you didn't talk about
simplicity or you didn't talk that's
fine that's you have when you do a talk
you're supposed to stick to kind of one
thing but I will wander around quite a
lot physically and intellectually
hopefully but people don't generally say
oh yeah that legacy system it's so small
they don't complain of like oh yeah the
legacy system whose code base is about
the right size that's that there's never
a sense there is always a sense of
there's too much stuff and that's an
enduring idea and so there's a question
of sustainability so I've shamed lessly
stolen the title in fact the inspiration
ultimately for this talk comes from this
book here and small is beautiful
published in the early 70s 97
three ants schumacher a an economist who
effectively created what we now think of
as a more responsible form of economics
the idea that you don't operate in a
vacuum and that perhaps there's a
question of sustainability and so and
it's a it's a lovely old book and I went
back to it again recently it's kind of
interesting seeing the view from the
1970s now that we're in the 21st century
and how much he got wrong but also how
much he got right and there's a subtitle
here a study of economics as if people
mattered so I can steal that I'm going
to go with small is beautiful and you
know by the way if you don't have kids
yet give them an Internet unique name
they will thank you at some point okay
my parents managed to do this before the
internet existed which is a major act of
pressings on their part but you have no
excuse I've done this for my children
okay I did this when they were born
I googled just to make sure that there
were no hits on first name last name and
you know when my older son joined
Twitter dad what name shall I use I said
just try a first name last name like I
do he says oh it's available I said yeah
that moment was thirteen years in the
making so you you can offer this service
to your children okay however a talk on
codes that people mattered this is an
interesting one one of the most amazing
things that we've discovered in the last
decade or so it turns out software
developers are human beings I know I
mean how did that happen but what that
means is that human beings have a
particular set of properties comfort how
they interact with one another that gets
reflected in the code we often try to
think of the code of somehow separated
from the people that work on it and this
also tells us about why things get out
of control and get bit but there's also
another aspect talk on codes of
economics mattered because there is this
idea that people are always chasing a
deadline there's always a reason why
they're doing something the way that it
is well
sometimes it's an excuse not a reason
but it's worth actually kind of pulling
this back and saying hang on does this
actually make sense is this the right
way to invest in development and do
things so that's the kind of the bigger
now I'm out of all this I've mentioned
the word sustainable development and
kind of Schumacher's work ultimately led
to this observation in the Brundtland
report headed by the former Norwegian
Prime Minister NGO harlem brundtland
in the late 80s that was looking at
sustainable agriculture sustainable
economies and so on and I'm doing
something far less interesting here I'm
talking about software but the
definition of sustainable development is
an interesting one because it's
something we can appropriate so when
eventually the words agile and lean lose
steam then perhaps we can use
sustainable as a way of kind of saying
the same thing but giving it a new label
sustainable development this development
that meets the needs of the present
without compromising the ability of
future generations to meet their own
needs so they'll immediately we see that
the challenge of development is to
strike this balance if you never have to
worry about future generations then
that's that's fine you can just put out
anything and you're done but normally we
do have to worry about future
generations even if it's not ourselves
there's normally somebody else who has
to pick up the code there's normally a
release - what's that going to look like
and that becomes the question it's not
but people are so focused on initial
release and first release and then the
next release becomes another first
release without realizing there's a
whole procession of releases and maybe
that this is a system of releases emini
there's a better way to approach this on
the other hands we end up with sometimes
architecture the big enterprise projects
the global projects I went through a
phase of noticing and visiting companies
either workshops or even contract work
when I noticed that the word global kept
appearing in their project names or
enterprise and when a project acquires
that name you know it's doomed okay it's
just like this is going to be
No so that's the point is that that's
the future that will never arrive
there's no present there so this is
balance and the balance is the bit
that's hard so let's explore this a
little bit that's first of all start out
with the idea that not everything is
going to be small but that everything
has an appropriate scale there is a
right size for certain things the
question is are we able to get to that
size this is ultimately an optimization
problem here is the system you would
like and here's the system you've got
how different are those not just in
terms of the physical sizing of things
like the source code which we're going
to come to as being something that you
need to put in your head but actually in
terms of the development effort um so we
can get a real sense of this with a bit
of Shakespeare and C soon as we're in
Denmark I will choose Hamlet Prince of
classic to be or not to be that is the
question this is fairly classic how
would we approach this if we were
writing this in marketing speak or
business speak okay well we have an
answer to this question Tom Burton and
his book long words bother me how'd it
go continuing existence or cessation of
existence those of the scenarios is it
more empowering mentally to work towards
an accommodation of the downsizings and
negative outcomes of fzero circumstance
or would it be a greater etc etc and you
know what a lot of developers write like
this as well actually they write like
crazy ideas about how to lay out their
code you know it's just says oh look
I've got an IMAX screen so I will push
everything out over here and I will use
really long names to be really helpful
as we see the bigger the word it's not
necessarily more impressive you end up
with homeopathic naming where you add
more and more to the identifier names
meaning further and further into
there's absolutely nothing left so let's
go back to this question of constraint
and stick with Shakespeare for a moment
this is the monstrosity and Lovelady
that the will is infinite the execution
can find that the desire is boundless in
the actor slave to limit so there is
this idea that human imagination is
pretty cool we can imagine all kinds of
things the problem is that we are
eventually confined to what we can
deploy what we can write what
importantly we can actually hold in our
heads that turns out to be the bandwidth
limitation what you can actually
reasonably in hold so there is this
concept that a good architecture a good
code base a good piece of code is
something that at some level you can
hold in your head and reason about
comfortably the minute it overflows the
harder it becomes to reason about and
suddenly you slow down you're more
likely to introduce defects you're more
you're more likely not to touch anything
at all because you're just a little bit
cautious you're not entirely 100% sure
you understand it and that is how you
create a little as one of the ways in
which you create a large system is that
kind of hesitancy that a fact that you
cannot reason about it this does not
mean we do not build large systems it
just means that we have created a system
that feels large whereas you want to
create a system that wherever you are
it feels local you can grasp the
concepts around you so it turns out that
this is your bandwidth now obviously we
can go to the extreme of saying well
let's talk about ultimate smallness an
ultimate constraint my favorite example
which I've actually got an update for is
this piece of code start the day with a
bit of assembler I think some of you are
still booting up so something fairly
low-level like this should probably suit
this is z80 code it runs on a zx81 runs
present-tense I guess ran it's written
in the early 80s 1982-83 by a guy called
David Horne this was referred to by
games developers today as frog Lee as
being the greatest program ever written
which is quite a claim but it's actually
got a fairly reasonable to consider that
this is a chess program I mean it's not
exactly you know deep blue but this is a
chess program and it's done to fit in a
1k memory space okay that appreciation
of one kilobyte in fact to be precise
this doesn't fit in 1k if it's it's 672
bytes because the runtime has to take
some space
this is LX 81 so 672 bytes that's 4.8
tweets you can't even describe the rules
of chess in 4.8 tweets yet this game
this I mean as I say you might go to a
few deep learning sessions it's not
exactly going to compete with those but
it can recognize a draw and a win it can
play the basic rules of chess so I don't
think it has I'll pass on basic bits of
pieces like that and it can play against
you that's just impressive obviously
somebody forgot to tell David Horne this
was not possible so he went ahead and
did it now why is this why is this even
remotely relevant well one it's kind of
fun to I'm not going to claim that this
code is brilliant but the chances are is
that the reasoning behind it the very
challenge that he has taken up is the
thing that we're interested in is when
somebody says how you can't do that
there's a very good chance that you can
and that the very constraint that you
said you can't do that with will allow
you to overcome this so there's a very
good case for this being exactly what it
is now I've used this example a number
of times before or being in a remarkable
example of cool code but actually
recently a couple years back indeed last
Olivier Pooh dad managed to create boot
chess which fits in the boot sector says
512 bytes so that's even smaller which
is really impressive so that's that's
that's half a K yeah now why is this
relevant why is this relevant at all
because obviously I'm not going to say
we should all write obscure code we're
already doing that anyway it's called
code for a reason and obviously there's
a particular aspect here such a thing is
result of unique temperament
but it is just a reminder that
ultimately when you are creating things
as the agile test david evans observed
your customers do not buy your software
by the line if they did that would be
great for some people and we just type
more just keep on typing because you
look at some systems and you think is
this are they actually incorporating and
encoding code is a word that we can play
with you are codifying business
knowledge you're codifying technical
together and that's sometimes you look
at a piece of code and you're thinking
this is a really long hand way of doing
think this is a ridiculously overbearing
and heavyweight interface there's so
much here and yet for so little
functionality so there's these questions
that come to mind but the challenge
should not be around this there's
there's something deeper we want to kind
of reason about and we have these lovely
examples about constraints one of my
sort of favorite classics and sense of
how people have created and reason about
these things is this book by the late
John Lyons who's a professor at the
University of New South Wales were
correctly in the 1970s he wanted to
write he wanted to run at a university
course on operating systems and he
wanted to run a real operating system he
designed for students so he wanted an
example from the real world and he so he
managed to get his hands on the recently
released code and you know there was no
internet to speak of but the the code at
the code from the six edition of UNIX
and and then annotate it he wrote it up
as she got published as a book it was
passed around by samizdat people would
basically photocopy the notes here is
your whole operating system and
annotations now for those of you who are
a little young a photocopier is a kind
of socialized scanning okay you'd have
yeah it's you don't scan alone you scan
other people you kind of hang out in the
university library hoping that somebody
you know and want to talk to will come
by and yes as a social experience so
that this was that this was the premise
here but the key thing here is that the
the colonel was under 10,000 lines of
95% of which was written in C was
written portably only 5% of which was
importantly so at the time this was
absolutely one this is it's something
you can't do you cannot write a whole
operating system that is retarget able
in a language that is retarget able this
was a basic article of faith it always
has to be assembler or most of it has to
be assembler but here we have the very
kernel not simply the shell being
expressed in a way that is portal and we
look back at the code and you look
through any otherwise fairly antiquated
C style it's an area of very different
conventions and what was defined to be
good and yet at the same time they did
things like write code in separate files
that 10,000 lines is not all in one
place and yet I see classes that have
this kind of line count if not larger
there are classes out there in systems
that people are paid good money for
there are classes out there that have
their own gravitational field and we
know about the planet is more theory of
planet formation you have something
there and it has a particular size other
things are drawn towards it because of
the gravitational field and it becomes
larger and larger until you've got one
hot Jupiter of a legacy system ok you've
ended up with this thing it just grew
nobody planned it that way I mean people
don't generally out this I don't believe
this happens but if you have anything to
contradict this people don't sit down
and say right ok we're going to build a
new system how we're going to do it I
think we should have really large
classes that with lots of duplication
absolutely should we make it testable
okay you know I think we should you know
I think we should you know if we're
struggling for a name for things I think
we should use names like manager and and
and processor and controller and stuff
like that and scatter the
responsibilities around the code which
would cause more duplication confusion
and I think she put a you know shed load
of people on the project as well and
we're going to make this really obscure
nobody will understand it okay
make it so that's not how architectures
are created I hope but there's a key
point here that there is a challenge
here that was met that there are under
10,000 lines they've been able to create
an operating system whereas sometimes
you look at something and what have you
managed to do we've managed to take a
message and process it into a different
format I had that with one client it's
like that's 20,000 lines good god I mean
who is this not a solved problem so we
have this observation this observation
was made fairly early on in UNIX is life
the late Dennis Ritchie and Ken Thompson
71-72 paper on UNIX there have always
been fairly severe size constraints on
the UNIX operating system and its
software given the partially
antagonistic desires for reasonable
efficiency and expressive power the size
constraint has encouraged not only
economy but a certain elegance of design
and this is the interesting thing is
that are actually working with the
constraint not simply against it and we
often assume that a constraint is a
negative thing to have but I want to try
and get back to the point of why I
showed David horns program it's not
because I'm saying you should write
obscure code and chess plan you know and
encode chess programs into small bits of
your enterprise system what I'm actually
saying is that you can use seemingly
arbitrary constraints as a benefit the
human imagination is indeed infinite the
word but the will is indeed confined if
you can find that constraint you will
probably find something of value to help
drive the architecture and that thing a
value may change but once you have that
you have a driver and we see this time
and again I mean Donald Knuth observed
one rather curious thing I've noticed
about aesthetic satisfaction is our
pleasure is significantly enhanced when
we accomplish something with limited
okay and you know as he says yeah makes
a person feel like a real virtuoso to
achieve something under such severe
restrictions but he's not the only
person to make this observation we're
going to talk about virtuosos to about
Stravinsky the most the more constraints
one imposes the more unfreeze oneself
and the arbitrariness of the constraint
serves only to obtain precision of
execution you need to find the
constraints in your system they're not
always obvious at the beginning this is
this is so this has becomes a
hillwalking problem what you're trying
to do is find the constraints that
characterize your system and find the
spaces that allow you your degrees of
freedom and once you found that then you
think to work against that will give you
a more natural sizing you can reveal the
natural sizing of a system so what is it
that in that case stands against our
assumptions well it's kind of a nice
observation from a program I know Chris
Allwood isn't a netbook screen a bit
small know if the code doesn't fit on it
it's too complicated and there's a big
case for I want to go back to this
observation I meant made earlier on the
way that sometimes people respond with
how they structure their code they
structure it according to their
available screen size and these days for
developers that's quasi a quasar IMAX
you know they're kind of sitting there
in front of like triple-headed 4k
resolution and they're like yeah that's
fine I've got no problem with a name
that is 120 characters long because I've
got the screen for it and it's like
you're solving the wrong problem the
problem here is that there is a human
being turns out we are humans much as
our screen sizes have evolved it turns
out that the visual system has yet to
catch up and when you actually have to
turn your head to read the end of an
identifier you know that's a bad sign
when our source line of code has its own
microclimate that's different at the end
of the line to the one at the beginning
that's not a good sign so you did this
with a client couple of years ago we're
just going through the code and actually
I'm going to say but their code was
it was within 8080 columns actually
except every now and then something
would just go off it was kind of kind of
funny looking like a looking at a like a
heartbeat something you know this is a
bit I'm goth screen a bit complex
algorithm a bit
what's that formula doing so we went in
search of the right-hand side you know
we we got our blunderbuss guns and we
got our appropriate hunting gear and we
went out into the wild we found a line
that was 360 columns long you know we
kind of almost wanted to make it it put
an extra five characters in there so we
have one column for every day of the
year and the point is that that's not
human that doesn't work with humans and
similarly the way that sometimes people
will hit 160 column limit and say are 80
columns that's just so 1970s it's like
no no that's to do with human visual
perception only if you actually care
about how people read it turns out I'm
going to give you bad news 60 columns so
by saying 80 to 100 we've been quite
generous it turns out that we're much
more comfortable around the 50 to 60
look at how web pages are organized yeah
in other words your code defines your
code source code actually defines an
experience for a user that user is a
developer and how does their visual
system work how are they able to chunk
and reason about things so there's a
sort of a call here some kind of balance
and reasoning but also this says
something about our ability to break a
problem down and to reason about it
clearly the sizing thing does matter is
how easily you can grasp stuff I recall
a visit to a company a couple years back
where they were complaining about the
technical debt on day one we were
talking about technical debt as well as
well I was there to talk about and then
there was this discussion in the evening
I went out for a meal and I said I'd
really like to see some code as we
talked about we've drawn diagrams on
whiteboards and you know talked about
things but I really like to see what
code is like so I then paired the
following warning with one of the
developers and you know just sort of
tweaked a couple of tests
looked at the code and code looked
pretty good to me I have to say and then
the moment that will always stay with me
is he pulled up on screen a method and
it fitted on the screen and he said this
Carolyn this is what I'm talking about
this is the problem yeah he said look at
that it's such a mess you know it's so
large it's like right okay thank you you
have calibrated your system for me if
this is the worst you can show me you do
not have a technical debt problem okay I
said yeah it's there are people that
were kill for methods that fit it on a
screen this is the worst he had to show
and yet you could reason through it yeah
it was a bit tricky in places but that
was actually a reflection of the nature
of the problem not a reflection of
source code chaos it was not a
reflection of muddled thinking that it
started down that road of like I'm not
sure what's going on here let me add
something rather than modify what's in
place and so there's that idea that
these seemingly arbitrary constraints
can help us to reason about things
obviously you have to spawn respond in a
reasonably rational way don't do what I
didn't meet this guy but it was at a
place that I used to work and it's far
enough back that when people talked
about the size of procedures or methods
in code they would talk about printed
pages and the idea is that the routine
should not go over a printed page so
this guy responded in the way that many
developers would with ingenuity and he
worked around what he saw as an
arbitrary constraint so he'd write the
code as he wanted printed all out and
then just mark off the boundaries and
those levels yeah so you have an
arbitrary point here and called version
the thing to at this point that did not
have the desired effect that's not the
kind of constraint we're talking about
but we can take this further I've talked
about code locally what about the idea
of creating a whole product based on
this kind of thinking what about the
idea of actually introducing an idea to
the world this is this is interesting
that John O'Dell former editor of bite
magazine a couple years back he was
reflecting on Ward Cunningham's wiki and
he said you know here's the original
email 19 years ago I received this email
from Ward Cunningham John so what do you
think of wiki I put it up a year and a
half ago when HTML authoring tools were
in short supply I think it has held up
pretty well that I suppose it's days are
numbered it turns out the numbers quite
large still there was something about it
we call wiki nature that is in short
supply on the net and in computers in
general and that is still true 20 years
on there's something really profound
what Ward did was you know this is the
top and tail of one of the early
versions of the wiki and I used to hang
out on the original c2 comm wiki it was
quite a little community there but it
actually turned out to be quite useful
and it was there was an ease of use
about it that came from the introduction
of arbitrary constraints you couldn't
just put HTML into it it was not an HTML
authoring tool and it had a very simple
model of links simplistic even in fact
it's so naive that it's amazing it got
popular at all but that's the whole idea
there's an idea here I want to pick up
at the end about the simplicity of the
approach that led to something that was
easy to discard we don't talk enough
about discardable code we sometimes talk
thrown away have you noticed that we use
we use metaphors so badly it's just like
yeah I've just written this throwaway
code now what normally happens is that
throw a bit code becomes your system or
in a good case you might have just
actually just best about with something
throwaway code your colleague comes to
you six months later and says oh you
know that's throwaway code you wrote six
months ago to do blah blah blah and you
say yes could I have a look at it yeah
absolutely no problem well I want you to
pause for a moment and think about the
words throw away I threw it away but
here it is six months later it did not
fall into a time vortex to reappear at
this point in time what happens is we
didn't throw it
way there's a very simple concept in the
real world of what throwaway means it
means when I throw it away I can't get
it back we don't throw these things away
we generally hold on to them but there
is an idea here of a property that we
don't value enough discard ability the
ability to say there's a piece of code
and that kind of worked out fine for us
but now I've got a better idea and I'm
happy and free to throw that away
just purely from a human point of view
the larger that thing is the less likely
you are to throw it away
okay the bigger something is this is
this leads to the sunk cost fallacy the
bigger something is the more effort it
acquires around it the less likely we
are to throw it away the less likely we
we feel because in truth we don't really
throw things away we just have different
versions of things that's one of the
fascinations of software development and
probably one of the reasons quite a lot
of people in software development enjoy
science fiction in one way or another
it's the fact that you get to play with
alternative realities on a daily basis
you get to play with constructed worlds
that's that's quite that's quite a big
deal yeah there's this whole idea that
in other disciplines they don't get to
time-travel whereas we can go all I
didn't mean to do that let's go back in
time step into the TARDIS unfortunately
we don't have that kind of ceremony that
makes it sound exciting but it would be
kind of cool if every time you did an
undo you kind of got the kind of warping
sound of the TARDIS yeah just to remind
you that what you're doing is actually
quite interesting you are travelling
back in time you can go into your repo
and go back in time and navigate into an
alternative reality we don't we don't
play enough with that idea it's a
fundamental advantage it allows us to
create small things and throw them away
so this is the top and tail of the
original Perl file for the wiki so the
wiki was made in Perl I mean good grief
it's my least favorite length I'm not
even sure it is a programming language
so friend of mine once described it as
executable line noise and I think he's
still right
but it's 150 lines of code that's the
top and the tail of it
and yet I want you to imagine if you
given that to a team in an enterprise ii
organization what we need is an
interactive editable web thing ok we'll
get back to you in eighteen months and
then they overrun by six months this is
just so small it's so ridiculous that
it's a it's an idea that surely could
never work and yet it does and it has
the opportunity to grow because of this
so Hemingway kind of had it right
s you the monumental shun the epic all
the guys who can paint great big
pictures can paint great small ones so
this is this problem that what we've
ended up with is a sizing thing today we
suffer from an almost universal idolatry
of giantism it is therefore necessary to
insist in on the virtues of smallest
where this applies but the problem is it
probably applies in more cases than we
respect it and that sometimes when you
get a conversation with people what
you'll end up with is you know I do I do
a session like this in a conference or I
say something at a workshop at a company
and somebody go yeah here we're saying
but our system is special it's ten
million lines it's half a million lines
it's it's whatever and we're special so
therefore we can't do that trust me
everybody is a special snowflake that's
the whole point of software software is
about software development exists to
create differentiation it's not a
manufacturing process so everybody is
ultimately going to be different and
everybody is working against interesting
constraints but is the problem is people
weigh in and say our system is special
without ever really asking why is it so
specially large why is it that you have
had to organize armies to do the work
which kind of leads to the question of
why do these systems get large they are
we looked at a couple of reasons but
let's take a take a different tack I'm
going to borrow from friend of mine
Hanlon Kelly who's been it's been making
a very strong case for this
understanding of the economics of
software and the economics of team
sizing and team effort and he says
software development does not have
economies of scale if anything
development has
economies of scale and this is it this
is this is quite this is quite difficult
to reason about because from a very
early age we are ingrained with the idea
of economies of scale it's kind of part
is this idea of if you get a he uses
milk cartons I'm more likely to use wine
glasses okay but there is this idea that
if you ever buy a bottle of wine in a
restaurant if you were to buy the
equivalent number of glasses the cost of
the glasses would be far more than the
bottle of wine there's a natural economy
of scale I'll sell you ten for the price
youngest son had to do this exercise at
school where they you know went away and
did this little project and invested
money and then try to sell things to
other kids and parents and all the rest
of it this is what happens when you do
brexit you know it's a you've suddenly
got to figure out where the hell's the
money gonna come from so now we're we
yeah child labor but if you want any
pencils you know my youngest son can do
a really good deal on them because that
was one of the things that he and his
best friend came up with was you know
yeah what we'll do is we'll split do a
special offer we'll give you five
pencils for the price of three it's not
Wow it was pretty good he's understood
economies of scale that's before he
turned ten it's ingrained into us very
very deeply so there's this idea that
when somebody says we need this software
system it's just that right we need
bigger architecture we need bigger teams
we need more people we don't even know
we're doing it because it's so hardwired
into the spine of our thinking it's not
even happening up here okay it's a much
it's brewing down in the reptile brain
so there's this challenge that we need
to kind of pause and think maybe this is
not the way we do maybe what we're doing
is we are thinking with or through ideas
and what we are call thinking is
generally the application of
pre-existing ideas to a given situation
or set of facts we look at a situation
through a lens of experience and habit
and perhaps we need to challenge that
much force the situation of your
existing ideas and also the organization
you exert now it has become traditional
to refer to this paper but so therefore
I do it slightly differently this is
actually Melvin Conway's original paper
how do committees invent in 1968 so it's
not some kind of retro look so I'm after
here this is actually genuine retro
before the moon landings just to put
this in perspective and a Melvin Conway
made this observation that has been
observed casually but also since
verified a number of times yeah the
basic thesis is that organizations which
design systems are constrained to
produce designs which are copies of the
communication structure of these
organizations in other words the way
that your system will come out is to do
with the way that you arrange your
people and how the people communicate
with one another and this is an
important idea because it is how people
actually communicate that he's saying
here the problem is that we have an
idealized view of organizations we think
that the way that people are organized
is follow something like an organic Ram
a hierarchical decomposition yeah you
have that you have Department you have
the route of the company and you have
department heads and then you have
people who actually to the work and it's
all beautiful and in some companies this
this you know this appears on PowerPoint
files and other companies it appears
plaster on the walls you know if you
wanted to know I just referred to brexit
the greatest achievement of the brexit
Department yes we have a department
apparently over the whole summer was to
produce an organic Ram of who was who
they had hired that was it they hadn't
done anything else but it was a very
pretty diagram it was the what they were
doing was offering the illusion that
there was some reality here that this
was the reality organizations do this
there's a problem though that's the
organization as you would like to
perceive it according to pay grade line
management and all the rest of it but
let's take a slightly different
observation it's taken from a book
designed after modernism which has a
slightly unusual cover to say the least
by john zacchara's collection of essays
is a rather interesting essay by
Christopher Alexander a city is not a
tree this gives us a different insight
into sometimes how we can think about
architecture when we talk about
single earth space within an urban space
we use the metaphor to refer to
buildings and we talk about buildings an
awful lot and indeed this is the same
Christopher Alexander that was
responsible for this idea
timeless we're building patterns this is
he came he's the guy responsible for
that it's the same guy and he's
interested in systems of thinking
overlapping thinking how one design idea
can lead to another or support or negate
another idea and that we're not thinking
just in terms of modular parts we're
thinking about how a system actually
overlaps and as he observes it says the
tree of my title as a city is not a tree
it's not a green tree with leaves it is
the name of an abstract structure at
which point the whole room suddenly
becomes comfortable ok yeah we can do
that yeah we're good because as as you
and I know trees their roots are at the
top their leaves are at the bottom
that's the true way and if trees have
color then it's red and black okay this
whole green trees with roots at the
bottom and leaves at the top that's some
kind of real-world aberration that we
just target with CG with CGI yeah but
the real trees are these they are
abstract structures and he wants to
contrast it with the semi lattice their
ways of thinking about how a large
collection of many small systems go to
make up a large and complex system and
as he then observes well look this is
this is actually what goes on in a real
organization people are not they don't
sit within a hierarchy that's not how
they live and breathe the system it's
not how they communicate with people
it's also not how the code grows because
the paths of communication will emerge
from that and you have a choice here you
can either acknowledge that this is what
happens and therefore you can organize
your architectural effort around that or
you can be surprised by it every single
time it turns out that where people sit
who people know who people trust who has
certain expertise in some things versus
others who is easy to communicate with
all of these kinds of things
we'll ultimately start shaping the
system and we haven't even started
talking about the requirements and how
you choose to deliver so there's this
idea of complexity which is why people
shy away from it semi-lattice is
potentially a much more complex and
subtle structure than a tree the reality
of today's structure is thick with
overlap the subsidy the systems of
friends and acquaintances form a semi
lattice not a tree which means that
although we may envisage a system and
you know decompose it rather elegantly
into smaller and smaller parts and if
you look at every single paradigm of
development they're always about
decomposition and somewhere in there is
the concept of a tree and we find it
valuable it informs the syntax of our
languages and informs the way that we
indent code it's certainly valuable on a
local basis the problem is the reality
is a lot Messier so you've got a choice
you can either say I deny reality and
we're going to force it this way you can
say no realities like this it is a bit
messy how do we organize the messiness
and yet still get localized order that's
non-trivial that is exactly this
Christopher Alexander says subtle
because if you don't respect it you will
end up with some surprises you will end
up with the surprises that that occur
when you don't respect this dynamism if
you like we've seen that this fact has
important implications for the
management of system design a design
effort should be organized according to
the need for communication if we don't
then our code will reflect this
communication it will reflect the
mismatched messy communication if you
what is communication it is the
transmission of knowledge mmm no that's
good communication there's an awful lot
of communication in organizations not
about the transmission of knowledge it's
to compensate for the organization's
inefficiencies and attempt to be
hierarchical and actually it isn't quite
that sounds like just pure business
speak but what I'm saying is it trickles
down into the code as was observed in
this paper in 2007 the MIT Sloan
management review is a website it has a
website and you can kind of go to it and
read one article if you're going to read
one article for free this is the article
you want to read it's still remarkably
relevant avoiding the alignment trap NIT
they make this observation
having looked at a number of large
companies I t's effort to satisfy its
various and sometimes conflicting
business constituencies created a set of
Byzantine overlapping systems that might
satisfy individual units for a while but
did not advance the company's business
as a whole what they're looking at is
the fact that an organization
organizations often ended up with
alignment that was local but also
against other groups and they say we
have a strategy we are business aligned
the thing you need to remember is that
in an organization an organization is
made up of people and we we have a nasty
habit of oversimplifying it's part of
the abstraction nature of humans we
sometimes talk about the customer as if
the customer was a single entity in most
cases the customer is an organization
filled with individuals who don't all
agree with each other and that that gets
quite interesting if you say we are
going to follow the organization's lead
we have a vision for the organization it
turns out everybody is pulling in
different directions which leads do this
which also starts telling us how systems
can get messy or tells us again how
systems can get messy obviously at this
time but down not gonna present anything
more complex in a quadrant diagram
because you know quadrant diagrams are
easy they divide the universe into four
and normally there's one very very good
quadrant or perhaps there's one very
very bad quadrant in this diagram we
have both okay and there is a simple
idea we're going to based on a survey of
over 500 companies that have some kind
of software development and IT capacity
within them as opposed to say software
houses or consultancies two axes
alignment with the organization is your
IT aligned with the organization
strongly or weakly and effectiveness how
good are your teams
how good are their practices this is not
a judgment on how good are the
individuals because as we know as we
know you can get a lot of people
together collectively and they can
actually act dumber than any of the
individuals in Britain we call this a
referendum for example and also there's
the England Football team individually
there are actually quite good players
collectively they can be beaten by
okay so the point there is that it is
quite possible to have a group of people
be far less effective than the
individual capacity suggests and this is
it so this is about coordination about
the necessary communication that you get
and so the observation here is one
element of the observation is
unsurprising most teams 3/4 of them are
in the bottom left-hand corner they're
not particularly aligned they're not
particularly effective but turns out
from a financial point of view that's
not particularly bad they're not really
gaining money they're not really losing
money in fact they're probably in
equilibrium with an equally inefficient
set of other departments in the
organization everybody's reached a kind
of comfortable equilibrium they're all
kind of they will suck equally but
they're happy with it that's good this
is where most companies are what people
want is the top right-hand corner which
is one tenth of the size ok in other
words these guys are they getting
cheaper and they're generating more
revenue this is fantastic everybody
wants to go up there now this there's
another reason I chose a chess program
near the beginning this is a bit like
chess except you don't get to be a
bishop you get to be a rook you can only
move horizontally or vertically if
you're in the bottom left-hand quadrant
would you like to become more aligned
with the organization or would you like
to become more effective at software
development the standard the standard
answer from many consultants is become
more aligned because everything else
will follow I'm sorry that's the
alignment trap you've just walked into
it it turns out that you have taken a
group of people who are not necessarily
coordinated as a whole who don't
necessarily have the best development
practices and you've just thrown them to
the Lions and they've got eaten that's
what those that's what that's what those
numbers are what happens is that they're
never able to recover they're always
worrying about delivering in the present
and they're never able to kind of say
you know what we could step outside this
vicious cycle that we've got ourselves
in and there's a different way to
approach the architecture there's a way
of approaching technical debt we can
improve the testing we can improve the
deployment we can improve our skills we
view but we don't have enough time so
none of that happens and it gets worse
you've become an amplifier for the bad
practices you haven't yeah sometimes
people describe certain development
processes this is a popular one in the
scrum world a process that shows you the
dysfunction of your organization that's
one of the most frustrating phrases you
can ever throw at a group you know
they're they're struggling we're trying
to do stuff well scrum is revealing your
inefficiencies to you yes we know but
that doesn't tell us how to solve them
there's a frustration there that all
we've done is said yeah you see that
problem you thought you had you have it
it's like thank you whereas it turns out
that if you decide to become better at
what you do it turns out that has its
own benefits and you can that doesn't
necessarily mean that you're you know
not delivering useful values to the
organization but you're more likely to
be able to do that and this goes back to
the release one versus release two verse
thinking but not everything is released
one the idea here is imagine simple
thought experiment a system that has
been developed by top left-hand and
bottom right-hand teams they've
developed the same functionality whose
do you think is going to be most likely
to meet the requirements
quite possibly these guys because
they've been aligned to the business in
fact the business won't shut up so they
really know what is required these guys
might miss they might miss some key
requirements and assumptions okay who's
got better code what's these guys better
deployment practices better quality
better management and explicitness of
all the things that are invisible in
software development these guys and so
what's their code look like those guys
over there are gonna have a nightmare
if you want to then join the team join
this team because when they say you know
what we realized we missed some key
requirements oh that's easy we've got
the test we've got the code it's all
good it's small it fits in our head we
know how to change it and you're done or
as though these guys over here they've
got this Jenga tower yes that is now a
style of architecture they've got this
Jenga tower of architecture of like
don't go near it
I need to fix a bug don't go near it
because if you fix the bug the whole
thing falls down and certainly don't add
any functionality to it so there is this
idea that the thing on the that side
over there it's going to start getting a
larger and Messier it's just going to
become a challenge so the sizing aspect
is what we're what we've walked into how
else do you create a big system we've
talked about the idea of communication
we've talked about the idea that
sometimes the code artifacts themselves
start attracting stuff but there's other
ways we can do this and this again goes
back to questions of economies of scale
how do we reason about how we organize
an effort so let's just imagine very
idealized example this is how much time
is going to take one person to complete
all the work all the work I'm going to
include in this all aspects of a system
including reasoning about the
requirements and all this kind of stuff
deployment testing the works so we then
apply classic division of labor we
divide by n we've got n people on the
project and most people stop there
problem is that's not how it works not
everything can be parallelized
not everything when you put more people
on it turns out they start stepping on
each other's toes sometimes that's a
physical sometimes it's in the code
sometimes it's how they work you end up
with this which starts looking a bit
scary when you start showing this to
managers it's like Oh equations it turns
out though that's quite important how
much can you actually do in parallel how
many weight states are there how much
flow can you get out of the system it
turns out that Kanban people were right
you can do this from first principles in
fact we already knew this this is am
dars law but I've got an robustly
applied it to people it turns out that
this is going to reflect on your
architecture but it's also going to
start creating is going to start
exerting a force back on your
architecture people will continue
working even when they're waiting they
will start creating things they will
start creating defenses in fact it gets
more interesting because there's more
equation they will start creating
defenses against the question of
because it turns out you have to talk to
other people I know who thought that was
going to be a consequence you got to
talk to other people that's the number
of connections you have the more people
you have the more connections you have
in it shoots up and there is a cost to
communication this roof is reflected in
the code code ultimately becomes a form
of communication code is a form of
communication you're transmitting
knowledge you're not merely telling a
machine how to execute you are
organizing your thoughts it's a
collective effort to create a belief
that executes this is how we believe
this technology will run and answer a
business need and the problem is it
interferes with the nature of humanity
if we are not careful you end up with a
nice curve like that which kind of fits
our intuition because it turns out them
you know if you have zero people on a
project it will take infinitely long to
develop but hey no bugs but then as you
start adding people you find some kind
of optimal size that actually you have
got the right number of people to do
this work to do this undertaking the
problem is that it's not a beauty very
idealized people are not all the same
they don't all have the same abilities
and you can't see the curve and it's not
perfectly smooth so you don't exactly
know when you're here and you may find
yourself wandering up the wrong side and
this is how you create large systems
of consultancies hang on quick check
over here the sponsor list to see the
vine whether or not I'm insulting
anybody actually I might be but I'll
keep them quiet a lot of consultancies
operate on the right hand side
explicitly in other words we've you took
your small problem we've made it a large
problem by adding enough people to make
it a large problem I bet I could make
this room develop a HelloWorld
application the like of which the world
has never seen before and would never
want to see simply by scaling the
problem up it's not a problem wants to
be scaled up it's a small problem and it
needs to be met with a small team and a
small technology and the point there is
we're very bad at doing this sizing but
once you start running up the right-hand
side then it becomes a self
filling prophecy because eventually that
system becomes the 60 or 70 person
were to rewrite it how many people will
we need well we currently got 60 or 70
people on this so maybe we need 60 or 70
about 10 and we have actually seen these
experiments run out in practice
accidentally sometimes and actually
seeing them in the real world where two
companies or in one case one company
that I turned a job down for because I
thought there is no way these guys are
ever going to deliver anything sane or
sensible and I don't want to I don't to
work for them
and then nine months later a friend of
mine went for a job interview there and
so I had a chat with him filled him in I
said this is what they were doing this
is probably what they're doing now he so
he knew he was walking into the Lions
Den but for him it was practical they
were 10 min 10 or 15 minutes walk from
his house so it's actually for his point
of view that's what he values being
close to home ok so some time later he
told me yeah yeah we've got about 90 to
100 people developing this system and
we've just discovered that in the north
of England the same company because
remember companies are not single
unified entities they have no idea what
they're doing the left and the right
hand are very very different
in the north of England another
development team had developed
functionality or a system with exactly
the same functionality with 10 people in
about six months these guys had spent 3
years and 100 people and there are loads
of examples like that over the industry
they had created by staffing it up to
100 people they had fulfilled their own
prophecy it did become a system that
needed 100 people to create and once you
are there it's very difficult to come
back down again I also had the benefit
there the joy of working for somebody in
my many years ago um who actually knew
this he took a team that was late
and half the team size and they went
back on schedule yeah you hear about
these things but I can say first hand
it's real I've seen it happen he was
hated by the rest of the management yeah
because everybody else was going by the
the easy bit of the equation T 1 divided
by n but he actually said now wait a
I think we've created the size the size
is not a reflection so there's a point
here this size is not a reflection of
the functionality once you reach a
particular size anything beyond that is
no longer a reflection of the
functionality and there's this idea that
a larger system has more functionality
actually we've seen time and again this
is not the case what we've seen is that
above a particular size the rest of that
size is there to compensate for
something else
it there to solve a problem of
organization so it came across this one
last year the facebook iOS app has over
18,000 objective-c classes 18,000
classes not 18,000 lines
18,000 classes and in a single week 429
people contributing to her now I'm going
to take a wild guess I don't run on iOS
a phone I've got an Android phone I'm
going to take a wild guess that it's not
radically different on Android but if
any iOS users want to tell me what the
extra few thousand classes are doing
that I'm somehow missing out on please
let me know because that is just insane
I wished earlier on we looked at an
operating system I know some trivial
thing called Unix whose original kernel
was less than 10,000 lines in fact the
fifth edition it was 5,000 lines and I'm
not saying everything can be made
trivial but there is a kind of a sense
of perspective that you need to get this
does not only have its own gravitational
field this is actually at the heart of a
galaxy this is one of those black holes
okay there are hundreds of millions of
stars orbiting it as a result and that's
you're just thinking about what
so they doing I mean it's one of those
things I don't want to make it sound too
trivial in the sense that what are you
doing well I'm looking at friends
pictures and you know pictures of cats
and goats and things like that and I'm
posting some things and I'm liking a few
things I mean how am I that I'm there's
something really special going on there
and I don't think that's 17,000 worth
17,000 classes worth of NSA stuff okay
so we have this kind of interesting
challenge that we are looking at a
number of interesting practices
there's organizational things but
habits are non-thinking where we started
this cargo cult programming style of
computer programming characterized by
the ritual inclusion of code or program
structures that serve no real purpose
cargo cult program can also refer to the
results of applying a design pattern or
coding style blindly without
understanding the reasons behind that
design principle and suddenly without
realizing it you've turned a one-line
problem oh that's too easy you've turned
fizzbuzz into fizzbuzz Enterprise
Edition one of my favorite sites I did
actually have to point out so I will
point out to you this is a joke I mean
it's got real code it does compile but
you know there's tongue-in-cheek
enterprise software marks a special high
grade class of software that makes
careful use of relevant software
architecture design principles this
project is an example of how popular
fizzbuzz game might be built where it's
subject to the high quality standards of
enterprise software the whole thing is
in quotes okay you have to read it like
that because you end up just looking at
one of the areas this is just some of
the interfaces in one corner of the
system yeah we're in the factories
interfaces section and we got such
fundamentally important plugins as
integer string returner Factory and
string string returner Factory oh yeah
there are factory factories as well
Andersen's law observes there are two
aspects to this law I have yet to see
any problem however complicated which
when you looked at in the right way did
not become still more complicated
one aspect of this is that it's Turtles
all the way down some problems genuinely
are deep but the other is a
self-fulfillment we can make a simple
thing hard we have we have decades of
experience doing this in fact we're not
alone in this one the American architect
building architect Philip Johnson
observes architecture is the art of how
to waste space and you know sometimes
you look at something at it yeah
actually that might might be the that
might be a good description of what's
going on that there's a very simple idea
here Paul McMahon kind of observed all
code is technical there some code just
has a higher interest rate that's what
you're interested in when people sub
sale how much technical debt if you've
got that turns out not to be the real
issue it's it's it's it's part of the
piece of code that has high technical
debt or other sorry if you have a piece
of code that you never touch then its
technical debt is low or its interest
rate is low because you never interact
with it it does not cause you pain on a
daily basis if you constantly interact
with something that has high technical
debt that has a higher interest rate
because you it's causing you pain on a
daily basis you're having to pay out
extra every single day but that code
that's not doing anything is it really
not doing anything does that not mean
that is it technical debt that we have a
problem with there it or is it something
else so I want to close with a sort of
consideration of a example that one of
the other problems that we have when we
talk about the dead past the dark matter
of our system okay in any large system
will be the galaxies are not simply
shaped by the black hole in the heart
there in their heart they are shaped by
dark matter stuff you cannot see code
you do not understand but also code that
is not executed Knight Capital Group
2012 experienced a slight blip I think
would be an understatement to say on the
first of August going from 31st of July
to thirst of August on the New York
Stock Exchange and there's a very good
but dug seven wrote this blog from a
DevOps perspective night capital group
realized a 460 million US dollar loss in
45 minutes
ten million dollars a minute excellent
yeah that's the great thing about
computers yeah they can amplify basic
human human error it's very difficult
for an individual human to actually lose
that okay Donald Trump spent years
losing twice as much as this okay but
you you what you can do is you can you
can actually optimize it computers can
let you do the wrong thing faster
ridiculous speed they realized something
was going wrong like quick pull the plug
45 minutes later they did actually they
actually lost more money this is how
much was left this is how much the loss
was after everything had been rolled
back that could reasonably be rolled
smart a system I think they had eight
service was intended to replace old
unused code refer to as power peg
functionality night hadn't used in eight
so there's dead code this is the problem
is that they had eight machines seven
updates were correct one of them was not
so it ran old code and what they did is
they recycled effectively a piece of
protocol we're going to reuse that
because nobody's using it yeah because
it's safe it's easier than adding
something new
we'll just recycle that problem as it
has meaning to the old system and it not
the right meaning yeah we just recycle
old ideas but the problem is we left the
interesting and where I disagree with
this assessment and what tells us a
little more about the grasp ability the
in your head ability of this why code
that had been dead for eight years we're
still present in the code base is still
a mystery but that's not the point
actually it is how did you end up with
this dead code this is not dead well
it's a zombie code basically if you
think you have dead code it's probably
zombie code what would cause this code
to resurrect and come back to life and a
particular set of circumstances caused
this so it is actually the point because
the idea is that this would not have
happened it was not just a bad update it
was a bad update and the presence of the
code the point here is this was a
perfect storm it was not this
or that it was this and that what
happens is in systems which we have
effectively lost intellectual control
over is you increase the probability of
cases like this that's what's going on
and then boom you get a big surprise so
ultimately just a demonstrate there is
no new idea Under the Sun I'm gonna go
back to the 1990s I mentioned patents
before which Gabriel key member of the
patents community wrote this book no
longer available PDF is available very
interesting series of essays one of
which he talks about his approach from
26 years ago worse is better I proposed
a theory called worse is better why
software would be more likely to succeed
if it was developed with minimal
invention problem is with the phrasing
worse is better you immediately think
well worse I mean it's the wrong name
for it but when somebody says it is far
better to have an under featured product
there's rock-solid fast and small than
one that covers what an expert will
consider the complete requirements he's
talking about something fundamentally
different he's talking about a
philosophy that fits the wiki way for
UNIX way and indeed where a lot of
people now think of as lean startup he
focuses on implementation
characteristics because that's where
developers live developers will
ultimately produce a system of course
the customer experience is important but
ultimately somebody needs to develop it
and you need to make that amenable
impossible it should be fast it should
be small because then you can throw it
away you can understand it and think of
better ways of doing it it should
interoperate with programs and tools the
expected users already using minimal
invention he's saying something very
modest wiki the wiki way for example did
not define a fundamentally new browser
concept or paradigm but it helped us
understand what we might want from one
it should be bug free this is what worse
means worse means fewer bugs slightly
counterintuitive but there's that idea
of if you want to make it smaller reduce
feature set that's easier to manage and
don't get carried away with abstractions
very simple idea so I want to close with
this kind of observation that
people we talk about incremental
development a lot I'm very interested in
an idea called decremental development
okay we should we should learn more
about how to throw things out more about
that was our first attempt now how do I
size it down to its natural sizing and
I'm sure over time it will get bigger
the problem is that what we do is we
start in the wrong place and I'll give
Schumacher the last word thank you very