The Entity Microservice Trap You May Be Doing It Wrong

Fred George

Recorded at GOTO 2016


Get notified about Fred George

Sign up to a email when Fred George publishes a new video

yes the title the talk would was the
engine micro-service trap you may be
doing it wrong actually should be you're
screwed if you're trying this technique
um just a little bit about me if you
haven't run across me before really the
only interesting stuff is it's kind of
on the right side I'm a technologist
I've been I've been sort of a bleeding
edge guy in almost every technology wave
since I got started in programming that
means a lot of things I do I tend to be
you know one of the first adopters so if
a lot of stuff I've been saying today
doesn't make sense to you it's kind of
okay and maybe another four or five
years before you get around to it but I
have a pretty good track record of
guessing guessing correctly what's
really my real job is the more
associated with this I'm basically a
disruptor the disruptors basically is
one of the actually official titles of
people in organizations now at the sea
level the job of disruptor is basically
to question everything they're doing
it's not a good guidance study having
all their meetings but you want to be a
part of that so I go into organizations
I do tend to make changes
this is actually my job description at
the Daily Mail in London I was described
as the hand grenade being thrown into
development if you never get this job
title you should take it because it's a
lot of fun so yeah I didn't have a lot
of radical ideas um at this moment of
time able to get to my age particularly
you you sort of reminisce about things
and and today in micro services kind of
feeling like about 15 years ago when we
were in the beginning the ages of agile
and and this sort of comes up that you
know agile is largely very productive I
mean people are doing agile and talk
about this is very productive this is
what Jeff basically Jeff Sutherland
instead of the conference about a year
ago and all his projects he at least
gets a four times improvement of that
and probably says I've done to the last
15 years I've got at least that sort of
productivity but then you see a number
like this so this is what a large
consulting firm here in the u.s. claims
that that's what they're seeing in agile
productivity at best
13% what's the difference between the
Forex guys where would you I see
do and there do in the 13% which seems
to be you know what the average seems to
be they're talking about and to some
degree it has to do with we're just not
doing it right
and I think microservices right now is
kind of in this heyday we're kind of in
the early days and and or we're gonna
have the same sort of you know fall that
we sort of have the same fate of agile
that all of a sudden the next year or so
gonna have all these articles in the
trade press about these micro service
failures and of course is in Microsoft's
or micro service certification process
will come up and you'll be Microsoft
scrum masters and and all these other
things like this will happen yeah we
will go through that phase it's
undeniable we're gonna do that too many
people are saying micro services too
many people are green is good idea
it's getting an appointment where CTOs
or CIOs have to say I understand micro
services to get hired even though they
don't so we're getting at that point
we're gonna do that so what's we're
gonna happen here and what's what I'm
the problems we're gonna run into that I
begin to see a little bit of already
lonnis goes back to this guy this is
Fred Brooks Turing Award winner very
early Turing Award winner I also to the
mythical man-month he is the guy who
basically did a lot of the design of the
early 360 operating systems for IBM and
I grew up an IBM myself I joined IBM
about 70 1974 work for there for a long
time but his philosophy was basically
when we adopted it says you know you get
your data structures right in the Koch
and write itself now remember he's
talking about operating systems that are
about 60 K which is a whole different
world than we live in but that was kind
of the idea behind it and so we actually
worked that way we'd have our data
structure so we'd have lay them out this
is all done in assembly language coding
by the way this is really Adi sect if
those understand what that term is
and then basically your write all the
code tend to write itself and this
became a really important thing in fact
we tend to put some somebody in charge
of just that data structure it was his
full-time job as a programmer was to
maintain that data structure actually I
just noticed I colored him in brown
which seems to be out of propria color
for the job because it wasn't a very fun
job and then of course then all restos
would try to write code and we go to the
sky and ask him to you know kind of have
another field he says well how you gonna
use it and we argue about it then and
he'd be very likely to make changes but
else was that was his job that was 1970s
role until the 1990s what I see is this
we have this relational database sitting
in the middle of the world and yeah if
you get to a relational database
structure right yeah you can write out
lots of code around this side it makes
it really good but oh my god this this
person in charge of that database oh by
the way it's to databases down the
reporting database versus the
operational because they're organized
differently for different purposes and
yes there's still that person in charge
of this you know they have all sorts of
new tiles DB DBAs of some sort
architects and oh my goodness you try to
get a change that done it's like you
know you're asking for the firstborn
child oh and for good reason because
once you start tweaking that thing yeah
nasty things can happen so migration
scripts and everything you have to be
approved I know and I've wrestled with
this and by the way why wasn't Fred
Brooks talking about the database back
then well in 1970 sequel was just a
concept I study the university nobody
actually really build something so
inefficient it exists so of course we
didn't talk about databases what you see
the picture looks uh very much too
similar and I'm feeling the same sort of
thing about that there what this is
holding us back so you sort of go back
to this approach and that and I talk
about a corollary because we figured it
out pretty quickly and I would say with
it be it's only by the 1990s that you
know Fred Brooks approached we did work
it's true you get that data structure to
write the code writes itself but it's
also a corollary and says yeah but you
can't change it without making all the
code suspect without having to go back
and retest everything and back in our
day I mean 60km software is not that
much you would think but we had to use
text editing tools even to find
references to field so it wasn't a nice
or concept of ID's this trying to find
this code was very hard and hence was
born out of this the object movement it
says oh by the way let's sort of take
that big giant dissect and carve it up
to little pieces and give each piece a
little piece of code and hide it they
didn't capsulation initially enough
historic
the guided data encapsulation Dave
partes actually worked for Fred Brooks
at the University of North Carolina and
basically Fred Brooks did admit yeah
Dave you got a better idea my idea
really does suck so we have doing the
right way so we broke that thing up into
objects and so we kind of broke that
de-sect up so that's kind of where we
got with that and by the way we've
played lots of tricks back then you
can't believe how clever we got of
course you don't want to touch the green
stuff so of course you put new fields at
the bottom but one time you put a
variable name field at the bottom so
what do we could do then it's like oh my
goodness when you do that well you put
fields at the top negative offsets
seriously you can do this in assembly
language it's quite easy you just go up
up for bikes and look at something else
this is I'm gonna be there so also just
tricks were pulled just to not touch
that middle data that's how volatile it
was I was in a presentation and actually
the Yale conference in Perth last year
and these guys stood up and said oh by
the way we're doing the way of the
centralized database we're doing micro
services and we're able to deploy you a
couple times a day and they kept
listening and says this is fascinating I
wonder how they did this and they said
oh yeah what we do is we can change
these as long as we don't touch the
database and I'm like oh that kind of
kind of eliminates almost all the
interesting stuff doesn't it oh but we
couldn't touch the database so yeah
they're the same problems begin to exist
we recognize that we have same problem
in micro services now there's books out
there is that how do you attack this
thing how do you sort of build your
database so if you haven't got these
sort of issues how do you get around
this sort of thing how you make some
sort of agile database and of course is
an entire book about this agile database
design and his story talks about here's
the tricks you could do where you don't
have to touch the schema so often one of
the tricks they talk about is this one
I've used this trick myself sort of this
flexible schema sort of concept and the
idea is instead of sort of defining are
your fields and columns you kind of
define yourself a hold on the table has
data in it and you have a column for
integer as a column four floats column
four your strings etc etc and you kind
of tag it which one it is and basically
you have to wait to a lot of these so
these guys are just point to each other
some sort of joint table
so yeah my data has these fields and I
want to add one in it's not a big deal
just kind of destroyed ad in there um
kind of cheating kind of makes the
queries run a little strange but it does
survive changes but that's the story
technique we have to get around in order
to allow ourselves to run at a pace it's
more than one to at least every four or
five months so along comes other
solutions that say okay let's try some
of these non sequel stuff so some of the
non sequel stuff no sequel stuff arises
out of this problem of having to
understand that data is so much and this
is the this whole industry has grown up
it's very powerful at this point I mean
there's a lot of interesting databases
out there and if you look at some of the
large companies especially in Silicon
Valley you find out that they've
abandoned sequel for the for the things
that don't seem appropriate for it yeah
there's probably still to do payroll
where they're sequel but they they're
not doing all the other stuff with it
they found it not to be the right sort
of data structure and then go in other
places so you have this whole hierarchy
of structures from simple key value
stores all the way through graph
databases and they have very very
appropriate business problems they're
trying to solve that are not payroll but
they sort of have these things in common
they're really all about trying to get
away from locking yourselves into a
particular schema format you're trying
to but you have that schema format you
basically trying to predict the future
and back when storage was really really
rare and we were trying to really push
as much stuff into those little fewer
bytes as possible it made sense that
sort of Smith some time think about it
let's get the design perfect let's
optimize ourselves to minimize our byte
footprint but when you've eliminated the
need for that storage to be so scarce
then some of these other solutions make
a lot more sense and we want to get away
from trying to predict the future that
says I need this field this field will
be useful for the next five years or
somebody's gonna need this in three
years from now I better put it in you'll
get a lot of arguments from that group
that says don't put it in now when we
need it we'll put it in custom may not
space is very valuable we got to save
our space and unfortunately our most of
thinking that way so now I got to make
sure out to my space space I got to make
single bit so now you move on to your
conclusion says
very nice and there he has some schemes
to do this but fundamentally in order to
get around this problem you got to break
the database apart we got to do the same
thing we did with the object movement
going into the fred brooks model and
staring tearing the data part putting it
into little paces you want do the same
thing with the database you got to kill
that central database unit it may be a
large graph database you still may want
to tear it apart so you gives a concept
says okay let's break this guy apart and
what I really want to do is you know get
rid of it and put little databases with
each service basically rebuild the
concept of an object model and it's
encapsulation into this new world now
this is the right approach to take so
you got a coordinator model any service
in some way so you probably have a
couple of choices if you're using
asynchronous architecture very much like
some things that Netflix need then of
course you got to be able to have the
services find each other so they can
communicate with each other you have
some services like that I'm a particular
fan of asynchronous architectures in
this case we we put an event bus in
place I'll talk a bit more about event
buses a little while but we use sort of
a bit bus and let people communicate
indirectly through the event bus but you
already guys scratch your head say okay
what am i tearing everything down what
am i trying to get out of this me what's
what's the goal behind this this whole
mic why am i doing micro services again
because it's getting really ugly and the
really answer is you're trying to get
some wicket wicket speed increase and by
speed I don't mean the execution time of
the code I mean from the time I have a
requirement
till I try to get out the door that's
the Holy Grail these days your
competitors are getting faster and
faster about trying ideas out so one of
the things I got myself around is I
don't want to just get speed you got to
reduce the coupling you want to make
sure the pieces of your code or as much
as possible completely separate it from
each other and for that I go back to I
like to look at a presentation by Jim
Wyrick the late Jim Wyrick unfortunately
we lost him a couple years ago and he
had a presentation called
talked about kinases he called it almost
the it was this global unified theory of
software we reconnaisance is kind of a
really fancy word in fact it may not be
your dictionary
it certainly doesn't translate to other
languages
but read that is almost coupling in
levels of coupling and he basically sort
of said that that in its presentation
that that there's various levels of
coupling necessary there's almost some
sort of hierarchy of these couplings and
there's stronger and weaker versions of
that and he has this rule locality the
closer things are physically the more
you can couple them together but if you
really want them apart and that's what
I'm really after there you had some very
very weak forms of coupling at best
that's by the way the Euro reference to
his presentation I definitely encourage
you to go listen to it because it's a
fascinating presentation it also does
make you feel kind of stupid I mean
there's as far people in the world and
he is definitely was one of them uh he
talks about various degrees of kinesins
you know sort of in the increasingly
things named kinases where the name of
something here has to agree with that so
I have if I'm calling a function I have
to know its name in order to get compile
let's see a position you know ordering a
parameters is very important so if you
have an ordering parameter thing it's a
positional Kinesis it's a bit of
stronger meaning when i say what i mean
read when you receive one you better
believe it means read very indirect or
relationship another kind of a strong
coupling you got to be very careful of
on down through these others so you're
all as it talks about those and and i
got a lot of my design space to fund
that information so we need to move to
some sort of way to break this database
up and your first impression is going to
be let's break it up along entity lines
it's kind of the natural way we think
about we already got tables name along
these lines it's and it would be the
first approach it's not really surprised
the approach at that level
it's not it's not surprising we struggle
a little bit mostly because
microservices involves other things
we're trying to aggressive deployments
and it's going to have its own set of
issues we got to reorganize ourselves
differently there's DevOps moving in and
out there's a lot of other things going
on in parallel to micro services that
creme la nose round this so you're gonna
have to struggles movie even to
instantly micro services but what you're
really trying to build is you know you
started by saying okay well let me break
it up along the database lines because
I'm really good at database design I've
been doing this for you know twenty
thirty years we know how to break up
things by database lines
so we'll have the entities out there
well grip them together you know if
that's a sort of relational model we
just kind of break it up along those
lines natural way of thinking in fact
first step I see many many organizations
take and it's kind of driven by a couple
of other reinforcing things this
Conway's law that says you should break
up things along organizational
boundaries which are kind of aligned up
very nicely with that so yeah of course
entities services that make sense
I own I own various bank accounts you
own client relationships well you own
that service I own this service and then
you got the rest guys jump again and of
course the rest API is in its pure form
is basically just a database access
language here's how you do an update
here's how you do an insert hi you
of verbs doing that so you got rest
sitting there saying oh yeah your
service should be like a database and
and you should have you know crud
transactions against it through these
restful interfaces um in fact here's a
quote from one of their services they're
they're very commonly used to create a
ps4 web-based applications I mean that's
the guidance we're getting so you
convert your model list of its the base
services that's the first thing you want
to do so take that those database
designs and and basically you know build
yourself a little service around each
one of those and then you get the
question K where you put the business
logic
how you gonna glue these things together
you know what's what is where you put
that do you kind of create a new service
as sort of the the moderator or the the
orchestration engine that does his work
that pretty much will turn into some
sort of God service pretty quickly with
massive methods and all of the business
logic there or you kind of stuff it in
one of the others other places rails is
basing big fan of stuffing the logic
back under the database elements so
maybe should do it like rails does it
and so you run across these dishes out
where you we are you gonna put this
business logic you and it's really not a
good answer
because all these answers lead to you
know big tar balls of a new nature and
they're even more issues coming up it's
one of the cons on performance I mean if
these used to be database tables and you
really want performance do you really
want to go to every wreck
every record of the per client pull them
in pull every record of the orders pull
those in figure out where they they
match up to create the join I mean the
whole idea of joins and databases is
trying to minimize the interaction with
that yeah if you sort of pull these
things in different databases you're
gonna be stuck with across the network
and restful interface is pulling this
stuff in yeah you may have a get 30 game
okay get back playing on your machine
but you're still gonna drown yourself at
how many bytes you're bringing across so
you if we tried to joins it and what do
there's really efficient things that run
these inserting algorithms and give you
back one or two records you're gonna be
pulling all the raw data in and trying
to make that thing so it's not gonna
work and of course the cap theorem comes
in as well so this these are problems
you're going to run into in fact Chris
is what clients are starting to run into
when they start going this way in fact
this is where SOA ran into a lot of
issues when they started trying to
separate out and along the lines of a
big databases over here and dig Bay I
own the customer database you own the
you go in the orders and we'll sort of
try to figure out how we're gonna
communicate with each other they ran
across the same issue so let me go back
and sort of say let's talk about what
problem you're really trying to solve
you're going to micro services all
particular problems I first thing I'm
sorry I asked my clients as I get into
this is sort of what type of problem she
was trying to solve and I use the
connection framework from Dave Snowden
Dave basically says the world is divided
into four types of problems it
classifies simple problems as having
very straightforward cause-and-effect
relationships not very interesting for
us complicated problems there's a is a
cause-and-effect relationship it's a bit
convoluted it would be like accounting
rules and stuff like that but there are
experts that can do this for you but he
doesn't stop there he says they're also
done the set of problems he's seen
called complex and chaotic and complex
problems there is a there's no
cause-effect relationship just because
something happened doesn't mean you can
figure out why yes for this particular
instances I can fear exactly why it
happened but it doesn't help me predict
the future your financial markets Google
advertising
all your recommendation engines are
these are sort of complex problems yeah
solve a lot today
sometimes with sequel databases so it
turns out of this sort of structure
consistency is very important in any a
rule-based thing if I I don't want my
bank account to have maybe you got five
dollars maybe you haven't got five
it didn't make it this is not a fuzzy
problem so it's a complicated problem
but you need consistency but a complex
problems it is fuzzy sure it alone your
money I don't know maybe I mean I can
have a thousand different answers to
questions like that that's still not
sure I should loan your money or not
just still some risk associated with
that so it turns out these are fuzzy
problems and because they're fuzzy I can
have eventual consistency with my data
I'm not stuck with having to have
perfect information perfect information
and yet a larger your organization's
about oh my I like to have perfect
entities so you have to be perfectly
synchronized otherwise the system won't
work actually for complex systems I
don't need that so this is a restriction
you need to kill that off if you go back
to people are beginning to microservices
something other success stories that are
being talked about a lot of them have to
do with taking some gigantic rails
applications that you know startups have
built up and all of a sudden they're
kind of like dying and not dying very
well and and if one day what I'm doing
is they wind up having to go into those
applications and convert to micro
services but the first step in that
process is usually just taking the
database and breaking it apart
don't worry about the services yet let's
just get this gigantic database into
small pieces so I can wrestle with it
better then I can start building
behavior oriented services around those
things and the key is behavior oriented
services not data organic sources or as
I want to service to actually do
something ask you the question it gives
you some conclusion is strong based upon
doing some analysis it's not a
conclusion to say I needed to know how
old you are you're going to tell me
that's not a service it's like what are
you gonna do with that age what are you
really curious about or you think I'm
gonna die too soon okay ask me that
question that's the sort of thing you
want to change the behavior around
services and so you'll wind up with
several services with
somewhat similar data and that's okay
and the guys have been doing this and
factors good presentations about this I
know that Chad Fowler is six wonder
candor talked about this two years ago
in Berlin that presentations online
about how they took his gigantic rails
application and tore it down the Gilt
Groupe has a very similar story as how
they did their their site for fashion
sales halo did this for their for their
taxi service in London in the Ireland
and they all did the same way they tore
the database apart and took it apart so
this is the approach you got to be
thinking about how you're doing my
favorite approach to this and I talked
about this quite a bit is the consummate
event bus one of the things I'm seeing
in happening in Silicon Valley
extensively is that getting rid of the
operational sequel database that's not
the core of the world anymore the core
of the world is a persistent event bus
the idea that things are happening and
they go put an event they're trying to
optimize the stories they're not trying
to filter anything they're putting all
of the raw data on the event bus because
data is cheap now that you go back and
analyze and reanalyze it that our
heart's content and that's kind of how
they build their systems so the movie
throw that a guy away and they put it in
an event bus now they probably still
have the operate the reporting store
because frankly reports and sequel work
really well together
and reports are things that you want to
make some business decisions based on so
we're still pushing information out to
the data warehouses we always have but
little databases are basically sort of
cached information or whatever
information that will serve as nice but
just just all it needs I talk about some
examples of this when I talk about
microservices where we kind of have lots
of little databases in fact we don't
necessarily choose the same database
technologies each service needs what it
can use what it has to you need so my
scenario I like to talk about I'd like
to talk about this advertising is called
an offer engine this is a case where you
basically take you want to put more
advertising on the site somebody comes
to your site you get a chance to sell
them more the McKenzie guys call this
share of wallet now how do you get a
little more
a customer that's already showing up at
your door and so in there yeah behind
this you would be I you know I somebody
shows up at my website I put myself sorr
an event bus in place put a couple of
services and so basically the guy will
come down and say you know you have some
ads for me and a little yellow guys my
answer yeah I got some ads and they'll
compete for the business it's kind of a
fuzzy problem what shad should I show to
get more your money and I may have other
an other little little micro services
start running they're providing
additional information about membership
of my free Corinne a program or market
segmentation but you sort of tear into
these little services look how they look
you'll see this little service right
here is actually consist of something I
would run in four containers there's a
little kids a little micro service or a
oh by the way I need some advertising
put some message on the bus there's
another piece of code in there sitting
listening for the answers it says here
are some answers coming back from
various services bidding for your
attention and I'm gonna judicata tween
those but I'm gonna do is basically
stick those a little red is database
cash up and after 300 milliseconds
because I'm running ie6 or some other
browser like that I'm gonna basically
look in the res database pick out the
best answers I found so far and push
them back to the user a micro service
but it has a little database little red
ass databases for pre-k cache I'm not
too worried about backing it up if you
lose it
final lose it I just rebuild it because
it's just a cache but it's tuned very
nicely for this environment similarly
the membership service this is where I
if I want to find out whether my user is
actually a member of my frequent render
programs because I turns out you
remember my program terms I want to give
you a slightly better deal if I give you
a better deal your life more likely to
stay with me nice rich cheaper to hold
on to you then try to win you back from
the competitor so we'll give you a
little better deal but of course you
open that little service up and you find
out there's two or three pieces of code
running in there again in this case
three containers I got one little
service there looking for somebody to
pop a message on there saying I have a
need for an advertising and who it is if
I can look up the person find their name
go by the way
Sally is a platinum we want to make sure
we try to rent to her and it's pulling
this information of a key value store
because it's basically just user ID
status user IDs status it's all it is
now that table it gets built every night
we go to the data warehouse we pull up
the latest status for everybody we put
into a little key value store little ETL
process runs as a cron job in the little
micro service again database if I tweak
that database that we'll definitely have
to tweak the blue code but I've minimal
I don't tweak the yellow code or the
green code just because I touched that
one
and again this tense if this database is
out of date fine so maybe I need run
last night fine so I'm working with day
old data I can live with that for that
problem
I will consistent is fine so I talked
about this sort of style of programming
in a workshop I ran we ran it yesterday
I think some of my students are here
today and I found it interesting because
I've talked about this sort of style of
programming for a while and people nod
their heads and then I put an animation
together watching you harlots all the
stuff work really well and people said
wow now I really understand it and then
I started running the workshop and
people say oh ok now I really really
understand it because it works out very
strange
it's again a workday workshop but
dungeon quite a few of these and I'll be
doing quite a few more late in the near
future but it almost to a person a lot
of these people in the class say yeah I
could buy companies doing micro services
and almost to a person when they walk
out of the class they say wow we're not
doing that we're doing entity based
something we're doing something else and
we're beginning to have trouble that's
why I'm here at the class because we
think we're doing it we'll be doing it
wrong and come to the right answers yeah
you should be suspicious of what you're
doing because basically it's not like
so it's kind of my you know my rant for
the day your micro services are very
cool I think it's a very powerful way to
work it's extremely powerful for solving
the complex problems which are in fact
sort of the problems that frankly you
make a lot of money yet we solve complex
problems that a company I worked in in
London fork Fork group they were doing
internet advertising
we made fifty million pounds with 50
people that year so solving these
complex problems and solving them
effectively in solving them very quickly
was compared to advantage for us and
these are the sort of techniques we were
using so it works very well so for
solving complex problems to get a
rethink it if you're just doing micro
services because it's cool yeah don't
don't start please we don't want to hear
about the failures if you're trying to
do entity based micro services you're
gonna hit a wall
I was prefer you stop now and rethink it