TalkFrom

Outcome Delivery w/ the Mobius Loop

Wolf, Tsielepi, Jakobsen, Benefield, Hagensen

Recorded at GOTO 2015

thank you so we're gonna have three
parts first part is a really quick
introduction to outcome delivery and the
Mobius loop which is the framework we're
using to create the outcomes and just be
clear when I talk about outcomes I'm
actually talking about goals but we like
to talk about where we're getting to and
ways to define and measure those the
second part will be her nave from Mickey
travel so she's talking about a
medium-sized company getting management
and the entire team to work in an
outcome driven way third talk we're very
lucky to have a mix of some suppliers
from systematic and the customer espen
so that's quite an interesting final
case so if you walk out of here you will
miss out talking about outcome driven
contracts which is a really big thing
for government to be doing in Denmark
and outcome delivery through it and a
lot of the trials and tips that we've
picked up along the way so the first
thing we can do is talk about outcomes
and what this means one of the easiest
ways to explain this is with a quick
exercise who's doing agile and the room
hands up okay use asterisk a lot of you
so I got some good experts in the room
see how good our techniques are so let's
take a simple user story we'll write
we just had a coffee break so I want a
coffee so that what's the reason that
you wanted a coffee in the break you're
allowed to talk back at this point
stay awake yeah what was the session you
went to before this one kidding don't
dare say Henriksen so that I okay we all
want to stay awake okay so that's the
reason we want our cup of coffee now
that we've got our good story we're
gonna be writing some tests for it so
when you get this cup of coffee team
delivers it and said there's your coffee
what are you gonna test for what are
some things you care about throw them
out
you don't like the coffee okay we have
we have remedies for that in the break
what other things in your coffee do you
care about sighs okay what else
temperature okay anything else you want
milk yeah anything else
caffeine no milk no milk okay so have we
if we deliver all this taste sighs
temperature milk caffeine is that a good
story have we fulfilled it why not
I'm hearing no what's wrong tell me you
meant to be experts erotic stories come
on we haven't got all day right right no
one got to are they actually awake this
was apparently a reason which we totally
forgot about so if we put an awake now
what do you think is that a good story
yeah they're unhappy with that yeah okay
and this in lies the problem we have at
the moment
we are really good at telling people how
to do things we talk a lot about
self-organizing teams that gel but
actually we have delivered a solution
here right I immediately told you we're
going to give this cup of coffee and
you've said so we stay awake and the big
thing you should always look at is the
question of why
why do you actually want that coffee in
the first place so tell me that you want
to stay awake but let's ask why do you
want to stay awake because
Henrique and I have been talking about
this lately you know conferences are
boring we want to try this new concept
where we have beds spread out in there
you get there and Henrik goes Lego and
drug and you get to go any here
scurrying scrum not evil and you like
you learnt it all after that session so
why do you want to stay awake why so you
can learn oh okay now that's kind of
interesting what we're finding out is a
deeper reason that we want to actually
learn something okay so we start with
why and this is something the framework
tries to get us to think through the
problems what are the reasons we're
doing something then the next thing we
have to do is actually make these more
tangible so when we say learn let's
start coming up with some outcomes how
would we test that you guys have learned
something I mean it's a conference and I
know we have ratings at the moment I
think at the end so if I look at the
ratings and I say okay as a starting
point
people are getting 50% I guess is the
average and if we want to test just a
quick do people feel like they've got
some value that they learned something
maybe we could make that you know 90%
or ninety one hundred percent when you
go out just remember that rating and
then we talk about value so we put in
what's the value of doing this value is
sometimes quite hard for us to qualify
or quantify and this is where we get
into how we prioritize things by
understanding a lot of the cost of value
trade-offs with reaching our outcomes so
some value with learning list today some
takeaways would be maybe knowing how to
do outcome delivery can get you a better
job maybe you can build better products
and you'll be able to
make happy customers maybe you even get
a raise and earn more money so these
start when you go through the framework
you start seeing things like value
they'll keep reinforce we understand
some of the more intrinsic things we're
going to measure against we're going to
look at what we call options so
personally I'm not a big fan of
requirements or anything else what I'm
looking at is what are the different
ideas that can help us reach our goal so
going back to this idea we want to learn
what are some different ways that you
can learn reading okay so you could do
some reading videos courses trying stuff
experiment right we have lots and lots
of options and one thing that's also
handy is to go back and flip the
question around ask why are we not
learning right we might find that in the
one of the previous sessions maybe we
found that new graphics tablet 10 people
fell asleep and we actually think that
for us we want to get that down to zero
so we can start having a hypothesis that
if we keep people awake we believe will
improve the learning outcome now instead
of just going off and saying to the team
right guys go off and build a cup of
coffee I can throw this to the team and
I say to them what are some different
ways that we can keep people awake and
they might turn around and say you know
what well one thing we can do is go and
give them these caffeine tablets another
idea is we can give people a can of coke
another idea is we can give them a
coffee right and they'll come up with
many different options so instead of
limiting it from the beginning with one
we can now look at all the different
ones out there but maybe there's
different cost points for example that
caffeine tabs cost few cents coke costs
$1 the coffee which is amazing I've
heard and the brakes cost $2 and this is
where options are interesting rather
than saying we have to do them
we keep them open and we say we don't
know I've kind of trained most my teams
now instead of calling it a product
backlog call it our giant list of
guesses we are completely clueless until
we validate and understand if they bring
us any value whatsoever so up here we go
into this delivery cycle and delivery is
made up of I call REI sort acronym so
everyone's about potentially shippable
product but I like having three states
that we're constantly running in one is
we're running an experiment oh sorry
research the next one is we're running
I'm an experiment we're trying something
and the last one is we're actually
implementing and delivering something
and the idea is that we want to be out
of validate ideas the least amount of
investment possible so in this case we
might say well we need to do some
research how many people are falling
asleep is this a real problem we might
say well let's run an experiment maybe
we try one so I go and give out all
these caffeine tablets but I find half
of you stay awake half you won't even
swallow the tablets half of you a manic
and you go home and clean the house all
night long so you come back to me and
there's all these various complaints
like how's okay and I might say well
let's try that kind of code so I give
out the cans of coke actually that's
kind of nice that we kind of like that
some people ethically won't drink it but
you know different reasons
and that's good until the weather turns
maybe it gets cold then we need to try
something like hot coffee
or if we're smart we might run three
experiments in quick succession give out
samples all of them then we can look at
the results and when we look at the
results we go into this idea of being
able to measure so measure is where we
want to say did we actually impact
the outcomes so if we go back to
original outcomes if we thought sleep
was a problem did we actually keep these
people awake not only did we keep them
awake but we want to show the
correlation to the high-level outcome so
in this case did people actually learn
something and for me I probably going
into jobs did it stick our people
actually applying this after the case so
we're constantly reassessing where we
are progressing against the outcomes it
helps us course-correct as we're
building our products and our strategy
so if we go back up to the last of the
curve we end up and adapt adapt is where
we now get to say you know what hmm that
was really great that little research or
the experiment we ran really paid off so
we think we actually are going to
deliver coffee this time or maybe none
of it paid off and we say hey we need
some more options or sometimes we say
you know what those are really pretty
rubbish outcomes we're not testing the
right thing we don't really understand
it um so this is a little simple
framework and if you look down here of
course it has to be on a canvas it's the
only thing that sells nowadays
apparently we have this little mobius
canvas that structures it nothing new
right these are all ideas we use but
four years of trying to get people to do
outcomes thinking I really had a wall
and I found back to shoe hurry we needed
some simple frameworks to step people
through so we needed a way that they
could structure their language and that
they could go through and kind of use
this as a mental model to do their work
so I'm not going to talk more about it
cuz I want to take two case studies
because it's enough to hear me going oh
it's really cool you want to see people
using it and practice what they've been
doing with it so first up we've got
Renee from key travel and Renee was
interesting she came to one of my
classes I still do scrum classes because
I pay the bills but I do a
bait-and-switch and teach a lot of the
outcome stuff in there and after it
Renee said to me hey can you come in and
it was interesting can you teach my
business to kind of be good citizens and
with us in the agile team so they've
been doing agile for a long time pretty
good at Joel and so there's a story of
how they really got the business on
board with that and I'm gonna try this
one for you see if this works and these
go over your ears and under your
beautiful blonde here and because you're
a girl like me you can either put on
your pants or put it on your thing thank
you
hi guys um so we are at Mickey travel
and Mickey travel is a leading global
wholesaler of travel providers so what
we do is my division specifically
focuses on the onward distribution of
hotels so we go out we contract hotels
directly as well as other travel
providing services and we make that
available for onward distribution so an
example of a well known brand that you
might be familiar with which isn't a
client of ours because I can't talk
about particular clients would be a
brand like Expedia so we would provide
our contracts and availability through
Expedia and they would then distribute
that onwards to other organizations or
otherwise directly to the consumer so in
preparing for this presentation I
actually went through a global strategy
conference that we did last year for the
organization and this was all about what
we did and what we achieved last year so
I went through that going wow 16
releases we did 258 bugfixes we did 204
user stories and 42 quick wins Wow
there we go how's that there we go is
that better
great thank you so we were delivering
all of this stuff and we were doing so
amazing so isn't this great so did this
actually return that 50 million that the
business expected from all of this
resources times money spent into this
hello okay way okay so much better there
okay so we went through like I said the
global strategy conference that we did
last year and we had delivered so much
stuff and we
she thought wow this is pretty amazing
right but looking back at this who
actually cares who cares about all the
stuff that we delivered what was the
value did this actually achieve the 50
million that the business expected so no
it didn't so what was the point of this
so we had really great fantastic scrum
teams going on as Gabe you mentioned
Thank You Gabby um and we're doing
everything right and delivering
we're delivering all these outcomes but
where was the value so the biggest
problems we had was we're delivering all
the stuff but we weren't seeing the
value being returned so a lot of the
stuff wasn't even being used there was a
long queue from the business the the the
value and the power struggles there was
a lot of power struggles between the
internal teams and there was a lot of
competitiveness going on
so this something had to change here but
we didn't quite know how to begin to
actually get the business working a lot
better with the scrum teams to actually
deliver that value so that's where my
Mobius loop journey began so I reached
out to Gabriella to actually conduct a
two-day workshop within our organization
so she came in and the the main goal was
really to stop the waste within the team
so every time I started talking to the
business about MVP they started getting
scared what was a Minimum Viable Product
this just means it takes not going to
deliver what I want this is just broken
promises you just want to deliver a
little piece of what I want isn't it so
they were very scared about this whole
concept but by actually going through
the loops of Mobius loop the actual
steps where you talk about things you
define the outcome and you actually have
a common goal and common ground
this really sticks with the business and
it really helps us communicate better
together between take and the business
it also gave us permission to experiment
more because we went while we were doing
MVP we were doing prototypes and a lot
of proof of concepts we weren't actually
experimenting nearly as much as we what
we do now so this workshop wasn't a tech
workshop it was all about the business
we had everyone there from our very
senior sales executive to our very
junior
whoops I beg your pardon that's gone
forward where's my clicker there we go
okay so sorry back to where we were we
had everyone there in the workshop from
the very junior sales executive to the
very senior executive called hero in the
bottom there and basically we were
working with with the whole team there
we even had an agile hater in the
audience there he's actually holding his
hand over his face in shame so what we
took a lot out of that workshop and but
Gabby actually said to us off to the
first 15 minutes
she actually said to us guys what are
you doing your guys are so competitive
and help each other things will work
better you are part of the same
organization right wow that's really hit
home for us it was pretty much a big
stab and you really felt it it really
rain true in terms of the
competitiveness and how we weren't
working together as much as we should so
within the workshops we actually took
some real-life examples of what we were
doing today so features and products
that we were actually developing and one
of the teams started working through the
Mobius loop the seven steps and it turns
out that one of the items were actually
already delivered it was just toggled
off and done many years ago so people
didn't even know it existed another
feature that the business were actually
talking to us about and wanting tech to
deliver on they found out that you know
what going through these steps it could
actually cause more damage than good if
we actually went ahead and implemented
it so this they told us not to implement
that feature and the business actually
wanted to conduct more research and more
experiments on this to see if it would
actually add value also our senior
executive hero over there um he said to
us at the beginning of the workshop you
know what Gaby I apologize I'm only
going to be able to be here for about
two hours out of these two days because
I've got really important things going
on turns out he was there for pretty
much the whole two days borrowing about
two of those hours so it was really
positive the way that it actually say
thin so within the workshop we found
that we were really super competitive
and we really all deduced this after the
shop and we needed to work together to
be more successful we were also building
a lot of the wrong things building too
much when you build the wrong things
those are then features that you've got
to maintain for eternity so there was a
lot of waste and very little cohesion
that was going on in the organization so
we needed to change this so we did and
this is when we move to an outcome-based
to the remodel so instead of focusing on
the deliver part over here like most
scrum teams do we looked at the whole
loop and actually worked with the
business from the outcome all the way
through through this loop and this
really allowed us to spend our time
working on valuable outcomes rather than
just delivering output all the time so
what didn't work well okay the first
time we started unfortunately we
couldn't keep Gabby there the whole time
we were there so once she did go on her
merry way we have to try and get this
working within the organization itself
so we did have a challenge the first
time we started we actually got our
product manager together we got all the
key people towards the problem together
in a room and we started thrashing out
the problem we started with the Y of the
problem and actually talking through
naming the problem etc so you spent the
first four hours just coming up with the
Y's and naming the problem that's a long
time right and at the end of those four
hours we had little take waves except
the takeaway of another four hour
session in the next few weeks and that
wasn't going to fly so we needed to
change something dramatically because we
can't continue with four hour workshops
like this you can see how excited
everyone is to be there so so that's
what we did we decided to change tactic
because we had to look at what wasn't
working well and adapt and make sure
that we were actually learning from our
mistakes and changing the way we were
working rather than forcing the the
whole seven steps process in Mobius and
going through PowerPoint presentations
and forcing everyone through this
process so the next approach we did was
we actually got the whole the whole key
people together that we did previously
and we talked about the problems in our
organizations the big why why weren't we
actually hitting our targets for this
year for 2015
why were we not hitting our far
your plan right now so this was pretty
important at the moment there was a lot
of conditions that were impacting us
within travel we had market conditions
like exchange rate fluctuations
with the euros and the yen we also had
terrorist attacks in in Europe in France
particularly and also in Africa that was
impacting travel so there was a lot of
outside competitive external factors we
also did some company competition
analysis competitive analysis and we
found out that our company potatoes were
also flat the same as us within Europe
so there was something going on here but
we needed to fix it so we discussed the
outcome and the value that we needed to
achieve to actually hit our targets and
we deduced that we needed to hit an
additional 21 percent revenue in 2015 as
well as maintain our gross profit ratio
in order to hit our targets so we mapped
out the outcomes and we discussed all of
this as a team and we deduced that this
was pretty large right so where do we
start where do we start to even try and
hit these outcomes so the next thing we
tried we actually got together and we
had our first big one-hour session where
we tried a guerilla style approach so we
talked about what we'd have to do all
our problems that we'd have to list out
an opportunities that would allow us to
meet our outcome of that 21 percent so
we literally posted it everything down
on a whiteboard we used green post-its
for the make revenue we use pink posters
save because there's normally around
make save or protect revenue but we
really used to make and protect because
this was focused on our outcome and we
thresh those all out we then had a very
big board with a lot of problems and
opportunities for us to make our targets
so we couldn't actually discuss feasibly
them all because we only had another 20
minutes left of the meeting of the our
meeting so that's when we took the
guerilla style approach as well and we
just used the the three dots so the
three dot voting system we just got
voted on the ones that we felt were the
highest outcomes or biggest problems and
we then set out the probably the top six
and then we
Gusto's in a little bit more detail we
had a few minutes each for each of them
to actually talk about them a little bit
more detail
turns out our search response time so
our speed was one of our biggest ones as
well as our errors so those are two that
we wanted to move forward so we needed
to understand the problems a little bit
more first Tech's work take were adamant
that the speed wasn't an issue and the
business thought that it was so we did
the five why's we did cause an effect
diagram and we also interviewed our
users to actually find out a little bit
more about the speed problem turns out a
lot of the cities that they were
searching were too slow so London Paris
Rome we were returning to many hotels in
those cities so it meant we were lot
slower than expected turns out after
calculating from those top customers we
were losing around five million pounds
revenue where we were where they were
turning us or for those big cities we
didn't know they were turning us off
because they were still getting search
volumes coming through but only after
interviewing them did we found out there
was turning us or four selected cities
so this was a big option for us to start
with so the next day we booked in a
second one-hour session and we focused
on naming the outcome and also defining
the outcome speed so we came up with
improve the search response speed for
all searches to under one second and we
also visualize this and put this on a
trailer board so that we could
constantly keep this focus so there we
have we have it up on the trailer board
and and the fun part was now going to
start and coming up with the options to
meet that speed so one of the options we
came up with was our XML file size we
had a hypothesis that our file size was
too large I mean who needs to know about
the toilet in the hotel lobby right
you're just worried about rooms that are
available so we were potentially
providing a lot more information then
the customers actually needed so that's
something that we wanted to look at
further and the next option we had was
specific cities which is where we
mentioned about the clients who was
stopping searching for particular cities
so we wanted to review this and see if
we wanted to reduce some of the cities
that we were actually returning the
other one was the image file hypothesis
quite a simple one we just thought that
the image sizes were too big so we
wanted to do some experimenting on that
and also look at some cloud-based
solutions to to speed that up we also I
decided from there that we needed to do
more user journey mapping to understand
really how our clients work and also
system mapping because potentially there
were problems deep down in the system
with business rules that we didn't even
know that we're being applied that we're
causing us problems so now it was time
to put all the options down and
experiment yeah we put all the options
down in Trello so you can see them there
and those are the options we were
actually exploring so what we did is we
had a scrum team that we put together
and we call them the improvement scrum
team that was the revenue improvement
scrum team we then had a customer to
come in and visit so this was our key
customer on the xml side so that we
could actually do some investigation
they came in and we actually did a deep
dive of the XML with them to go through
all the options it turns out they only
wanted about 20% of what we were
providing in the XML so that was a very
good exercise that we did with them so
we managed to reduce the file size from
five makes of the xml down to 522
kilobytes which was actually a 25%
improved response times and from that we
predict predicted that we would actually
get a 2.5 million uplift and that was
just from the pilot client that was
testing out the pilot for us so we did a
lot of experiments as Gaby mentioned the
research experiments and implements that
and we knocked up the prototype the
client is now currently implementing our
new XML at the moment and as mentioned
we've hit our we'll be expecting an
additional 2.5 million as a result of
this and we've improved the speed by 0.5
of a second so we actually halfway
through just by one option implementing
one of those options that we came up
with so we also put all the tasks down
in Trello so instead of using the
trailer board like you would
traditionally in a scrum team we
actually use it for all the tasks such
as
getting the user to come out
interviewing the user so it was a lot of
the business passed the research side
too as well as the implementation that
really helped us to track that so the
whole process we're conducting
experiment of the experiment with our
business and one of the experiments we
conducted was with the customer where we
we switched we we turned off a switch
for customer and they resulted in 90,000
euro uplift just from a switch a simple
toggle that we changed for them we
actually conducted another experiment
over a 48 hour period during the weekend
and this is when it's not quieter period
so we didn't do it you in the week
because it's more busy for us and that
resulted in 175 euro uplift from that
experiment a very dodgy experiment to
that was very important so that resulted
in an additional 35 million per annum
uplift the important part is to measure
so measuring is crucial at times it can
be harder than others to measure
especially when you're talking about
response times so there's so many worth
traveled is so many outside influences
such as terrorist attract mavet mother
nature even you all heard about the the
upheaval that the volcano created a
number of years ago but it's so
important for us because it's the very
core of our business and we need to be
the most fastest and most efficient
service possible sometimes it's based on
our get best gas measuring and sometimes
most of the time it's based on
historical data the most important part
is to measure each time you deliver an
option measure it so that you can
actually see what outcome what what you
are against your outcome and what to do
next so there are other outcomes that a
lot easier to measure so our error
outcome that I talked about earlier
that's a lot easier to measure we
actually took particular clients with
that and this was resulting in incorrect
searches or product codes or searching
same-day arrivals this was after we did
that and cast him a review to understand
what the customers were doing and one of
the one of the customers that we asked
them to change the way they were doing
their searches resulted in a 226
thousand euro in a month so that's a
potential 2.7 million in a year another
customer that we also asked him to
change the behavior in the way they were
searching we found we've gotten an
additional 1.7 1 million per month which
could be an additional 12 million per
year so that's just from asking the
customers to change the way they
searching for us changing their
behaviors and what they did what they
actually doing but doing that deep dive
so that's a really 15 point 5 million
just in one year just by changing the
way we're doing things experimenting
more taking risks so we then reconvened
for our final our session and here we
talked about how well we were doing on
our existing outcome whether we wanted
to change or pick up an additional
outcome so there was some outcomes that
were still yielding very positive
results that we wanted to continue with
those there were outcomes that had
reached the point of diminishing return
so we stopped working with those we
decided we wanted to work on one of the
new outcomes that came as a result of
some of the previous experiments that we
were doing which was the search response
times sorry I beg your pardon that was
the same day arrivals because what
happened was we found that customers
were searching for the same day which we
didn't actually support in our system so
if we had actually looked at
implementing this this could net us an
additional 6 million so we were busy
root-cause was mapping that out and
while we were doing that time came up it
was an hour we were getting kicked out
of the meeting room I'm sure some of you
will relate to that and so we had to go
so I was trying to wrap up the meeting
and then our big agile hater let's call
him Jim said to us no look look we can't
stop this let's go to the boardroom
we've got to continue this discussion
that's when I know we had achieved
something really special here so this
was one of the most out positive
outcomes from all of us aside from the
obvious value
we're finally working together as an
organization towards a common outcome
it wasn't take versus sales sales versus
purchasing purchasing versus
reservations it was every single one of
us working together on that common
outcome we had one language a common
language and by putting all of those
simples
together and working through that
framework that really helped naming the
problem defining the outcome all of
these steps really allowed us to work
together they made people think more
they made people talk more to
collaborate so prioritizing is now
easier than ever it's clearer what
technique to work on and we don't have
conflicts in priority we focus on
delivering valuable outcomes which makes
all of our jobs more rewarding too so
now as a group we conduct experiment
after experiment just to see what would
happen before doing anything else so now
we collaborating as an organization on
common outcome and leaving any
competitiveness and blame games behind
thank you yes hello everybody my name is
Espen I'm a program manager in the
municipality of wolves and I'm currently
heading a program where we are we are
getting ourselves a new electronic
healthcare system for the most needy
elder care we have some 25,000 citizens
in all receiving this kind of services
running around and trying to provide
this healthcare and all these 6,000
people they need to know what is
everybody else is doing so that the
people they get the service they deserve
and we need an IT system for that and
that's what we are doing
my name is Tobin Haynes I'm I'm the
product owner from the supplier side and
we are working very hard on using the
methods that that the municipality has
started and that Gabriel has helped us a
lot
doing this work working on goals and
outcomes instead of working with
specific requirements and it has been a
really nice challenge and I hope you'll
hear about it in a minute and my name is
Carsten Jacobson I've been working as a
lean and agile coach for systemic for
the power
ten years and I've been deeply involved
in starting up the product organization
that we are using to provide this new
product to the municipality of Falls
okay oh joy yeah huh yeah it's always
backwards it's this one okay you're
gonna be photo guy okay so we've done
the project okay so it's been why did
you take this outcome delivery approach
well it's it's pretty banal really
because I've been there before and done
this many times and what we usually do
is that that we who do not know very
much about IT systems we try to design
them in detail and then leave to the
supplier who do not know anything about
our business to understand what we
really want so if we do as we usually do
everybody is doing his worst and and
that's that's a very bad idea so also
I've been pestered over the last years
with suppliers suggesting that we try to
focus on our our needs and our outcomes
and the functions that we need support
it instead of trying to design IT
systems which we are not very good at so
I decide that this is this is the time
now we're going to do it so that's why
really and tell us what happened when
you first started trying to put together
this outcome based approach well we we
started we started we did a lot of work
shopping in our organization large-scale
we tried to figure out how can we become
more efficient not talking about IT
systems but more talking about the way
we conduct our daily work our daily
business and and and learned a lot of
parts that we actually could work
smarter and then we started asking the
whys basically we knew why we were going
to have a new IT system because the
contract of the old one was was becoming
extinct it was dying of old age so we
were going to have a new contract anyway
so I was a good opportunity and then we
started asking ourselves the whys why
are we actually doing this
and you have to keep on asking the whys
until it becomes absurd and then go back
a little so how did you get by them
because you know outcome thinking is
very new and one of the biggest hurdles
I have is in procurement and actually
and the lawyers as well and it's
interesting that's the first time I've
actually been able to really work
closely with both the customer and
supplier wanting to work in this way
yeah you know this organization that I'm
working for
they hired they hired me and our our
head IT architects they hired us
specifically for this job so we came
into the organization from the outside
so it was pretty easy for us to sort of
maintain that what what what you used to
do what you've been doing over the last
is that doesn't work this rubbish now
you need to listen because we come from
the outside work we know what's modern
we know how to do it nowadays and they
started listening and and so they just
said okay we have been there before we
have seen that these kind of projects
they fail we sit in our own organization
we have seen it in in other public
organizations so so they've just
listened and and it was not a big issue
really what was the it did mention some
quite well-known Danish project that I
think if you're if you are a Danish you
will probably know about this text depth
systems it's a good example right it's
it's just worse worth remembering the
the basic features they started
designing it in 2004 really and now it's
15 and they finally decided that to
ditch it it's not going to work and and
in m79 and 13 they postponed the system
again is a nice figures
7 9 13 every day and can relate to that
and and and I think that over these
years the the budgets are tripled or
quadrupled or something like that and
and now they actually decided they were
going to ditch the system it doesn't
work and this is just one amongst many
many examples and we've almost gotten
used to it that well we are we have
wasted another five billion something
said the bar was low see it nothing and
how did you create the initial tender
because you've got all these supplies
asking you
for an outcomes-based approach so what
happened next well when they decided to
figure out what are the outcomes really
so we started asking the whys and at at
some point we found ourselves discussing
the nature of God and the meaning of
life and then we knew we've gotten a
little too far and that's what you have
to you have to ask the whys until you
get a little too far and then go back to
a level where it makes sense and we
found out that there were six six basic
outcomes that that we really really
wanted and I think I think I can list
them by heart by at this time we wanted
happy usage really do we want satisfied
as well because if we don't have happy
users we won't have happy citizens then
we we are on the constant pressure to be
ever more efficient every year so we
need to have support for this
ever-growing efficiency and then we also
need to innovate all the time so we need
we need also to how good good basis for
innovation we need to rethink the way we
conduct our business so we need
innovation and also we need a freedom to
develop because what we have experienced
in the past is that we have these
wall-to-wall IT systems which are
effective logins every time we find some
new IT feature out there that we knew
little IT system a new app that we would
really like to add to our portfolio
because we think it might give us some
value then the current supplier our main
system he looks very tired and then he
talks about very lengthy projects and
lots of money and then everybody gets
tired and then we just forget about it
so what we effectively have experienced
over the last many years it that the the
IT the healthcare IT development it just
passes by outside our windows and we
can't reach out and get it so we need to
have a solution we need to have a
solution that gives us the freedom to
develop along with the technical end and
I see development out there so that was
very important it's
ask a question now turbine and custon
what's your side of the story
because you get this you know it's like
Nirvana we suddenly get this
outcome-based tender so what happens
next
yeah we said Gray's yes finally someone
that wants to work the way we want to
work and then with this was a dialogue
based tender so we have some dialogue
meetings and then they start asking some
really really difficult questions and
and we find it's very very hard to find
out what to do about this because they
said happy users okay then now they ask
us how do you want to measure that we
have had happy users what to do about
that we didn't know so well Caston and I
had met Gabriel at the point and we
joined the hot housing course in in
London and and we will add something
about well there was a method for doing
this so well we panicked and called
Gabriel and and and God arranged the
meeting so we took to London and talked
to Gabriel well first our first answer
that we gave it the first toilet meeting
it was quite clear that ystem was not
satisfied with our answer even though we
have tried quite hard to make it yeah I
got a funny emails saying we need to get
hold of you soon then the next email I
think was we hope you're there on Monday
cuz we've booked tickets and we're
coming over yeah tickets Friday night
and we came Monday and we hadn't really
talked that much so we were not sure
that Gabriel was there but anyway we
needed to do that so we made an Argo and
and the result was never and and how did
you want lawyers deal with this whole
idea because again when you look at this
contract it's really to me I do a lot in
the contract space it's I'd say it's
shared risk but it's a very different
model
things aren't locked down you don't have
the same measurements this is incredibly
radical
and you know for both sides I think
you're taking a big step into the
unknown so how did you sell that
internally well it was quite obvious
that this was a good great opportunity
for us to build a great system and well
it was quite obvious that there were
windows at
in the marketplace already that had some
systems that actually tried to solve
this problem and if we should have any
reason to be there we needed to do
something radically different than these
vendors and one of the ways was to
attack this whole problem in a new way
and and actually start saying what is
the problem and try to fix that instead
of just making a system like the other
ones and spen how did you decide who
toward the contract to see what's the
difference in the responses you get back
with this new approach as opposed to
traditionally well what we did was that
we took our our outcomes these six
outcomes and we made two breakdowns one
break down into a requirement
specification trying to figure out how
do happy users translate into
requirements of functions and things
like that and the other part breakdown
was trying to say we need to break them
into some keep a key performance
indicators because we need that the
supplier takes his share of
responsibility for the fact that we need
happy users so these key performance
indicators they are at the very heart of
this contract so that's what we did we
broke these things into P performance
indicators and then within the contract
we actually focused on the incentives
and we focused mainly on positive
incentives because I have observed that
negative incentives in the way of fines
and and and punishments in contracts
doesn't work very well so you meet it
you need to make it attractive for the
supplier to help us reach these outcomes
that we need and then the the the lower
part the contract part with the lowest
was not really a big issue because what
we did was that we got some really good
lawyers and that's that's a point of
advice here get yourself some really
good lawyers good lawyers who understand
the nature of HR process understand the
know something about it because we we
often hear in this con
that these standard contracts they don't
fit in with HR processes and and
outcome-based
IT projects and so on and that's just
it does not simply true you can
easily do it you just need some good
lawyers and how did you choose the
supplier in the end what were the
responses
well we closely we did as a part of the
tender we told we had four bidders we
told them we want you to tell us what
these KPIs
should be we want you to do the
breakdown because you are the guys who
are getting measured on the fact are we
getting happy users or are we not so you
should choose some measures that you
feel comfortable about right so you tell
us and that was a part of the beauty
contest and so we had three three
companies who clearly didn't know what
to do and then we had one company who
had serious problems but then again also
seriously tried so the pic was easy
because this was very very important of
course there were other issues also that
we ever evaluated on but this was a very
important one and a systematic they they
they did a hard job and I think that
percent of the way which was pretty
impressive
because obviously everybody was in deep
trouble okay let's switch into some
practicals because I know I don't got
much time that show us the things that
so far you've been doing to weave and
the outcomes into the work you're doing
so how you went from this contract has
been awarded what did you do next to put
it together and start framing it and
that comes away yeah well well one of
the issues was that well even though
that this is an agile project it's still
a project that has a fixed budget so
what to do about that so we need to find
out what about what is what in this and
there were some minimal requirements
that that actually the minimal
requirement could translate into well
you need to have the
every day working for the municipality
and that's that's not an option we
needed to do that
so actually we fixed that and said okay
we will do that in the first part of the
project and then we will have a second
part of the project where we improve a
lot and get happy users happy citizens
and so on and when we had to do that we
had to start discussing how how to work
that and we said ok let's do some of the
things that he liked talked about before
let's have a cadence of in our case six
weeks where we talked about the goals
and these goals these goals are
discussed with the customer and after
these six weeks we get back and see how
far did we get with these girls did we
did we satisfy all of these goals and if
we did not what to do then oh I work
with goals all the way through and
actually just leave it to the team to
design the solution that's not our
problem we had nineteen to focus on the
codes so so that's basically what we are
trying to doing is really really
difficult but we have tried anyway but
before that we do need to have a product
backlog so that our EDI teams can work
so we had some very frustrating weeks
because we have not been awarded a
contract the contract have no solution
design then no requirements so what we
used to do would be to take out the
feature survey through from the contract
put them all on the board and then
prioritize them and then we would be
ready to kind of start and then we could
elaborate on the features but it took us
a while to remember that what we were
actually looking at was not features
what we were looking at was a very good
description of needs what is it that the
municipality of almost needs and how can
we create a parapet lock out of that
then we wanted to understand from the
möbius model how can we value this
product so the value obviously can be
different from the sales price
so what is the value well it's a
replacement system system so it seems
that the value of the system comes from
all kinds of abilities that you get when
you change the system and then we
started reflections on okay
how can we attribute this kind of value
to specific functionalities in the
system that are required and we were
kind of stuck until we understood that
okay what we are building is truly a
skyline of let's see so we try to build
a skyline of these needs
initially we sort of them as features
but then we realized that what we call
features were really needs so what we
were searching for words where is the
reliability where is it that I have to
choose about choose between options so
all of these different features are
mandatory because these are mandatory
needs and what needs to be done is
design design is about creating options
how do we create options but we create
options by doing field studies
investigating what are what are the
contents of these workflows that they
are doing at the mentality of ORS and
then we can look into what kind of
options in the system would support that
workflow best maybe show us what you
mean by field studies so what's
different between normal field study
results and when you're working in this
way you have any examples I think Tom
would can well of course
field study is about being in the field
but but what we did a lot of work on was
that when they came back we tried to
apply some of the techniques that
Gabriel has been talking about and start
talking about the the outcome of this
and and when we when we were working I I
actually took the first one and and
interviewed the the two persons that
have been in the field and start talking
about okay what is the context well what
is the problems that you have what are
the outcomes that come from that and how
to measure that so that was a
in an interview that it didn't take very
long but it brought us a structure into
getting out the information out of their
head and into our own heads and actually
the guy was sitting there he was doing
heard at one time so he could do it then
and we actually did it one more time in
front of the developers so that they
would be able to do it when when when
someone had been on a field study of
course the developers need to go out
there and see it as well but we need to
focus on getting the information
sufficiently amount sufficient amount of
information back into the team so that
we can find out what to ask so yeah I
think the difference of this when they
were doing this it was interesting that
it had gone from saying a nurse needs to
say enter information when they go to
someone's home so 1/8 older person put
it in and you know there's obvious
problems that I can't always log into
the current system they can't capture
was some other things like the fact that
they go into a house and it was a tablet
based system and the houses in these not
great areas were very cluttered there's
nowhere to actually sit down so they
couldn't even use the tablet in the
first place so a lot of these things
came out as you know ultimately the
outcomes for these nurses was they
wanted to spend time with the patients
that are the citizens they didn't
actually want to be typing things in so
the outcomes became about can they get
this information and quickly can they
see the key information how to design a
more usable system so the difference I
think traditionally would have been they
could have said yes we built this
functionality to enter it in check the
difference is now can the nurse actually
provide better quality care can they
actually get it done quickly without all
this frustration can they learn a new
system fast right learnability was one
of the things that we started baking in
so that you don't have big upfront
training costs for the youth health
municipality and when we did this yes
but I mean he's an architect right he's
a techie and he said oh when I did this
I had a lump in my throat it was so sad
what these people have to do and you're
like wow okay you
the techie and he was so enthusiastic he
said see I'm getting excited because I
love seeing the 10 gets into it and he
said he said you know what I'm gonna
train the team up we're gonna take turns
so when they come in we're all going to
interview them and everyone will now
know what matters when we build the
system so again it's not these metrics
it's the act of going through their
understanding that makes these teams now
care about building a very usable system
and this is what ultimately I think
you're hoping for when you build it or I
would be a puppy and and what what we
could see there was that well if you
take it in in in the in the big scale
it's well the system was a it had the
information just at least most of it
available to the users but they didn't
use it very well because they couldn't
find things they had to browse around to
well I'm not something had changed since
the last time they were there so
actually it was quite obvious if you
could see the line requirements that was
being had been satisfied but you had
done all those things that you should do
but it didn't fit the workflow that
people have it's gonna be the hardest
facilitator we're kind of at a time I'm
gonna do a quick wrap-up then if you're
interested we crammed a lot in we know
that but we kind of want to talk about
the stuff so maybe we do a spin a quick
wrap-up on just some thoughts takeaways
interesting comments we'll go through
and then this is funny this slide right
instead of building minimal viable
products minimal lovable I wanted to
change it when I saw it it's like should
be maximum lovable right anyway let's
see that yeah well that's true and and
as said just now well you know these
nurses and these assistants they are low
esteem people they are poorly paid still
they do their job and that you would
happily why because it makes sense to
them they have people they really need
jobs that make sense to them and they
feel deeply about their jobs and we want
to pass on that feeling to the techies
even that's very important so we want to
give that feeling to the techies to the
team to the guys at our supply
but what we also want to give them being
outcome based contract mind we want to
give them trust so we want to trust
these guys not in the sense that we want
to be naive but we want them to do what
they're best at and we want them to give
them we want to give them a chance to do
what they're best at so we don't want to
check the nitty gritty little details
you just want them to give us happy
users and the other things we want and
and and this trust thing is very
complicated to talk about when it comes
down to contracts and law and stuff like
that but but we need to address it and
we need to build this kind of trust into
our ways of working because otherwise
we're not going to get these outcomes so
I think that's the wrap-up from from my
side and on the first issue I think one
of the things that are essential here is
that well we have the same goal if we
don't have the same goal and it's then
it's difficult to to build trust so we
know that if this is a success for one
party then it's a success for the other
is not something you have it's something
that you need to build so we have been
working quite a bit on that and in that
reasoner world we would have forty days
in an easy tender to do system design
and flesh out the outline of the
features in this way of working we have
the entire schedule of the project to go
out in few studies to talk with our
customer to figure out what is the right