Say Hello to Offline First

Ola Gasidlo

Recorded at GOTO 2015


Get notified about Ola Gasidlo

Sign up to a email when Ola Gasidlo publishes a new video

okay so let's say my name is low I'm a
developer for more than 10 years now and
the core member of hoodie and Corgan
Iser of open tech school conference and
reject us for a misprint to type and
thank you for having me I'm super
excited about that so let's start not
much time today I will tell you what the
often first concept actually is and why
we need that so badly where the problems
occur on the client and on the server
side how we as developers we have to
rethink our application logic and I will
show you one way to implement the
concept because there are a lot but yeah
I will show you one so if I first it's
like the new cool kid in town but what
does offline first actually mean and why
do we need it the Internet it turned 25
last year in this time our relationship
with the Internet it changed a lot we do
different things with different devices
than like twenty five ten five years ago
and it's a part of our everyday life so
to wake you up a little bit I will ask
you some questions and just raise your
hands who of you used a phone today for
more than five different tasks more than
10 more than 15 we rely on our devices
on the internet but our needs changed
like you see here we've reached the
turning point since last year there are
more mobile than desktop users out there
in some countries there already are just
mobile only users like Andy tip for
example just make to make sure there's
no desktop device usage at all it might
surprise you but think about it mobile
devices are much cheaper than a laptop
for example and for many people this is
totally enough considering what they
actually using their devices for so to
wake you up a little bit more because
it's early again some questions raise
your hands who have you ever tried to
work on a train or a plane how well the
dead girl who of you ever did a
screenshot of the applications because
they were too scared of losing the data
right and hear of you ever stood like in
the supermarket I wanted to ask someone
because if I got what to buy like a
friend someone from the family and
realized damn I'm offline all the time
what if you traffic limit is reached
your hotel wife is way too expensive or
the roaming costs I like not reasonable
those are issues we all know even from
the wealthy countries for living today
our work a whole social interactions our
lives are built on being online when
you're not online it almost seems that
you don't even exist we need new ways to
deal with these issues and be able to
stay a part of the online community and
families and the situation is way worse
in the rural areas with poor or even no
network coverage in zero reception
unreliable and inconsistent connections
are still an issue no matter where we go
we need to accept that technology has
reached the limits go away from the
desktop mindset of a permanent and fast
connectivity by accepting that being
offline it's not a buck it's just a
reoccurring State and let's start and
design applications was keeping often
first in mind there are already some
really great applications of my first
applications and services out there
starting with emails which sent the
image when we write when you're online
again writing down notes during meetings
with minute i/o or creating lists with
create lists helps us to stay productive
without sending error messages as soon
as our internet connections lost sending
pictures tweets messages during a
concert ever done that it's really hard
and gets tricky when everyone is doing
that at exact same time applications
like Twitter the store at least our
messages and which is great so they're
not lost but why not sending it when
we're online again whatsapp sink works
it pretty well at least on Android I
heard yesterday so I see really great
implications in the future Google Maps
already implemented some of the stuff
which is super amazing like imagine
you're in an area you've never been
before and I'm sure we'd love to explore
that right so what if your application
would realize you somewhere you've never
been before enough space on your on your
device would realize okay I have Wi-Fi I
will download the map for you and show
you like the sightseeing points the next
bar or your cafe with Wi-Fi or the
nearby ATM which we all need and what
kind of help you to feel like almost at
home so this sounds great but what are
the real problems here
let's compare mobile first it's a design
strategy that helped cope you with the
huge Verity of devices and capabilities
like you see here in the same way
offline first it's a design strategy to
help you cope with unknowable
circumstances in connection States you
may your user my parent find themselves
in offline first is simply their
knowledge man that lack of certainty
extends a bit further further than we
previously thought with an offline mode
it's only it's only a contingency for an
air scenario but a fundamentally more
flexible fault tolerant way of
application design offline is more than
just error handling it's about accepting
the reoccurring state so you have to ask
ourselves does the application even need
to inform the user of the current
connection state it's this information
even relevant for the user and if so how
can this best be done like you see here
we don't have an internet connection on
Instagram how can this be a good
practice
why not showing the user data we saved
in the cache or stored locally
all data can be valuable to the user
bill just can I give the user more trust
in my application and leaving them safe
the last state I was in and that it
won't lose any data regardless of the
connectivity situation so the needs
don't need to think about making
screenshots of the applications and how
can I communicate to the human that
creating data within the application is
still possible and that it will be
posted dispatched and deal properly with
in the future this is one of the
concepts for the safe I think I really
love for a new one there's the unsafe
State the saving site with seven options
and the safe State and this is how it
works you see the icon in the upper
right corner the user type something the
stage gets changed and changed then it
gets updated and saved again
automatically and this looks like a
really good practice to me organized
data
how can my interface convey changes that
occur in the users current view when
they reconnect and the server pushes new
and changed data what to do with deleted
items think that can't be organized in
lists or objects that that aren't mutant
bull and themselves like emails how can
I make the unavoidable resolution of
conflicts for the human as painless and
such as soon as possible this is an
error message my data hasn't been stored
I got that but this won't help anyone
it's great that at least know that my
data hasn't been saved but what now
doing this all over again I would drop I
wouldn't do that
and finally decides can the application
make any preferred decisions of the part
of the human and preload any data they
might need later
what metrics possibly behavioral are
there to base decisions on and how can i
word all of the scenarios in a way that
doesn't worry the user that embraces
offline as the reality without causing
problems those are the issues we know
from the user sides we have to think
about what but what does this mean for
the search
and the client behavior so traditionally
the server is like the source of truth
prophet it communicates always with
server sends requests waited for answers
and told the server I have a thing that
happened and when the server is not
available the client panic was helpless
and didn't know what to do but we won't
post some data the client sends out
requests and waited for answers the data
was stored the view was branded and
preloaded again so with an async request
the client at least wasn't called and
didn't need to wait for the all the time
for the server a step forward but still
it was on call so let's make the client
independent times has changed and so the
Internet and empowered us and also the
client so with offline first we want to
decouple this relationship but how shall
we deal with the new states and what are
actually the new states a scenario we
all know imagine urine in a restaurant
the client is the guest and would love
to order some food and enjoy the dinner
the kitchen is the server the kitchen
knows all the recipes ingredients
prepares all the foods for the guests
the waiter gets the request serves the
responses independent if it's a success
or something failed so as we know the
concept it works like this the client
decides on a dish orders the waiter
serves two requests to the kitchen
kitchen process requests server gets
back results here won't be a
well-prepared dish or the kitchen might
be on fire well so the waiter would get
back and tell the guests something fails
sounds familiar all right
what's offline first it's like the
guests would be at home and not the
restaurant with all the ingredients and
recipes needs to be stored at home when
the person so the client decides on the
dish they can look it up and cook it by
themselves this works for all the
ingredients which are available and
recipes that are known but what if the
person decides to try something
knew as a passionate cook the person
needs to ask for the recipe first the
kitchen allows sharing the knowledge so
the waiter gets the recipe gives it to
cook right away because they realized
hey all the greetings they're awesome
so offline first means you can basically
cook at home anything you want just it
just works but for anything totally new
we need to send out requests and here
are the news scenarios the waiter is not
available
so the requests will fail right away the
waiter but well then storing the draft
making a queue and trying to send it
again like 1 2 5 10 seconds again and
annoying the waiter blocking the request
showing an error or just ignoring it and
make it disappear like it never happened
another scenario a part of the recipe
makes it back but not all of it because
the waiter has so much to do that they
need to rush back to the restaurant so
person couldn't understand all of it so
should the person cook just half of the
recipe and see if it's enough for a good
meal just don't cook at all cook as far
as possible and try it to call the
waiter again from time to time can the
person change the request food recipe
while he waits because craving for wheat
got replaced by craving for beans and
how will the way to proceed with the new
requests how can we keep the frustration
level as low as possible while the guest
is making a request like often for the
recipe the interface here the
interaction between the person and the
waiter should start by showing the
process of unspecified duration has
started usually done with this who loves
that you can enjoy it a bit yeah
annoying I know based on a different era
scenarios the user interface needs to
account for the fact that the requests
failed immediately or after a timeout or
only a partial response was received
10 out seeds to be chosen in a way that
is useful to the end-user what do you
think is a common default on interfaces
what a timeout just raise your hand and
say number what do you think it's a
common default come on just take a guess
shout I don't mind in real life like 30
seconds ok something else 10 seconds
something else ok it's a minute it's
seriously a minute so what's your way
deadline and always remember when it's
time for more than a second you have to
keep the user busy preload images load
data step by step out of it's available
and so on and so on in a case of a time
out there should be a user interface
affording for canceling the operation so
that user can apply a shorter time out
if it makes sense to the situation and
partial responses can be super tricky
but still very useful for example
starting to show starting to start like
the top half of the webpage or an email
or other messages the time it takes the
user to read through that might be
enough time to like fetch the rest of
the data sometimes there's not even
enough data provided to do anything at
all and the user interface should at
least account for that so we'll see the
other first concept super important we
have to rethink how we concept and build
applications so how can we implement
that we need a new approach we talked
about before like decoupling the client
and the server side and I show you one
way the hoodie team came up with hoodie
as an open source and free JavaScript
library it offers to like the whole
back-end with user account storage
shares emails it's offline first
including purge to be an automatic sync
and has a dream code API every developer
understands in a second hoodie abstracts
all away old boring for the plate stuff
you don't want to think about so let's
take a look at the architecture
therefore we have like three stages we
have the application
then we have the hoodie layer here we
who'd use hoodie store the application
only ever talks to the hoodie IP I never
ever directly to the site a server-side
code to the database or not even to the
in browser storage
feel free to replace the local storage
here with any umbrella source of your
choice this by itself should be enough
for an application then we have the sync
layer restful error and the database we
can do that because each user has their
own private database by default they
only can access so we can't share data
by accident and this is very simple for
us to decide what's get synced and how
to handle change data so I will show you
like I explained to you a direct message
moving through system you create a
message in the application and send it
to the hoodie layer the hoodie layer
sends out the task to the locker storage
so the message can be stored locally and
results with the promise at the exact
same time it sends out the same task
through the same layer when the device
is online again sync layer pushes the
message to the couch to be it could
stored and tasks are sent out to other
devices to update the clients because we
changed some data and the plugin gets
the task to send the message so the main
point here is that the client and the
server they never ever speak directly to
each other they all leave each other
tasks and messages and this all very
loose couples and event based which
means it can be interrupted at any stage
without breaking and messages and tasks
will be delivered and exit warm-up on
whenever possible it's designed for
eventual contingency and if you want to
chew off on capability you up can never
ever talk directly to the server only
through think and always you want to
keep a local copy in your browser and
sync is really hard it's super hard
don't try to do that by yourself but
there's a lot of stuff you can use so
let's talk about the implementation we
have to take care of two things which we
store an art
Isis files and data let's take a first
look at defaults because without files
notification and yeah no data so who
have you ever heard about the app
manifest who's using it
who's loving it the manifest gives you
the possibility to fetch the files would
love to load from the cache when you're
offline I won't go any further here
because it's super easy a lot of
tutorials around just need just one
thing you need to know here is that app
cache to see you to douchebag and very
aged I'm working with that like four
years now and it sucks a very great
article by Jake Archibald why app cache
just a douche bag and I'll sum it up a
little bit in four points first one when
you visit a site the files
always come straight from the cache so
after the rendering the browser will
look for updates to the money fast and
cached files so you need to reload the
page every time to get new files second
app cache only updates if the manifest
has changed HTTP already has a caching
model so as a slightly unusual
workaround the browser will only look
for updates if the files listed in the
manifest or if the manifest file itself
has changed since the browser's last
checked so any change that makes the
body the manifest different by provider
will do so if you put a space in it it's
fine the app cache is an additional
catch it's not an alternative one so
when the browser updates the application
cache its request or else like usually
would it'll based regular caching
instructions so if an item's Hatter says
I'm totally cool on to 2022 the browser
assumed resource is indeed good until
then and won't trouble serve any updates
without specifics the browser is taking
a guess and we all know what happens
when browsers are taking a guess and a
Feist you serve should always have cache
headers especially when they're very
likely to be updated
and if you think that's better to surf
with no cash but always have a hatter
and unless one is non cashed resources
will not load on a cash patch so if you
just cash like the index.html but don't
cache the background image it won't be
displayed even when you're online and
these are like the seriously super
simple pain points of the app cache
there's a solution for that if you still
have to work with app cache that's the
application any Gregor Martinez a
colleague of mine taught the app cache
some manners and develop the application
and it helps us to deal with most of the
problems the app cache still has using
an iframe for the money fast so files
are from a server when you're online and
the text of the manifest is up-to-date
and there's another civil lighting so
you might have heard about the
serviceworker already which seems to be
like the new holy grail of offline first
the serviceworker is a very specific new
of that worker it sounds great but what
is it that worker actually most people
don't know so let's take a look at that
first the back worker was introduced
which the html5 specs and it's a part of
the standard it runs predefined
JavaScript browser scripts in your
browser the support is pretty good like
you can see here it's supported by all
major breath okay okay gotcha well this
is like the worst case this is what
happens if you're running just too many
scripts your pace gets super slow and
sometimes scripts you can get stopped by
after a timeout it's like the worst case
so sometimes we need to do things like
that
for example loading a huge J'son file or
running a killer algorithm on so numbers
or preload sets of data
imagine a slide that needs to handle
like UI events query it some processing
large amounts of API data and manipulate
the Dom I mean this is pretty common
right
we all know that so unfortunately all of
that can't be done at excessive
time due to limitations in browsers
JavaScript runtime script execution
happens within a single threat so this
means a youthful add logic can block
your whole application and developers
love dominican-american currencies so
ASIC events are processed after the
current executing script is done
I think but non-locking doesn't
necessarily mean concurrency workers
utilize thread like messages passing to
achieve parallelism they're perfect for
keeping your I refreshed performant and
responsive to the users so application
you see it's a very happy one the moment
a huge pile pulling your each part of
data you might run into your time out
what do you think it's a time out like
you know when popping up like the script
has to be stopped
what is the common thing in the browser
normal time out what do you think just
shout times in the browser the time out
for script not the UI we had that before
the scripts 30 seconds five seconds 90
it's 80 milliseconds then you should
start to use workers 15 milliseconds is
when you this is kind of common default
and when you're on on mobile devices
it's 30 milliseconds then you should
start using that workers so the web
worker specs defines an API for spawning
background scripts in the web
application it allows you to do things
like fire up long-running scripts to
handle in terms of tasks like Oh note
said soon as the web worker threads runs
the code from top to bottom and entered
this sink face racing face in which they
respond to the events and timers so you
can still interact with your application
like you normally would but outsource
the heavy operations and if they fail
and get a timeout the applique
still up and running some salsa right so
what can we use to work us for encoding
and decoding large amounts lot strings
complex mathematical calculations like
prime numbers encryptions and
assimilated in healings prefetch and
order caching data cool right there's
more Network requests resulting data
processing especially processing large
errors are you Jason responses
calculations and data manipulation on
local and session starts image
manipulations and can get any better yes
code syntax highlighting or other
real-time text analysis like spell check
to real-time search analyzing or
processing video or other data including
face and voice recognition background IO
and pauline.dub services I'm sure you
found more than two things already right
who did more than two so I guess you
found love I did and even this guy has
limits so let's take a look what we
can't and we can't do first what we
a lot we can't access the window object
the document object and the parent
object so no access to page script
Global's or functions
this also means you can't use libraries
that depend on the objects to work nope
no Jerry or at Google and sorry but you
can still use those in your eye this
just means you can't use those in your
worker and upcoming react.js version
won't rely on the browser but still do
you really need it
think about it so bummer what can we
actually do because web workers have a
multi-threaded behavior we only can
access a subset of JavaScript features
the bad workers still can access the
navigator object use the read-only
location object execute xml httprequest
there to send HTTP and HTTPS requests
please keep that in mind for later we
believe that and set time or interval
for any activity access the application
cache
later imported scripts spawned the
others workers we can do that too
so still lost how we start basically a
worker like this little amazing helpers
that you spawn when you can pass them
some work to after they're done you get
back a result and you can spawn as many
workers as you like but to be careful
they do not block your eye but that can
use still a lot of CPU so basically
freeze at all I mean you all know the
chaotic minions workers how basic they
can get so we have two types of workers
dedicated and shared workers dedicated
web workers I'll link to the Creator the
Creator is the script that called and
loaded them shed workers are web workers
that allow any number of scripts to
communicate within a single worker there
indentified in two ways either URL or
explicit name and normally when articles
talk about that worker it's about
dedicated workers but chat workers are
pretty important to us because service
worker are kind of our specific shared
you need to check if the web worker
property in the global window object is
available if not will be undefined they
run into is later threat so as a result
the code that they execute needs to be
contained in a separate file and you
need to create an object in your main
scripts constructor takes the name of
the work scripts so if the specified
file exists the browser will spawn in
your thread which is downloaded and the
worker will not begun until the file
completely downloaded and then it's
executed and if the path to your work
returns as a poor four it will fail
silently silly so I will put up the
slides later I won't show you the demos
because much time the only thing you
need to know which is super important as
when you check out like the code
a lot of examples which is super amazing
locally you need to nail occurs because
of security reasons so we talked about
the worker it's now focused look service
worker the often first concept became
one of the most importance wants to
follow we know that already
Jake Archibald got it right the other
day of one first means and getting on
screen without a network request the
more that it's there before the network
the more of my first it is so this means
every web application should work
without an connection after the very
first visit and just use the network to
get and set new data
everything else is offline first online
first sorry and we all know that app
cache is a douchbag a lot of issues here
we need to handle the major issue with
app cache is not just a number of
gotchas the design works really well
from singer page applications but not
for multi page sites so the
serviceworker
is a new browser feature that provides
event-driven scripts that run
independently and that pages it's a
programmable network proxy allows you to
control how network requests from your
page are handled and help developers to
build like URL friendly always available
application in a sane and layered way
but why can't we use the web workers for
that the difference is that service
workers you can fetch your assets and
serve them to the caches so we replace
the app cache on top of that we
outsource the heavy operations like
pulling the huge pile of jason imagine
on a page request your page request
something new the request gets through
the serviceworker then the network
because it's new and the network serves
back the response to the page and to the
cache so because we told the
serviceworker before to cache the fat to
catch the the response it's really great
facets and here we have the same use
case but
Kano's we have already stored request
acid or data in the cache already so
it's served straight from the cache so
we can add the behavior it should go to
the network and check if there's an
update available and which is awesome so
it saves a lot of time and requests
because you're able to control the whole
behavior and one more feature how it
saves resources
unlike other workers the worker can be
shut down at any end of events all the
workers are just running all the time in
the background the service worker is not
and we were terminated when not in use
you can't rely on global stayed within
the service work on fetch and on message
handlers but if you need to persist and
reduce across research and the
informations service worker live access
to the indexdb API so that's fine
service workers have scriptable caches
along with the ability to respond to
network requests from certain pages via
script to provide a way for applications
to go fine the main advantages are that
service worker doesn't just cache the
files
it also supports like background sync
and increases your performance
enormously and gives you support for
push notifications so workers make
extensive use of promises you should be
familiar with the concept before
starting to use service workers and as
the last part of the Service Worker I
will tell you how the Service Worker
live Celica works because it's really
nice inside to install a service worker
from a site you need to register it like
we've seen before with workers it will
cause the browser to start the service
worker and still step in the background
typically during the install step you
want to cache some static assets so if
all fires are a cache successfully then
the service worker becomes installed and
if any of that fails like the download
the assets and catch them then it'll
step will fail silently
but and the service worker won't
activate but don't worry about that
we'll try next time when we installed
the activation step we'll follow and
this is a great
Trinity for you to handling any
management of the old caches after the
activation step the service worker will
control all pages that fall under that
scope the page that registered the
service worker for the first time won't
be controlled until it's loaded again
so once the service worker is in control
it will be in one of two states either
it's terminated safe memory or will
handle fetch and message events which
occur when a network request or message
is made from your page service web is
still in development but pretty far
already they finally get Safari on the
boat and at as developing - so this is
great and it's partially shipped I can
offer chrome edge Safari and if you want
to dive deeper in serviceworker talk
project archibald last year just come
for you his talks are super fun
legendary and he's very amazing guys so
watch that link will be online and
there's also the cookbook so we have to
fast we have to still take thing take
care of one thing the data so finally
first law of robotics
yes do not harm any humans do you know
what the first law of offline first is
take a gas pretty close pretty close
very good it's do not lose data this is
like the very first law of offline first
we decouple the client and the server
side so we really need to store it on
both sides
therefore we trust pouch for the client
and couch for the server side there are
like a perfect team for offline first
make it quick and easy couchdb is no
spell database that replicates which is
exactly what we need for sync couch to
be comes with the suit of features like
on-the-fly document transformation
real-time change notification we rely on
with our messages and tasks system it
also has a very basic version control
system which helped us to
you've managed the data and when your
data gets out of sync it makes kind of
intelligent automatic decisions and help
you to merge the data paths to be as an
open-source JavaScript database inspired
by couch that it's designed to run well
within the browser and stores all your
data locally while offline
pedda fold it uses index to be in Chrome
and Firefox web SQL for Safari to store
the data and you also can add adapters
for local session starts it syncs with
couch to be and compatible service which
speeds the CouchDB replication protocol
like Cloudant or coach precinct getaway
when the application is back online
it also works on mobile devices for
native applications for example with
PhoneGap cordova and many many more the
special thing about hoodie is that every
user has their own private database like
I mentioned before and it's pirate party
folds so soft a lot of problems from the
get-go mm-hmm
when we have one database per user you
always know where to sing to when the
version of the document isn't up-to-date
we can check by app active member
version ID and what's Justin first and
how to merge the data without real
conflicts when we receive partial data
we store the version of the document and
can grab that before more during and
thanks to the event listener sand task
you can sync the data across devices so
this means we have multi device on usage
on the fly
if you want to share the data sure when
you want to share some private data you
can publish it and it gets just
replicated and others can read or read
and manipulate the data and we are able
to sync it again in our application
logic into the private storages pouch DB
is also a couchdb client and you are
able to switch between a local database
and an online CouchDB instance without
changing any of application code so a
question I get asked a lot how much data
can pouch to be store
so I prepared something nice in Firefox
pouch to be uses indexdb which will ask
you if data can be stored first time it
is attempted then after 50 MB every time
stored the amount of data can be stored
as unlimited chrome very cool calculates
the amount of your available storage on
calculate the limits
Oprah works the same as Chrome because
blink Internet Explorer above 10 hard to
not 50 below don't talk about that
Mobile Safari is hard 50 desktop Safari
prompt more than when you want to store
more than 5 up to 500 MB Android same as
Chrome 4.4 and above older hard 200 and
photo Cordoba unlimited on both iOS and
Android by using the SQLite plug-in so a
lot of information I hope I could give
you a small insight any concept of
offline first and how to build
applications that handle off like fruit
not just a bug but it's a feature
actually and if you like to say hi or
grab some hoodie stickers I have some
here so I'll be around so thank you very