Swift, Frameworks & Modules, Learnings from CocoaPods

Orta Therox & Marius Rackwitz

Recorded at GOTO 2015


Get notified about Orta Therox & Marius Rackwitz

Sign up to a email when Orta Therox & Marius Rackwitz publishes a new video

- Hello everybody.
I am going to use my slide for this one.
We are talking about the CocoaPods Frameworks.
This is something that most of you will have heard of,
but we are going to go into it, both: the details
of what the actual transition was for CocoaPods Frameworks,
and the bit of advice on how you could do it yourself
(because we have pragmatic time
from when we have had to transition applications to Frameworks).
CocoaPods has a very large amount of contributors.
More importantly, he is Marius Rackwitz and he did all
of the actual Swiftification and Frameworkification
to the new way).
He is Orta Therox
(everybody knows him as Orta, @orta on Twitter).
He used to introduce himself
as doing just the design work and being the design dictator,
but that is actually not right.
He has done much more for CocoaPods,
(e.g. CocoaDocs, which is a huge project on it is own),
and maintaining our websites
(not just the design side,
but more!).
(Big inside stuff, and much more:
plug-ins, weird plug-ins)
- ([Orta] I do all the weird things).
(Whenever it is like frustrating
and CocoaPods will not do something and I have to wait
for a release, it is time to make a plug-in for it).
(It has a beautiful structure for doing this kind of stuff).
What is CocoaPods?
CocoaPods is primarily a dependency manager, and also
a community (really important
about that part.
It is really easy to think that a single tool
can allow you to do something, but one of the cool things
about CocoaPods is it allows people to interact).
(In 2013 it had
the most amount of committers
to a single repo on GitHub, which is huge.
That meant that, at that point, we were the only people
actively using GitHub as a source for individual contributions).
It is a great reflection on how big the community is,
in terms of what people are doing with it.
That is also reflected in the complexity of the tool.
It has existed for five years now;
Now that we talk about Frameworks,
we are going to talk about some of the terminology.
First off, this is an Xcode project.
(You may have seen one of these before).
We are going to talk about workspaces.
In general workspaces (the bit that you see on this side)
is where your project, files and groups are kept.
We are talking about targets;
they are the apps or the extensions
that you normally have in your project.
Later, we will be talking
about copy or resource build phases
(you can get to these when you click
on a target and you see build phases).
CocoaPods does a few of its own
(instead of the normal Xcode ones).
(We wanted to make sure that you knew
all the bits that we will be talking about).
Integration history:
Frameworks have not been there forever.
Before frameworks, we had static libraries
(static libraries have been the common denomiator
until i07, because until i07 they were not available for iOS).
(At least not for the App Store).
You could use them unethically and standard Frameworks
with a link, but you could not distribute your own frameworks
We use them for the integration
because they were unavailable on OSX and iOS
(were the only platforms
we had to support back then).
(two platforms. Could you imagine?)
(Just two platforms).
What are static libraries?
(It seems obvious but,
question to the audience:
(Please raise the hands).
Who of you would think they could explain
how static linking works,
or a static library is linked?)
(I see some hands raised).
(It is a trick question).
Static libraries are not linked at all.
It is not obvious in the main page
of LD but static libraries
are just a collection of object files,
and object files are compiled
with a lot of meta information.
The machine code you have there is independent
from its position (and some other stuff you have to know
in detail, but they are not linked).
They are just compiled object files.
(And how we get there?)
When we build our source code with Clang,
we get the object files and put them
into Ranlib (making an archive out of them).
And archive has contents saying
which object files at which position in the whole file,
maintains the file information,
and how this object file was originally named.
We do this for every architecture.
to a fat binary.
When we built enough of that,
this could look like that (simplifications
are a applied here for the sake of simplicity).
We have here a library banana kit,
which has a dependency to the other library, monkey.
And a seal eye tool
as our program we want to build,
which depends on both of those libraries.
When we build this program
it is rather simple.
We just put everything together, compile it and then link it
(that is our program).
For CocoaPods,
we set up the exclude project
(everything works like that).
We set up static library targets. In addition
(because static libraries do not contain or support
resources,
or other associated files),
we would need for example,
correlater models.
CocoaPods has solved that with an additional build files
(used for the integrated target),
named CocoaPods resources build files.
It is nothing different than run script build files,
executes a shell script
(which lives in a separate shell file,
as it is easier
to regenerate the separate shell file
instead of having them living in the Xcode project
in a pods Xcode project;
that would cause bigger change to the integration
and it is not that easy to review
just with that on).
We want to make it as easy as possible,
if you take in the pods project.
But that is all
about the integration scenario of static libraries.
What we needed then were dynamic frameworks;
they are required for Swift.
If you want to build for Swift,
from iOS 8 for iOS 7,
they are distributions after standard libraries,
aesthetic libraries (only available for iOS 7).
If you want to target a later platform,
you will need to use the dynamic libraries.
That is the way for the future,
especially when Swift once reach that point
where it comes bundled together with the OS,
or other standard libraries like foundation
(where you are used to a dynamic link up).
We adopted the dynamic linking with the frameworks together.
First of all: what is a framework, how it looks like?
If you tab that in the finder (in your build product),
it is just a file bundle.
Because it has this file suffix, it is displayed
in another way and find out now some meta information
about frameworks.
If you right click on it, you can tab
into it and browse it (like any other directory).
Here we have the full expanded free.
We see at the top afield example
of a very popular networking library, Alamofire
(if networking for Swift build).
The first entry is the executable,
belsky, a dynamic library
with all the file suffix down low
(because it is clear,
executable through
living in the framework and having the same name)
Actually, they are static frameworks.
You can have a static library at this place
and it just magically works (is
undocumented that it works that way).
(But they are just side information here).
The headers
are automatically on the right,
pushing header Swift
or for object of seed to interface
the Swift parts of the model.
(together with the model map) because it is responsible
for defining the public interface.
The umbrella header is referenced in the model map.
Everything transitively imported
by the umbrella header
is part of the public model interface and available
for everything linking against it
(that place we have all been using model imports,
which are available with at import object of c,
and
for Swift. They are based on what we see below,
as in the directory models
and Alamofire Swift model).
We also see
that there is some extra documentation information,
which is used by another part of the tooling.
Those are platform specific
(as we see by the file names).
In addition to that, we have the info p list,
which contains some meta information.
There is information (e.g. the version and the copyright)
in the umbrella header, which
is going to rate by default.
There is a foundation expert, your library name version
and it is a constant.
The version information
from the info p list file is pulled at build time of the framework,
and read. A c file is going to rate it;
the constant is not read at wrong time
from the info p list,
but is part of the binary and is compiled into that
(otherwise it would take a long time to assess
the version information
and that would not be helpful).
This information
are only necessary for import and linking.
If you distribute
your app for App store builds,
they would give more information
about what is going on in your libraries
and there could be protected information
across your app and your extension.
That is ripped out.
What is dynamic linking,
or what are dynamic libraries?
In comparison to the static linking
(from the main page this important detail),
it is a file linked image (the linker run on it).
We compile
the source code and then we run the linker over it.
We have the Mac object file, which has a header
with some meter information (e.g. dependencies),
and we repeat the process again
for every architecture we need.
(a little deeper)
With Swift files
the same, different artifacts;
in the first step
we compiled monkey swift
and then we exported words
in our public interface.
Next, we compiled banana kit
(making use of foreign symbols:
MKMonkey - we have to link that
to our monkey swift, the library, and annotate
where it comes from).
We have this other symbol, which is a ray
and comes from Swift's and a library.
(I will stop for sake of simplicity).
We compile all my program
and obviously we cannot export there any symbols
because the program is linked by itself
and no one links to a ready program
(which is built for running).
Then we have to resolve the symbol c again
to both of our libraries.
With frameworks because we linked them
and annotated in our macro header
(there are other dynamic libraries
or frameworks around),
we have to embed them and make sure
that they are available on runtime. The dynamic link
can be resolved when the program runs.
They have to be embedded in the bundle,
frameworks folder.
Build files are
a usual copy file,
and it is named.
You can rename the files in Xcode
and the destination is set to frameworks from the comma box.
That is not enough for CocoaPods.
We have some special possibilities,
Little demonstration how that would look like for the pod file.
That makes sense if you want to include some frameworks
which are not suitable for App Store
(e.g. user testing,
something you cannot ship in the App Store
because Apple has some obvious (and not that obvious) restrictions).
We needed to come up
with a script which allows
to embed the pods that
are active for the current build configuration
(Xcode is not able
to allow us a set up).
That is the reason why we found in a configuration
a new build files (which looks a bit different
than the default case if you would set it up yourself).
(But what is happening there is still the same).
Headers are stripped and the frameworks need to be signed.
Everything is signed by itself:
frameworks are assigned individually
and your other executables assigned individually.
(that is the way it is).
Now, a comparison. You have seen static linking (before)
and dynamic linking (now):
what is better?
Frameworks are easier to distribute
and integrate into the app once they are built and compiled
because they bring everything together and they allow
to bundle resources.
They have a separate name scope.
Even at runtime,
you see the difference
(e.g. when referencing to a bundle,
you have a separate name scope).
Another advantage is
they allow to reduce the file size
of your whole bundle if you share the same framework
across your app into your extensions.
We use it in both places,
but if you have to use
different API availabilities
(like in iOS), there is this restriction
that the extension API
may only be used in extensions.
They have limited access on, for example, UI application -
you need to ship two versions,
which is not been available
since the very beginning
of when extensions were introduced,
(I cannot say exactly).
But you can have separate frameworks folder
in your apex:
you can separate frameworks
and have two versions of the same framework.
(what I have said before)
You have to really separate
bundle, which allows you to assess resources by name
only from your framework bundle and you do not get by accident
any other framework or any other framework resources
or the app resources just because they have the same name,
(that is nice).
There are disadvantages.
You have a dynamic library
as a final linked binary:
you limit possibilities for optimization,
because it is linked at that time when you build it
(it could be on a different computer
if it is some closed source third party library
which is rarely the case for CocoaPods).
In the same category,
dead-code stripping is limited with static linking.
You can import the object files,
or not included.
A real disadvantage,
because with object of c
you have
the object files included
(or at least the vast majority of them),
because the compiler is unable to figure out at compile time
what is called and what is needed at runtime
(because it is a dynamic language).
(something very important to mention here)
You will have increased load times.
When you do it at runtime,
there is overhead for resolving the dynamic links
to other binaries.
(I go over back to Orta).
- (I get control again).
For CocoaPods
we have to spell out different things
and we ended up getting freebies like the Clang modules.
To pull it off we had to extend
the code that you would write when you are using CocoaPods
(e.g. use frameworks)
We did that initially to make sure that people knew
that they were using frameworks (and you have been pitching
lately that we should remove it eventually).
- (We could recommend to use it).
(In some cases we could be a bit smarter about it),
I think
we need integration scenarios;
it makes sense to have it in the pod file end,
to have to use it explicitly.
- (Yes,
We have had to to work around Apple tools
(they are as good as you expect them be
around using third party stuff).
Code signing has been difficult,
(even months after,
we are still seeing problems with code signing).
Apple does not give
any notice upfront about what is going to change.
On top of that,
they will make changes on their server
that you will find out
when you are deploying your code to Apple
(at the end of your build).
(Things that previously
were not problems before using frameworks).
- You know probably yourself
from building your application and insuring
that everything runs smooth, there are limited possiblities
to really catch everything in your continuous integration;
with an open source project,
we are even further limited. We cannot, obviously,
deploy ultimately to the App Store
in our continuous integration.
We rely on user feedback
- (a lot of it).
I have transitioned three major applications
from CocoaPods static libraries to CocoaPods Frameworks.
(I wanted to give you a rough gist of the process,
of how I have ended up doing it.
Marius says I should not be recommending this entirely
because we have a proper way of migrating projects).
I use plug-ins
that completely nukes CocoaPods from your project.
Cocoapods-deintegrate and then I run it.
I use this when I am switching between frameworks
and libraries and from libraries to frameworks.
You have a clean slate
when you running pod-install the next time.
(is what I would recommend).
(This is what it ends up looking). It goes
through all these different parts (where CocoaPods
touches your Xcode project and shows you what has changed
and what is changing), and gives you a good idea
of what has created a clean slate.
That is: deleting the pods,
changing some of the build phases,
and removing any empty Xcode groups
that CocoaPods will have made.
From there, it is as simple as these three steps:
use frameworks in your Podfile,
that will dictate that the next time
to frameworks.
Then, run your tests
(where you find out
The most common problems are:
1) pods that depend on other libraries
that are not yet dynamic.
Most people have seen this from like Google Analytics,
or other
binary distributed libraries,
(Flurry is another one.
it is nearly always analytics libraries).
(To some extent, you just cannot work around that).
I have ended up adding app code
that links two pods together
in order to make that work.
The other one is to bug
whoever is distributing that library, that maybe
they should start supporting frameworks
(now that it is been a year).
(To some extent on their perspective,
it is also a bad thing because they have to
dictate that they are going iOS 8 and above only;
reasons for people not migrating
in that direction).
We often find that libraries do not look up
their own bundle resources.
They presume that the main app bundle is already inside
which means that you end up
with missing images (most of the time).
we have pods that give us our fonts,
that we had to
dynamically link them (we could not even
rely on putting
font names in our info dot p list).
Pound defies would allow them to
do whatever they want
with someone else's code
(e.g. with custom prefixes,
where someone would have a pound define
that would allow you to remove the custom prefix):
(Shame).
One of the apps that we translated
was one of the fist apps to use CocoaPods
as a frameworks (the moment you had a commit
that was workable, I shipped it
on one of our apps).
(I have the pull request).
Where I work
everything is open source
(you can go look at the pull request
of the transition from a static library
to a frameworks).
(This one is the first time we started doing it).
Eidolon
is a Swift app. It was nice and easy,
because it was already in that process.
The next one was our oldest app
was about four and a half years old).
There was a lot of legacy code
in moving it over.
That transition probably took me
about two or three days of editing pods
and changing our internal structure.
(Again, this is open source, so you can also go
and see the pull request changing these things).
Finally, we had our most complicated application:
over 60 pods,
a bunch being internal,
testing, open source ones;
transitioning was difficult.
(This one took about a week,
of someone who knows how to use CocoaPods well's time).
(part of that was made extremely difficult
because) A week or two later,
we realized that the launch time was so bad
that every time you would click the app
for the first time,
it would time out.
(The watchdog would just kill it).
(It would stay there for the full, eight seconds
before watchdog kills it?)
- I do not know, except - [Orta] - Six?
(Ten? it is a large amount of time;
really frustrating to watch).
We transitioned it and it turned out
to be the switch to frameworks.
Finding this,
having 60 pods linked into your app
at launch time was not working for us.
There is a big pull request that is still not finished,
where we have multiple developer tech support issue requests
with Apple (there is many radars about this now).
(That is a prohibitive problem).
This and my eventual fix was
to convert it back. As an application,
at this point we were not using Swift,
we were not using extensions - there were no reasons
for us to actively use it, we were just switching to frameworks
to be ready in case we wanted to.
Even now you can ship an app that does not require
you to be using frameworks.
It is nice that they are there, but you do not have to use it
(if you do not need to).
Remember to rate this session
and rate CocoaPods Frameworks as being awesome.
As it is two of us, you will have to remember
that we are extra super cool.
And you can come help us out at CocoaPods.
This links that we made a second ago will take you
to what we call the easy first issues,
which are things where, people that do not know Ruby
can start making a stab at some of these smaller bugs,
or smaller feature requests that we
(as the actual people building CocoaPods)
do not try, so that other people
can come and get them.
I strongly recommend doing this,
and helping us out.
- Please come and help yourselves.
you are very welcome.
- Thanks.
from the audience.
The first one is, what is your opinion on Carthage
and why is CocoaPods the better dependency model?
Marius: This question is difficult to answer,
(at least the latter part,
without taking here too much time).
I would say both dependency managers are very different
in the way they try to achieve their goals
and CocoaPods tries to make the whole integration
as transparent as possible to the user.
Everything is inspectable and
is as you used to have it in Xcode
and you would set it up on your own.
Even the build process is used
as what Xcode is giving you.
That makes it easy
to ignore Xcode about that,
because you do not need to have a
working Xcode project to have our working pods spec.
If you do not like
to have state and maintain information in Xcode projects,
pod specs are a better solution for you.
Carthage is going a different way at this point.
With four platforms,
I think we prefer
to continue the way we have chosen
(we are happy with that).
- [Voiceover] I think you covered this, but they said
how manage with float times through the experience
with apps being converted?
Orta: Yes, the Tumblr app they posted
that they said they went from roughly
9.3 seconds to five or six.
Which yes, at that point we were...
- [Voiceover] The takeaway from your session
is you can use frameworks, just don't.
Orta: There is a lot of apps that should be shipping with it
I guess the answer is maybe, you are going to have to have
a crazy, hybrid solution of some frameworks
and some static libraries.
Marius: I feel it depends on your use case and how you use
dependencies and how many of them and know about
the outside projects that you are using.
Dependencies and third party dependencies,
which are not actually third party because they maintain
open source projects on their own.
If you have
codes scattered over
small libraries, it will definitely
have an impact on loading times.
But if you have a small amount
of big dependencies,
you could profit in the loading times
(a big executable takes longer to load
than a small executable,
which has big dependencies).
Orta: Yes, n squares, the more
it gets significant, but while it is still small,
We are still shipping some apps
the have frameworks turned on, or multiple.
But the big ones, no.
For the moment.
- [Voiceover] Questions from the audience?
that you tried to try the Swift version
for CocoaPods in your own apps.
Do you have a test app that you send off to the App store
with every version of CocoaPods?
Just to make sure that Apple still accepts
what you are doing new?
Marius: No, we do not do anything like that
because there is just no proper way to do it.
- [Voiceover] If there is a problem,
somebody iwill call you "hey this just happened
where Apple said no".
Marius: Exactly, yes.
We try to prevent it
by having experienced
developers in place for code review in our code team,
and using the technology we built on our own,
but there will be always edge cases,
especially with f extensions
and previous stuff like sharing dependencies,
on watchkit extensions and the watchkit app
and having all the stuff in place, it is just not too much
as you are really able to test it
by one big integration scenario.
- [Voiceover] As an idea for somebody who has
free time on their hands, why not create a server that just
checks out their latest version of CocoaPods,
builds a project and sends it off to the App Store?
Just to make sure it goes through again.
Marius: if you can make that work without being able
to discard versions from iTunes Connect?
Orla: because you have no guarantee
that Apple are going to... Marius: Especially
to tooling on the rep hand side of things,
is going to be problematic,
but with the tutorials around fast lane,
it could be eventually doable.
Orla: Let's get Felix. Felix can do it.
He will probably just do it in a day if we ask him politely.
- [Voiceover] One of the standard questions
I get when I recommend people to CocoaPods
is what should I check in to get,
check in just support file, the entire pulse.
what is your current--
Orla: How about you, because I have got different opinions.
Marius: We can both answer this question.
I would say it depends,
but I generally prefer having the pods directory
in place because that is nice for continuous integration.
It does not have to run pod install
and you make sure that even
on some pods, a force push to master
would be run and there is a different version up there,
but just new commits for the same version
that could help them.
We rely on techs in the repository
and you cannot change the tech
when you do something bad (e.g. a force push).
You should not do that but the code could change
and you could get something different,
because once you have the pods directory in place,
it is no need to run CocoaPods at all,
or to have CocoaPods even installed.
You are much more independent
with having the pods directory in place
(in some way at least).
I know some developers who came up with using
GitHub's models for a pods directory specifically.
Orla: it is kind of ironic, right?
Marius: Not that much.
Your answer.
Orla: In all of our apps, we do not.
In all of our libraries, we do.
Libraries, you actually get Carthage support
for free (more or less);
a lot of the time you will come back to a library
after a long amount of time in between.
There is also the fact that you cannot entirely rely
on other people hosting your code that you will be bringing in;
checking in your pods means that you will actively
have the code that is necessary for your app every time,
in case Twitter decided to take over the Twitter pod
and suddenly decided they are not going to support
all the old versions
of this unofficial Twitter SDK.
If you have checked your pods in before hand
you would still be able to rely on that code existing
in your own fork (in your own repo, effectively).
But in general, the applications we work on (where I work),
these are permanently developed applications
that are constantly going to have development done on them.
When a pod disappears, it is very likely we will
have found out by the next day and we will have
found a mitigation strategy for that problem.
It does not have the clutter of it being
inside your code reviews,
as well as we include our application keys
inside our pods directories;
we have sensitive data in there too.
It is a bunch of yes and no reasons,
which is why people ask the question.
In my opinion there is no strong answer for one.
The multiple paragraphs we have
on the guides about it,
there are reasons for doing it and reasons for not doing it.
You should find our own space for it.
Marius: Definitely.
(faint voice from audience)
(Orla: Right? that is the most important part).
(faint voice from audience)
Orla: (I would like someone)
I want to build a new website on CocoaPods,
but I do not want to be the one building it.
I just want to do all the designing;
if someone feels up for building more Ruby websites,
that would be cool.
Want to build something like medium
but for talking about pods.
I have not found the name or the terminology
but I think that would be a nice way of collecting them together.
You could say these are the certain pods that I think
go well together or they compete.
There are many functional reactive programming pods;
maybe five or six
are worth your time.
But all of them have trade-offs and this website
should be the way in which people can express those.
That is what I would like to see.
Marius: I would like to see some
of the easy issues tackled.
But also
frameworks built for us.
We want to still enable configuration dependent pods;
would be nice if we could ultimately recognize
when no configuration dependent pod is used
that we just use
the copy files resource files instead
and let Xcode do their job
where we tried to reproduce to do the same
because there is no officer to or from Apple
that functionality is exposed
so that they could reverse that).
They have to replicate everything what is done effectively
and do it on our own in the shell script.
That has shown to be error prone in the past,
especially with Apple changing our own things
and tweaking stuff.
I would like to have the integration as close
as possible to what is the way Apple would do it.
Orla: I would like to double that.
That is all good and proper, but every time you press build
on a CocoaPods project you see this little running script phase.
Fixing that problem would remove that problem for everybody.
There are roughly 40,000 projects using CocoaPods.
If you could do that, you would save maybe a second or so
from every single person pressing build, every time.
That is a lot of time.
And it is totally worth someone's time to build.
(But build my website first).
Thanks.
Marius: Thanks.