Splitting Up a Monolith: From 1 to 25 Swift Packages

For the past month, I have spent the majority of my working time in Xcode (and Visual Studio Code), working on a major refactor of Foodnoms.

I've been considering this refactor for a long, long time. What motivated me to finally bite the bullet? A few things influenced me:

• Good timing. I am waiting on some backend changes to finish for a new set of features (coming soon). Also, WWDC is going to be here before we know it, and that may influence my plans. It's usually a good time in the months leading up to WWDC to address any technical debt in order to prepare for potential new APIs.
• Concerns about iCloud. While I have no plans for Foodnoms to stop using iCloud, in the past year I've had more concerns about the app's reliance on it. Too much of the Foodnoms codebase directly depends on CloudKit. This has started to feel more like a liability, in the case one day I wish to use another backend syncing service.
• Sharing code with another app. For the past four months or so, I've been working on another app. This app was able to share a lot of code with Foodnoms. This was done via a shared Swift package: the monolith, "CoreFoodNoms". While I was able to share a lot of code successfully, there were some global side-effects and assumptions that were tricky to workaround. (Note: I have decided to pause work on this app for the time being.)
• Troubles with SwiftUI previews and compile times. SwiftUI previews for Foodnoms has always been unusable. This was mostly due to the incredibly slow compile times. I had heard that using SwiftUI previews in a smaller build target with fewer dependencies can help with this. However, this didn't work for me. The problem is that a lot of my SwiftUI code depends on core models, such as 'Food' and 'Recipe'. The thing is, these models were not 100% pure. Some of them referenced global singletons that required some sort of static/global initialization. As a result, SwiftUI previews of these views in smaller Swift packages would immediately crash, due to those singletons not being properly initialized.

The Process

At the start of the refactor, I felt overwhelmed. Where do I start? What would stay a singleton, and what should become dependency-injected? Did I want to go all in on swift-composable-architecture? What should be global and what shouldn't? What's the scope of this refactor? How do I make sure that I actually finish it in a reasonable time frame?

Instead of trying to design the "perfect" architecture, I just decided to start with a smaller, practical problem: I wanted to decouple CloudKit from my core models. This took a lot of work – essentially building a new layer of abstraction between my models and CloudKit. This led to the following packages:

• FoodnomsRecords: the most "core" package of the app. This package holds the core Codable models of the app. I refer to these models as records. They are the canonical, forward-compatible representations of the core objects that power the app.
• FoodnomsCloudKitSync: this package connects the dots between Foodnoms, the app, and CloudSyncSession, the CloudKit sync engine I open sourced last year.

Before, the records themselves would directly import CloudKit and implement conversion to and from CKRecord. After the refactor, I introduced a layer of indirection. Now, all records implement conversion to and from a new struct, AnyRecord, which, as the name suggests, is a type-erased value type that can represent any record. FoodnomsCloudKitSync implements a generic conversion to and from CKRecord and AnyRecord.

After finishing this first step towards splitting up the monolithic CoreFoodNoms package, I still felt overwhelmed. Again, I tried to find a smaller, more-manageable surface area to focus on. I realized that the app's QuickLook extension, which is relatively simple, would serve as a great place to focus next. All the QuickLook extension needs to do is render a simple UIViewController. No database work – just doing some model computation and rendering. It didn't need to import all the code for HealthKit, CloudKit, performing database work, etc., like it had been. This naturally led to the extraction of a few more packages.

With each new package, I made sure to re-integrate it into the final product. Usually, this wasn't too involved, but it ensured that I wasn't creating a big mess that would be a huge pain to clean up at the end.

A notable challenge was the "Preferences" singleton, which stored common user settings, e.g. preferred energy unit (Calories or kilojoules). The problem was that this singleton was used in a lot of low-level model code, which caused multiple issues with unit testing and SwiftUI previews. The solution I arrived at involves introducing a new Preferences model struct, which serves as a snapshot of the current user's preferences. All model code needed to be updated to accept a Preferences instance (yay dependency injection). The main app target would then be responsible for instantiating the Preferences structs when necessary, based on current values from the preferences store. As a nice side-effect of making this a value-based struct, preferences can now be passed in as a SwiftUI environment value.

As I progressed, I noticed each package belonged to one of two separate classes:

1. Pure and reusable: These packages had minimal third-party dependencies, higher unit test coverage, were unlikely to perform major file I/O, and were more inclined towards using dependency injection.
2. Foodnoms-specific: For these packages, I was more accepting of singleton usage, file I/O, and third-party dependencies.

As a secondary benefit to all of this refactoring, the other Foodnoms app extensions, including the intents and widget extensions, became simpler and more efficient. They no longer do any gross static initialization of globals, instead creating and opening resources on-demand. This eliminates a variety of possible errors, and likely comes with improved memory and resource usage.

Bit by bit, I whittled CoreFoodNoms down to just a dozen or so files. Most of the remaining files felt best suited to just go back to being part of the main iOS app target. Finally, I was able to delete the CoreFoodNoms folder completely. RIP.

You might think that ending up with over two dozen packages is overkill, but honestly, it feels just right. The package hierarchy evolved naturally. Each package has its own set of concerns, and together, they fit seamlessly into a modular hierarchy.

Adoption of os_log and OSLogStore

One other aspect of the refactor that I figured I would mention: logging. Previously, Foodnoms used Nike's Willow library for logging, which is a fantastic little library. Long ago, I integrated Willow with a file writer, so all log lines would be appended to a shared, rotating set of log files.

This setup has served me well over the years. It's helped me solve so many hard-to-debug problems, both in development and with customers.

I use logging everywhere, making it one of the aspects I had to consider from the outset. Was I going to make Willow a shared dependency for all packages? This would go against several of my guiding principles for the refactor.

Whenever Apple offers an API or way to do something, there tends to be a pressure to adopt that thing, even if there's perhaps a better solution out there. I have felt that way for a while with os_log, which I'll get into here in a bit. But at this point, I felt that it was a better decision to fully adopt os_log, rather than compromise on the principles of the refactor in a major way.

So with each package migration, I had to manually convert all log calls to os_log. I just checked and there's 508 of them total. Since os_log requires C-style formatted strings, I had to manually replace all the nice string interpolation I had with StaticStrings. I was not very happy about this, because a) it was a lot of work and b) it was less safe, as it could lead to crashes if I made a mistake in the string format. But I committed and just pushed through, one os_log at a time.

Finally, it came to querying the os_logs dynamically to render them in the app's diagnostics report, which is a file that customers can send to me for support problems.

There's a few issues with os_log and OSLogStore that I want to highlight. I already knew about these tradeoffs going into this refactor, but I'm still unhappy about them, because they are going to make it more difficult for me to solve customer issues going forward.

• Logs are only collected for the current process. In order to retrieve logs from app extensions, I'll have to ask a customer to perform a sysdiagnose.
• Logs can be persisted, but they tend to be purged quite quickly. I was able to store multiple days worth of logs with Willow. With os_log and OSLogStore, I often only get logs for the current process execution.
• On watchOS, no logs are captured at all unless the device has a sysdiagnose profile installed. While I understand the performance concerns that led Apple to the decision to make it this way, this is a really annoying thing to deal with. I can't stress enough how helpful logs are as a developer! Especially when dealing with a local-first app like Foodnoms.

These are some really big compromises. I pray that I don't encounter some obscure bug with a customer where I wish I had my old logging solution back.

On the positive side, adopting os_log has helped me remove a third-party dependency, and source of possible errors. I also expect it to be more performant than my previous solution (note: I didn't do any testing to confirm this).

Results

I am very pleased with the outcome of this refactor. Perhaps a bit self-indulgent to my overengineering tendencies, but I think it was time worth spent.

The hairiest, most error prone singletons have been eliminated. A lot of code that was unnecessarily coupled is now cleanly separated.

The packages themselves provide nice layers of abstraction, building on top of each other. Another nice side effect of all this work, is I was able to split up my unit tests into their respective package. I updated the Xcode scheme to test all these packages. It's nice to have all unit tests organized in such a way, and comforting to know that they are all run with every Xcode Cloud CI build.

There are some wins in terms of productivity as well. I ran a few rudimentary tests to evaluate the impact on build performance. Note these results are in no way scientific, but illustrative. YMMV.

Warm Compile Time

For these tests, I compared the git commit before the refactor and afterwards. For both, I compiled the app locally in a debug build configuration.

When I say "warm" compile time, I'm talking about performing a build that leverages caches from a previous build.

Here was my procedure:

1. In Xcode, clean the project.
2. Run a full build.
3. Make one small change.
4. Build again.

To get a well-rounded comparison, I found two places in the codebase that would serve as distinct data points for comparison:

1. Code that was previously in the monolith, CoreFoodNoms, and is now in CoreRecords, which is a package that is depended on by almost every other package.
2. Code in the iOS app target itself.

For the first test, I saw a roughly 2x speedup: median 14.4 -> 7.3 seconds. For the second test, I observed a >3x speedup: median 18 -> 5.5 seconds.

I'm very happy with these results.

Cold Compile Time

I compared the same git commits for this test. Except this time, I used code from XcodeBenchmark to do the comparison.

The shell script in XcodeBenchmark uses xcodebuild to trigger the compilation. It overrides the DerivedDate directory in order to control for any cached work by simply ensuring that there is no cache.

With these tests, I observed a 1.18x slowdown: 147 -> 174 seconds.

I was curious to understand if the reason for the slowdown was due to more time spent in the package resolution step, which notably includes network time. To account for this, I modified the XcodeBenchmark script to run this before xcodebuild:

xcodebuild -resolvePackageDependencies -project "$PATH_TO_PROJECT" -scheme "$SCHEME_NAME" -derivedDataPath "$PATH_TO_DERIVED"

For the pre-refactored code, package resolution took 29 seconds. After the refactor, the package resolution step took 46 seconds.

This partially accounts for the slowdown, but not entirely. The time spent building the project is still slower: 118 -> 128 seconds.

To me, this is only a marginal penalty. I'd much rather trade small performance losses with cold build times for significant gains in warm build times.

Bundle Size

Unfortunately, the app's bundle size increased with the refactor.

To compare real-world impact, I look at the download and install size on the latest iPhone models.

I anticipated an increase in the bundle size, given that Swift packages are compiled into static libraries by default. This approach leads to the code from these libraries being included with each target that relies on them, effectively boosting the overall size of the application bundle. This effect is particularly noticeable when dependencies form a complex tree. The precise impact of this architectural decision was somewhat of a mystery to me. Despite searching, I couldn’t find detailed explanations or practical resources online that shed light on the extent of the impact this choice might have.

I briefly tried the new mergeable libraries feature, but it didn't seem to reduce bundle size at all. Maybe I wasn't doing something right? I only tried "automatic mode". Also, on Mac (Catalyst), I was getting a dyld crash on app launch.

I also briefly tried compiling the Swift packages as dynamic libraries, but that also resulted in dyld crashes.

Ultimately, I consider download size a lot more of an important metric than install size, and honestly, I'm not going to lose much sleep over a few additional MB.

SwiftUI Previews

SwiftUI previews have become somewhat usable again, hurrah! Although a significant portion of my SwiftUI code is still within the app target and thus slow to build, I now enjoy faster SwiftUI previews for highly reusable views in the 'Design System' package. Looking ahead, I aim to create new feature-specific UI packages that will allow for quicker iteration with SwiftUI Previews in upcoming projects.

I hope this writeup was helpful for you. Before I embarked on this adventure, I was hoping to hear about others experiences doing something similar, but couldn't find but a few anecdotes. Have you done something similar for your project? I'd be curious to hear how it went! What did you learn? Any regrets? Let me know, you can @ me on Mastodon or Threads.