Write-Ahead Logging is the default journaling mode for Core Data SQLite stores since iOS 7 and OS X Mavericks. Journaling in Core Data is best explained as the way data transactions are saved into the underlying SQLite store. The WAL mode is significantly faster in most scenarios compared to the previous default “rollback” journal mode and is the underlying reason that makes it possible to have concurrency in Core Data.
With the WAL mode, Core Data keeps the main SQLite store file untouched and keeps track of data transactions in a -wal file within the same location of the SQLite store. This means that in some cases, changes are not yet visible in the SQLite store itself. Because of this, you might want to force commit those changes into the SQLite store. In this blog post, I’ll give you both an example and an explanation of how you can do this.
How does Write-Ahead Logging work?
You might have always wondered what those .sqlite-wal and .sqlite-shm files meant when you opened up your app’s directory to find your Core Data .sqlite file. Those two files exist next to your .sqlite file since iOS 7 and OS X Mavericks and are part of the current default Write-Ahead Logging journaling mode of Core Data.
Journaling modes in Core Data and SQLite prevent data loss by writing new transactions in an in-between journal file. Before WAL was introduced, SQLite was using the traditional rollback journal which works by keeping a copy of the original unchanged database content in the journal file. New changes are written directly into the database file and the journal file is used to roll back changes in the case of a crash. The final commit saving the changes occurs when the rollback journal is deleted.
WAL works exactly in the opposite way. The original content is preserved in the database file and new changes are added into the WAL file. Commits finalizing data changes can actually happen in this WAL file which means that data can be saved while the original database stays untouched. So-called “Checkpointing” eventually takes place to transfer all the transactions from the WAL file into the database.
The .sqlite-shm file exists as the shared memory between multiple SQLite database connections and is used as an index for the WAL file.
You can read the in-depth explanation regarding WAL mode if you like to learn more.
Core Data Changes not visible when opening a SQLite database file
In some cases, you might find yourself opening an SQLite database file realizing certain changes are missing. This could be caused by the fact that you’ve moved an SQLite file without its matching -wal and -shm file. Most database reader apps support WAL mode and automatically show changes that are not yet committed to the SQLite store.
Reasons to force commit changes into the SQLite store
I bet you wonder why you would want to force commit changes into the SQLite store. It could also be you ended up in this post because you already have a reason in which case I would love to hear more.
The Collect by WeTransfer contains a lot of data and uses Core Data as its storage. Over time, we had to write multiple Core Data migrations which in rare cases failed with errors like the following:
\“addedDate is a required value.\” UserInfo={NSValidationErrorObject=<NSManagedObject: 0x2819fba70> (entity: ImageContent; id:
In most cases, you would solve this by writing a correct migration policy that sets the right values during the migration itself. However, we found cases in which we liked to do preprocessing on the local store just before the migration to delete those known invalid cases that were a result of bugs in the past.
Writing the preprocessor was as easy as creating a persistent container using the source database to perform the required transactions. However, in our experience we found out that those changes were not always visible for the Core Data migration and required us to do a force commit to make sure those changes are applied to the SQLite database file.
How to force commit changes into the SQLite database
Forcing a commit, also known as checkpointing, means merging the transactions that exist in the -wal file into the store file. By using the traditional rollback journaling mode we can force Core Data to perform a checkpoint operation for Write-Ahead Logging databases.
To do this, we can add a persistent store by setting the journal mode to DELETE:
private func forceWALCheckpointingForStore(at storeURL: URL, model: NSManagedObjectModel) throws {
do {
let persistentStoreCoordinator = NSPersistentStoreCoordinator(managedObjectModel: model)
let options = [NSSQLitePragmasOption: ["journal_mode": "DELETE"]]
let store = try persistentStoreCoordinator.addPersistentStore(ofType: NSSQLiteStoreType, configurationName: nil, at: storeURL, options: options)
try persistentStoreCoordinator.remove(store)
} catch let error {
throw Error.forcedWALCheckFailed(error: error)
}
}This method takes the source store URL and database model as input and is enough to force commit the changes into the local database file.
Disabling WAL mode completely in an NSPersistentContainer
If you like to write changes directly into the SQLite file you can decide to directly commit changes into the local SQLite file by setting up your persistent container with the rollback journaling mode:
let persistentContainer = NSPersistentContainer(name: "Database_Name", managedObjectModel: model)
let storeDescription = NSPersistentStoreDescription(url: storeURL)
storeDescription.setValue("DELETE" as NSObject, forPragmaNamed: "journal_mode")
persistentContainer.persistentStoreDescriptions = [storeDescription]Verifying the configured journal mode using SQLite debugging
Although this section is best explained in my detailed Core Data Debugging post, there’s one launch argument I’d like to highlight. By using -com.apple.CoreData.SQLDebug 4 as an argument passed on launch you will get detailed information on the transactions happening in Core data.
Within those logs you’ll find the following information regarding the journal mode:
CoreData: sql: pragma journal_mode=DELETEWhat to do when the delete journal mode is resulting in a dead lock
A dead lock can occur if a persistent container with a different journaling mode is currently active and using the database file. I don’t have an exact explanation why this happens but it’s likely caused due to an active lock on the database files. Therefore, make sure to only force commit when there are no active connections to your database.
Conclusion
Core Data allows us to save data in a SQLite store using high-level APIs. After reading this blog post, you hopefully learned more about the underlying structure which is using the Write-Ahead Logging journaling mode. Force commits allow you to write changes into the SQLite file to make sure changes actually exist in the local database file.
If you like to improve your Swift knowledge, even more, check out the Swift category page. Feel free to contact me or tweet to me on Twitter if you have any additional tips or feedback.
Thanks!