Is a GCD dispatch queue enough to confine a Core Data context to a single thread

http://stackoverflow.com/questions/7718801/is-a-gcd-dispatch-queue-enough-to-confine-a-core-data-context-to-a-single-thread?lq=1

Concern
One big advantage of GCD dispatch queues is that it manages and makes use of multiple threads to process its FIFO tasks as needed. Each task gets executed by one thread. However, the it “may” be a different thread that processes the next task.

So – if I understand this right – tasks I hand off to one and the same dispatch queue, could end up running in different threads, potentially handing off a core data context from one thread to another, and having things go wrong. Is that right?

Answer
The accepted answer to that question, using GCD queues, does ensure that a new context is created on each thread, but does not point out the necessity of doing this.

The big thing you need to remember is that you must avoid modifying the managed object context from two different threads at the same time.

That could put the context into an inconsistent state, and nothing good can come of that. So, the kind of dispatch queue that you use is important: a concurrent dispatch queue would allow multiple tasks to proceed simultaneously, and if they both use the same context you’ll be in trouble.

If you use a serial dispatch queue, on the other hand, two or more tasks might execute on different threads, but the tasks will be executed in order, and only one task will run at a time. This is very similar to running all the tasks on the same thread, at least as far as maintaining the context’s consistency goes.

See this question and answer for a much more detailed explanation.

This is how Core Data has always worked. The Concurrency with Core Data section of the Core Data Programming Guide gives advice on how to proceed if you do decide to use a single context in multiple threads. It talks mainly about the need to be very careful to lock the context any time you access it. The point of all that locking, though, is to ensure that two or more threads don’t try to use the context simultaneously.

Using a serialized dispatch queue achieves the same goal: because only one task in the queue executes at a time, there’s no chance that two or more tasks will try to use the context at the same time.

thread confinement

https://www.quora.com/What-is-Thread-Confinement

Thread confinement is the practice of ensuring that data is only accessible from one thread. Such data is called thread-local as it is local, or specific, to a single thread.

For example, declaring local variables is a form of thread confinement because each thread has its own local stack. The local variables you declare, are to be used for say thread A only. If another thread B comes in, it cannot use thread A’s local stack variables. Each thread has its own stack, and manipulates its own stack. This is called Thread local data.

Thread-local data is thread-safe, as only one thread can get at the data, which eliminates the risk of races. And because races are nonexistent, thread-local data doesn’t need locking. Thus thread confinement is a practice that makes your code safer (by eliminating a huge source of programming error) and more scalable (by eliminating locking).

An example of this would be a Servlet’s GET or POST method. In such a GET/POST method, if you were to declare local variables, these variables are to be confined to the specific thread that comes in. No other thread are sharing those variables.

Let’s say you declare member variable for the Servlet class. That member variable has global scope, and thus, can be shared by threads. This will lead to lots of problems, such as race conditions, or locking issues, and thus, we do NOT have thread confinement.

Core Data Stack #2 – multiple contexts and resolving conflicts

https://developer.apple.com/library/ios/documentation/Cocoa/Conceptual/CoreData/ChangeManagement.html
http://stackoverflow.com/questions/4800889/what-does-apple-mean-when-they-say-that-a-nsmanagedobjectcontext-is-owned-by-the

Multiple Contexts manipulating the Database

source download

Let’s see what happens when we have 2 contexts making changes to the database. First let’s add in an extra NSManagedObjectContext called privateQueueContext2. Implement the custom set method and create our context to execute tasks on a queue.

Then synthesize it and create the custom set method.

CoreDataStack.m

How we want to test it

We’re going to test what happens when 2 contexts edit a database by having the first context get the first Person and change its name to “Ricky”, then save the context. Then we’re gunna have the second context get the first Person and change its name to “Shirley”. The change, and its related saves will be overlapped.

But first, let’s implement the code where the contexts edit a common object (0-th indexed object of the fetched result array), then do a context save.

We do this for private context queue 1:
1) We have changeFirstPersonPrivateQueue method which responds to a button that changes the 0th Person object’s name attribute to Ricky.
2) Then we have privateQueueContextOneSave method which responds to a button that wants context 1 to save.

I want to emphasize that

We do this for private context queue 2:
1) We have changeFirstPersonPrivateQueue2 method which responds to a button that changes the 0th Person object’s name attribute to Shirley.
2) Then we have privateQueueContextTwoSave method which responds to a button that does context 2 save.

Now that the implementation is ready hook up these methods to buttons.

Specifically what we’re trying to do is

to have context 1 be able to make a change to the first Object, then save it.
to have context 2 be able to make a change to the first Object, then save it.

Hence we should have 4 buttons.

2contexts-view

Now, we know if one context makes “changes and save immediately”, then it gets reflected in the database. The next context that either reads or make further changes will see the updates. However, what if the 2 contexts’s changes and saves overlap each other?

Let’s look at this example:

  • First, click on the “add bulk” button to add Person objects into Core Data.
  • Then, click on the “Display All” button to show all the first/last name of the Person objects.
  • Then, we use context 1 to change the 1st Person object’s first name to “Ricky”. Click the “Context1, make change to Ricky” button to edit Object 1’s name to “Ricky”.
  • Then, we use context 2 to change the 1st Person object’s first name to “Shirley”. Click the “Context2, make change to Shirley” button to edit Object 1’s name to “Shirley”.
  • Then click on “Context 2 save”
  • Then click on “Context 1 save”
  • In this case, you’ll get an error:


    2015-10-27 00:29:56.801 CoreData2[48750:5178513] Unresolved error Error Domain=NSCocoaErrorDomain Code=133020 “(null)” UserInfo={conflictList=(
    “NSMergeConflict (0x7f96bbc0f100) for NSManagedObject (0x7f96bbd0aa90) with objectID ‘0xd000000000040000 ‘ with oldVersion = 1 and newVersion = 2 and old object snapshot = {\n firstName = \”remark: 1445930979.058863\”;\n lastName = \”1445930979.058863\”;\n} and new cached row = {\n firstName = SHIRLEY;\n lastName = \”1445930979.058863\”;\n}”
    )}, {
    conflictList = (
    “NSMergeConflict (0x7f96bbc0f100) for NSManagedObject (0x7f96bbd0aa90) with objectID ‘0xd000000000040000 ‘ with oldVersion = 1 and newVersion = 2 and old object snapshot = {\n firstName = \”remark: 1445930979.058863\”;\n lastName = \”1445930979.058863\”;\n} and new cached row = {\n firstName = SHIRLEY;\n lastName = \”1445930979.058863\”;\n}”
    );
    }

    ..and you’ll come to the abort method:

    ref – https://developer.apple.com/library/ios/documentation/Cocoa/Conceptual/CoreData/ChangeManagement.html

    If your application contains more than one managed object context and you allow objects to be modified in more than one context, you need to be able to reconcile the changes. This is a fairly common situation when an application is importing data from a network and the user can also edit that data.

    Ultimately there can be only one truth, and differences between these views must be detected and reconciled when data is saved. When one of the managed object contexts is saved, its changes are pushed through the persistent store coordinator to the persistent store. When the second managed object context is saved, conflicts are detected using a mechanism called optimistic locking. How the conflicts are resolved depends on how you have configured the context.

    When Core Data fetches an object from a persistent store, it takes a snapshot of its state. A snapshot is a dictionary of an object’s persistent properties—typically all its attributes and the global IDs of any objects to which it has a to-one relationship. Snapshots participate in optimistic locking. When the framework saves, it compares the values in each edited object’s snapshot with the then-current corresponding values in the persistent store.

    • If the values are the same, then the store has not been changed since the object was fetched, so the save proceeds normally. As part of the save operation, the snapshots’ values are updated to match the saved data.
    • If the values differ, then the store has been changed since the object was fetched or last saved; this represents an optimistic locking failure. The conflict must be resolved.

    Say in our example, when our contexts gets the first object in our Core Data, it does so using

    as you can see in the code. Basically, the NSArray you get back is a snapshot of its state. Within the NSArray, we get the 0-th index object, and change its attribute. What we’re changing here is just a snapshot. Now, when you decide to save after you change the attribute in the situation of the 2nd context changing the name to Shirley, it compares the values in its object’s snapshot with the then-current corresponding values in the persistent store.

    Context 2 – no conflict

    In our case for context 2, before it saves, it sees that the current corresponding value in the persistent store is the same as the values in the snapshot it received. In both cases, it is the default value “remark: whatever today’s date is”. Since they are the same, then the store has not been changed by other context(s) since context 2 fetched it, and thus the save proceeds normally.

    Context 1 – conflict

    It does not work for context 1 because when context 1 fetches the object, its snapshot is “remark: whatever today’s date is”.

    However, because context 2 previously managed to change the 0th Person object’s name attribute to “Shirley” and then save. This means the current corresponding values in the persistent store is Shirley.

    This means the values differ, and thus, for context 1, the store has been changed since the object was fetched or last saved. This represents an optimistic locking failure. And the conflict must be resolved.

    Resolving conflicts by synchronizing changes between our contexts

    If you use more than one managed object context in an application, Core Data does not automatically notify one context of changes made to objects in another. In general, this is because a context is intended to be a scratch pad where you can make changes to objects in isolation, and you can discard the changes without affecting other contexts. If you do need to synchronize changes between contexts, how a change should be handled depends on the user-visible semantics you want in the second context, and on the state of the objects in the second context.

    Consider an application with two managed object contexts and a single persistent store coordinator. If a user deletes an object in the first context (moc1), you may need to inform the second context (moc2) that an object has been deleted. In all cases, moc1 automatically posts an NSManagedObjectContextDidSaveNotification notification via the NSNotificationCenter that your application should register for and use as the trigger for whatever actions it needs to take. This notification contains information not only about deleted objects, but also about changed objects. You need to handle these changes because they may be the result of the delete. Most of these types of changes involve transient relationships or fetched properties.

    Code for Context 1

    First, let’s look at the context setter methods. Whenever other contexts save, we want to make sure our own context updates on their saves. Thus we add an addObserver to observe saves from other contexts. Whenever other contexts save, the message NSManagedObjectContextDidSaveNotification gets sent, and we capture it like so:

    What this means is that whenever object self.privateQueueContext2 sends a message name NSManagedObjectContextDidSaveNotification, then let’s call custom method mergeChangesFromContext2.

    Note that for the object parameter, if you can it to the sender of the notification such as self.privateQueueContext2, then we will then only be notified of self.privateQueueContext2’s events.

    If we set to “nil” you will get all notification of this type (regardless who sent them).

    In our particular case, we set the object parameter to context 2, so that we can be updated on whatever context 2 just did, or is doing.

    The custom method gets called whenever context 2 saves, because we as context 1, wants to merge changes from context 2 if and when context 2 does save. We do a simple check that if the notification is from context 2, then we simply have our context 1 merge via the method mergeChangesFromContextDidSaveNotification method:

    Run the program, add your bulk of People object. Then Display them. You’ll see the first object which is 0th index.

    reportOnAllPeopleToLog method called
    2015-10-27 11:17:59.555 CoreData2[49227:5232602] ———— RESULTS FROM DATABASE ————
    2015-10-27 11:17:59.556 CoreData2[49227:5232602] There are 10 number of entries so far
    2015-10-27 11:17:59.556 CoreData2[49227:5232602] 0 —————-
    2015-10-27 11:17:59.556 CoreData2[49227:5232602] firstName = remark: 1445969859.502338
    2015-10-27 11:17:59.557 CoreData2[49227:5232602] lastName = 1445969859.502338
    2015-10-27 11:17:59.557 CoreData2[49227:5232602] object address is: 0x7fd358d8f710
    2015-10-27 11:17:59.558 CoreData2[49227:5232602] 1 —————-
    2015-10-27 11:17:59.558 CoreData2[49227:5232602] firstName = remark: 1445969859.822105
    2015-10-27 11:17:59.558 CoreData2[49227:5232602] lastName = 1445969859.822105
    2015-10-27 11:17:59.559 CoreData2[49227:5232602] object address is: 0x7fd358d83d40


    etc.

    Press button Context 1, Change to Ricky
    Press button Context 2, Change to Shirley
    Press Context 2 save
    Press Context 1 save.

    You’ll then see that Context’s RICKY is the latest save

    result:


    reportOnAllPeopleToLog method called
    2015-10-27 11:19:53.481 CoreData2[49227:5232602] ———— RESULTS FROM DATABASE ————
    2015-10-27 11:19:53.481 CoreData2[49227:5232602] There are 10 number of entries so far
    2015-10-27 11:19:53.481 CoreData2[49227:5232602] 0 —————-
    2015-10-27 11:19:53.481 CoreData2[49227:5232602] firstName = RICKY
    2015-10-27 11:19:53.481 CoreData2[49227:5232602] lastName = 1445969859.502338
    2015-10-27 11:19:53.481 CoreData2[49227:5232602] object address is: 0x7fd358c31a90
    2015-10-27 11:19:53.482 CoreData2[49227:5232602] 1 —————-


    etc

    The result is now correct. When context 2 saved Shirley, context 1 merged itself with context 2’s changes, and thus when our context 1 saves, it will see that the persistent store value and its snapshot value are both “Shirley”, and thus goes on to save its own changes of “Ricky”. Thus, after the save, we see that the value “Ricky” is reflected in the database.

    Some Upkeep

    However, this only applies for context 1 merging if and when context 2 saves. What if context 2 need to merge on what context 1 does? That means we have to do the same thing for context 2.

    Also, what if we have x number of contexts? That means for whatever context n that’s saving, we just make sure all other contexts merge with the saving context.

    Notice object is set to nil in order to observe NSManagedObjectContextDidSaveNotification messages from any object. Then in our mergeChanges custom method, we simply make all contexts merge with the saving context.

    and…

    Using the Queue and performing your edits and saves via blocks

    Apple states:
    [A context] assumes the default owner is the thread or queue that allocated it—this is determined by the queue that calls its init method. You should not, therefore, initialize a context on one queue then pass it to a different queue.

    This means that in our custom setter method for a context, we create the context with an init to a concurrency queue type. This means default owner of privateQueueContext is the queue used via the init:

    Hence privateQueueContext and the init-ed queue are now pinned together. This pattern is thread confinement or isolation.

    We created privateQueueContext with this queue, and must remember to always use privateQueueContext with this queue.

    For example, say we have a task: “use privateQueueContext to get a Person object to manipulate and later save into Core Data”. We want to queue this task because we’re doing this 10,000 times and needs to be run on a background. We queue this task on privateQueueContext because privateQueueContext is what is involved.

    Notice we use privateQueueContext on all. If in the task of fetching the Object using privateQueueContext but queueing the task on say privateQueueContext2 (which has its own queue via an init), then we have “have violated thread confinement by exposing the MOC object reference to two queues. Simple. Don’t do it. Don’t cross the streams.” (http://stackoverflow.com/questions/4800889/what-does-apple-mean-when-they-say-that-a-nsmanagedobjectcontext-is-owned-by-the)

    Part of the reason is because multiple queues process tasks concurrently. So whatever context data you are executing in queue A, may also be executed in queue B at the same time. If you were to confine tasks to its attached queue, then no context data will ever be shared because each task is processed one after another via FIFO.

    furthermore

Core Data Stack #2 – 1 context, background task

  • http://www.cocoabuilder.com/archive/cocoa/290810-core-data-conflict-detection.html
  • http://joelparsons.net/blog/2013/09/02/background-core-data-with-privatequeuecontext

Updated Demo 5/5/2016

demo 10/26/2015

Have the context process in the background

First step is to add a private queue context where it will save a lot of data into the database.

  1. add a NSManagedObjectContext privateQueueContext object, synthesize it
  2. custom set method where we create the context with a queue
  3. queue up tasks such as bulk add
full source for CoreDataStack.h

addBulkPeople method is the process of using our privateQueueContext object and throwing a block on the queue and processing the saving of data into the database.

reportOnAllPeopleToLog method is using our mainQueueContext to simply get data from CoreData and see what’s in the database.

CoreDataStack.m

First, we set up our init method and init our queue contexts via custom setter methods.

Notice when we alloc and init NSManagedObjectContext, we initialize the context with concurrency type NSPrivateQueueConcurrencyType

Essentially, instead of the old way of attaching a context to a thread, we now have the option of attaching a context to a queue. This means that whatever job and/or tasks are lined up for this context gets queued. Thus, whatever operation needs to be done by that context will be completed FIFO via blocks.

Now that we have our queue context ready to be used, use queue up tasks by adding in blocks of code to the queue. We want to add a bulk of people so we queue up the task of “adding 10,000 people”. This is obviously a very time consuming task, and by adding it onto our context queue, we make it run in the background. Thus, freeing up the main thread for UI operations.

reportOnAllPeopleToLog

…simply reads the Core Data and displays the results.

Keep in mind that in the demo, the mainQueueContext is initialized with
NSMainQueueConcurrencyType, which runs on the main thread.

Thus, you SHOULD put tasks that update UI here.

If you are to use a time consuming task like a loop that displays all the People (like how we’re doing it), expect the main UI to freeze.

If you want to change it so that logging is done in the background and not freeze the UI, then change the NSMainQueueConcurrencyType to NSPrivateQueueConcurrencyType.

Then, put your code inside [self.mainQueueContext performBlock:^{…}.

Below are just some of the standard methods for the core stack.

Now, run the app. Make sure there is no previous version installed because we want a clean database.

Click on the add bulk button so we let it add 10,000 people into the database. Then play around with the scrollbar, you’ll notice that its responsive. Hence, we’ve successfully used the context to queue up jobs and run them on a background queue. The main thread is then responsive.

add_bulk

If you want to display all the Persons you’ve added, click on the display all button. The display all button uses the mainQueueContext, which runs on the main UI. Hence, put all the code that updates the UI, into that queue.

display_all

For demonstration purposes, we are using NSPrivateQueueConcurrencyType for the mainQueueContext. Hence, anything you do on on that context SHOULD BE TO UPDATE THE UI.

If you do labor intensive job such as logging all the People you’ve added, the UI will freeze…as demonstrated in the updated demo project.

IF you want to do the logging in the background, you can change it to NSPrivateQueueConcurrencyType:

then place your logging task in the performBlock like so:

Then, your logging will be executed in the background and not block the UI.

Next, using multiple contexts >>

Core Data Stack #1 – everything on main queue

https://blog.codecentric.de/en/2014/11/concurrency-coredata/

Everything on Main Queue

Create a iOS project and check “using Core Data”.

This is how it works:

basic_core_data_diagram

At the most basic level we have a CoreDataStack class.
In its implementation file, we first have:

NSManagedObjectModel – Managed Object Model

We start with the managed object model.

  • Essentially, it is the schema of our model.
  • Our model comes from the .momd file. In xcode, you can create your model and it will save to the .momd file.
  • Then we load it into a NSURL, which in turn is used to initialize a NSManagedObjectModel object.

NSPersistentStoreCoordinator – Persistent Store Coordinator

The persistent store coordinator is responsible for coordinating access to multiple persistent object stores. As an iOS developer you will never directly interact with the persistent store coordinator and, in fact, will very rarely need to develop an application that requires more than one persistent object store. When multiple stores are required, the coordinator presents these stores to the upper layers of the Core Data stack as a single store.

This is essentially the database connection. It coordinates between our model and a .sqlite file.

The class has attribute that points to our previous defined NSManagedObjectModel. Then it stores a NSURL to a .sqlite file. From there on, it is used by either one or many NSManagedObjectContexts in order to do DB operations.

NSManagedObjectContext – managed object context

This is the scratch pad for what goes into and comes out of the database. We use this to do many operations, then use save context, predicates, and other special operations to make changes to the database. There can be either 1 or more context used at the same time.

Full Source

CoreDataStack.h

CoreDataStack.m

Inserting into Core Data

Notice this method:

insertNewObjectForEntityForName returns a NSManagedObject * for you to manipulate. All you have to do is set the attributes of the object and then call the saveContext method.

NSManagedObject are the objects that are created by your application code to store data. A managed object can be considered as a row or a record in a relational database table. For each new record to be added, a new managed object must be created to store the data. Similarly, retrieved data will be returned in the form of managed objects, one for each record matching the defined retrieval criteria. Managed objects are actually instances of the NSManagedObject class, or a subclass thereof. These objects are contained and maintained by the managed object context.

When NSEntityDescription returns an object for you to insert. All you have to do is set the attributes of the object and then call the saveContext method.

Every object that is returned to you by the method insertNewObjectForEntityForName will be monitored by your context. Once you finish updating those objects, and you call saveContext, your context will check up on those objects and then save them accordingly.

Group changes together

However, calling saveContext for every change takes a bit longer because you are having the persistent store coordinator communicate with the database off of one change. You can group your changes together, and then tell your context to save once.

The Downside

While this stack is very simplistic, the downside is that if you record large amount of data, the UI will be unresponsive because you are running off of the main thread.

coredata-ui-clogged

For example, in our example, we have a for loop of 10. Change it to 10,000, and try to play with the UI controls on your viewController. They won’t work because your main thread is busy processing the 10,000 core data operations.

basic inheritance

If the public method is NOT over-ridden, your class extends the hierarchy, then call the public method, you’ll use the public method declared FARTHEST from you

Say you have a Base class with a public method called “publicMethod”

Then, you have a Child1 class that derives from Base class. It does not over-ride publicMethod in Base class.

Then, you have a Child2 class that derives from Child1 class.

When you go:

It will call on publicMethod in the Base class.

If the public method IS over-ridden, your class extends the hierarchy, then call the public method, you’ll use the public method declared CLOSEST to you

Child2 derives from Child1

Then you go:

Then it will call Child1’s anotherPublicMethod

Cascade – every inheritance node needs to run

If you want a certain method tor un from Base to Child1, to Child2, make sure you use [super …] in the beginning of your call. A great example would be the init method.

Base’s init

Child1’s init

Child2’s init

Allocating a Child2 object

would have the result:

— Base.m – creating Base object —
— Child1.m, init Child1 object —
— Child2.m – init Child2 object

Child2’s init calls [super init], which is Child1’s init
Child1’s init calls [super init], which is Base’s init
Base’s init calls [super init], which is NSObject’s init

When NSObject’s init returns, our execution moves forward and displays Base init method’s log
When Base init method returns, our execution moves forward and displays Child1’s init method’s log
When Child1’s init method returns, execution moves forward and displays Child2’s init method’s log.

Hence, we have

— Base.m – creating Base object —
— Child1.m, init Child1 object —
— Child2.m – init Child2 object

Core Data Part 5 – deleting

Core Data part 4 – rollback

After you get an object and change its attributes, IF YOU DO NOT saveContext, you can rollback the data to its original value.

For example, say we get the first indexed Person object. You change the first name to Ricky. If you use [coreDataStack saveContext], that means the change is permanent in the database and you will be able to see the change when you restart up.

But if we comment out [coreDataStack saveContext], that means the change is not permanent and is only in cache context. You will see the change in the current usage of the app, but if you restart the app, you will see the old value.

In other words, in the database, we have NOT updated the value.

When [coreDataStack saveContext] is commented out, the changes you make is ONLY reflected in the current usage of the app, and not in the database. Thus, you can rollback to the original value by first checking to see if there was a change on the context by using
[[coreDataStack managedObjectContext] hasChanges]. Then if there was a change, you rollback the value by doing [[coreDataStack managedObjectContext] rollback]