Not blocking the UI with FMDB Queue

Not blocking the UI with FMDB Queue

xCode 7.3 concept demo

So notifications are coming in rapidly…each one is a thread, and the threads’s data are getting inserted into your FMDB. Your FMDB ensures thread safety because FMDB’s queue uses dispatch_sync:

FMDatabaseQueue.m

It gets the block of database code you pass in, and puts it on a SERIAL QUEUE.

The dispatch_sync used here is not a tool for getting concurrent execution, it is a tool for temporarily limiting it for safety.

The SERIAL QUEUE ensures safety by having each block line up one after another, and start their execution ONLY AFTER the previous task has finished executing. This ensures that you are thread safe because they are not writing over each other at the same time.

However there is a problem. Let’s say your main thread is processing a for loop with the inDatabase method. The main thread places the block on FMDB Queue’s serial queue. This means the dispatch_sync that FMDB Queue is using will block your main thread as it processes each task. By definition, dispatch_sync DOES NOT RETURN, until after it has finished its executing task.

Proving FMDB does block

We need to prove that FMDB does indeed block our UI so we first put a UISlider in ViewController for testing purposes. If we are concurrently processing all these incoming notifications in the background, then this UISlider should be responsive.

You put a slider on your UI like so:

When you run a simple for loop outside of method say executeUpdateOnDBwithSQLParams:, you are essentially adding a dispatch_sync on your main thread. This will block, and your UI will NOT be responsive.

In order to solve this, we do 2 things:

  1. Use a concurrent queue and have main thread work on it to ensure concurrency and that the UI is not blocked
  2. Inside of that concurrent queue, we queue up db jobs to FMDB’s thread safe serial queue

Solution

dispatch_sync does not return until its task is finished. Thus, while the task is executing, the main queue can’t do anything because the dispatch_sync has not returned. That’s the just of the issue.

What we did to solve this issue is to

dispatch_async FMDB tasks on a concurrent queue.

This is the basic setup that enables fmdb to be non-blocking.

1) We set up a block on a concurrent queue first. This ensures that whatever runs inside of that concurrent block will be able to run
concurrently with the main thread.

2) The block starts off with executing its log. Then the PRE-TASK. Then it syncs its own block of the “DB Task”. This sync means that it blocks whatever is trying run with it, on conQueue. Hence, that’s why POST-TASK will run after DB task.

3) Finally, after PRE-TASK, then DB task, finish running, POST-TASK runs.


— start block (concurrent queue) —
—  Task PRE-TASK on concurrent queue start  —
PRE-TASK on concurrent queue – 0
PRE-TASK on concurrent queue – 1
PRE-TASK on concurrent queue – 2
^^^ Task PRE-TASK on concurrent queue END ^^^
— start block (sync queue) —
—  Task DB task start  —
DB task – 0
DB task – 1
DB task – 2
DB task – 3
DB task – 4
DB task – 5
DB task – 6
DB task – 7
DB task – 8
DB task – 9
^^^ Task DB task END ^^^
— end block (sync queue) —
—  Task POST-TASK on concurrent queue start  —
POST-TASK on concurrent queue – 0
POST-TASK on concurrent queue – 1
POST-TASK on concurrent queue – 2
^^^ Task POST-TASK on concurrent queue END ^^^
— end block (concurrent queue) —

The dispatch_sync is to simulate FMDB like so:

So BOTH tasks

  • dispatch_sync is running its db tasks (fmdb)
  • free to move your UI (main queue)

are processing on the concurrent queue via dispatch_async.
Thus, that’s how you get FMDB to be non-blocking.

Tidbit: changing dispatch_async to dispatch_sync on the concurrent queue

If you were to change from dispatch_async to dispatch_sync on the concurrent queue “conQueue”, it will block the main queue
when it first starts up because by definition, dispatch_sync means it does not return right away. It will return later
when it runs to the end, but for now, it blocks and you’re not able to move your UI.

Thus, it runs to PRE-TASK, and executes that.

Then it moves down, and runs the “DB task” block via dispatch_async on serial queue “queue”.

The dispatch_async returns immediately, starts executing “DB task” on serial queue “queue”, and then it executes
POST-TASK. Thus, DB task and POST-TASK will be executing together.

After POST-TASK finishes, our concurrent block has ran to the end, and returns (due to dispatch_sync).
At this point, you will be able to move the UI. “DB task” continues its execution because its part of the task that’s still sitting on concurrent queue “conQueue”.
Since its a concurrent queue, it will be processing this task that’s still sitting on its queue, and you will be able to move around the UI because nothing is blocking anymore.

Other details when you have time

where concurrencyQueue is created and initialized like so:

But what about the database writing that is dispatch_sync on the serial queue? Wouldn’t that block?

No. The dispatch_sync on the serial queue only blocks against thread(s) that touches FMDB Queue’s serial queue. In this case, it would be FMDBQueue’s serial queue’s own thread, and the concurrent queue’s thread.

does_not_block_main

max of 64 threads running concurrently

ref – http://stackoverflow.com/questions/34849078/main-thread-does-dispatch-async-on-a-concurrent-queue-in-viewdidload-or-within

Note that when you are using custom concurrent queues, you are only allowed to use 64 concurrent working threads at once. Hence, that’s why when your main thread calls on the concurrent queue and queue up tasks, the system starts blocking your UI after 64 tasks on the queue.

The workaround is to put the task of placing db tasks onto the concurrent queue onto your main queue like so:

Then simply call the utility method run_async_with_UI and place your database calls in there.

Proof of concept

The dispatch_sync(serialQueue,….) is essentially the FMDB Queue.
We just added dispatch_async(concurrencyQueue…). Now, you can see that we are manipulating the database in a thread-safe manner, in the background, without clogging up the UI.

result:

So, the dispatch_async throws all the tasks onto the concurrent queue without returning (aka blocking). That’s why all the task blocks log “concurrentQueue inserts task n”.

The task the dispatch_async throws onto the serialQueue will start executing immediately via dispatch_sync. Dispatch_sync by definition means it won’t return until its block has finished executing. Hence, this means that “concurrentQueue’s END task n” message won’t be logged until after the block on serialQueue has been executed”.


Notice how serialQueue FINISHED 1, then concurrentQueue logs END task 1.
serialQueue FINISHED 0, then concurrentQueue END task 0…

its because dispatch_sync does not return until it has finished executing.
Once it returns, it continues onto the “concurrentQueue END task n” log message.

In other words, due to dispatch_sync, line 10-16 must be run, before line 20 is run.

Another important note is that notice serialQueue has started to execute. But by definition, dispatch_sync blocks and does not return until the current executing task is finished…so how does concurrentQueue keeps inserting?

The reason is that the blocks on serialQueue is running in the background. The dispatch_sync that’s not returning happens in the background, and thus, does not affect the UI. The enqueueing of the “db simulate write” onto the serialQueue is done on the background queue concurrentQueue.

Say we switch it

So now we dispatch_sync a block onto a queue, it will not return until it finishes enqueueing. The key point here is that “due to dispatch_async throwing the task onto the serialQueue and returning immediately”, enqueueing will be:

lots of fast enqueueing of blocks onto the concurrent queue, thus, logging of
line 5, and line 20.

example:

1. block task 1 goes onto concurrent queue via dispatch_sync, WILL NOT RETURN UNTIL WHOLE TASK BLOCK IS FINISHED
2. “simulate DB write” task block goes onto serial Queue via dispatch_async, RETURNS RIGHT AWAY.
3. block task 1 finished, thus RETURNS control to concurrent queue.

4. block task 2 goes onto concurrent queue via dispatch_sync, WILL NOT RETURN UNTIL WHOLE TASK BLOCK IS FINISHED
5. “simulate DB write” task block goes onto serial Queue via dispatch_async, RETURNS RIGHT AWAY.
6. block task 2 finished, thus RETURNS control to concurrent queue.



etc.

continues until the serialQueue, being a background queue, starts processing its first block. Hence it will display:

serialQueue – START 0———
serialQueue – FINISHED 0——–

Hence, the situation is that all the tasks of putting “simulate write tasks onto the serial Queue” are enqueued onto the concurrent queue quickly.

Then, when the serial Queue decides to execute its first task, thats’ when it does its first “DB write simulate”. This DB write simulate does not block UI because its being done in the background.

result:

Then after all the tasks are being enqueued onto the concurrent queue…the serialQueue processing task one by one.

Using Cocoapods

ref – http://stackoverflow.com/questions/23755974/cocoapods-pod-install-takes-forever

http://blogs.triffort.com/?p=309

https://www.raywenderlich.com/156971/cocoapods-tutorial-swift-getting-started

Install cocoapods

First, install pod:

The verbose option logs progress as the process runs, allowing you to watch the process instead of seeing a seemingly “frozen” screen.

Note: If you have previously installed pod and want to do a re-install do this:

Create a xCode project

Open Terminal and navigate to the directory that contains your project by using the cd command:

cd ~/Path/To/Folder/YourProject

Next, enter the following command:

Go into the root directory of the project.

This creates a Podfile for your project.
Finally, type the following command to open the Podfile using Xcode for editing:

Note: You shouldn’t use TextEdit to edit the Podfile because it replaces standard quotes with more graphically appealing typeset quotes. This can cause CocoaPods to become confused and throw errors, so it’s best to use Xcode or another programming text editor to edit your Podfile.

The default Podfile looks like this:

Delete the # and space before platform, and delete the other lines starting with #.
Your Podfile should now look like this:

This tells CocoaPods your project is targeting iOS 9.0 and will be using frameworks instead of static libraries.
In order to use CocoaPods written in Swift, you must explicitly include use_frameworks! to opt into using frameworks. If you forget to include this, and CocoaPods detects you’re trying to use a Swift CocoaPod, you’ll get an error when you try to install the pods.

Installing your Dependency

It’s finally time to add your first dependency using CocoaPods. Add the following to your Podfile, right after use_frameworks!:

This tells CocoaPods you want to include Alamofire version 4.4.0 – the latest, stable version at the time of writing this tutorial – as a dependency for your project.
Save and close the Podfile.

You now need to tell CocoaPods to install the dependencies for your project. Enter the following command in Terminal, after ensuring you’re still in the directory containing your project and Podfile:

You should see output similar to the following:
Analyzing dependencies
Downloading dependencies
Installing Alamofire (4.4.0)
Generating Pods project
Integrating client project

[!] Please close any current Xcode sessions and use YourProject.xcworkspace for this project from now on.
Open the project folder using Finder, and you’ll see CocoaPods created a new YourProject.xcworkspace file and a Pods folder in which to store all the project’s dependencies.

instance variable strong references?

ref – http://stackoverflow.com/questions/16461343/ivars-references-strong-weak-or-what

Default behavior of instance variable

Instance variable maintains a strong reference to the objects by default

Why don’t we specify weak/strong for iVar like properties?

Local variables and non-property instance variables maintain a strong references to objects by default. There’s no need to specify the strong attribute explicitly, because it is the default.
A variable maintains a strong reference to an object only as long as that variable is in scope, or until it is reassigned to another object or nil.

If you don’t want a variable to maintain a strong reference, you can declare it as __weak, like this:

Function Objects, and closures (javascript)

Function definitions

Use new to create instances, you are having objects returned to you.

Thus, the objects would be referenced by t1 and t2. If you were to access property name, you’d get the results:

t1 this.name – Test1() – hooray works!

public and private

this.property is public. var properties are private.

Given a function prototype without a closure, this.property is public, while a var is private

Here comes the Closure

Now, add the function definition growls into Test2.

By adding a function inside of a function, the whole thing becomes a closure.
2 things happen:

  • the inner function now has references to the local variables, such as any variables you declare
  • Outside cannot access the object’s properties. The properties have become private.

Hence, having our reference t2.name will give us undefined:

Because this is referenced differently in functions, depending on whether:

  • object that’s attached to the function, calls the function
  • scope calls the function
  • object uses addListener, attach…etc

CLOSURE EXPLAINED

A closure is when you have a function inside of a function. It automatically sets all properties to private.

Result

Test2() Constructor: Jabba the Hut
undefined yells out hadoken

The reason why this happens is because the this of Test2() and the this are different. The this in the growls function refers to the object that’s bound to the function.

The this of the Test2() is bound to its scope only, and does not affect the this of function growls. It can be used within Test2()’s function scope only and is used privately.

The this.name in the functions use the object that the function is attached to.

You can assign public attributes to the calling object by doing t2.name = “Ken”.

Hence

would give you


Test2() Constructor: Jabba the Hut
ricky yells out hadoken

because the this in function growls is referring to the object (in our case t2), which has attribute name of “ricky”.

Using var or let

If you were to use var or let, you can access the values because by definition, the var’s scope is the closest outer functional scope, and the let’s scope is the closest enclosing block scope.


Test2() Constructor: Jabba the Hut
ricky yells out hadoken
age is: 36
type is: human

read:

http://stackoverflow.com/questions/133973/how-does-this-keyword-work-within-a-javascript-object-literal

Using ‘self’ for non strict js

If you are not using strict mode, use self and use self in your inner functions in order to use properties like OOP.

result:


Test2() Constructor: Jabba the Hut
Jabba the Hut yells out hadoken
age is: 36
type is: human

The reason why it works is because self is a var, which means var’s scope extends all the way into the inner function’s scope as well.

NOT using New…our scope becomes global


Test2() Constructor: Jabba the Hut
Ryu.. yells out hadoken
age is: 36
global.name: Jabba the Hut

So as you can see because we are NOT using new, it means the attached this reference is the global object to our Test2 function.
this.name automatically attaches the attribute name to the global object.

We put object var t2 onto the stack, and assign Test2 function to this object t2.

t2.name = “Ryu…” basically assigns the attribute name to the t2 object.

t2 object then invokes the growls function. Inside the growls function, the this refers to the object that’s invoking it, which in our case, is t2. Hence, that’s why this.name will get “Ryu”.

Using “use strict” and “=>”

Until arrow functions, every new function defined its own this value (a new object in case of a constructor.

It is undefined in strict mode function calls, the context object if the function is called as an “object method”, etc.).

This proved to be annoying with an object-oriented style of programming.

So you’ll get

“Jabba the Hut yells out hadoken”
“Jabba the Hut yells out shoooo ryu ken!”

So what’s going on

We have to see how Test2 is created. We see that we use New to create it. So right away we know 2 things:

1) Test2 object’s this is lexically scoped to itself. The this used is private and lexically scoped within Test2 only.

2) On the stack we have a reference variable called t2. It now points to the object Test2 in the heap.

In growls function, the this in that function will refer to t2’s public attributes. Hence that’s why when you use this.name, it will be ‘Ryu’, because ryu is assigned to the public name attribute as defined for t2.

using self.name will give you “Jabba the Hut” because self references the this that’s privately scoped.

In growls2 function, the this is lexically tied to the this that’s scoped for the Test2 object on the heap. IT DOES NOT TIE ‘THIS’ TO THE OBJECT THAT’S CALLING THIS FUNCTION ANYMORE’ Hence, typing the this to the private this of the object in the heap is more object oriented behavior.

Now, if the object was created like

It means Test2()’s this scope is tied to global, or window.

Thus, all the other behavior is the same. The only difference is that in Test2’s this.name, you are referring to the window or global object, and thus adding the name attribute to it.

If you go

you will see the result.

Function Constructor, Prototype, and mix

ref – http://a-developer-life.blogspot.com/2011/11/7-ways-to-create-objects-in-javascript.html

Creates a Person object, which has attribute name, and function getName.

The constructor pattern is compact, reusable and gives a classical OO syntax.

However, here’s the problem:

In our example the field getName is a function.

In Javascript functions are objects. Thus, 10 object will have 10 function. And this means we define ten extra objects.

Escalate this to a thousand and realize how much memory is being wasted!

It would be nice if all instances of Person share the same getName object, since this holds behavior and not data.

Prototype

Functions are very special in Javascript. They are objects, they can create other objects and they automatically get a field called prototype.

A prototype is a plain object with a single field, called constructor, pointing to the function itself.
What makes it special is that every object created through a function inherits the function’s prototype.

Thus, personOne and personTwo will both have attribute name initialized to “Diego”. Therefore, all instances share the same fields.

Function constructor gives you objects with the instantiation’s own functions, but numerous instances take up too much space because each function is like an object.

Prototype let’s you escape that by using the prototype property. This let’s the instantiations share attributes and functions.

The function/prototype combination let’s you take advantage of both approaches.

While sharing behavior (the getName function), each object has its own data and state in the attribute ‘name’.

attach functions to objects

First declare your objects

Then you declare a function…

You can literally attach functions to objects like so:

Where the function uses ‘this’ to access the calling object’s attributes.
Using them like this:

would give:

name is: gary, age is: 25
name is: ricky, age is: 35

‘this’ in function, and in object

this is not part of the closure scope, it can be thought of as an additional parameter to the function that is bound at the call site.

If the method is not called as a method then the global object is passed as this. In the browser, the global object is identical to window. For example, consider the following funciton,

and the following object,

If you call the function using method syntax such as,

then this is bound to obj.

If you call someFunction() directly, such as,

then this is bound to the global object, that is window.