are indexeddb/localforage reads resolved from a synchronous buffer?

Question

Taking the following pseudo code

localforageStore.setItem('foo', 'bar')
    .then(console.log('foo is persisted to disk')); 
localforageStore.getItem('foo')
    .then(v => console.info('foo is '+v));   // A, B or C? 

Is the console.info:-

• A. Guaranteed to display 'bar'

• B. Guaranteed to display 'undefined'

• C. Indeterminate

i.e. even though the write to disc is async, will a synchronous read be resolved from a buffer internal to indexeddb and/or localforage?

Ekg

Without knowing anything at all about localForage, my guess is C. The way your example is worded, the read is not waiting until the write completes. Therefore the read could settle first or second. Also, that example looks like an asynchronous read to me, despite your question about a synchronous read. – Turbary

when I described the read as "synchronous", I mean that the read is initiated as soon as the write returns. Had I placed the read inside the "then", I'd refer to that as "asynchronous". I agree this is possibly a poor choice of words. I know the write hasn't completed, however localforage and indexeddb both have the result available in memory, hence the question. Will localforage/indexeddb use this in-memory data to satisfy the read request? – Ekg

Fairly sure there is no buffering in indexedDB, but not sure about localforage. – Turbary

Answer 1

I took a look at the localForage indexedDB driver at https://github.com/localForage/localForage/blob/master/src/drivers/indexeddb.js. I do not see any buffering. Therefore, there is nothing for the getItem to grab from some buffer.

More specifically, looking at the localForage source again, I can see that setItem and getItem are basic promise wrappers around indexedDB transactions. Those wrappers resolve when the transactions complete. This tells me that indexedDB is what controls the non-blocking behavior, not localForage.

So, because indexedDB is in charge, this means we can look at indexedDB behavior to help answer your question. We are issuing two transactions, each with a request. The first is a readwrite transaction from setItem, and the second is a readonly transaction from getItem.

Ordinarily, transactions can overlap. For example, you can have 100 readonly transactions all running at the same time. However, readwrite transactions block other transactions in order to ensure data integrity. readwrite transactions cannot overlap.

This gets complicated a bit by the fact that you can fire and forget call things. You can init two transactions at the same time, without waiting to start the second until after the first completes. Note the difference between starting to run something, and actually have it be considered running.

So, looking at your code, setItem('foo', 'bar') starts a readwrite transaction, and getItem('foo') starts a readonly transaction. You are not waiting for the readwrite transaction promise wrapper to settle before starting the readonly transaction.

While that is indeed a non-blocking approach on the surface, it is still technically blocking within the indexedDB layer, because the readonly transaction will block (wait, indefinitely) for the prior readwrite transaction on the same object store to settle.

And this is what I think causes confusion. Because we know the readonly transaction was started after the readwrite was started, we know that the readonly transaction will technically always resolve only after the readwrite transaction, because it cannot possibly resolve before it, nor at the same time as it.

Therefore, you can say the answer would be A. Because the readonly transaction has to wait for the readwrite transaction to complete. It is indeterminate at the localForage promise layer, but determinate at the indexedDB transaction layer.

Look at the spec for some more technical explanation at https://www.w3.org/TR/IndexedDB-2/#transaction-construct

Here is a relevant section (emphasis mine):

If multiple read/write transactions are attempting to access the same object store (i.e. if they have overlapping scope), the transaction that was created first must be the transaction which gets access to the object store first. Due to the requirements in the previous paragraph, this also means that it is the only transaction which has access to the object store until the transaction is finished.

Any transaction created after a read/write transaction must see the changes written by the read/write transaction. So if a read/write transaction, A, is created, and later another transaction B, is created, and the two transactions have overlapping scopes, then B must see any changes made to any object stores that are part of that overlapping scope. Due to the requirements in the previous paragraph, this also means that the B transaction does not have access to any object stores in that overlapping scope until the A transaction is finished.

Generally speaking, the above requirements mean that any transaction which has an overlapping scope with a read/write transaction and which was created after that read/write transaction, can’t run in parallel with that read/write transaction.

Turbary

As a followup, this just gets hilariously more complex and confusing because browser vendors vary in behavior. For example what happens if you read then write? See github.com/w3c/IndexedDB/issues/253 – Turbary

Many thanks, Josh. My app wants A, so I'm thinking of adding a layer above localForage to guarantee this behaviour. I will cache each object as it is setItemed, and remove it from the cache when the write promise resolves. Any getItem will resolve from the cache if present, else fetch from indexedDb. Does this sound a reasonable approach, or would you be sufficiently confident to say that indexedDb's transaction isolation is enough to guarantee A? – Ekg

I do not enjoy adding a layer of indirection when the foundation is unclear. I personally would rely on indexedDB and only add a layer of indirection when I want to introduce invariant behavior with respect to the underlying implementation's potential for change. It is quite difficult to ascertain whether the spec has settled. I cannot really suggest what you should do, just relay my personal thinking. – Turbary

In other words, is your caching proposal increasing or decreasing complexity? Is performance even an issue here? – Turbary