Application crashes with java.util.concurrent.TimeoutException io.objectbox.BoxStore.close:707

[avarughese-tripactions]

Is there an existing issue?

• [] I have searched existing issues

Build info

• ObjectBox version: 4.0.3
• OS: Android 13, 14 and 15
• Device/ABI/architecture: happening across various OEM's and models. Some samples is Pixel 8 pro, Pixel8, Pixel 6a, Open CPH2551 etc

Steps to reproduce

We cannot reproduce the issue so far. It's happening at random in production with the new version upgrade from 3.7.1 to 4.0.3. The majority cases are happening right at the start/initialization of Object Box. But there are cases where we are seeing this crash quite randomly while the app is running

Expected behavior

The app should not crash

Actual behavior

The app is mostly crashing right at startup and sometimes randomly while in middle of execution.

Code

Code

This is how we do the initialization 

try {
   
    val builder = MyObjectBox.builder()
    boxStore = builder.androidContext(context)
        .name(custom_name)
        .build()
} catch (error: DbException) {

   boxStore?.removeAllObjects()
    if(!retry){
		// try it one more time
    }
}

Logs, stack traces

java.util.concurrent.TimeoutException: io.objectbox.BoxStore.finalize() timed out after 10 seconds
at io.objectbox.BoxStore.close(BoxStore:707)
at io.objectbox.BoxStore.finalize(BoxStore:525)
at java.lang.Daemons$FinalizerDaemon.doFinalize(Daemons.java:370)
at java.lang.Daemons$FinalizerDaemon.processReference(Daemons.java:350)
at java.lang.Daemons$FinalizerDaemon.runInternal(Daemons.java:322)
at java.lang.Daemons$Daemon.run(Daemons.java:131)
at java.lang.Thread.run(Thread.java:1012)

Logs

[Paste your logs here]

[greenrobot-team]

Thanks for reporting! See #1190 (comment).

Note: I labeled this issue with "more info required" so it will auto-close in a few days if there are no follow-up comments.

[avarughese-tripactions]

can you please provide some pointers on what kind of info you are looking for?

[RobbWatershed]

Interesting! I've been analyzing that very same issue on my app for weeks without any clear clue either.

I can't completely blame ObjectBox v4 as I've been migrating all my code to Kotlin at the same time, but I can't rule it out either.

[RobbWatershed]

One more info : I've been extensively reviewing threaded Box operations and made sure to call BoxStore.closeThreadResources() on the same thread that did the operations, using a finally block. That didn't really help in my case.

[avarughese-tripactions]

We figured out the problem. So basically what we see is while creating the Boxstore object it checks if a file is already open. If it is open it creates a checker thread which calls the system.gc while acquiring lock to the openFiles object. It tries to run finalization by looping. Meanwhile if the gc gets invoked then close() function is called via finalize(). This GC thread tries to acquire the lock on openFiles object which is not available. They get in to a deadlock situation causing a timeout
This is the call stack scenario:
BoxStore constrauctor-> verifyNotAlreadyOpen-> isFileOpen -> checkerThread
checkerThread -> isFileOpenSync-> locks openFiles
GC thread -> Finalize -> Close -> waits to acquire lock on openFiles

we had a corner case in the code which might try to create a new instance while previous one is not closed hence triggering this. Although we think the architecture of ObjectBox creation might need a change to avoid this deadlock
Also, it doesn't look like lib version specific.
I will let the ObjectBox to comment

[RobbWatershed]

we had a corner case in the code which might try to create a new instance while previous one is not closed hence triggering this.

If you mean an instance of BoxStore, aren't we supposed to create it inside a Singleton?

[avarughese-tripactions]

Yes, it should be singleton but we had a bug and that's triggering this issue for us. That's what we think

[greenrobot-team]

@avarughese-tripactions Thank you very much for the analysis and details! From what I remember it sounds plausible and we will have a look!

Edit: for our reference, the internal issue for this is objectbox-java#240

[RobbWatershed]

Hello @greenrobot-team! Any update on that one since january?

My users are still regularly reporting related crashes.

Thanks in advance~

[greenrobot-team]

@RobbWatershed Not yet. As previously explained, a workaround to avoid the assumed deadlock might be to make sure that if your app creates a new BoxStore instance that any previous one has been explicitly closed. Do not let your code rely on a previous instance getting garbage collected.

[RobbWatershed]

I happen to have two different BoxStores (A and B) in the same app. They have a distinct lifecycle and do not share the same file (differentiated by using BoxStoreBuilder.name upon initialization).

Is that issue supposed to happen if BoxStore A tries to get a lock just before BoxStore B is being GC'ed? Or is it only linked to BoxStores using the same file?