Login/Register
Sonotype Nexus Repository/OrientDB can't connect to database
Asked Answered
orientdbcorruptsonatype
C

1

9

Yesterday our nexus repository instance went sideways due to a disk full event. After freeing some space and attempting a restart, nexus is no longer accessible. As a result, the database became corrupted (presumably) and cannot be connected to. When I run the nexus-orient-console.jar to connect, here's what I get:

orientdb> CONNECT plocal:/nexus-data/db/component admin admin        

Connecting to database [plocal:/nexus-data/db/component] with user 'admin'...
2018-08-09 22:04:03:769 WARNI {db=component} Storage 'component' was not closed properly. Will try to recover from write ahead log...
2018-08-09 22:04:03:775 WARNI {db=component} Record com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OCheckpointEndRecord{lsn=LSN{segment=34, position=52}} will be skipped during data restore$ANSI{green {db=component}} Exception during storage data restore
java.lang.NullPointerException
    at com.orientechnologies.orient.core.storage.impl.local.paginated.wal.ODiskWriteAheadLog.cutTill(ODiskWriteAheadLog.java:919)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.makeFullCheckpoint(OAbstractPaginatedStorage.java:3718)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.recoverIfNeeded(OAbstractPaginatedStorage.java:3920)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:294)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:259)
    at com.orientechnologies.orient.console.OConsoleDatabaseApp.connect(OConsoleDatabaseApp.java:234)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.orientechnologies.common.console.OConsoleApplication.execute(OConsoleApplication.java:405)
    at com.orientechnologies.common.console.OConsoleApplication.executeCommands(OConsoleApplication.java:260)
    at com.orientechnologies.common.console.OConsoleApplication.run(OConsoleApplication.java:131)
    at com.orientechnologies.orient.console.OConsoleDatabaseApp.main(OConsoleDatabaseApp.java:145)
    at org.sonatype.nexus.orient.console.Main.main(Main.java:58)
Exception `1FDE0371` in storage `plocal:/nexus-data/db/component`: 2.2.31 (build 285537d2767275f460df32c6a3be01bfff6a517c, branch 2.2.x)
java.lang.NullPointerException
    at com.orientechnologies.orient.core.storage.impl.local.paginated.wal.ODiskWriteAheadLog.cutTill(ODiskWriteAheadLog.java:919)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.makeFullCheckpoint(OAbstractPaginatedStorage.java:3718)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doClose(OAbstractPaginatedStorage.java:4397)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.close(OAbstractPaginatedStorage.java:580)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:316)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:259)
    at com.orientechnologies.orient.console.OConsoleDatabaseApp.connect(OConsoleDatabaseApp.java:234)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.orientechnologies.common.console.OConsoleApplication.execute(OConsoleApplication.java:405)
    at com.orientechnologies.common.console.OConsoleApplication.executeCommands(OConsoleApplication.java:260)
    at com.orientechnologies.common.console.OConsoleApplication.run(OConsoleApplication.java:131)
    at com.orientechnologies.orient.console.OConsoleDatabaseApp.main(OConsoleDatabaseApp.java:145)
    at org.sonatype.nexus.orient.console.Main.main(Main.java:58)
$ANSI{green {db=component}} Error during storage close
java.lang.NullPointerException
    at com.orientechnologies.orient.core.storage.impl.local.paginated.wal.ODiskWriteAheadLog.cutTill(ODiskWriteAheadLog.java:919)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.makeFullCheckpoint(OAbstractPaginatedStorage.java:3718)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doClose(OAbstractPaginatedStorage.java:4397)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.close(OAbstractPaginatedStorage.java:580)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:316)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:259)
    at com.orientechnologies.orient.console.OConsoleDatabaseApp.connect(OConsoleDatabaseApp.java:234)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.orientechnologies.common.console.OConsoleApplication.execute(OConsoleApplication.java:405)
    at com.orientechnologies.common.console.OConsoleApplication.executeCommands(OConsoleApplication.java:260)
    at com.orientechnologies.common.console.OConsoleApplication.run(OConsoleApplication.java:131)
    at com.orientechnologies.orient.console.OConsoleDatabaseApp.main(OConsoleDatabaseApp.java:145)
    at org.sonatype.nexus.orient.console.Main.main(Main.java:58)
Exception `260A3A5E` in storage `plocal:/nexus-data/db/component`: 2.2.31 (build 285537d2767275f460df32c6a3be01bfff6a517c, branch 2.2.x)
com.orientechnologies.orient.core.exception.OStorageException: Cannot open local storage '/nexus-data/db/component' with mode=rw
    DB name="component"
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:323)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:259)
    at com.orientechnologies.orient.console.OConsoleDatabaseApp.connect(OConsoleDatabaseApp.java:234)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.orientechnologies.common.console.OConsoleApplication.execute(OConsoleApplication.java:405)
    at com.orientechnologies.common.console.OConsoleApplication.executeCommands(OConsoleApplication.java:260)
    at com.orientechnologies.common.console.OConsoleApplication.run(OConsoleApplication.java:131)
    at com.orientechnologies.orient.console.OConsoleDatabaseApp.main(OConsoleDatabaseApp.java:145)
    at org.sonatype.nexus.orient.console.Main.main(Main.java:58)
Caused by: java.lang.NullPointerException
    at com.orientechnologies.orient.core.storage.impl.local.paginated.wal.ODiskWriteAheadLog.cutTill(ODiskWriteAheadLog.java:919)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.makeFullCheckpoint(OAbstractPaginatedStorage.java:3718)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.recoverIfNeeded(OAbstractPaginatedStorage.java:3920)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:294)
    ... 11 more

Error: com.orientechnologies.orient.core.exception.OStorageException: Cannot open local storage '/nexus-data/db/component' with mode=rw
    DB name="component"

Error: java.lang.NullPointerException

Because I am not able to even connect to the database, I haven't been able to attempt the recovery steps suggested here > OrientDB corruption state in Nexus Repository version 3.2.0-01. Is there any way to repair and recover the databases or are we just S.O.L.? Is there any way to start nexus in a safe-mode that ignores database issues?

Collaboration answered 9/8, 2018 at 22:27 Comment(3)
You can try to repair database as suggested. – Grab
@Lance Held did you find a solution? I'm having the same problem. – Claireclairobscure
@Claireclairobscure We eventually hacked our way through it by upgrading to the latest version of nexus. It worked for the most part, but there are aspects of the application that don't work properly still. We have been limping along for a few months now and have gotten to the point where we have decided to install a fresh instance and transcribe our settings an repos. :-/ – Collaboration
S
11

This thread must fixed our issue. https://groups.google.com/a/glists.sonatype.com/g/nexus-users/c/64W_y9fDGzQ/m/o0FKRgiDAwAJ?pli=1

To summarize, this steps fixed my error (same as you) :

rm /mnt/sonatype-work/nexus3/db/config/*.wal

Then try to start the console and repair the database

At the orientdb prompt run the following:

connect plocal:/mnt/sonatype-work/nexus3/db/confg admin admin
repair database --fix-links
rebuild index *
disconnect 
exit

For Nexus 3.2.1+ it is possible to use the included OrientDB client.

Spend answered 8/9, 2020 at 8:8 Comment(1)
Here it was enough to just delete the *.wal files and restart Nexus (3.34.0). Nexus than started to rebuild the index on its own. – Portfolio

© 2022 - 2024 — McMap. All rights reserved.