How to actually set up basic Titan + Rexster + Cassandra?

Asked 31/10, 2013 at 17:5 Answered 9/8, 2015 at 4:33

I am trying to set up a completely basic Titan Rexster Cassandra instance, but I can't seem to break the code. I have tried a whole lot of things now to get it to work but I just can't seem to get it to work. No matter how much I read about it I am not able to set it up properly.

What I want is a Titan-rexster-cassandra instance running in embedded mode with a few indexes including elastic search. After all the stuff I have read it seems that this is what I should get when i download titan-server-0.4.0 and run the bin/titan.sh start command. An this also starts the server. However: When I try to add an index to this, nothing happens. When I try to populate it over RexPro nothing is added.

When I restart the server my graph is gone. It is no longer in the Rexster list of graphs when I go to http://localhost:8182/graphs. So it appears that my data does not persist, or at least disappears for rexster.

I feel like I have tried just about everything to get this to work:

Changing the .properties to include the search-index like so: storrage.index.search.backend=elasticsearch...
Changing the .properties files (all of them) to use cassandra, embeddedcassandra and cassandrathrift for storage.backend
Trying to start the server with properties as indicated in this question to point to specific config files.
I have looked through the titan.sh file to see what actually happens, then gone to the config files indicated by these and had a look to see what goes on there, upon which I have tried a lot of things such as the above.

I have struggled with this for well over a week, probably two or even more and I am starting to lose faith. I am considering going back to neo4j, but unfortunately I really need the scalability of Titan. However if I can't get it to work then it is no use. I feel like there might be some trivial but essential thing that I have not figured out, or forgot.

Do anyone know of a guide out there that brings you from absolute scratch (eg. starting a fresh VM or something), or close to it, to getting a titan-rexster-cassandra instance running with elastic search index? Or perhaps, if you are awesome, provide such a guide? I feel lost :(

Key Points:

Ubuntu 12.04 (also tried 13.10. Same issue)

Titan 0.4.0

Goal: To get persistance, index a vertex name property with Elastic search, and get edges with weight.

Connecting with ruby rexpro like this:

require "rexpro" #the "rexpro" gem
rexpro_client = Rexpro::Client.new(host: 'the.ip.of.my.machine.running.rexster', port: 8184)
results = rexpro_client.execute("g.getClass()", graph_name: "graph").results
#=> returns the following: class com.thinkaurelius.titan.graphdb.database.StandardTitanGraph

The steps I follow to create the problem where the DB does not persist:

On WindowsAzure: Create a new small (1 core, 1.75GB ram) VM with Ubuntu 12.04 LTS with name vmname (or whatever).
Log on to this VM with SSH when it is ready (ssh [email protected] -p 22)
Run: sudo apt-get update
Run: sudo apt-get install openjdk-7-jdk openjdk-7-jre p7zip-full
Run: mkdir /home/azureuser/Downloads
Run: wget -O /home/azureuser/Downloads/titan-server-0.4.0.zip "http://s3.thinkaurelius.com/downloads/titan/titan-server-0.4.0.zip"
Run: cd /home/azureuser/Downloads/
Run: 7z x titan-server-0.4.0.zip
Run: cd /home/azureuser/Downloads/titan-server-0.4.0
Run: sudo bin/titan.sh -c cassandra-es start
Run: sudo bin/rexster-console.sh
In rexster console, run: g = rexster.getGraph("graph"), returns titangraph[cassandra:null]
CTRL-C out of rexster consloe
Run: sudo bin/titan.sh stop
Run: sudo bin/titan.sh -c cassandra-es start
Run: sudo bin/rexster-console.sh
In rexster console, run: g = rexster.getGraph("graph"). Now this returns null, not a graph.

There appears to be some issues here when shutting down and starting up againt:

On shutdown

[WARN] ShutdownManager - ShutdownListener JVM Shutdown Hook Remover threw an exception, continuing with shutdown

On Startup #2

Starting Cassandra...
xss =  -Dtitan.logdir=/home/azureuser/Downloads/titan-server-0.4.0/log -ea -javaagent:/home/azureuser/Downloads/titan-server-0.4.0/lib/jamm-0.2.5.jar -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms840M -Xmx840M -Xmn100M -XX:+HeapDumpOnOutOfMemoryError -Xss256k
Starting Titan + Rexster...
 INFO 12:00:12,780 Logging initialized
 INFO 12:00:12,805 JVM vendor/version: OpenJDK 64-Bit Server VM/1.7.0_25
 INFO 12:00:12,806 Heap size: 870318080/870318080
 INFO 12:00:12,806 Classpath: /home/azureuser/Downloads/titan-server-0.4.0/conf:/home/azureuser/Downloads/titan-server-0.4.0/build/classes/main:/home/azureuser/Downloads/titan-server-0.4.0/build/classes/thrift:/home/azureuser/Downloads/titan-server-0.4.0/lib/activation-...
 INFO 12:00:13,397 JNA mlockall successful
 INFO 12:00:13,419 Loading settings from file:/home/azureuser/Downloads/titan-server-0.4.0/conf/cassandra.yaml
 INFO 12:00:14,093 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
 INFO 12:00:14,093 disk_failure_policy is stop
 INFO 12:00:14,101 Global memtable threshold is enabled at 276MB
 INFO 12:00:14,878 Initializing key cache with capacity of 41 MBs.
 INFO 12:00:14,892 Scheduling key cache save to each 14400 seconds (going to save all keys).
 INFO 12:00:14,894 Initializing row cache with capacity of 0 MBs and provider org.apache.cassandra.cache.SerializingCacheProvider
 INFO 12:00:14,955 Scheduling row cache save to each 0 seconds (going to save all keys).
 INFO 12:00:15,273 Opening db/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ib-2 (167 bytes)
 INFO 12:00:15,347 Opening db/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ib-1 (264 bytes)
 INFO 12:00:15,376 Opening db/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ib-11 (717 bytes)
 INFO 12:00:15,387 Opening db/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ib-9 (6183 bytes)
 INFO 12:00:15,392 Opening db/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ib-10 (687 bytes)
 INFO 12:00:15,411 Opening db/cassandra/data/system/schema_columns/system-schema_columns-ib-2 (209 bytes)
 INFO 12:00:15,416 Opening db/cassandra/data/system/schema_columns/system-schema_columns-ib-1 (3771 bytes)
 INFO 12:00:15,450 Opening db/cassandra/data/system/local/system-local-ib-3 (109 bytes)
 INFO 12:00:15,455 Opening db/cassandra/data/system/local/system-local-ib-2 (120 bytes)
 INFO 12:00:15,521 Opening db/cassandra/data/system/local/system-local-ib-1 (356 bytes)
Processes forked.  Setup may take some time.
Run bin/rexster-console.sh to connect.
azureuser@neugle:~/Downloads/titan-server-0.4.0$  INFO 12:00:16,705 completed pre-loading (8 keys) key cache.
 INFO 12:00:16,777 Replaying db/cassandra/commitlog/CommitLog-2-1383479792488.log, db/cassandra/commitlog/CommitLog-2-1383479792489.log
 INFO 12:00:16,802 Replaying db/cassandra/commitlog/CommitLog-2-1383479792488.log
 INFO 12:00:17,178 Finished reading db/cassandra/commitlog/CommitLog-2-1383479792488.log
 INFO 12:00:17,179 Replaying db/cassandra/commitlog/CommitLog-2-1383479792489.log
 INFO 12:00:17,179 Finished reading db/cassandra/commitlog/CommitLog-2-1383479792489.log
 INFO 12:00:17,191 Enqueuing flush of Memtable-local@1221155490(52/52 serialized/live bytes, 22 ops)
 INFO 12:00:17,194 Writing Memtable-local@1221155490(52/52 serialized/live bytes, 22 ops)
 INFO 12:00:17,204 Enqueuing flush of Memtable-users@1341189399(28/28 serialized/live bytes, 2 ops)
 INFO 12:00:17,211 Enqueuing flush of Memtable-system_properties@1057472358(26/26 serialized/live bytes, 1 ops)
 INFO 12:00:17,416 Completed flushing db/cassandra/data/system/local/system-local-ib-4-Data.db (84 bytes) for commitlog position ReplayPosition(segmentId=1383480016398, position=142)
 INFO 12:00:17,480 Writing Memtable-users@1341189399(28/28 serialized/live bytes, 2 ops)
 INFO 12:00:17,626 Completed flushing db/cassandra/data/system_auth/users/system_auth-users-ib-1-Data.db (64 bytes) for commitlog position ReplayPosition(segmentId=1383480016398, position=142)
 INFO 12:00:17,630 Writing Memtable-system_properties@1057472358(26/26 serialized/live bytes, 1 ops)
 INFO 12:00:17,776 Completed flushing db/cassandra/data/titan/system_properties/titan-system_properties-ib-1-Data.db (64 bytes) for commitlog position ReplayPosition(segmentId=1383480016398, position=142)
 INFO 12:00:17,780 Log replay complete, 12 replayed mutations
 INFO 12:00:17,787 Fixing timestamps of schema ColumnFamily schema_keyspaces...
 INFO 12:00:17,864 Enqueuing flush of Memtable-local@1592659210(65/65 serialized/live bytes, 2 ops)
 INFO 12:00:17,872 Writing Memtable-local@1592659210(65/65 serialized/live bytes, 2 ops)
[INFO] Application - .:Welcome to Rexster:.
 INFO 12:00:18,027 Completed flushing db/cassandra/data/system/local/system-local-ib-5-Data.db (97 bytes) for commitlog position ReplayPosition(segmentId=1383480016398, position=297)
 INFO 12:00:18,036 Enqueuing flush of Memtable-schema_keyspaces@1453195003(527/527 serialized/live bytes, 12 ops)
 INFO 12:00:18,038 Writing Memtable-schema_keyspaces@1453195003(527/527 serialized/live bytes, 12 ops)
[INFO] RexsterProperties - Using [/home/azureuser/Downloads/titan-server-0.4.0/conf/rexster-cassandra-es.xml] as configuration source.
 INFO 12:00:18,179 Completed flushing db/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ib-3-Data.db (257 bytes) for commitlog position ReplayPosition(segmentId=1383480016398, position=1227)
[INFO] Application - Rexster is watching [/home/azureuser/Downloads/titan-server-0.4.0/conf/rexster-cassandra-es.xml] for change.
[WARN] AstyanaxStoreManager - Couldn't set custom Thrift Frame Size property, use 'cassandrathrift' instead.
 INFO 12:00:18,904 Cassandra version: 1.2.2
 INFO 12:00:18,906 Thrift API version: 19.35.0
 INFO 12:00:18,906 CQL supported versions: 2.0.0,3.0.1 (default: 3.0.1)
[INFO] ConnectionPoolMBeanManager - Registering mbean: com.netflix.MonitoredResources:type=ASTYANAX,name=ClusterTitanConnectionPool,ServiceType=connectionpool
[INFO] CountingConnectionPoolMonitor - AddHost: 127.0.0.1
 INFO 12:00:19,087 Loading persisted ring state
 INFO 12:00:19,097 Starting up server gossip
 INFO 12:00:19,162 Enqueuing flush of Memtable-local@114523622(251/251 serialized/live bytes, 9 ops)
 INFO 12:00:19,169 Writing Memtable-local@114523622(251/251 serialized/live bytes, 9 ops)
 INFO 12:00:19,314 Completed flushing db/cassandra/data/system/local/system-local-ib-6-Data.db (238 bytes) for commitlog position ReplayPosition(segmentId=1383480016398, position=51470)
 INFO 12:00:19,369 Compacting [SSTableReader(path='db/cassandra/data/system/local/system-local-ib-3-Data.db'), SSTableReader(path='db/cassandra/data/system/local/system-local-ib-2-Data.db'), SSTableReader(path='db/cassandra/data/system/local/system-local-ib-4-Data.db'), SSTableReader(path='db/cassandra/data/system/local/system-local-ib-1-Data.db'), SSTableReader(path='db/cassandra/data/system/local/system-local-ib-6-Data.db'), SSTableReader(path='db/cassandra/data/system/local/system-local-ib-5-Data.db')]
 INFO 12:00:19,479 Starting Messaging Service on port 7000
 INFO 12:00:19,585 Using saved token [7398637255000140098]
 INFO 12:00:19,588 Enqueuing flush of Memtable-local@365797436(84/84 serialized/live bytes, 4 ops)
 INFO 12:00:19,588 Writing Memtable-local@365797436(84/84 serialized/live bytes, 4 ops)
 INFO 12:00:19,666 Compacted 6 sstables to [db/cassandra/data/system/local/system-local-ib-7,].  1,004 bytes to 496 (~49% of original) in 286ms = 0.001654MB/s.  6 total rows, 1 unique.  Row merge counts were {1:0, 2:0, 3:0, 4:0, 5:0, 6:1, }
 INFO 12:00:19,796 Completed flushing db/cassandra/data/system/local/system-local-ib-8-Data.db (120 bytes) for commitlog position ReplayPosition(segmentId=1383480016398, position=51745)
 INFO 12:00:19,810 Enqueuing flush of Memtable-local@1775610672(50/50 serialized/live bytes, 2 ops)
 INFO 12:00:19,812 Writing Memtable-local@1775610672(50/50 serialized/live bytes, 2 ops)
 INFO 12:00:19,967 Completed flushing db/cassandra/data/system/local/system-local-ib-9-Data.db (109 bytes) for commitlog position ReplayPosition(segmentId=1383480016398, position=51919)
 INFO 12:00:20,088 Node localhost/127.0.0.1 state jump to normal
 INFO 12:00:20,108 Startup completed! Now serving reads.
^C
azureuser@neugle:~/Downloads/titan-server-0.4.0$ sudo bin/rexster-console.sh[WARN] GraphConfigurationContainer - Could not load graph graph. Please check the XML configuration.
[WARN] GraphConfigurationContainer - GraphConfiguration could not be found or otherwise instantiated: [com.thinkaurelius.titan.tinkerpop.rexster.TitanGraphConfiguration]. Ensure that it is in Rexster's path.
com.tinkerpop.rexster.config.GraphConfigurationException: GraphConfiguration could not be found or otherwise instantiated: [com.thinkaurelius.titan.tinkerpop.rexster.TitanGraphConfiguration]. Ensure that it is in Rexster's path.
    at com.tinkerpop.rexster.config.GraphConfigurationContainer.getGraphFromConfiguration(GraphConfigurationContainer.java:137)
    at com.tinkerpop.rexster.config.GraphConfigurationContainer.<init>(GraphConfigurationContainer.java:54)
    at com.tinkerpop.rexster.server.XmlRexsterApplication.reconfigure(XmlRexsterApplication.java:99)
    at com.tinkerpop.rexster.server.XmlRexsterApplication.<init>(XmlRexsterApplication.java:47)
    at com.tinkerpop.rexster.Application.<init>(Application.java:96)
    at com.tinkerpop.rexster.Application.main(Application.java:188)
Caused by: java.lang.IllegalArgumentException: Could not instantiate implementation: com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager
    at com.thinkaurelius.titan.diskstorage.Backend.instantiate(Backend.java:339)
    at com.thinkaurelius.titan.diskstorage.Backend.getImplementationClass(Backend.java:351)
    at com.thinkaurelius.titan.diskstorage.Backend.getStorageManager(Backend.java:294)
    at com.thinkaurelius.titan.diskstorage.Backend.<init>(Backend.java:112)
    at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:682)
    at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.<init>(StandardTitanGraph.java:72)
    at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:40)
    at com.thinkaurelius.titan.tinkerpop.rexster.TitanGraphConfiguration.configureGraphInstance(TitanGraphConfiguration.java:25)
    at com.tinkerpop.rexster.config.GraphConfigurationContainer.getGraphFromConfiguration(GraphConfigurationContainer.java:119)
    ... 5 more
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at com.thinkaurelius.titan.diskstorage.Backend.instantiate(Backend.java:328)
    ... 13 more
Caused by: com.thinkaurelius.titan.diskstorage.TemporaryStorageException: Temporary failure in storage backend
    at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager.ensureKeyspaceExists(AstyanaxStoreManager.java:429)
    at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager.<init>(AstyanaxStoreManager.java:172)
    ... 18 more
Caused by: com.netflix.astyanax.connectionpool.exceptions.BadRequestException: BadRequestException: [host=127.0.0.1(127.0.0.1):9160, latency=42(60), attempts=1]InvalidRequestException(why:Keyspace names must be case-insensitively unique ("titan" conflicts with "titan"))
    at com.netflix.astyanax.thrift.ThriftConverter.ToConnectionPoolException(ThriftConverter.java:159)
    at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:65)
    at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:28)
    at com.netflix.astyanax.thrift.ThriftSyncConnectionFactoryImpl$ThriftConnection.execute(ThriftSyncConnectionFactoryImpl.java:151)
    at com.netflix.astyanax.connectionpool.impl.AbstractExecuteWithFailoverImpl.tryOperation(AbstractExecuteWithFailoverImpl.java:69)
    at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:256)
    at com.netflix.astyanax.thrift.ThriftClusterImpl.executeSchemaChangeOperation(ThriftClusterImpl.java:146)
    at com.netflix.astyanax.thrift.ThriftClusterImpl.addKeyspace(ThriftClusterImpl.java:246)
    at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager.ensureKeyspaceExists(AstyanaxStoreManager.java:424)
    ... 19 more
Caused by: InvalidRequestException(why:Keyspace names must be case-insensitively unique ("titan" conflicts with "titan"))
    at org.apache.cassandra.thrift.Cassandra$system_add_keyspace_result.read(Cassandra.java:33158)
    at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
    at org.apache.cassandra.thrift.Cassandra$Client.recv_system_add_keyspace(Cassandra.java:1408)
    at org.apache.cassandra.thrift.Cassandra$Client.system_add_keyspace(Cassandra.java:1395)
    at com.netflix.astyanax.thrift.ThriftClusterImpl$9.internalExecute(ThriftClusterImpl.java:250)
    at com.netflix.astyanax.thrift.ThriftClusterImpl$9.internalExecute(ThriftClusterImpl.java:247)
    at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:60)
    ... 26 more
[WARN] GraphConfigurationContainer - Could not instantiate implementation: com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager
java.lang.IllegalArgumentException: Could not instantiate implementation: com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager
    at com.thinkaurelius.titan.diskstorage.Backend.instantiate(Backend.java:339)
    at com.thinkaurelius.titan.diskstorage.Backend.getImplementationClass(Backend.java:351)
    at com.thinkaurelius.titan.diskstorage.Backend.getStorageManager(Backend.java:294)
    at com.thinkaurelius.titan.diskstorage.Backend.<init>(Backend.java:112)
    at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:682)
    at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.<init>(StandardTitanGraph.java:72)
    at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:40)
    at com.thinkaurelius.titan.tinkerpop.rexster.TitanGraphConfiguration.configureGraphInstance(TitanGraphConfiguration.java:25)
    at com.tinkerpop.rexster.config.GraphConfigurationContainer.getGraphFromConfiguration(GraphConfigurationContainer.java:119)
    at com.tinkerpop.rexster.config.GraphConfigurationContainer.<init>(GraphConfigurationContainer.java:54)
    at com.tinkerpop.rexster.server.XmlRexsterApplication.reconfigure(XmlRexsterApplication.java:99)
    at com.tinkerpop.rexster.server.XmlRexsterApplication.<init>(XmlRexsterApplication.java:47)
    at com.tinkerpop.rexster.Application.<init>(Application.java:96)
    at com.tinkerpop.rexster.Application.main(Application.java:188)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at com.thinkaurelius.titan.diskstorage.Backend.instantiate(Backend.java:328)
    ... 13 more
Caused by: com.thinkaurelius.titan.diskstorage.TemporaryStorageException: Temporary failure in storage backend
    at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager.ensureKeyspaceExists(AstyanaxStoreManager.java:429)
    at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager.<init>(AstyanaxStoreManager.java:172)
    ... 18 more
Caused by: com.netflix.astyanax.connectionpool.exceptions.BadRequestException: BadRequestException: [host=127.0.0.1(127.0.0.1):9160, latency=42(60), attempts=1]InvalidRequestException(why:Keyspace names must be case-insensitively unique ("titan" conflicts with "titan"))
    at com.netflix.astyanax.thrift.ThriftConverter.ToConnectionPoolException(ThriftConverter.java:159)
    at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:65)
    at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:28)
    at com.netflix.astyanax.thrift.ThriftSyncConnectionFactoryImpl$ThriftConnection.execute(ThriftSyncConnectionFactoryImpl.java:151)
    at com.netflix.astyanax.connectionpool.impl.AbstractExecuteWithFailoverImpl.tryOperation(AbstractExecuteWithFailoverImpl.java:69)
    at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:256)
    at com.netflix.astyanax.thrift.ThriftClusterImpl.executeSchemaChangeOperation(ThriftClusterImpl.java:146)
    at com.netflix.astyanax.thrift.ThriftClusterImpl.addKeyspace(ThriftClusterImpl.java:246)
    at com.thinkaurelius.titan.diskstorage.cassandra.astyanax.AstyanaxStoreManager.ensureKeyspaceExists(AstyanaxStoreManager.java:424)
    ... 19 more
Caused by: InvalidRequestException(why:Keyspace names must be case-insensitively unique ("titan" conflicts with "titan"))
    at org.apache.cassandra.thrift.Cassandra$system_add_keyspace_result.read(Cassandra.java:33158)
    at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
    at org.apache.cassandra.thrift.Cassandra$Client.recv_system_add_keyspace(Cassandra.java:1408)
    at org.apache.cassandra.thrift.Cassandra$Client.system_add_keyspace(Cassandra.java:1395)
    at com.netflix.astyanax.thrift.ThriftClusterImpl$9.internalExecute(ThriftClusterImpl.java:250)
    at com.netflix.astyanax.thrift.ThriftClusterImpl$9.internalExecute(ThriftClusterImpl.java:247)
    at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:60)
    ... 26 more
[INFO] HttpReporterConfig - Configured HTTP Metric Reporter.
[INFO] ConsoleReporterConfig - Configured Console Metric Reporter.
[INFO] HttpRexsterServer - HTTP/REST thread pool configuration: kernal[4 / 4] worker[8 / 8] 
[INFO] HttpRexsterServer - Using org.glassfish.grizzly.strategies.LeaderFollowerNIOStrategy IOStrategy for HTTP/REST.
[INFO] HttpRexsterServer - Rexster Server running on: [http://localhost:8182]
[INFO] RexProRexsterServer - Using org.glassfish.grizzly.strategies.LeaderFollowerNIOStrategy IOStrategy for RexPro.
[INFO] RexProRexsterServer - RexPro thread pool configuration: kernal[4 / 4] worker[8 / 8] 
[INFO] RexProRexsterServer - Rexster configured with no security.
[INFO] RexProRexsterServer - RexPro Server bound to [0.0.0.0:8184]
[INFO] ShutdownManager$ShutdownSocketListener - Bound shutdown socket to /127.0.0.1:8183. Starting listener thread for shutdown requests.
^C

Flannelette answered 31/10, 2013 at 17:5 Comment(0)

As a first note, embeddedcassandra is no longer what you want in Titan 0.4.0. You can read more about that here. In the Titan Server distribution for 0.4.0 cassandra and rexster run in separate JVMs and should generally run out-of-the-box from the distribution.

Also note, that I would say it is recommended to create types/indices by way of the Gremlin Console directly. I like being "close to the graph" when working with TypeMaker. You can read more about such production implementation patterns here.

As for your specific problem, your issue helped uncover a hole in the documentation (which has since been remedied). To ensure that elasticsearch gets started with Titan Server make sure that you do:

bin/titan.sh -c cassandra-es start

At this point you can connect via Rexster to construct and query elasticsearch indices. Here's an example from Rexster Console:

rexster[groovy]> g = rexster.getGraph("graph")
==>titangraph[cassandra:null]
rexster[groovy]> g.makeKey("name").dataType(String.class).indexed("search",Vertex.class).make()
==>v[74]
rexster[groovy]> g.commit()
==>null
rexster[groovy]> g.addVertex([name:'marko'])
==>v[4]
rexster[groovy]> g.addVertex([name:'stephen'])
==>v[8]
rexster[groovy]> g.commit()
==>null
rexster[groovy]> g.V.has('name',PREFIX,'mar')
==>v[4]

Note that by starting Titan Server in this mode, elasticsearch is running in embedded mode to the Titan instance started by Rexster which means that:

Elasticsearch will not be accessible from outside of this particular Titan instance, i.e., remote connections will not be possible

So if you are trying to connect via a Titan Gremlin Console, I don't believe it will work. Connections have to run through Rexster.

Gauthier answered 31/10, 2013 at 23:32 Comment(13)

Alright, that seems to be starting. What index name should I use? In the documentation it says that you make elasticsearch indices like this: graph.makeKey("reason").dataType(String.class).indexed("search",Vertex.class).indexed("search", Edge.class).make(). When I do this I get an error, which I guess is because the search index does not exist (the stack trace won't show). How do I fix this? – Flannelette 1/11, 2013 at 11:33

Where are you trying to create the index from? Rexster Console? – Gauthier 1/11, 2013 at 12:2

Expanded my answer a bit to show index creation. Again, no extra configuration was required here...works out of the box. – Gauthier 1/11, 2013 at 12:17

Ahh.. I was using the gremlin console and TitanFactory('tmp/titan'). I thought I could not do this with rexster. Thanks Stephen! – Flannelette 1/11, 2013 at 14:4

Hate to bother you, but there is one last thing: When I stop the server with bin/titan.sh stop and then start it again with bin/titan.sh -c cassandra-es start the graph is gone. In rexster console when I do g = rexster.getGraph("graoh") I get null in return. Is there maybe something wrong with my setup? – Flannelette 1/11, 2013 at 14:29

By default, Titan Server creates its data directory in TITAN_SERVER_HOME/db. As long as that directory isn't being deleted between runs I can't think of why you would see it disappear. Is it possible you changed the defaults or are you just working with the out-of-the-box distribution as I was (I'm not seeing that problem by the way...my data is clearly persisted between shutdown/restart). I assume you mistyped the command you were using to get the graph in Rexster Console as the graph name is "graph" and not "graoh". – Gauthier 1/11, 2013 at 15:29

Yeah, that was a typo here on SO. I am using a brand spanking new download from github, no alterations. Maybe there is something strange going on with the machine I am prototyping on. I will try this on a new VM on azure and see how that goes. Will get back if I still have the issue. – Flannelette 1/11, 2013 at 17:54

The graph keeps on disappearing after I shut down the titan server and start it again. If you want I can give you the exact steps and tech stack (OS etc.) I am using all the way from the booting of the VM. I am simply doing bin/titan.sh -c cassandra-es start and then logging in with rexster console. Then I do bin/titan.sh stop and then do the same startup again, open the rexster console and then g = rexster.getGraph("graph") gives me null. – Flannelette 2/11, 2013 at 21:7

is the TITAN_SERVER_HOME/db directory still present after shutdown? how about on restart? i would verify that you know exactly where the data directory is and what its state is at each given moment. if you find that on some OS/JVM version it is somehow disappearing between stop/restart then that should be reported so it can be fixed. – Gauthier 2/11, 2013 at 21:11

The DB directory is there, it has two folders: cassandra and es. It appears that there is data in the cassandra system folder, but not in the titan folder. I am using openjdk-7-jdk and openjdk-7-jre. Where should I report this in the case it might be a bug? Is it GitHub that is used, or is it somewhere exsternal? – Flannelette 2/11, 2013 at 22:21

I can't recreate that problem even on that jdk. i will say that i have data in the titan TITAN_HOME/db/cassandra/data/titan directory though. Is there ever any data in that directory at any point during the time titan.sh is running? – Gauthier 3/11, 2013 at 11:26

Have a look at my update to the question. It has complete steps for recreating the problem. – Flannelette 3/11, 2013 at 12:8

@Cort3z The Titan Astyanax adapter (enabled by default when talking to Cassandra) sometimes throws that exception on startup if the Cassandra process hasn't completely started by the time Rexster + Titan tries to talk to it. 1.75 GB RAM is little for Rexster + Titan + Cassandra + ES. Try either extending the sleep time in titan.sh's start() method or switching to the "cassandrathrift" storage.backend (or both). – Jaguar 5/11, 2013 at 9:0

Maybe I'm a little late to the party, but i thought anyone else coming to this question could benefit.

Using this script will automatically setup embedded cassandra for "Titan 0.5.4 with Hadoop 2" I didn't test it with "titan-0.9.0-M2-hadoop1.zip", but if it works please let me know.

The script takes the current hostname ip and replaces the network ip settings in all the relevant configuration files (The Titan directory must be at: /opt/titan/, change accordingly).

#!/usr/bin/env bash
cd /tmp
wget http://s3.thinkaurelius.com/downloads/titan/titan-0.5.4-hadoop2.zip -O titan.zip
unzip titan.zip -d /opt/titan

//queries for current hostname ip - check your /etc/hosts file or this possibly won't work
titan_node_ip=$(hostname -i)
#/opt/titan/conf/cassandra.yaml
sed -i 's/seeds:.*$/seeds: "'$titan_node_ip'"/g' /opt/titan/conf/cassandra.yaml
sed -i 's/listen_address:.*$/listen_address: '$titan_node_ip'/g' /opt/titan/conf/cassandra.yaml
sed -i 's/rpc_address:.*/rpc_address: '$titan_node_ip'/g' /opt/titan/conf/cassandra.yaml

#/opt/titan/conf/elasticsearch.yml
sed -i 's/network.host:.*/network.host: '$titan_node_ip'/g' /opt/titan/conf/elasticsearch.yml

#/opt/titan/conf/rexster-cassandra-es.xml
sed -i 's/<base-uri>.*<\/base-uri>/<base-uri>http:\/\/'$titan_node_ip'<\/base-uri>/g' /opt/titan/conf/rexster-cassandra-es.xml
sed -i 's/<storage.hostname>.*<\/storage.hostname>/<storage.hostname>'$titan_node_ip'<\/storage.hostname>/g' /opt/titan/conf/rexster-cassandra-es.xml
sed -i 's/ <index.search.hostname>.*<\/index.search.hostname>/<index.search.hostname>'$titan_node_ip'<\/index.search.hostname>/g' /opt/titan/conf/rexster-cassandra-es.xml

#/opt/titan/conf/titan-cassandra-embedded-es.properties
sed -i 's/storage.backend=.*/storage.backend=embeddedcassandra/g' /opt/titan/conf/titan-cassandra-embedded-es.properties

#/opt/titan/bin/titan.sh
sed -i 's/ELASTICSEARCH_IP:=.*}/ELASTICSEARCH_IP:='$titan_node_ip'}/g' /opt/titan/bin/titan.sh
sed -i 's/REXSTER_IP:=.*}/REXSTER_IP:='$titan_node_ip'}/g' /opt/titan/bin/titan.sh

if all is well you should now be able to run titan successfully in cassandraembedded mode:

/opt/titan/bin/titan.sh start

Sublunary answered 9/8, 2015 at 4:33 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Recommended topics

Hot tags