Dynamo Local from Node-aws: all operations fail "Cannot do operations on a non-existent table"

Question

I have a local dynamo-db running. I have set up my tables using the JavaScript console and they list OK from there.

I can also put and get items to my tables from the JavaScript console:

var params = { TableName:"environmentId", Item: { environmentId: {"S":"a4fe1736-98cf-4560-bcf4-cc927730dd1b"} }};
dynamodb.putItem(params, function(err, data) {
    console.log("put : err was " + JSON.stringify(err) + " and data is " + JSON.stringify(data));
});

prints put : err was null and data is {} which I'm assuming is "success" because

params = { "Key":{"environmentId":{"S":"a4fe1736-98cf-45e0-bcf4-cc927730dd1b"}},"TableName":"environmentId"}
dynamodb.getItem(params, function(err, data) {
    console.log("get : err was " + JSON.stringify(err) + " and data is " + JSON.stringify(data));
});

prints get : err was null and data is {"Item":{"environmentId":{"S":"a4fe1736-98cf-45e0-bcf4-cc927730dd1b"}}} i.e. it retrieves the object I just put to the table.

However, if it fire up the node REPL and type:

var AWS = require('aws-sdk');
AWS.config.loadFromPath("./config/credentials.js");
endpoint = new AWS.Endpoint("http://localhost:8000");
var dynamoOpts = {apiVersion: '2012-08-10', 'endpoint':endpoint};
var dynamodb = new AWS.DynamoDB(dynamoOpts);
var params = { TableName:"environmentId", Item: { environmentId: {"S":"a4fe1736-98cf-4560-bcf4-cc927730dd1b"} }};
dynamodb.putItem(params, function(err, data) {
    console.log("put : err was " + JSON.stringify(err) + " and data is " + JSON.stringify(data));
}

I get a resource not found error:

{ "message":"Cannot do operations on a non-existent table",
    "code":"ResourceNotFoundException",
    "time":"2015-04 10T10:01:26.319Z",
    "statusCode":400,
    "retryable":false,
    "retryDelay":0
}

The ASW.request object returned from the putCommand has the correct endpoint:

{ protocol: 'http:',
    host: 'localhost:8000',
    port: 8000,
    hostname: 'localhost',
    pathname: '/',
    // etc.

The same thing happens from my Node app however the same code connecting to the real AWS hosted dynamo works.

Shuddering

this could be tagged with other languages too, same behavior for .net. – Sphygmic

Answer 1

The problem is that the JavaScript console and your app use different profiles (credential and region) and therefore DynamoDB local will create separate database files for them. By using the -sharedDb flag when starting the local DynamoDB, a single database file will be shared for all clients.

From the doc:

-sharedDb — DynamoDB Local will use a single database file, instead of using separate files for each credential and region. If you specify -sharedDb, all DynamoDB Local clients will interact with the same set of tables regardless of their region and credential configuration.

Shawnna

Thanks. The fact that the JavaScript console would use a different db was not at all clear from that doc line for me. – Shuddering

You can use a docker container with volumes setted - docker run -d -p 8000:8000 -v /tmp/data:/data/ dwmkerr/dynamodb -dbPath /data/ run-amazon-dynamodb-locally-with-docker Note: You need to setup a Access Key Id on Dynamo JS shell to: DynamoDB Local loses tables? – Cherian

There is an official docker image available from Amazon now on Docker hub. hub.docker.com/r/amazon/dynamodb-local – Egidius

Thank you so much! I wish I saw this earlier – Misquote

I personally had control over everything, so it was better for me to keep the in-memory tables and just make sure that regions and credentials were identical in all test fixtures. Though, I did have to run tcpdump on port 8000 before I found the last set of bad credentials. – Geographical

Answer 2

Those who are using the official DynamoDB Local Docker image should use this line to start it to enable sharedDb:

docker run -p 8000:8000 amazon/dynamodb-local -jar DynamoDBLocal.jar -inMemory -sharedDb

The original ENTRYPOINT and CMD used by the image can be seen in docker inspect amazon/dynamodb-local output and are:

"Entrypoint": [
    "java"
]
"Cmd": [
    "-jar",
    "DynamoDBLocal.jar",
    "-inMemory"
]

So we basically need to copy them and add -sharedDb.

Gilpin

dear god you are a life saver! – Impromptu

well what can i say, i owe you like 40 hours of life – Kirsti

I am facing this error 'Unrecognized option: -inMemory ' – Elviaelvie

Alternatively, make sure all your clients use the same access key ID and the right region. – Expunge

Answer 3

If you are using CircleCI, here is what your executor should look like:

# using service containers on CircleCI
# https://circleci.com/docs/2.0/databases/
executors:
  with-dynamodb:
    docker:
      # image used to install source code,
      # run our server and run Cypress tests
      - image: cypress/base:14.17.3

      # image used to run Dynamodb in a separate container
      - image: amazon/dynamodb-local:1.17.0
        # HERE IS THE IMPORTANT PART
        command: ["-jar", "DynamoDBLocal.jar", "-inMemory", "-sharedDb"]

The command option allows you to override the default CMD in the Dockerfile, pointed out by @madhead.

Fleece