Truncate DynamoDb or rewrite data via Data Pipeline

Question

There is possibility to dump DynamoDb via Data Pipeline and also import data in DynamoDb. Import is going well, but all the time data appends to already exists data in DynamoDb.

For now I found work examples that scan DynamoDb and delete items one by one or via Batch. But at any rate for big amount of data it is not good variant.

Also it is possible to delete table at all and create it. But with that variant indexes will be lost.

So, best way would be to override DynamoDb data via import by Data Pipeline or truncate somehow. Is it possible to do? And how is it possible if yes?

Carroty

Possible duplicate of Deletion from amazon dynamodb – Respectability

Answer 1

Truncate Table functionality is not available in DynamoDB, So kindly consider deleting the table & creating again,

Reason : DynamoDB Charges you based on ReadCapacityUnits & WriteCapacityUnits which you have used. If you delete all items using BatchWriteItem function, it will use WriteCapacityUnits. So, to save these WriteCapacityUnits for deleting items, It will be better if you truncate the table & recreate it agian.

Steps to Delete & Create DynamoDB Tables as follows :

Delete Table via AWS CLI :

aws dynamodb delete-table --table-name *tableName*

Delete Table via AmazonDynamoDB API :

Sample Request

POST / HTTP/1.1
Host: dynamodb.<region>.<domain>;
Accept-Encoding: identity
Content-Length: <PayloadSizeBytes>     
User-Agent: <UserAgentString>
Content-Type: application/x-amz-json-1.0
Authorization: AWS4-HMAC-SHA256 Credential=<Credential>, SignedHeaders=<Headers>, Signature=<Signature>
X-Amz-Date: <Date> 
X-Amz-Target: DynamoDB_20120810.DeleteTable 

{
    "TableName": "Reply"
}

Creating DynamoDB Table via AmazonDynamoDB API :

POST / HTTP/1.1
Host: dynamodb.<region>.<domain>;
Accept-Encoding: identity
Content-Length: <PayloadSizeBytes>     
User-Agent: <UserAgentString>
Content-Type: application/x-amz-json-1.0
Authorization: AWS4-HMAC-SHA256 Credential=<Credential>, SignedHeaders=<Headers>, Signature=<Signature>
X-Amz-Date: <Date> 
X-Amz-Target: DynamoDB_20120810.CreateTable 

{
    "AttributeDefinitions": [
        {
            "AttributeName": "ForumName",
            "AttributeType": "S"
        },
        {
            "AttributeName": "Subject",
            "AttributeType": "S"
        },
        {
            "AttributeName": "LastPostDateTime",
            "AttributeType": "S"
        }
    ],
    "TableName": "Thread",
    "KeySchema": [
        {
            "AttributeName": "ForumName",
            "KeyType": "HASH"
        },
        {
            "AttributeName": "Subject",
            "KeyType": "RANGE"
        }
    ],
    "LocalSecondaryIndexes": [
        {
            "IndexName": "LastPostIndex",
            "KeySchema": [
                {
                    "AttributeName": "ForumName",
                    "KeyType": "HASH"
                },
                {
                    "AttributeName": "LastPostDateTime",
                    "KeyType": "RANGE"
                }
            ],
            "Projection": {
                "ProjectionType": "KEYS_ONLY"
            }
        }
    ],
    "ProvisionedThroughput": {
        "ReadCapacityUnits": 5,
        "WriteCapacityUnits": 5
    }
}

Summary : Delete the table & Create it again would be the best solution.

Respectability

Thank you. So - yes. Delete and then create way is best way for now. Before when I create new table via cli command I had problem with indexes. But now fixed it and everything is fine. – Carroty

Don't follow this advise if you have partitioned your dynamodb table. Re-Partitioning could be way more expensive. – Overstretch