How do I implement a DynamoDB Global Secondary Index with Infrastructure As Code in CloudFormation

Question

I am working on implementing a GSI in CloudFormation with Infrastructure As Code. All I want to do is use this table to keep count of the entries in the main DynamoTable. Here is what the main tale looks like:

Resources:
  CaseRecords:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: ${self:custom.tableName}
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: userId
          AttributeType: S
        - AttributeName: caseRecordId
          AttributeType: S
      KeySchema:
        - AttributeName: userId
          KeyType: HASH
        - AttributeName: caseRecordId
          KeyType: RANGE

I do not need the keys from the original table and all I want is to create a new HASH key for the new GSI that will tell me which table the count I am keeping track of came from, ie the table above.

Below is how I have tried to implement the GSI so far:

# Implement a GSI to handle item count totals
      GlobalSecondaryIndexes:
          - IndexName: gsiCaseCountTable
            KeySchema: 
              - AttributeName: table-name
                KeyType: HASH
            ProvisionedThroughput:
              ReadCapacityUnits: 5
              WriteCapacityUnits: 5

However, the error I get is as follows:

An error occurred: CaseRecords - Property Projection cannot be empty..

When I include the PROJECTION I was, with is only the userId from the original table simply to keep track of entry counts in the original table per user, I try the following:

 Implement a GSI to handle item count totals
      GlobalSecondaryIndexes:
      - IndexName: gsiCaseCountTable
        KeySchema:
        - AttributeName: table-name
          KeyType: HASH
        Projection:
          NonKeyAttributes:
          - userId
          ProjectionType: INCLUDE
        ProvisionedThroughput:
          ReadCapacityUnits: 5
          WriteCapacityUnits: 5

However this returns an error also:

An error occurred: CaseRecords - Property AttributeDefinitions is inconsistent with the KeySchema of the table and the secondary indexes.

How can I correctly implement a Global Secondary Index in Dynamo using a CloudFormation Template so I can record a count of entries in the original table????

Thanks.

UPDATE

In case anyone is wondering this is how I was able to deploy it. It is not a perfect solution but it lets me keep track and count the entries to the item table:

# NOTE: DynamoDB Serverless Configuration
# NoSQL Table for CaseRecord DB

Resources:
  CaseRecords:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: ${self:custom.tableName}
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: userId
          AttributeType: S
        - AttributeName: caseRecordId
          AttributeType: S
      KeySchema:
        - AttributeName: userId
          KeyType: HASH
        - AttributeName: caseRecordId
          KeyType: RANGE
      # Set the capacity based on the stage
      # ProvisionedThroughput:
        # ReadCapacityUnits: ${self:custom.tableThroughput}
        # WriteCapacityUnits: ${self:custom.tableThroughput}
      # Implement a GSI to handle item count totals
      GlobalSecondaryIndexes:
      - IndexName: gsiCaseCountTable
        KeySchema:
        - AttributeName: userId
          KeyType: HASH
        Projection:
          ProjectionType: KEYS_ONLY

UPDATE #2 - FAILING

Based on the info provided by @Pedro Arantes below, I am trying to implement the GSI with the Attribute Definitions that I want to use. This too however is failing. Below is the implementation and here is the link to the AWS Doc that I used: AWS GSI Doc and here is the failing implementation:

Resources:
  CaseRecords:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: ${self:custom.tableName}
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: userId
          AttributeType: S
        - AttributeName: caseRecordId
          AttributeType: S
        - AttributeName: table-name
          AttributeType: S
        - AttributeName: count
          AttributeType: N
      KeySchema:
        - AttributeName: userId
          KeyType: HASH
        - AttributeName: caseRecordId
          KeyType: RANGE
      # Set the capacity based on the stage
      # ProvisionedThroughput:
        # ReadCapacityUnits: ${self:custom.tableThroughput}
        # WriteCapacityUnits: ${self:custom.tableThroughput}
      # Implement a GSI to handle item count totals
      GlobalSecondaryIndexes:
      - IndexName: gsiCaseCountTable
        KeySchema:
        - AttributeName: table-name
          KeyType: HASH
        Projection: 
          NonKeyAttributes: 
            - userId
            - count
          ProjectionType: INCLUDE

How can I get this to work with only the NonKeyAttributes that I have declared in the AttributeDefinitions???

Tardif

Came here to fix my IaC GSI template, but on a side note this count functionality would be better architected using the new DynamoDB streams to store table totals in one record. This would reduce read costs and can store more interesting data. Streams triggers a DynamoDB whenever (under criteria) a record is Created/Updated/Deleted. – Esp

Answer 1

You need to add table-name at AttributeDefinitions property. From docs:

AttributeDefinitions

A list of attributes that describe the key schema for the table and indexes. Duplicates are allowed.

So even if you don't use some attribute in the original table, you must declare to be able to use in your GSIs.

UPDATE #2 - FAILING

You're using keys attributes userId and count that you defined at AttributeDefinitions as NonKeyAttributes (but they are keys attributes) at Projection. You don't need to add them because they're automatically projected. From docs:

AWS::DynamoDB::Table Projection

Represents attributes that are copied (projected) from the table into an index. These are in addition to the primary key attributes and index key attributes, which are automatically projected.

FINAL TEMPLATE

Resources:
  CaseRecords:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: ${self:custom.tableName}
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: userId
          AttributeType: S
        - AttributeName: caseRecordId
          AttributeType: S
        - AttributeName: table-name
          AttributeType: S
      KeySchema:
        - AttributeName: userId
          KeyType: HASH
        - AttributeName: caseRecordId
          KeyType: RANGE
      # Set the capacity based on the stage
      # ProvisionedThroughput:
      # ReadCapacityUnits: ${self:custom.tableThroughput}
      # WriteCapacityUnits: ${self:custom.tableThroughput}
      # Implement a GSI to handle item count totals
      GlobalSecondaryIndexes:
        - IndexName: gsiCaseCountTable
          KeySchema:
            - AttributeName: table-name
              KeyType: HASH
          Projection:
            NonKeyAttributes:
              - count
            ProjectionType: INCLUDE

Considerations:

1. count should not be on AttributeDefinitions because you're not using it as key.

2. You don't need to add userId at Projection because it'll be project automatically since it's defined in AttributeDefinitions.

Squatness

So if I want to key a field that only increments a counter, do I have to include that as an attribute in the original table? I am worried now that the implementation I completed above is now duplicating fields. – Tardif

would you mind taking a look at the second update I included in my question. It is still failing even though I have used the example shown in the docs here: docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/… What do you think???? – Tardif

You're adding keys attributes count and userId as NonKeyAttributes – Squatness

Here is the ERROR MSG: An error occurred: CaseRecords - Property AttributeDefinitions is inconsistent with the KeySchema of the table and the secondary indexes. – Tardif

Im trying to have a HASH key called table-name which I only will use in the GSI. I added it to the AttributeDefinitions so I can use it in GSI. That is the only Key I want. From there I really just want userId and count as normal fields without being keys.... Where am I going wrong here? – Tardif

Let us continue this discussion in chat. – Squatness