Amazon DynamoDB :- Invalid UpdateExpression: Expression size has exceeded the maximum allowed size dynamodb

Question

I am trying to update an item in was dynamoDB using nodes, db.updateItem(query).

I am getting the following error :

Invalid UpdateExpression: Expression size has exceeded the maximum allowed size dynamodb

On reading few posts, I realised that dynamoDB allows itemSize to be 400KB and that might be a problem here. But if that is the problem, why did it allow to insert the item in the first place.

I am not sure what exactly the issue. Any help would be appreciated.

Please let me know if I missed any required information

Saphra

Answer 1

You are probably hitting Expression Parameters limits. Please refer to:

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-expression-parameters

Stockbreeder

Yes, I am crossing the Expression Parameters limit. The object I am updating is huge and have more than 40 embedded objects to be updated. The query being formed is absolutely correct, but the parameters size is big. Any idea how to make it work? – Saphra

Maybe you have to split your query? Update less objects per query? – Stockbreeder

Downvote because this doesn't offer any solution. – Chaussure

Answer 2

If you are getting this exception

software.amazon.awssdk.services.dynamodb.model.DynamoDbException: Item size has exceeded the maximum allowed size

This exception is due to AWS Dynamodb limits mentioned here

in my case, I compressed the record using gzip and stored binary zipped data, and uncompressed it back after reading that record.

please see below sample code to compress and decompress (I am using enhanced dynamodb client library)

public CompletableFuture<Boolean> storeItem(MyBeanClass object) {

    CompletableFuture<Boolean> completableFuture = CompletableFuture.supplyAsync(() -> false);
    try {
        byte[] serialized = objectMapper.writeValueAsString(object.getLargeData()).getBytes(UTF_8);
        if (serialized.length >= 10000) { //large record, gzip it

            try (ByteArrayOutputStream bos = new ByteArrayOutputStream(serialized.length);
                 GZIPOutputStream gzip = new GZIPOutputStream(bos)) {
                gzip.write(serialized);
                gzip.close();
                MyBeanClass newObject = new MyBeanClass();
                newObject.setPrimaryId(object.getPrimaryId());
                newObject.setGzData(SdkBytes.fromByteArray(bos.toByteArray()));
                completableFuture = enhancedDynamoDbTable.putItem(newObject)
                        .thenApply(res -> true)
                        .exceptionally(th -> {
                            th.printStackTrace();
                            return false;
                        });
            }

        } else { //no compression required
            completableFuture = enhancedDynamoDbTable.putItem(object).thenApply(res -> true)
                    .exceptionally(th -> {
                        th.printStackTrace();
                        return false;
                    });
        }
    } catch (IOException e) {
        e.printStackTrace();
    }

To fetch record and unzip

public CompletableFuture<MyBeanClass> getItem(String id) {

    return enhancedDynamoDbTable
            .getItem(Key.builder().partitionValue(id).build())
            .thenApply(record -> {
                if (record.getGzData() != null) {
                    try (ByteArrayInputStream arrayInputStream = new ByteArrayInputStream(record.getGzData().asByteArray());
                         GZIPInputStream inputStream = new GZIPInputStream(arrayInputStream);
                         ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream()) {

                        byte[] buffer = new byte[1024];
                        int length;
                        while ((length = inputStream.read(buffer)) != -1) {
                            byteArrayOutputStream.write(buffer, 0, length);
                        }
                        record = objectMapper.readValue(byteArrayOutputStream.toString(UTF_8), MyBeanClass.class);
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                }
                return record;
            });
}

Hope that helps.

Ventilate