How to set limit of matching items returned by DynamoDB using Java?

Question

In my Android app, I want to query data from DynamoDB. There will be a thousand matching items, but I just want to get only the first 10 of them. I don't know how to set this limit. I found these lines in documentation:

The DynamoDB Query and Scan APIs allow a Limit value to restrict the size of the results.

In a request, set the Limit parameter to the number of items that you want DynamoDB to process before returning results.

In a response, DynamoDB returns all the matching results within the scope of the Limit value. For example, if you issue a Query or a Scan request with a Limit value of 6 and without a filter expression, DynamoDB returns the first six items in the table that match the specified key conditions in the request (or just the first six items in the case of a Scan with no filter). If you also supply a FilterExpression value, DynamoDB will return the items in the first six that also match the filter requirements (the number of results returned will be less than or equal to 6).

But I cannot find the way to set limit for response. I found the method SetCount of QueryResult:

QueryResult result2 = dynamodb.query(request);
    result2.setCount(5);

It said that: then Count is the number of items returned after the filter was applied

But I think it is not what I want. Because DynamoDb still returns all the matching items before calling setCount. Can any one help me?

Nickelic

You need to use AWS SDK I guess docs.aws.amazon.com/amazondynamodb/latest/developerguide/… – Impaste

@Nambari Could you please give me more information? I don't see the Limit clause in your link. – Nickelic

mobile.awsblog.com/post/Tx1U4RV2QI1MVWS/… – Impaste

@Nambari Thanks for the link, but it is not what I want. I want the limit of returned matching items. – Nickelic

Answer 1

You need to apply the limit as part of the request that you send to the API, not on the response.

I assume the request object you are submitting to the dynamodb object is a QuerySpec. What you will want to do is is call withMaxResultSize to pass in the limit you want applied before the query is run against the API.

However, as you mentioned in your question you need to make sure you understand the behavior of limit as described in the DynamoDB documentation on Limits:

In a response, DynamoDB returns all the matching results within the scope of the Limit value. For example, if you issue a Query or a Scan request with a Limit value of 6 and without a filter expression, DynamoDB returns the first six items in the table that match the specified key conditions in the request (or just the first six items in the case of a Scan with no filter). If you also supply a FilterExpression value, DynamoDB will return the items in the first six that also match the filter requirements (the number of results returned will be less than or equal to 6).

What this means is that if you are not using a FilterExpression you are likely fine. However, if you are filtering the results you will likely receive fewer than your limit because the limit is technically not the number of results to return rather the number of items DynamoDB could potentially return.

It sounds like you are asking for a way to have DynamoDB limit the number of results it returns to an exact number while having a FilterExpression applied. Unfortunately this is currently not possible with DynamoDB.

Apteryx

Thank you very much. I understand. Currently I want to use DynamoDBQueryExpression and DynamoDBMapper to get Items. But the method setLimit(integer) of DynamoDBQueryExpression doesn't work. For example, if I call setLimit(10), it still returns all the 50 matching items. – Nickelic

I do not believe setLimit is the correct method, you should be able to call withMaxResultSize on the QuerySpec to specify the limit so it applied as part of the request. If it is still not working an additional code snippet may help clarify. – Apteryx

The docs for DynamoDBQueryExpression.setLimit explicitly say "Note that when calling DynamoDBMapper.query, multiple requests are made to DynamoDB if needed to retrieve the entire result set. Setting this will limit the number of items retrieved by each request, NOT the total number of results that will be retrieved. Use DynamoDBMapper.queryPage to retrieve a single page of items from DynamoDB." – Blackburn

Note: MaxResultSize is just a syntactic sugar which internally uses N PageSized network calls to retrieve the results. DynamoDB currently offers no way to do this in a single network call. Ref code from aws sdk: github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-dynamodb/… – Pleven

Answer 2

This is how you can limit a query result using aws-java-sdk ver 1.10.61

import com.amazonaws.AmazonClientException;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.auth.DefaultAWSCredentialsProviderChain;
import com.amazonaws.regions.Region;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBAsyncClient;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.ItemCollection;
import com.amazonaws.services.dynamodbv2.document.QueryOutcome;
import com.amazonaws.services.dynamodbv2.document.spec.QuerySpec; 



AWSCredentialsProvider provider = new DefaultAWSCredentialsProviderChain();
AmazonDynamoDBAsyncClient client = new AmazonDynamoDBAsyncClient(provider);
Region region = Region.getRegion(Regions.fromName(DYNAMODB_REGION));
client.setRegion(region);
dynamoDB = new DynamoDB(client);

QuerySpec querySpec = new QuerySpec();

/* MAX 1 item */
querySpec.setMaxResultSize(1);

querySpec.withHashKey("myPartitionKeyName", partitionKeyValue);

ItemCollection<QueryOutcome> query = dynamoDB.getTable(DYNAMODB_TABLE).query(querySpec);

Redwing

There is also a .withMaxResultSize(x) if you are using the builder pattern to put queries together. – Stpierre

Answer 3

This is how you can limit your data which is coming from dynamodb using java.

ItemCollection<ScanOutcome> collection = null;

try {
    
    DynamoDB dynamoDB = DynamoDBClientUtil
           .getDynamoDBClient(accessKeyId, secretAccessKey, arnRole);          
    
    Table table = dynamoDB.getTable(tableName);
    ScanSpec specification = new ScanSpec();        
    specification.setMaxResultSize(10);             
    collection = table.scan(specification);

} catch (Exception ex) {
    log.error(ex.getMessage());
}

return collection;

Preoccupy