Keeping AWS CloudSearchDomain in sync with DynamoDB

Question

Im trying to add some flexible search functionality to my dynamoDB, so I set up the AWS CloudSearchDomain service. Which I believed could add a wrapper around my dynamoDB and retrieve documents with flexible search options.

Since implementing I realised that adding an item to my dynamoDB does not automatically add the item to the searchable CloudSearchDomain documents. The AWS docs advise to sync the DB items with the CloudSearchDomain periodically, such as at the end of each day. But I want my newly added items to be available for search as soon as its in the DB.

I also realise the documents that get returned from a CloudSearchDomain endpoint are not my plain json dynamoDB items. Each search result is wrapped in a 'hits' array, and all the property values are wrapped in arrays and its just a bit messy to handle on the front end. So is this the right tool for flexible searching the dynamodb. How are others using CloudSearchDomain with DynamoDB?

Im using the javascript sdk with node.js.

Industrialism

Have you considered using DynamoDB streams that trigger lambda, which then pushes updated records to CloudSearch? This should be fairly fast as lambdas are usually triggered within seconds – Italian

That sounds like a good workaround solution @TofigHasanov, but I feel like it's too many things going on just to search a DB. I haven't looked into pricing but I'd imagine having a dynamoDB + cloudsearch (with the same DB documents in it) could double the price. And with the lambda functions it sounds a bit over complicated then it needs to be – Industrialism

Then you need to reconsider using DynamoDB, as it is not designed for flexible searching. If you need to have storage and searching in one place think about going back to relational databases. They might not scale so well, but definitely will be cheaper and simpler – Italian

Also, it is a standard pattern for DynamoDB to have streams into S3 or some other storage so that these can be later processed for searching or analytics, so I wouldn't call it a complicated solution. – Italian

I see, thanks for your input. I'm new to AWS and the original goal was to have simple set up and deployments, I'm sure it will get easier to manage these different services over time. For now I'm testing out mongoDB deployed on an EC2 service, which so far is going well, and I'm hoping to use an elastic search wrapper – Industrialism