amazon-kinesis Questions
5
Solved
Before sending the data I am using JSON.stringify to the data and it looks like this
{"data": [{"key1": value1, "key2": value2}, {"key1": value1, "key2": value2}]}
But once it passes through AWS...
Feminine asked 12/1, 2018 at 12:38
2
Solved
As reported in the AWS documentation
The maximum size of the data payload of a record before base64-encoding is up to 1 MiB.
Since that I need to process records that may have the size larger ...
Carding asked 7/12, 2018 at 14:50
14
I am writing record to Kinesis Firehose stream that is eventually written to a S3 file by Amazon Kinesis Firehose.
My record object looks like
ItemPurchase {
String personId,
String itemId
}
...
Cacophonous asked 26/12, 2015 at 3:48
6
Solved
I am working on an application that will read and analyze the logs of payment transactions. I know I will use Kinesis Analytics as per my requirements, which takes the input from the Data Str...
Lelahleland asked 27/9, 2020 at 21:3
5
I have used to be able to send a record to firehose without any problem like this
aws firehose put-record --delivery-stream-name my-stream --record='Data="{\"foor\":\"bar\"...
Cubital asked 8/7, 2020 at 21:19
2
I'm trying to use Boto3 to get a video stream from kinesis and then use OpenCV to display the feed and save it to a file at the same time.
The process of getting the signed URL and then the Getmed...
Marsha asked 27/4, 2018 at 11:40
4
Solved
I'm trying to set up Lambda transformations with a Firehose delivery stream. I have an IAM role defined for the Firehose which includes the following policy document:
{
"Statement": {
"Action": ...
Mobocracy asked 8/3, 2018 at 19:8
4
Solved
I have a very simple lambda function (nodeJS) which put the event received in kinesis stream. Here is the source code:
'use strict';
const AWS = require('aws-sdk');
const kinesis = new AWS.Ki...
Mesoglea asked 11/1, 2019 at 9:45
4
Solved
I am trying to connect to my RDS instance from a lambda. I wrote the lambda locally and tested locally, and everything worked peachy. I deploy to lambda, and suddenly it doesn't work. Below is the ...
Layla asked 31/5, 2016 at 21:9
2
Solved
What is the difference between the two? Also, what does it mean to have a kinesis iterator age of 12 hours and lambda iterator age of 2 hours. The configuration is very simple & basic (i.e only...
Bicarbonate asked 26/3, 2022 at 10:15
13
Solved
I have a use case where there will be stream of data coming and I cannot consume it at the same pace and need a buffer. This can be solved using an SNS-SQS queue. I came to know the Kinesis solves ...
Stilt asked 29/10, 2014 at 6:1
1
I am using Kinesis Analytics to read in JSON from Kinesis Firehose. I am successfully filtering out some of the records and writing a subset of the JSON properties to another Firehose.
I wanted to...
Perfectionism asked 6/10, 2017 at 12:34
6
Amazon claims their Kinesis streaming product guarantees record ordering.
It provides ordering of records, as well as the ability to read and/or replay records in the same order (...)
Kinesis ...
Taft asked 31/8, 2016 at 10:8
1
The scenario is I would be writing large volumes of data ( terabytes per day) to kinesis stream.I want to know which is a better way to achieve high write throughput.
I am considering the below two...
Dielectric asked 22/4, 2019 at 16:53
6
Solved
Firehose->S3 uses the current date as a prefix for creating keys in S3. So this partitions the data by the time the record is written. My firehose stream contains events which have a specific event...
Skep asked 9/2, 2017 at 6:22
4
I have an AWS Kinesis Firehose stream putting data in s3 with the following config:
S3 buffer size (MB)* 2
S3 buffer interval (sec)* 60
Everything works fine. The only problem is that Firehose c...
Hinz asked 28/4, 2016 at 17:9
3
Solved
I read you can have multiple consumer apps per kinesis stream.
http://docs.aws.amazon.com/kinesis/latest/dev/developing-consumers-with-kcl.html
however, I heard you can only have one consumer per s...
Traction asked 29/12, 2015 at 1:34
3
Solved
I'm very new to Amazon Kinesis so maybe this is just a problem in my understanding but in the AWS Lambda FAQ it says:
The Amazon Kinesis and DynamoDB Streams records sent to your AWS Lambda func...
Gapin asked 10/9, 2015 at 12:30
3
Solved
They seem to be doing the same thing to me. Can anyone explain to me the difference?
Expense asked 25/7, 2016 at 15:33
3
Solved
I was reading about AWS Kinesis. In the following program, I write data into the stream named TestStream. I ran this piece of code 10 times, inserting 10 records into the stream.
var params = {
D...
Auric asked 23/1, 2018 at 10:52
3
Solved
I am trying to setup a sync between AWS Aurora and Redshift. What is the best way to achieve this sync?
Possible ways to sync can be: -
Query table to find changes in a table(since I am only doi...
Bullnose asked 16/6, 2017 at 21:53
2
OK, I'll start with an elaborated use-case and will explain my question:
I use a 3rd party web analytics platform which utilizes AWS Kinesis streams in order to pass data from the client into the...
Soddy asked 17/9, 2014 at 12:44
1
Solved
I'm very confused with the concept of ParallelizationFactor.
My understanding
https://stackoverflow.com/a/57534322/13000229
In the past, one KDS shard can send data to only one Lambda instance/inv...
Bouldon asked 20/2, 2022 at 11:52
4
I have a Kinesis Firehose configuration in Terraform, which reads data from Kinesis stream in JSON, converts it to Parquet using Glue and writes to S3.
There is something wrong with data format con...
Enclose asked 25/6, 2021 at 4:36
3
Solved
We have a write-intensive table (on AWS RDS MySQL) from a legacy system and we'd like to stream every write event (insert or updated) from that table to kinesis. The idea is to create a pipe to war...
Ishmael asked 17/8, 2016 at 23:24
1 Next >
© 2022 - 2025 — McMap. All rights reserved.