Exception on SubscribeToShard Command

Question

I'm trying to subscribe to events from Kinesis Shard. But the execution of SubscribeToShardCommand hangs for 5 minutes (timeout for subscribe) and then throws an error:

(node:2667) UnhandledPromiseRejectionWarning: SyntaxError: Unexpected token  in JSON at position 0
    at JSON.parse (<anonymous>)
    at /home/mjpolak/Documents/company/project/node_modules/@aws-sdk/client-kinesis/protocols/Aws_json1_1.ts:4020:19
    at processTicksAndRejections (internal/process/task_queues.js:88:5)
    at Object.deserializeAws_json1_1SubscribeToShardCommand (/home/mjpolak/Documents/company/project/node_modules/@aws-sdk/client-kinesis/protocols/Aws_json1_1.ts:2583:21)
    at /home/mjpolak/Documents/company/project/node_modules/@aws-sdk/middleware-serde/src/deserializerMiddleware.ts:20:18
    at /home/mjpolak/Documents/company/project/node_modules/@aws-sdk/middleware-signing/src/middleware.ts:26:22
    at StandardRetryStrategy.retry (/home/mjpolak/Documents/company/project/node_modules/@aws-sdk/middleware-retry/src/defaultStrategy.ts:125:38)
    at /home/mjpolak/Documents/company/project/node_modules/@aws-sdk/middleware-logger/src/loggerMiddleware.ts:21:20
    at /home/mjpolak/Documents/company/project/kinesis-subscribe.ts:85:29

I'm looking for help that allows me to subscribe with success.

Complete code:

const { StreamDescription } = await client.send(
        new DescribeStreamCommand({
            StreamName: 'streamName',
        }),
    );
    const { StreamARN } = StreamDescription;

    const { Consumers } = await client.send(
        new ListStreamConsumersCommand({
            StreamARN,
        }),
    );

    let Consumer = Consumers.find((x) => x.ConsumerName == 'tester');

    if (Consumer == null) {
        const consumerData = await client.send(
            new RegisterStreamConsumerCommand({
                ConsumerName: 'tester',
                StreamARN,
            }),
        );
        Consumer = consumerData.Consumer;
    }

    console.log(`StreaARM: ${StreamARN}`);

    console.log(`Consumer: ${Consumer.ConsumerARN}`);

    const { EventStream } = await client.send(
        new SubscribeToShardCommand({
            ConsumerARN: Consumer.ConsumerARN,
            ShardId: 'shardId-000000000000',
            StartingPosition: {
                Type: 'LATEST'
            },
        }),
    );
    console.log("Consuming");

By investigating communication via Wireshark, I can confirm that HTTP connection is established, and AWS pushes some data to my client. So I'm guessing that there is some kind of bug in client library that hold block instead of returning the result.

I Also created bug on SDK repository: https://github.com/aws/aws-sdk-js-v3/issues/2418

Untie

Answer 1

Logs shows that error is thrown for JSON parsing and this occurs when you are receiving HTML or XML format response but expected is JSON response. Try to log the response on the console and see if it's in correct format.

Housman

Sorry, but I would like to get an answer that explains how to use AWS SDK that allows to use all of its features, not necessarily dive into implementation details. – Untie