AWS SDK S3 node.js connect to local MinIO server

Question

I have application server written in Node.js which upload files to AWS S3 storage. For that I'm using https://www.npmjs.com/package/aws-sdk and when I'm connecting to and uploading to my AWS production storage it is working fine. However during development I want to upload files to local MinIO server (create from docker image https://hub.docker.com/r/minio/minio/):

docker run -p 9000:9000 -p 9001:9001 \
  quay.io/minio/minio server /minio --console-address ":9001"

MinIO server itself is working fine, I'm able to login there locally, create buckets, etc. However when I tried to upload file to my local MinIO storage using AWS SDK for node.js I'm unable to do so:

import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';

#s3Client = new S3Client({
    credentials: {
        accessKeyId: config.getAwsStorageAccessKeyId(), // from env variables
        secretAccessKey: config.getAwsStorageSecretAccessKey(),
    },
    ...(globalConfig.isReleaseVersion() && {
        endpoint: config.getAwsStorageEndpoint(), // from env "http://localhost:9001"
    }),
});

And later I'm trying to upload object:

await this.#s3Client.send(new PutObjectCommand({
    Bucket: config.getAwsStorageWebflowBucketName(), // from env "test" value
    Key: 'example.json',
    Body: body, // a buffer
}));

I'm receiving error:

ERROR EndpointError: Custom endpoint `test.127.0.0.1://9001/` was not a valid URI

However I'm able to reach address http://test.localhost:9001/ from my browser. When I try to pass directly value 'http://localhost:9001/' as an endpoint to S3Client configuration, I'm getting error:

ERROR Error: getaddrinfo ENOTFOUND test.localhost

Looks like port value is ignored my SDK. How can this be fixed?

<Yes, I know there is separate MinIO SDK - I want to use one provided by AWS, existence of endpoint parameter in configuration implies that it should be possible>

Maquis

Answer 1

http://localhost:9001 is not the correct URL

• 9001 is the one where you access the minio console
• 9000 is the one where S3 API is served

Console output when running minio

Try following & it should work

const s3Client = new S3Client({
  region: "region_set_in_minio",
  credentials: {
    accessKeyId: "minio_access_key",
    secretAccessKey: "minio_secret_access_key",
  },
  endpoint: "http://127.0.0.1:9000",
});

You can set minio region from minio console.

Go to Administrator -> Settings -> Region

Change minio region

Uigur

Answer 2

All you need to do is set s3ForcePathStyle config option to true. Alternatively (in sdk V3) it can have name forcePathStyle. In my code I do it this way:

const awsConfig: S3.Types.ClientConfiguration = {
  s3ForcePathStyle: true
};

I understand that there's a similar answer but it did resolve my issue.

Additional options that I also supplied:

sslEnabled: false
endpoint: http://localhost:port/
accessKeyId: dummy
secretAccessKey: dummydummy

Situated

Answer 3

Have you tried the option "s3ForcePathStyle" ? I had to use this config to make it work on a local environment

  options.s3ForcePathStyle = true 

See https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Config.html#s3ForcePathStyle-property

So in your code

...(globalConfig.isReleaseVersion() && {
    endpoint: config.getAwsStorageEndpoint(), // from env "http://localhost:9001",
    s3ForcePathStyle: true,
}),

Teeth

Yes I'm still getting ERROR Error: getaddrinfo ENOTFOUND test.localhost - port is still missing – Maquis

Answer 4

the listener endpoint for S3 API is port 9000, 9001 is for dashboard, also remove region when using Minio, because I think Minio doesn't need it. by so, sdk will not give a prefix 'test' for the endpoint.

Hackamore

Answer 5

I saw here correct answers (@gladix and @p2t2p - I have upvoted, of course). But I think we should make it a bit clearer:

AWS SDK V3

In the question I see:

import { S3Client } from '@aws-sdk/client-s3';

It means he uses V3 of the AWS SDK. This is important, because in V3 the config passed to the constructor of the client is of type S3ClientConfig.

import { S3ClientConfig } from "@aws-sdk/client-s3";

And there, there is a member forcePathStyle: boolean.

AWS SDK V2

However, if user uses V2 (like import * as AWS from 'aws-sdk'), then the config passed to the constructor of the client is of a different type, ClientConfiguration.

import { ClientConfiguration } from "aws-sdk/clients/s3";

And there, there is a member s3ForcePathStyle: boolean.

Hope that makes things clearer :)

Quackenbush